升级obproxy异常

OceanBase
#1

【产品名称】oceanbase

【产品版本】3.1.2

【问题描述】升级obproxy异常

obd 1.1.1升级到1.2.1正常；

oceanbase-ce 3.1.1升级到3.1.2正常；

obproxy 3.1.0升级到3.1.2异常：

[admin@obp1 ~]$ obd cluster upgrade product -c obproxy -V 3.2.0 Get local repositories and plugins ok ±--------------------------------------------------------------------------------------------------------+ | Search obproxy 3.2.0 Result | ±--------±--------±--------±-------±-----------------------------------------------------------------+ | name | version | release | arch | md5 | ±--------±--------±--------±-------±-----------------------------------------------------------------+ | obproxy | 3.2.0 | 1.el7 | x86_64 | c2ad988860a0fe5c07cbe3755fab0e2c165cf7a771e6c1f11741cc7208bf13d3 | ±--------±--------±--------±-------±-----------------------------------------------------------------+ Download obproxy-3.2.0-1.el7.x86_64.rpm (7.80 M): 100% [####################################################################################] Time: 0:00:05 1.38 MB/s install obproxy-3.2.0 for local ok Open ssh connection ok [WARN] No such upgrade_route plugin for obproxy-3.1.0 [WARN] No such upgrade_check plugin for obproxy-3.1.0 [WARN] No such upgrade_check plugin for obproxy-3.2.0 ±----------------------------------------------------------------------------------------+ | Packages Will Be Used | ±--------±--------±--------±-------±-----------------------------------------±------+ | name | version | release | arch | md5 | mark | ±--------±--------±--------±-------±-----------------------------------------±------+ | obproxy | 3.1.0 | 1.el7 | x86_64 | 0b17cf0459a3b53c5a2febb6572894d183154c64 | start | | obproxy | 3.2.0 | 1.el7 | x86_64 | 8d5c6978f988935dc3da1dbec208914668dcf3b2 | dest | ±--------±--------±--------±-------±-----------------------------------------±------+ If you use a non-official release, we cannot guarantee a successful upgrade or technical support when you fail. Make sure that you want to use the above package to upgrade. [y/n]: y Remote obproxy-3.2.0-8d5c6978f988935dc3da1dbec208914668dcf3b2 repository install ok Remote obproxy-3.2.0-8d5c6978f988935dc3da1dbec208914668dcf3b2 repository lib check ok Stop obproxy ok Start obproxy ok obproxy program health check ok Connect to obproxy x #升级异常

再次升级仍旧报错

[admin@obp1 ~]$ obd cluster upgrade product -c obproxy -V 3.2.0 Get local repositories and plugins ok Open ssh connection ok Remote obproxy-3.2.0-8d5c6978f988935dc3da1dbec208914668dcf3b2 repository install ok Remote obproxy-3.2.0-8d5c6978f988935dc3da1dbec208914668dcf3b2 repository lib check ok Stop obproxy ok Start obproxy ok obproxy program health check ok

#2

日志截图：

#3

此时通过obproxy连接db正常：

[admin@obp1 ~]$ obclient -hobp1 -P2883 -uroot@sys#product oceanbase -c -A -p'mw)))pwd5w/m'
Welcome to the OceanBase. Commands end with ; or \g.
Your MySQL connection id is 1048577
Server version: 5.6.25 OceanBase 3.1.2 (r10000392021123010-d4ace121deae5b81d8f0b40afbc4c02705b7fc1d) (Built Dec 30 2021 02:47:29)

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MySQL [oceanbase]> show databases;
+--------------------+
| Database      |
+--------------------+
| oceanbase     |
| information_schema |
| mysql       |
| SYS        |
| LBACSYS      |
| ORAAUDITOR     |
| test        |
+--------------------+
7 rows in set (0.034 sec)

MySQL [oceanbase]> exit
Bye

查看版本也升级成功：

[admin@obp1 ~]$ /home/admin/obproxy/bin/obproxy -V
/home/admin/obproxy/bin/obproxy -V
obproxy (OceanBase 3.2.0 1)
REVISION: 58-local-b91a4006ba52e107d1a3edb03a50f3d14ef63cd8
BUILD_TIME: Nov 3 2021 06:51:33
BUILD_FLAGS: -g -O2 -D_OB_VERSION=1000 -D_NO_EXCEPTION -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -D__USE_LARGEFILE64 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILE -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -Wall -Wextra -Wunused-parameter -Wformat -Wconversion -Wno-deprecated -Wno-invalid-offsetof -finline-functions -fno-strict-aliasing -mtune=core2 -Wno-psabi -Wno-sign-compare -DGCC_52 -DSUPPORT_SSE4_2 -DHAVE_SCHED_GETCPU -DHAVE_REALTIME_COARSE -DOB_HAVE_EVENTFD -DHAVE_FALLOCATE -Werror

Copyright (c) 2021 OceanBase
OceanBase Database Proxy(ODP) is licensed under Mulan PubL v2.
You can use this software according to the terms and conditions of the Mulan PubL v2.
You may obtain a copy of Mulan PubL v2 at:
     http://license.coscl.org.cn/MulanPubL-2.0
THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
See the Mulan PubL v2 for more details.


但升级过程不能完成

[admin@obp1 ~]$ obd cluster display product
Deploy "product" is upgrading


该套环境是生产环境，还请尽快处理。

#4

日志补充：

[2022-01-08 15:12:03.144176] INFO [PROXY.CS] ob_mysql_transact.cpp:4925 [25649][Y0-7F809327BB00] [lt=47] [dc=0] succ to global variable proxy_route_policy(policy=)
[2022-01-08 15:12:13.154946] INFO [PROXY.CS] ob_mysql_transact.cpp:4925 [25649][Y0-7F809327BB00] [lt=5] [dc=0] succ to global variable proxy_route_policy(policy=)
[2022-01-08 15:12:13.161347] INFO [PROXY.CS] ob_mysql_transact.cpp:4925 [25649][Y0-7F809327BB00] [lt=28] [dc=0] succ to global variable proxy_route_policy(policy=)
[2022-01-08 15:12:13.164714] INFO [PROXY] ob_server_state_processor.cpp:1329 [25649][Y0-7F809327BB00] [lt=31] [dc=0] rs list maybe update by rs refresh task, current used dummy entry maybe old one. We need stop to update server list and dummy entry this time(cluster_name_=obcluster, last_rs_list_hash=0, last_server_list_hash=17872844444216291577, server_list=[[0]{server:"192.168.80.242:2881", is_dup_replica:false, role:"LEADER", type:"FULL"}, [1]{server:"192.168.80.243:2881", is_dup_replica:false, role:"FOLLOWER", type:"FULL"}, [2]{server:"192.168.80.244:2881", is_dup_replica:false, role:"FOLLOWER", type:"FULL"}])
[2022-01-08 15:12:13.164869] INFO [PROXY.CS] ob_mysql_transact.cpp:4925 [25649][Y0-7F809327BB00] [lt=64] [dc=0] succ to global variable proxy_route_policy(policy=)
[2022-01-08 15:12:33.138624] INFO [PROXY.CS] ob_mysql_transact.cpp:4925 [25649][Y0-7F809327BB00] [lt=17] [dc=0] succ to global variable proxy_route_policy(policy=)
[2022-01-08 15:12:33.143075] INFO [PROXY.CS] ob_mysql_transact.cpp:4925 [25649][Y0-7F809327BB00] [lt=29] [dc=0] succ to global variable proxy_route_policy(policy=)
[2022-01-08 15:12:33.145376] INFO [PROXY] ob_server_state_processor.cpp:1373 [25649][Y0-7F809327BB00] [lt=30] [dc=0] ObServerStateRefreshCont, update sys tennant's __all_dummy succ((tmp_entry={{this:0x7f8093341830, ref_count:4, cr_version:0, cr_id:0, create_time_us:1641625953145365, last_valid_time_us:1641625953145264, last_access_time_us:1641625953145264, last_update_time_us:0, schema_version:0, tenant_version:0, time_for_expired:0, state:"AVAIL"}, this:0x7f8093341830, is_inited:true, is_dummy_entry:true, is_entry_from_rslist:false, is_empty_entry_allowed:false, is_need_force_flush:false, has_dup_replica:false, cr_id:0, name:{cluster_name:"obcluster", tenant_name:"sys", database_name:"oceanbase", package_name:"", table_name:"__all_dummy"}, table_id:1099511627911, table_type:"UNKNOWN", part_num:1, replica_num:3, buf_len:32, buf_start:0x7f8093341930, tenant_servers:{this:0x7f8093344640, is_inited:true, server_count:3, replica_count:3, partition_count:1, next_partition_idx:0, server_array:0x7f808d77eb80, server_array_:[[0]{server:"192.168.80.244:2881", is_dup_replica:false, role:"FOLLOWER", type:"FULL"}, [1]{server:"192.168.80.242:2881", is_dup_replica:false, role:"FOLLOWER", type:"FULL"}, [2]{server:"192.168.80.243:2881", is_dup_replica:false, role:"FOLLOWER", type:"FULL"}]}})
[2022-01-08 15:12:33.145628] INFO [PROXY.CS] ob_mysql_transact.cpp:4925 [25649][Y0-7F809327BB00] [lt=128] [dc=0] succ to global variable proxy_route_policy(policy=)
[2022-01-08 15:12:45.245113] INFO [PROXY.CS] ob_mysql_transact.cpp:4925 [25649][Y0-7F809327BB00] [lt=21] [dc=0] succ to global variable proxy_route_policy(policy=)
[2022-01-08 15:12:45.248016] INFO [PROXY.CS] ob_mysql_transact.cpp:4925 [25649][Y0-7F809327BB00] [lt=10] [dc=0] succ to global variable proxy_route_policy(policy=)
[2022-01-08 15:12:45.249415] INFO [PROXY] ob_server_state_processor.cpp:1329 [25649][Y0-7F809327BB00] [lt=22] [dc=0] rs list maybe update by rs refresh task, current used dummy entry maybe old one. We need stop to update server list and dummy entry this time(cluster_name_=obcluster, last_rs_list_hash=0, last_server_list_hash=17872844444216291577, server_list=[[0]{server:"192.168.80.242:2881", is_dup_replica:false, role:"LEADER", type:"FULL"}, [1]{server:"192.168.80.243:2881", is_dup_replica:false, role:"FOLLOWER", type:"FULL"}, [2]{server:"192.168.80.244:2881", is_dup_replica:false, role:"FOLLOWER", type:"FULL"}])
[2022-01-08 15:12:45.249570] INFO [PROXY.CS] ob_mysql_transact.cpp:4925 [25649][Y0-7F809327BB00] [lt=62] [dc=0] succ to global variable proxy_route_policy(policy=)


#5

从日志上来看，升级是完成了。不会影响到业务使用。OBD报升级失败的原因是应该是OBD机器不能链接到obproxy的proxysys上。猜测原因有二：

1.proxysys禁用了非127.1的ip访问，导致obd所在的机器不能建链，这种情况下请先使用客户端链接到proxysys上后执行alter proxyconfig set skip_proxy_sys_private_check = true。

2.proxysys的密码与obd中存储的不一致，这种情况下请先修改proxysys密码与obd中存储的一致，完成升级后在使用edit-config修改密码后使用reload使其生效。

#6

通过进行上述两项操作，目前obproxy升级完成，谢谢！