[2024-05-12 05:33:13] INFO __init__.py:1611 parameters from cmd: host="10.10.22.37", port=2881, user="root", password="observer", timeout="0", log-file="upgrade_checker.log"
[2024-05-12 05:33:13] INFO __init__.py:1611 succeed to execute query: select distinct value from GV$OB_PARAMETERS  where name='min_observer_version', rowcount = 1
[2024-05-12 05:33:13] INFO __init__.py:1611 check observer version success, version = 4.1.0.0
[2024-05-12 05:33:13] INFO __init__.py:1611 succeed to execute query: select distinct value from GV$OB_PARAMETERS  where name='min_observer_version', rowcount = 1
[2024-05-12 05:33:13] INFO __init__.py:1611 succeed to execute query: select distinct value from oceanbase.__all_virtual_tenant_parameter_info where name='compatible', rowcount = 1
[2024-05-12 05:33:13] INFO __init__.py:1611 succeed to execute query: select count(*) from oceanbase.__all_tenant, rowcount = 1
[2024-05-12 05:33:13] INFO __init__.py:1611 succeed to execute query: select count(*) from __all_virtual_core_table where column_name in ('target_data_version', 'current_data_version') and column_value = 17179934720, rowcount = 1
[2024-05-12 05:33:13] INFO __init__.py:1611 check data version success, all tenant's compatible/target_data_version/current_data_version is 4.1.0.0
[2024-05-12 05:33:13] INFO __init__.py:1611 succeed to execute query: select count(1) as unsync_cnt from GV$OB_LOG_STAT where in_sync = 'NO', rowcount = 1
[2024-05-12 05:33:13] INFO __init__.py:1611 check paxos replica success
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select count(1) as cnt from DBA_OB_TENANT_JOBS where job_status='INPROGRESS' and result_code is null, rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select count(1) as rebalance_task_cnt from CDB_OB_LS_REPLICA_TASKS, rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 check rebalance task success
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select count(1) from CDB_OB_MAJOR_COMPACTION where (GLOBAL_BROADCAST_SCN > LAST_SCN or STATUS != 'IDLE'), rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select distinct value from GV$OB_PARAMETERS  where name='min_observer_version', rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select /*+ query_timeout(1000000000) */ count(1) from __all_virtual_tablet_compaction_info where max_received_scn > finished_scn and max_received_scn > 0, rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 check cluster status success
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select distinct value from GV$OB_PARAMETERS  where name='min_observer_version', rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select count(*) as count from DBA_OB_TENANTS where status != 'NORMAL', rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 check tenant status success
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select count(*) as count from oceanbase.__all_virtual_tenant_info where tenant_role != 'PRIMARY'  and tenant_role != 'STANDBY', rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 check tenant info success
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select count(1) from CDB_OB_RESTORE_PROGRESS, rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 check restore job success
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select distinct value from GV$OB_PARAMETERS  where name='min_observer_version', rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select count(1) from __all_virtual_ddl_task_status, rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 check ddl task execut status success
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select count(1) from CDB_OB_BACKUP_JOBS, rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 check backup job success
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select distinct value from GV$OB_PARAMETERS  where name='min_observer_version', rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select distinct value from GV$OB_PARAMETERS  where name='min_observer_version', rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select distinct value from GV$OB_PARAMETERS  where name='min_observer_version', rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select count(*) from oceanbase.__all_server where (start_service_time <= 0 or status != "active"), rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 check observer status success
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select if (a.cnt = b.cnt, 1, 0) as passed from (select count(*) as cnt from oceanbase.__all_virtual_server_schema_info where refreshed_schema_version > 1 and refreshed_schema_version % 8 = 0) as a join (select count(*) as cnt from oceanbase.__all_server join oceanbase.__all_tenant) as b, rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 check schema status success
[2024-05-12 05:33:14] INFO __init__.py:1611 succeed to execute query: select distinct(substring_index(build_version, '_', 1)) from __all_server, rowcount = 1
[2024-05-12 05:33:14] INFO __init__.py:1611 check server version success
[2024-05-12 05:33:14] ERROR __init__.py:1585 run error
Traceback (most recent call last):
  File "/tmp/10.10.22.37:2882_10.10.22.38:2882_10.10.22.39:2882/8f4d120bbf5dda9f278187b6528ebb10efb015a7/upgrade_checker.py", line 635, in do_check
    check_fail_list()
  File "/tmp/10.10.22.37:2882_10.10.22.38:2882_10.10.22.39:2882/8f4d120bbf5dda9f278187b6528ebb10efb015a7/upgrade_checker.py", line 597, in check_fail_list
    raise MyError(error_msg)
MyError: 'upgrade checker failed with 5 reasons: [1 replicas unsync, please check] , [1 rebalance tasks is doing, please check] , [1 tenant is merging, please check] , [80078 tablet is merging, please check] , [There are DDL task in progress] '
[2024-05-12 05:33:14] ERROR __init__.py:1585 normal error
Traceback (most recent call last):
  File "/tmp/10.10.22.37:2882_10.10.22.38:2882_10.10.22.39:2882/8f4d120bbf5dda9f278187b6528ebb10efb015a7/upgrade_checker.py", line 639, in do_check
    raise e
MyError: 'upgrade checker failed with 5 reasons: [1 replicas unsync, please check] , [1 rebalance tasks is doing, please check] , [1 tenant is merging, please check] , [80078 tablet is merging, please check] , [There are DDL task in progress] '
[2024-05-12 05:33:14] ERROR __init__.py:1585 normal error
Traceback (most recent call last):
  File "/tmp/10.10.22.37:2882_10.10.22.38:2882_10.10.22.39:2882/8f4d120bbf5dda9f278187b6528ebb10efb015a7/upgrade_checker.py", line 670, in <module>
    do_check(host, port, user, password, timeout, upgrade_params)
  File "/tmp/10.10.22.37:2882_10.10.22.38:2882_10.10.22.39:2882/8f4d120bbf5dda9f278187b6528ebb10efb015a7/upgrade_checker.py", line 648, in do_check
    raise e
MyError: 'upgrade checker failed with 5 reasons: [1 replicas unsync, please check] , [1 rebalance tasks is doing, please check] , [1 tenant is merging, please check] , [80078 tablet is merging, please check] , [There are DDL task in progress] '