生产环境合并卡住,SYS租户业务租户均卡住

OceanBase 技术问题
#1

【 使用环境 】生产环境
【 OB or 其他组件 】ob
【 使用版本 】4.3.5.2
【问题描述】清晰明确描述问题
sys租户合并问题，已经合并了24h，还在合并
mysql> SELECT * FROM oceanbase.CDB_OB_MAJOR_COMPACTION;
±----------±--------------------±---------------------------±---------------------±--------------------±---------------------------±---------------------------±-----------±---------±-------------±-----+
| TENANT_ID | FROZEN_SCN | FROZEN_TIME | GLOBAL_BROADCAST_SCN | LAST_SCN | LAST_FINISH_TIME | START_TIME | STATUS | IS_ERROR | IS_SUSPENDED | INFO |
±----------±--------------------±---------------------------±---------------------±--------------------±---------------------------±---------------------------±-----------±---------±-------------±-----+
| 1 | 1769998993204134000 | 2026-02-02 10:23:13.204134 | 1769998993204134000 | 1769968802328946000 | 2026-02-02 02:08:08.326617 | 2026-02-02 10:23:13.299600 | COMPACTING | NO | NO | |
| 1001 | 1770055203283143000 | 2026-02-03 02:00:03.283143 | 1770055203283143000 | 1770055203283143000 | 2026-02-03 02:07:38.704211 | 2026-02-03 02:00:03.396810 | IDLE | NO | NO | |
| 1002 | 1770055200817697000 | 2026-02-03 02:00:00.817697 | 1770055200817697000 | 1769796002825923000 | 2026-02-02 11:24:54.875280 | 2026-02-03 02:00:00.881022 | COMPACTING | NO | NO | |
±----------±--------------------±---------------------------±---------------------±--------------------±---------------------------±---------------------------±-----------±---------±-------------±-----+
3 rows in set (0.03 sec)

mysql> select * from CDB_OB_MAJOR_COMPACTION where STATUS != “IDLE”;
±----------±--------------------±---------------------------±---------------------±--------------------±---------------------------±---------------------------±-----------±---------±-------------±-----+
| TENANT_ID | FROZEN_SCN | FROZEN_TIME | GLOBAL_BROADCAST_SCN | LAST_SCN | LAST_FINISH_TIME | START_TIME | STATUS | IS_ERROR | IS_SUSPENDED | INFO |
±----------±--------------------±---------------------------±---------------------±--------------------±---------------------------±---------------------------±-----------±---------±-------------±-----+
| 1 | 1769998993204134000 | 2026-02-02 10:23:13.204134 | 1769998993204134000 | 1769968802328946000 | 2026-02-02 02:08:08.326617 | 2026-02-02 10:23:13.299600 | COMPACTING | NO | NO | |
| 1002 | 1770055200817697000 | 2026-02-03 02:00:00.817697 | 1770055200817697000 | 1769796002825923000 | 2026-02-02 11:24:54.875280 | 2026-02-03 02:00:00.881022 | COMPACTING | NO | NO | |
±----------±--------------------±---------------------------±---------------------±--------------------±---------------------------±---------------------------±-----------±---------±-------------±-----+
2 rows in set (0.02 sec)

SELECT * FROM oceanbase.__all_virtual_core_meta_table WHERE role = 1;
±----------±------±--------------±---------±---------±-----±-------------------------------------------------------------±------------±-------------±---------------±---------------±-----------------±--------±------±---------------------±----------±--------------±-------------±--------+
| tenant_id | ls_id | svr_ip | svr_port | sql_port | role | member_list | proposal_id | replica_type | replica_status | restore_status | memstore_percent | unit_id | zone | paxos_replica_number | data_size | required_size | learner_list | rebuild |
±----------±------±--------------±---------±---------±-----±-------------------------------------------------------------±------------±-------------±---------------±---------------±-----------------±--------±------±---------------------±----------±--------------±-------------±--------+
| 1 | 1 | xxx.xx.xx.22 | 2882 | 2881 | 1 | xxx.xx.xx.23:2882:1,xxx.xx.xx.22:2882:1,xxx.xx.xx.26:2882:1 | 3 | 0 | NORMAL | 0 | 100 | 3 | zone3 | 3 | 0 | 0 | | 0 |
±----------±------±--------------±---------±---------±-----±-------------------------------------------------------------±------------±-------------±---------------±---------------±-----------------±--------±------±---------------------±----------±--------------±-------------±--------+

grep “replica not merged” rootservice.log* | grep “T_MergeSche”

select * from __all_tablet_meta_table where tenant_id=1 and compaction_scn < 1769998993204134000;
1769968802328946000 | 0 | 0 | 1769968802328946000 | 0 |
| 2026-01-16 15:03:35.177038 | 2026-02-02 02:05:57.302342 | 1 | 101114 | 1 | xxx.xx.xx.26 | 2882 | 1769968802328946000 | 0 | 0 | 1769968802328946000 | 0 |
| 2026-01-16 15:04:19.358586 | 2026-02-02 02:37:19.890124 | 1 | 200001 | 1 | xxx.xx.xx.26 | 2882 | 1769968802328946000 | 512 | 4096 | 1769968802328946000 | 0 |
| 2026-01-28 16:46:46.806321 | 2026-02-02 02:37:19.912209 | 1 | 200002 | 1 | xxx.xx.xx.26 | 2882 | 1769968802328946000 | 448 | 4096 | 1769968802328946000 | 0 |
| 2026-01-16 15:04:19.370379 | 2026-02-02 02:37:19.890124 | 1 | 1152921504606846977 | 1 | xxx.xx.xx.26 | 2882 | 1769968802328946000 | 427 | 4096 | 1769968802328946000 | 0 |
±---------------------------±---------------------------±----------±--------------------±------±--------------±---------±--------------------±---------

select * from __all_virtual_compaction_diagnose_info where tenant_id=1;

mysql> select * from __all_virtual_compaction_diagnose_info where tenant_id=1;
±--------------±---------±----------±------------±--------------------±--------------------±---------------±---------------------------±----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| svr_ip | svr_port | tenant_id | type | ls_id | tablet_id | status | create_time | diagnose_info |
±--------------±---------±----------±------------±--------------------±--------------------±---------------±---------------------------±----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| xxx.xx.xx.22 | 2882 | 1 | MAJOR_MERGE | 1 | 1 | RS_UNCOMPACTED | 2026-02-03 11:06:41.006050 | server=“xxx.xx.xx.26:2882”,status=“compaction_scn_not_update”,frozen_scn=1769998993204134000,compaction_scn=1769968802328946000,report_scn=1769968802328946000 |
| xxx.xx.xx.22 | 2882 | 1 | MAJOR_MERGE | 1 | 3 | RS_UNCOMPACTED | 2026-02-03 11:06:41.006055 | server=“xxx.xx.xx.26:2882”,status=“compaction_scn_not_update”,frozen_scn=1769998993204134000,compaction_scn=1769968802328946000,report_scn=1769968802328946000 |
| xxx.xx.xx.22 | 2882 | 1 | MAJOR_MERGE | 1 | 4 | RS_UNCOMPACTED | 2026-02-03 11:06:41.006058 | server=“xxx.xx.xx.26:2882”,status=“compaction_scn_not_update”,frozen_scn=1769998993204134000,compaction_scn=1769968802328946000,report_scn=1769968802328946000 |
| xxx.xx.xx.22 | 2882 | 1 | MAJOR_MERGE | 9223372036854775807 | 9223372036854775807 | RS_UNCOMPACTED | 2026-02-03 11:06:41.006060 | table_id=3 |
| xxx.xx.xx.22 | 2882 | 1 | MAJOR_MERGE | 9223372036854775807 | 9223372036854775807 | RS_UNCOMPACTED | 2026-02-03 11:06:41.006061 | table_id=100001 |
| xxx.xx.xx.22 | 2882 | 1 | MAJOR_MERGE | 9223372036854775807 | 9223372036854775807 | RS_UNCOMPACTED | 2026-02-03 11:06:41.006061 | table_id=100002 |
±--------------±---------±----------±------------±--------------------±--------------------±---------------±---------------------------±----------------------------------------------------------------------------------------------------------------------------------------------------------------+
6 rows in set (4.60 sec)

select * from __all_virtual_dag_warning_history where tenant_id=1;

alter system change tenant META$1002;
select * from __all_tablet_meta_table where tenant_id=1002 and compaction_scn < 1770055200817697000;
Empty set (0.02 sec)

select * from __all_virtual_tablet_meta_table where tenant_id=1002 and compaction_scn !=1770055200817697000;
| 1002 | 1152921504606898120 | 1139 | xxx.xx.xx.26 | 2882 | 2026-02-01 20:29:16.427266 | 2026-02-02 11:24:54.865068 | 1769796002825923001 | 0 | 0 | 1769796002825923000 | 0 |
| 1002 | 1152921504606898121 | 1139 | xxx.xx.xx.26 | 2882 | 2026-02-01 20:29:16.428320 | 2026-02-02 11:24:54.865068 | 1769796002825923001 | 0 | 0 | 1769796002825923000 | 0 |
| 1002 | 1152921504606898122 | 1139 | xxx.xx.xx.26 | 2882 | 2026-02-01 20:29:16.429382 | 2026-02-02 11:24:54.865068 | 1769954145149422002 | 69293 | 73728 | 1769796002825923000 | 0 |
| 1002 | 1152921504606898159 | 1001 | xxx.xx.xx.21 | 2882 | 2026-02-03 10:34:35.294808 | 2026-02-03 12:46:16.753512 | 1770085319881784002 | 12470634 | 12470634 | 1770055200817697000 | 0 |
| 1002 | 1152921504606898159 | 1001 | xxx.xx.xx.27 | 2882 | 2026-02-03 10:29:24.643266 | 2026-02-03 12:46:16.753512 | 1770085319881784002 | 12470634 | 12470634 | 1770055200817697000 | 0 |
| 1002 | 1152921504606898159 | 1001 | xxx.xx.xx.25 | 2882 | 2026-02-03 10:52:05.437386 | 2026-02-03 12:52:08.588427 | 1770085319881784002 | 12470634 | 12470634 | 1770055200817697000 | 0 |
| 1002 | 1152921504606898160 | 1001 | xxx.xx.xx.21 | 2882 | 2026-02-03 10:34:35.379386 | 2026-02-03 12:46:16.753512 | 1770085319881784002 | 579924295 | 579924295 | 1770055200817697000 | 0 |
| 1002 | 1152921504606898160 | 1001 | xxx.xx.xx.27 | 2882 | 2026-02-03 10:29:24.946269 | 2026-02-03 12:46:16.753512 | 1770085319881784002 | 579924295 | 579924295 | 1770055200817697000 | 0 |
| 1002 | 1152921504606898160 | 1001 | xxx.xx.xx.25 | 2882 | 2026-02-03 10:52:05.496512 | 2026-02-03 12:52:08.598987 | 1770085319881784002 | 579924295 | 579924295 | 1769796002825923000 | 0 |
±----------±--------------------±------±--------------±---------±---------------------------±---------------------------±--------------------±-----------±--------------±--------------------±-------+
15507 rows in set (0.34 sec)

select * from __all_virtual_compaction_diagnose_info where tenant_id=1002;

mysql> select * from __all_virtual_compaction_diagnose_info where tenant_id=1002;
±--------------±---------±----------±------------±--------------------±--------------------±---------------±---------------------------±--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| svr_ip | svr_port | tenant_id | type | ls_id | tablet_id | status | create_time | diagnose_info |
±--------------±---------±----------±------------±--------------------±--------------------±---------------±---------------------------±--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| xxx.xx.xx.23 | 2882 | 1002 | MAJOR_MERGE | 1004 | 1152921504606898123 | NOT_SCHEDULE | 2026-02-03 14:57:00.255761 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1,last_major_scn=0 |
| xxx.xx.xx.23 | 2882 | 1002 | MAJOR_MERGE | 1004 | 226828 | NOT_SCHEDULE | 2026-02-03 14:57:00.255913 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.23 | 2882 | 1002 | MAJOR_MERGE | 1004 | 226838 | NOT_SCHEDULE | 2026-02-03 14:57:00.256028 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.23 | 2882 | 1002 | MAJOR_MERGE | 1004 | 226837 | NOT_SCHEDULE | 2026-02-03 14:57:00.256146 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.23 | 2882 | 1002 | MAJOR_MERGE | 1004 | 1152921504606898124 | NOT_SCHEDULE | 2026-02-03 14:57:00.256225 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1,last_major_scn=0 |
| xxx.xx.xx.23 | 2882 | 1002 | MAJOR_MERGE | 1004 | 1152921504606898141 | NOT_SCHEDULE | 2026-02-03 14:57:00.256313 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1,last_major_scn=0 |
| xxx.xx.xx.23 | 2882 | 1002 | MAJOR_MERGE | 1004 | 226834 | NOT_SCHEDULE | 2026-02-03 14:57:00.256421 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.23 | 2882 | 1002 | MAJOR_MERGE | 1004 | 226836 | NOT_SCHEDULE | 2026-02-03 14:57:00.256513 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.23 | 2882 | 1002 | MAJOR_MERGE | 1004 | 1152921504606898130 | NOT_SCHEDULE | 2026-02-03 14:57:00.256615 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1,last_major_scn=0 |
| xxx.xx.xx.23 | 2882 | 1002 | MAJOR_MERGE | 1004 | 1152921504606898142 | NOT_SCHEDULE | 2026-02-03 14:57:00.256700 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1,last_major_scn=0 |
| xxx.xx.xx.23 | 2882 | 1002 | MAJOR_MERGE | 9223372036854775807 | 9223372036854775807 | SPECIAL | 2026-02-03 14:57:00.259522 | diagnose info type=“MAJOR_NOT_SCHEDULE”,count of tablets with the same problem=37 |
| xxx.xx.xx.22 | 2882 | 1002 | MAJOR_MERGE | 1004 | 1152921504606898123 | NOT_SCHEDULE | 2026-02-03 14:57:00.256448 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1,last_major_scn=0 |
| xxx.xx.xx.22 | 2882 | 1002 | MAJOR_MERGE | 1004 | 226828 | NOT_SCHEDULE | 2026-02-03 14:57:00.256597 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.22 | 2882 | 1002 | MAJOR_MERGE | 1004 | 226838 | NOT_SCHEDULE | 2026-02-03 14:57:00.256702 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.22 | 2882 | 1002 | MAJOR_MERGE | 1004 | 226837 | NOT_SCHEDULE | 2026-02-03 14:57:00.256794 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.22 | 2882 | 1002 | MAJOR_MERGE | 1004 | 1152921504606898124 | NOT_SCHEDULE | 2026-02-03 14:57:00.256872 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1,last_major_scn=0 |
| xxx.xx.xx.22 | 2882 | 1002 | MAJOR_MERGE | 1004 | 1152921504606898141 | NOT_SCHEDULE | 2026-02-03 14:57:00.256952 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1,last_major_scn=0 |
| xxx.xx.xx.22 | 2882 | 1002 | MAJOR_MERGE | 1004 | 226834 | NOT_SCHEDULE | 2026-02-03 14:57:00.257057 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.22 | 2882 | 1002 | MAJOR_MERGE | 1004 | 226836 | NOT_SCHEDULE | 2026-02-03 14:57:00.257153 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.22 | 2882 | 1002 | MAJOR_MERGE | 1004 | 1152921504606898130 | NOT_SCHEDULE | 2026-02-03 14:57:00.257243 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1,last_major_scn=0 |
| xxx.xx.xx.22 | 2882 | 1002 | MAJOR_MERGE | 1004 | 1152921504606898142 | NOT_SCHEDULE | 2026-02-03 14:57:00.257340 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1,last_major_scn=0 |
| xxx.xx.xx.22 | 2882 | 1002 | MAJOR_MERGE | 9223372036854775807 | 9223372036854775807 | SPECIAL | 2026-02-03 14:57:00.259965 | diagnose info type=“MAJOR_NOT_SCHEDULE”,count of tablets with the same problem=37 |
| xxx.xx.xx.25 | 2882 | 1002 | MAJOR_MERGE | 1001 | 226829 | NOT_SCHEDULE | 2026-02-03 14:57:00.259986 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.25 | 2882 | 1002 | MAJOR_MERGE | 1001 | 226818 | NOT_SCHEDULE | 2026-02-03 14:57:00.260085 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.25 | 2882 | 1002 | MAJOR_MERGE | 1001 | 226825 | NOT_SCHEDULE | 2026-02-03 14:57:00.260158 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.25 | 2882 | 1002 | MAJOR_MERGE | 1002 | 226826 | NOT_SCHEDULE | 2026-02-03 14:57:00.263592 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.25 | 2882 | 1002 | MAJOR_MERGE | 1002 | 226830 | NOT_SCHEDULE | 2026-02-03 14:57:00.271747 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.25 | 2882 | 1002 | MAJOR_MERGE | 1002 | 226831 | NOT_SCHEDULE | 2026-02-03 14:57:00.271833 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.25 | 2882 | 1002 | MAJOR_MERGE | 1002 | 226842 | NOT_SCHEDULE | 2026-02-03 14:57:00.271913 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.25 | 2882 | 1002 | MAJOR_MERGE | 1003 | 226823 | NOT_SCHEDULE | 2026-02-03 14:57:00.272580 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.25 | 2882 | 1002 | MAJOR_MERGE | 1003 | 226821 | NOT_SCHEDULE | 2026-02-03 14:57:00.273626 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.25 | 2882 | 1002 | MAJOR_MERGE | 1003 | 226824 | NOT_SCHEDULE | 2026-02-03 14:57:00.273701 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.25 | 2882 | 1002 | MAJOR_MERGE | 1005 | 204844 | RS_UNCOMPACTED | 2026-02-03 14:57:00.274375 | server=“xxx.xx.xx.26:2882”,status=“compaction_scn_not_update”,frozen_scn=1770055200817697000,compaction_scn=1769796002825923000,report_scn=1769796002825923000 |
| xxx.xx.xx.25 | 2882 | 1002 | MAJOR_MERGE | 1006 | 204852 | RS_UNCOMPACTED | 2026-02-03 14:57:00.274379 | server=“xxx.xx.xx.26:2882”,status=“compaction_scn_not_update”,frozen_scn=1770055200817697000,compaction_scn=1769796002825923000,report_scn=1769796002825923000 |
| xxx.xx.xx.25 | 2882 | 1002 | MAJOR_MERGE | 1004 | 204856 | RS_UNCOMPACTED | 2026-02-03 14:57:00.274382 | server=“xxx.xx.xx.26:2882”,status=“compaction_scn_not_update”,frozen_scn=1770055200817697000,compaction_scn=1769796002825923000,report_scn=1769796002825923000 |
| xxx.xx.xx.25 | 2882 | 1002 | MAJOR_MERGE | 9223372036854775807 | 9223372036854775807 | SPECIAL | 2026-02-03 14:57:00.274397 | diagnose info type=“MAJOR_NOT_SCHEDULE”,count of tablets with the same problem=14 |
| xxx.xx.xx.21 | 2882 | 1002 | MAJOR_MERGE | 1001 | 226829 | NOT_SCHEDULE | 2026-02-03 14:57:00.256422 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.21 | 2882 | 1002 | MAJOR_MERGE | 1001 | 226818 | NOT_SCHEDULE | 2026-02-03 14:57:00.256562 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.21 | 2882 | 1002 | MAJOR_MERGE | 1001 | 226825 | NOT_SCHEDULE | 2026-02-03 14:57:00.256674 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.21 | 2882 | 1002 | MAJOR_MERGE | 1002 | 226826 | NOT_SCHEDULE | 2026-02-03 14:57:00.266334 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.21 | 2882 | 1002 | MAJOR_MERGE | 1002 | 226830 | NOT_SCHEDULE | 2026-02-03 14:57:00.271929 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.21 | 2882 | 1002 | MAJOR_MERGE | 1002 | 226831 | NOT_SCHEDULE | 2026-02-03 14:57:00.272015 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.21 | 2882 | 1002 | MAJOR_MERGE | 1002 | 226842 | NOT_SCHEDULE | 2026-02-03 14:57:00.272097 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.21 | 2882 | 1002 | MAJOR_MERGE | 1003 | 226823 | NOT_SCHEDULE | 2026-02-03 14:57:00.282843 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.21 | 2882 | 1002 | MAJOR_MERGE | 1003 | 226821 | NOT_SCHEDULE | 2026-02-03 14:57:00.294444 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.21 | 2882 | 1002 | MAJOR_MERGE | 1003 | 226824 | NOT_SCHEDULE | 2026-02-03 14:57:00.294537 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.21 | 2882 | 1002 | MAJOR_MERGE | 9223372036854775807 | 9223372036854775807 | SPECIAL | 2026-02-03 14:57:00.295335 | diagnose info type=“MAJOR_NOT_SCHEDULE”,count of tablets with the same problem=14 |
| xxx.xx.xx.27 | 2882 | 1002 | MAJOR_MERGE | 1001 | 226829 | NOT_SCHEDULE | 2026-02-03 14:57:00.256728 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.27 | 2882 | 1002 | MAJOR_MERGE | 1001 | 226818 | NOT_SCHEDULE | 2026-02-03 14:57:00.256848 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.27 | 2882 | 1002 | MAJOR_MERGE | 1001 | 226825 | NOT_SCHEDULE | 2026-02-03 14:57:00.256950 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.27 | 2882 | 1002 | MAJOR_MERGE | 1002 | 226826 | NOT_SCHEDULE | 2026-02-03 14:57:00.265281 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.27 | 2882 | 1002 | MAJOR_MERGE | 1002 | 226842 | NOT_SCHEDULE | 2026-02-03 14:57:00.265878 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.27 | 2882 | 1002 | MAJOR_MERGE | 1002 | 226830 | NOT_SCHEDULE | 2026-02-03 14:57:00.265975 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.27 | 2882 | 1002 | MAJOR_MERGE | 1002 | 226831 | NOT_SCHEDULE | 2026-02-03 14:57:00.266060 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.27 | 2882 | 1002 | MAJOR_MERGE | 1003 | 226823 | NOT_SCHEDULE | 2026-02-03 14:57:00.266157 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.27 | 2882 | 1002 | MAJOR_MERGE | 1003 | 226832 | NOT_SCHEDULE | 2026-02-03 14:57:00.266253 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.27 | 2882 | 1002 | MAJOR_MERGE | 1003 | 226821 | NOT_SCHEDULE | 2026-02-03 14:57:00.266335 | major not schedule for long time=“NO_MAJOR_SSTABLE”,max_receive_medium_snapshot=0,compaction_scn=1770055200817697000,tablet_snapshot=1770055200817697001,last_major_scn=0 |
| xxx.xx.xx.27 | 2882 | 1002 | MAJOR_MERGE | 9223372036854775807 | 9223372036854775807 | SPECIAL | 2026-02-03 14:57:00.302605 | diagnose info type=“MAJOR_NOT_SCHEDULE”,count of tablets with the same problem=14 |
±--------------±---------±----------±------------±--------------------±--------------------±---------------±---------------------------±--------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
58 rows in set (0.08 sec)

【复现路径】问题出现前后相关操作
【附件及日志】推荐使用OceanBase敏捷诊断工具obdiag收集诊断信息,详情参见链接（右键跳转查看）：

【SOP系列 22 】——故障诊断第一步(自助诊断和诊断信息收集)

【备注】基于 LLM 和开源文档 RAG 的论坛小助手已开放测试，在发帖时输入 [@论坛小助手] 即可召唤小助手，欢迎试用！

1 个赞
#3

mysql> select * from __all_virtual_compaction_diagnose_info where tenant_id=1;
±--------------±---------±----------±------------±--------------------±--------------------±---------------±---------------------------±----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| svr_ip | svr_port | tenant_id | type | ls_id | tablet_id | status | create_time | diagnose_info |
±--------------±---------±----------±------------±--------------------±--------------------±---------------±---------------------------±----------------------------------------------------------------------------------------------------------------------------------------------------------------+
| xxx.xx.xx.22 | 2882 | 1 | MAJOR_MERGE | 1 | 1 | RS_UNCOMPACTED | 2026-02-03 14:57:51.983586 | server=“xxx.xx.xx.26:2882”,status=“compaction_scn_not_update”,frozen_scn=1769998993204134000,compaction_scn=1769968802328946000,report_scn=1769968802328946000 |
| xxx.xx.xx.22 | 2882 | 1 | MAJOR_MERGE | 1 | 3 | RS_UNCOMPACTED | 2026-02-03 14:57:51.983590 | server=“xxx.xx.xx.26:2882”,status=“compaction_scn_not_update”,frozen_scn=1769998993204134000,compaction_scn=1769968802328946000,report_scn=1769968802328946000 |
| xxx.xx.xx.22 | 2882 | 1 | MAJOR_MERGE | 1 | 4 | RS_UNCOMPACTED | 2026-02-03 14:57:51.983593 | server=“xxx.xx.xx.26:2882”,status=“compaction_scn_not_update”,frozen_scn=1769998993204134000,compaction_scn=1769968802328946000,report_scn=1769968802328946000 |
| xxx.xx.xx.22 | 2882 | 1 | MAJOR_MERGE | 9223372036854775807 | 9223372036854775807 | RS_UNCOMPACTED | 2026-02-03 14:57:51.983595 | table_id=3 |
| xxx.xx.xx.22 | 2882 | 1 | MAJOR_MERGE | 9223372036854775807 | 9223372036854775807 | RS_UNCOMPACTED | 2026-02-03 14:57:51.983596 | table_id=100001 |
| xxx.xx.xx.22 | 2882 | 1 | MAJOR_MERGE | 9223372036854775807 | 9223372036854775807 | RS_UNCOMPACTED | 2026-02-03 14:57:51.983596 | table_id=100002 |
±--------------±---------±----------±------------±--------------------±--------------------±---------------±---------------------------±----------------------------------------------------------------------------------------------------------------------------------------------------------------+
6 rows in set (0.01 sec)

grep “replica not merged”
log/rootservice.log*|tail -10
所有主机均为空

#4

针对您描述的情况，推荐使用 obdiag 工具进行日志采集，以便确认导致合并卡住的具体原因。以下是具体的日志采集命令：

日志采集场景: 卡合并
日志采集命令: obdiag gather scene run --scene=observer.major_hold

请您先执行上述日志采集命令，并将采集到的日志提供给我们以便进一步分析。此外，为了更准确地定位问题，还需要您提供一些额外的信息：

  1. 您当前使用的存储介质是什么？（如 SSD、HDD 等）
  2. 在合并过程中是否有大量的读写操作？
  3. 您的租户配置参数是否进行了特别的调整？
  4. 当前集群的负载情况如何？（如 CPU 使用率、I/O 状态等）
  5. 是否有其他告警或异常日志记录？

希望这些信息能够帮助我们更快地找到问题的原因并给出有效的解决方案。感谢您的配合！

附上敏捷诊断工具 obdiag 使用帮助链接

#5

image
查询下这些分区id的表是不是都是一个分区表。

#7

mysql> SELECT table_id FROM oceanbase.__all_virtual_tablet_to_table_history WHERE tablet_id in (
→ select tablet_id from __all_virtual_compaction_diagnose_info where tenant_id=1002) and tenant_id=1002;
±---------+
| table_id |
±---------+
| 505237 |
| 505245 |
| 505272 |
| 578204 |
| 578207 |
| 578209 |
| 578211 |
| 578212 |
| 578214 |
| 578215 |
| 578216 |
| 578217 |
| 578218 |
| 578219 |
| 578220 |
| 578221 |
| 578222 |
| 578223 |
| 578224 |
| 578226 |
| 578228 |
| 531486 |
| 531486 |
| 532302 |
| 531792 |
| 531792 |
±---------+
26 rows in set (0.34 sec)
不是同一张表

#8

[2026-02-03 11:06:00.051163] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:916) [1290792][T1_L1_G10000][T1][YB420A8C7316-00064882EEDF77B9-0-0] [lt=25] get_stable_disk_usage(ret=0, capacity(MB):=11290, used(MB):=8997)
[2026-02-03 11:06:00.051178] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:54) [1290823][T1_L0_G10000][T1][YB420A8C7316-000648838B8B089E-0-0] [lt=28][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=9, len=128, pos=96)
[2026-02-03 11:06:00.051206] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:916) [1290792][T1_L1_G10000][T1001][YB420A8C7316-00064882EEDF77B9-0-0] [lt=22] get_stable_disk_usage(ret=0, capacity(MB):=104857, used(MB):=0)
[2026-02-03 11:06:00.051233] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:916) [1290792][T1_L1_G10000][T1002][YB420A8C7316-00064882EEDF77B9-0-0] [lt=14] REACH SYSLOG RATE LIMIT [bandwidth]
[2026-02-03 11:06:00.052099] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:54) [1290823][T1_L0_G10000][T1][YB420A8C7316-000648838B8B089E-0-0] [lt=18][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=85, len=563, pos=538)
[2026-02-03 11:06:00.052997] WDIAG [COMMON] parse (ob_cell_reader.cpp:531) [1289809][T1_L5_G0][T1][YB420A8C0619-0006487FDB1795A7-0-0] [lt=17][errcode=0] Unexpected lob scale(*version=1, *scale=0, ret=0)
[2026-02-03 11:06:00.053016] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:54) [1290823][T1_L0_G10000][T1][YB420A8C7316-000648838B8B089E-0-0] [lt=16][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=2, len=72, pos=51)
[2026-02-03 11:06:00.053028] WDIAG [COMMON] parse (ob_cell_reader.cpp:531) [1289809][T1_L5_G0][T1][YB420A8C0619-0006487FDB1795A7-0-0] [lt=22][errcode=0] Unexpected lob scale(*version=1, *scale=0, ret=0)
[2026-02-03 11:06:00.053062] WDIAG [COMMON] parse (ob_cell_reader.cpp:531) [1289809][T1_L5_G0][T1][YB420A8C0619-0006487FDB1795A7-0-0] [lt=9][errcode=0] Unexpected lob scale(*version=1, *scale=0, ret=0)
[2026-02-03 11:06:00.053078] WDIAG [COMMON] parse (ob_cell_reader.cpp:531) [1289809][T1_L5_G0][T1][YB420A8C0619-0006487FDB1795A7-0-0] [lt=13][errcode=0] Unexpected lob scale(*version=1, *scale=0, ret=0)
[2026-02-03 11:06:00.053087] WDIAG [COMMON] parse (ob_cell_reader.cpp:531) [1289809][T1_L5_G0][T1][YB420A8C0619-0006487FDB1795A7-0-0] [lt=7][errcode=0] Unexpected lob scale(*version=1, *scale=0, ret=0)
[2026-02-03 11:06:00.053946] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:54) [1290823][T1_L0_G10000][T1][YB420A8C7316-000648838B8B089E-0-0] [lt=19][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=1, len=27, pos=18)

这是lob列损坏了？

#9

[2026-02-03 11:06:00.079328] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:85) [1289700][T1_DDLTaskExecu][T1][YB420A8C7316-0006488171FEF639-0-0] [lt=0][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2026-02-03 11:06:00.079839] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:54) [1290823][T1_L0_G10000][T1][YB420A8C7316-000648838B8B089E-0-0] [lt=16][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=426, pos=388)
[2026-02-03 11:06:00.080723] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:54) [1290823][T1_L0_G10000][T1][YB420A8C7316-000648838B8B089E-0-0] [lt=15][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=12, len=166, pos=144)
[2026-02-03 11:06:00.081519] WDIAG [RPC] on_timeout (ob_async_rpc_proxy.h:85) [1289703][T1_DDLTaskExecu][T1][YB420A8C7316-00064886EDA226C0-0-0] [lt=1][errcode=0] some error in rcode and enter on_timeout(AsyncCB::rcode_.rcode_=0)
[2026-02-03 11:06:00.081540] INFO [SHARE.LOCATION] renew_cache_ (ob_tablet_ls_service.cpp:412) [1289705][T1_DDLTaskExecu][T1][YB420A8C7316-00064881719EC437-0-0] [lt=1] [TABLET_LOCATION]success to renew tablet cache(tablet_cache={cache_key:{tenant_id:1002, tablet_id:{id:1152921504606898076}}, ls_id:{id:1004}, renew_time:1770087960081486, transfer_seq:0})

#10

select * from GV$OB_COMPACTION_PROGRESS where STATUS != “FINISH” \G;
*************************** 1. row ***************************
SVR_IP: xxx.xx.xx.23
SVR_PORT: 2882
TENANT_ID: 1002
TYPE: MAJOR_MERGE
ZONE: zone1
COMPACTION_SCN: 1770055200817697000
STATUS: NODE_RUNNING
TOTAL_TABLET_COUNT: 15307
UNFINISHED_TABLET_COUNT: 93
DATA_SIZE: 1417192173069
UNFINISHED_DATA_SIZE: 590614626710
COMPRESSION_RATIO: 0.75
START_TIME: 2026-02-03 02:00:02.747540
ESTIMATED_FINISH_TIME: 2026-02-04 02:00:02.747540
COMMENTS: COMPACTION_FINISH_CHECK:cost_ts=952076,batch_check_stat={succ_cnt:10212, finish_cnt:0, fail_cnt:0, filter_cnt:0},is_inited:1,real_finish_cnt:15214
*************************** 2. row ***************************
SVR_IP: xxx.xx.xx.25
SVR_PORT: 2882
TENANT_ID: 1002
TYPE: MAJOR_MERGE
ZONE: zone1
COMPACTION_SCN: 1770055200817697000
STATUS: NODE_RUNNING
TOTAL_TABLET_COUNT: 15289
UNFINISHED_TABLET_COUNT: 80
DATA_SIZE: 1478285062701
UNFINISHED_DATA_SIZE: 606060153830
COMPRESSION_RATIO: 0.77
START_TIME: 2026-02-03 10:22:10.747530
ESTIMATED_FINISH_TIME: 2026-02-04 10:22:10.747530
COMMENTS: RS_FINISH_CUR_LOOP:cost_us=1497648,progress={this:0x7fb523268088, merge_finish:false, unmerged_tablet_cnt:0, merged_tablet_cnt:15299, total_table_cnt:13601, INITIAL:7526, COMPACTED:676, CAN_SKIP_VERIFYING:15, INDEX_CKM_VERIFIED:0, VERIFIED:5383},remain_table_id_count=8202,remain_table_ids=[505237, 505248, 505264],remain_tablet_ids=[204844, 204852, 204856],total_time_guard=PREPARE_UNFINISH_TABLE_IDS=7.24ms|,validator_statistics={query_ckm_sql_cnt:10610, use_cached_ckm_cnt:1948, write_ckm_sql_cnt:423, update_report_scn_sql_cnt:423, checker_validate_idx_cnt:1079},is_inited:1,real_finish_cnt:15209
*************************** 3. row ***************************
SVR_IP: xxx.xx.xx.27
SVR_PORT: 2882
TENANT_ID: 1002
TYPE: MAJOR_MERGE
ZONE: zone2
COMPACTION_SCN: 1770055200817697000
STATUS: NODE_RUNNING
TOTAL_TABLET_COUNT: 15289
UNFINISHED_TABLET_COUNT: 6
DATA_SIZE: 1481911325315
UNFINISHED_DATA_SIZE: 605959489916
COMPRESSION_RATIO: 0.77
START_TIME: 2026-02-03 02:00:03.880985
ESTIMATED_FINISH_TIME: 2026-02-03 18:55:53.251295
COMMENTS: SCHEDULER_LOOP:schedule_stats={all_ls_weak_read_ts_ready:true, start_timestamp:1770055203895738},schedule_tablet_cnt={schedule_dag_cnt:0, submit_clog_cnt:0, finish_cnt:15295, loop_tablet_cnt:15309, force_freeze_cnt:0, wait_rs_validate_cnt:13},is_inited:1,real_finish_cnt:15283
*************************** 4. row ***************************
SVR_IP: xxx.xx.xx.22
SVR_PORT: 2882
TENANT_ID: 1002
TYPE: MAJOR_MERGE
ZONE: zone3
COMPACTION_SCN: 1770055200817697000
STATUS: NODE_RUNNING
TOTAL_TABLET_COUNT: 15307
UNFINISHED_TABLET_COUNT: 93
DATA_SIZE: 1417187870586
UNFINISHED_DATA_SIZE: 590614627655
COMPRESSION_RATIO: 0.75
START_TIME: 2026-02-03 02:00:01.466175
ESTIMATED_FINISH_TIME: 2026-02-04 02:00:01.466175
COMMENTS: COMPACTION_FINISH_CHECK:cost_ts=452518,batch_check_stat={succ_cnt:5078, finish_cnt:0, fail_cnt:0, filter_cnt:0},is_inited:1,real_finish_cnt:15214
*************************** 5. row ***************************
SVR_IP: xxx.xx.xx.21
SVR_PORT: 2882
TENANT_ID: 1002
TYPE: MAJOR_MERGE
ZONE: zone3
COMPACTION_SCN: 1770055200817697000
STATUS: NODE_RUNNING
TOTAL_TABLET_COUNT: 15289
UNFINISHED_TABLET_COUNT: 6
DATA_SIZE: 1478809643059
UNFINISHED_DATA_SIZE: 605959489194
COMPRESSION_RATIO: 0.77
START_TIME: 2026-02-03 02:00:03.055005
ESTIMATED_FINISH_TIME: 2026-02-03 19:01:23.280975
COMMENTS: SCHEDULER_LOOP:schedule_stats={all_ls_weak_read_ts_ready:true, start_timestamp:1770055203071451},schedule_tablet_cnt={schedule_dag_cnt:0, submit_clog_cnt:0, finish_cnt:15295, loop_tablet_cnt:15309, force_freeze_cnt:0, wait_rs_validate_cnt:13},is_inited:1,real_finish_cnt:15283
5 rows in set (0.01 sec)

ERROR:
No query specified