【 使用环境 】生产环境
【 OB 】
【 使用版本 】OceanBase_CE 4.2.5.1 (r201000022024111819-1f8b94ab0822c17344e181dce36e72e756f87799) (Built Nov 18 2024 19:35:41)
【问题描述】 当前集群中架构1-1-1 ，x86架构，当前集群中发现磁盘空间占用率异常升高。
查询磁盘对象占比如下
obclient [oceanbase]> SELECT svr_ip, svr_port, SUM(size)/1024/1024/1024 data_size_gb
→ FROM __all_virtual_table_mgr
→ WHERE table_type > 10
→ GROUP BY svr_ip, svr_port
→ ORDER BY data_size_gb DESC;
±--------------±---------±-----------------+
| svr_ip | svr_port | data_size_gb |
±--------------±---------±-----------------+
| 172.29.16.2 | 2882 | 196.787795352749 |
| 172.29.16.3 | 2882 | 80.566095604561 |
| 1172.29.16.4 | 2882 | 72.565392533316 |
±--------------±---------±-----------------+
obclient [oceanbase]> SELECT table_type, SUM(size)/1024/1024/1024 data_size_gb
→ FROM __all_virtual_table_mgr
→ WHERE svr_ip=‘172.29.16.2’ group by table_type
→
→ ORDER BY data_size_gb DESC;
±-----------±-----------------+
| table_type | data_size_gb |
±-----------±-----------------+
| 11 | 196.624491893686 |
| 10 | 28.471801130101 |
| 0 | 2.390625000000 |
| 12 | 0.175000721588 |
| 1 | 0.076356410979 |
| 2 | 0.000000000000 |
| 3 | 0.000000000000 |
±-----------±----------------
obclient [oceanbase]> select tablet_id,sum(size)/1024/1024/1024 from __all_virtual_table_mgr where table_type=11 and svr_ip=‘172.29.16.2’ group by tablet_id order by 2 desc limit 10;
±--------------------±-------------------------+
| tablet_id | sum(size)/1024/1024/1024 |
±--------------------±-------------------------+
| 49402 | 196.132095831446 |
| 202434 | 0.063564237207 |
| 1152921504606854674 | 0.053563952445 |
| 1152921504606855045 | 0.035223263315 |
| 202371 | 0.031651915981 |
| 1152921504606855044 | 0.021027406677 |
| 1152921504606851142 | 0.012570331804 |
| 204446 | 0.012133625336 |
| 202433 | 0.011101170442 |
| 1152921504606851393 | 0.005889265798 |
±--------------------±-------------------------+
enum TableType : unsigned char
{
// < memtable start from here
DATA_MEMTABLE = 0,
TX_DATA_MEMTABLE = 1,
TX_CTX_MEMTABLE = 2,
LOCK_MEMTABLE = 3,
// < add new memtable here
// < sstable start from here
MAJOR_SSTABLE = 10,
MINOR_SSTABLE = 11,
MINI_SSTABLE = 12,
META_MAJOR_SSTABLE = 13,
DDL_DUMP_SSTABLE = 14,
REMOTE_LOGICAL_MINOR_SSTABLE = 15,
DDL_MEM_SSTABLE = 16,
// < add new sstable before here, See is_sstable()
MAX_TABLE_TYPE
};
目前确认的是 MINOR_SSTABLE 导致磁盘占用率异常增高的
查询当前minor merge
MySQL [oceanbase]> select * from oceanbase.GV$OB_MERGE_INFO where tenant_id=1002 and svr_ip=‘172.29.16.2’ and tablet_id=49402 order by start_time desc limit 10;
±-------------±---------±----------±------±----------±------------±--------------------±---------------------------±---------------------------±------------------±----------±----------------+
| SVR_IP | SVR_PORT | TENANT_ID | LS_ID | TABLET_ID | ACTION | COMPACTION_SCN | START_TIME | END_TIME | MACRO_BLOCK_COUNT | REUSE_PCT | PARALLEL_DEGREE |
±-------------±---------±----------±------±----------±------------±--------------------±---------------------------±---------------------------±------------------±----------±----------------+
|172.29.16.2 | 2882 | 1002 | 1011 | 49402 | MINI_MERGE | 1791513145764675000 | 2026-10-09 10:32:26.319691 | 2026-10-09 10:32:26.327315 | 1 | 0.00 | 1 |
|172.29.16.2 | 2882 | 1002 | 1004 | 49402 | MINI_MERGE | 1791513145806856000 | 2026-10-09 10:32:26.319434 | 2026-10-09 10:32:26.328742 | 1 | 0.00 | 1 |
|172.29.16.2 | 2882 | 1002 | 1027 | 49402 | MINI_MERGE | 1791513145803692000 | 2026-10-09 10:32:26.319181 | 2026-10-09 10:32:26.360594 | 2 | 0.00 | 1 |
|172.29.16.2 | 2882 | 1002 | 1 | 49402 | MINI_MERGE | 1791513145256988000 | 2026-10-09 10:32:26.318985 | 2026-10-09 10:32:26.324977 | 1 | 0.00 | 1 |
|172.29.16.2 | 2882 | 1002 | 1011 | 49402 | MINI_MERGE | 1791513124732822000 | 2026-10-09 10:32:05.120373 | 2026-10-09 10:32:05.147831 | 1 | 0.00 | 1 |
|172.29.16.2 | 2882 | 1002 | 1004 | 49402 | MINI_MERGE | 1791513124734939000 | 2026-10-09 10:32:05.118239 | 2026-10-09 10:32:05.141016 | 1 | 0.00 | 1 |
|172.29.16.2 | 2882 | 1002 | 1027 | 49402 | MINOR_MERGE | 1791513025904398000 | 2026-10-09 10:32:05.111730 | 2026-10-09 10:32:05.179496 | 21 | 85.71 | 1 |
|172.29.16.2 | 2882 | 1002 | 1 | 49402 | MINOR_MERGE | 1791513025173573000 | 2026-10-09 10:32:05.105537 | 2026-10-09 10:32:05.130494 | 1 | 0.00 | 1 |
|172.29.16.2 | 2882 | 1002 | 1011 | 49402 | MINI_MERGE | 1791513025763499000 | 2026-10-09 10:30:26.353130 | 2026-10-09 10:30:26.359329 | 1 | 0.00 | 1 |
|172.29.16.2 | 2882 | 1002 | 1004 | 49402 | MINI_MERGE | 1791513025777087000 | 2026-10-09 10:30:26.352941 | 2026-10-09 10:30:26.361224 | 1 | 0.00 | 1 |
±-------------±---------±----------±------±----------±------------±--------------------±---------------------------±---------------------------±------------------±----------±----------------+
ySQL [oceanbase]> SELECT svr_ip, ls_id, start_time, finish_time, type, compaction_scn, occupy_size, ROUND(occupy_size / 1024 / 1024 / 1024, 2) AS occupy_gb, macro_block_count, participant_table, comments FROM oceanbase.GV$OB_TABLET_COMPACTION_HISTORY WHERE tenant_id = 1002
AND type IN (‘MAJOR_MERGE’, ‘MEDIUM_MERGE’) AND start_time >= ‘2026-10-09 06:50:00’ ORDER BY start_time DESC limit 20;
±--------------±------±---------------------------±---------------------------±-------------±--------------------±------------±----------±------------------±-------------------------------------------------------------------------------------------------------------------------±-----------------------------------------------------------+
| svr_ip | ls_id | start_time | finish_time | type | compaction_scn | occupy_size | occupy_gb | macro_block_count | participant_table | comments |
±--------------±------±---------------------------±---------------------------±-------------±--------------------±------------±----------±------------------±-------------------------------------------------------------------------------------------------------------------------±-----------------------------------------------------------+
|172.29.16.2 | 1027 | 2026-10-09 10:38:30.858118 | 2026-10-09 10:38:32.112804 | MEDIUM_MERGE | 1791513252679171000 | 22147040 | 0.02 | 11 | table_cnt=4,[MAJOR]snapshot_version=1791509208693262001;[MINI]start_scn=1791506659900569000,end_scn=1791513387622282002; | extra_info=“time_guard=EXECUTE=1.25s|(0.99)|total=1.27s;”; |
|172.29.16.2 | 1027 | 2026-10-09 10:38:30.249670 | 2026-10-09 10:38:31.448679 | MEDIUM_MERGE | 1791513252679171000 | 19057460 | 0.02 | 10 | table_cnt=4,[MAJOR]snapshot_version=1791509208693262001;[MINI]start_scn=1791507123168247000,end_scn=1791513387622282000; | extra_info=“time_guard=EXECUTE=1.20s|(0.99)|total=1.21s;”; |
|172.29.16.2 | 1027 | 2026-10-09 10:38:30.055174 | 2026-10-09 10:38:30.836351 | MEDIUM_MERGE | 1791513252679171000 | 16091803 | 0.01 | 8 | table_cnt=4,[MAJOR]snapshot_version=1791509208693262001;[MINI]start_scn=1791506659900569000,end_scn=1791513387659189000; | extra_info=“time_guard=total=802.64ms;”; |
|172.29.16.2 | 1027 | 2026-10-09 10:38:29.965140 | 2026-10-09 10:38:31.830462 | MEDIUM_MERGE | 1791513252679171000 | 184997624 | 0.17 | 91 | table_cnt=4,[MAJOR]snapshot_version=1791503932606725000;[MINI]start_scn=1791500421995745000,end_scn=1791513388231574000; | extra_info=“time_guard=EXECUTE=1.86s|(0.99)|total=1.88s;”; |
|172.29.16.2 | 1027 | 2026-10-09 10:38:29.952855 | 2026-10-09 10:38:30.236227 | MEDIUM_MERGE | 1791513252679171000 | 4017215 | 0.00 | 2 | table_cnt=4,[MAJOR]snapshot_version=1791509208693262001;[MINI]start_scn=1791507123260806000,end_scn=1791513388278970000; | extra_info=“time_guard=total=296.56ms;”; |
|172.29.16.2 | 1027 | 2026-10-09 10:38:28.967121 | 2026-10-09 10:38:30.001476 | MEDIUM_MERGE | 1791513252679171000 | 19352868 | 0.02 | 10 | table_cnt=4,[MAJOR]snapshot_version=1791509208693262001;[MINI]start_scn=1791506659900569000,end_scn=1791513387623343000; | extra_info=“time_guard=EXECUTE=1.03s|(0.98)|total=1.05s;”; |
|172.29.16.2 | 1027 | 2026-10-09 10:38:28.967077 | 2026-10-09 10:38:30.038012 | MEDIUM_MERGE | 1791513252679171000 | 16163909 | 0.02 | 8 | table_cnt=5,[MAJOR]snapshot_version=1791509208693262001;[MINI]start_scn=1791509163912239000,end_scn=1791513388203178000; | extra_info=“time_guard=EXECUTE=1.07s|(0.98)|total=1.09s;”; |
|172.29.16.2 | 1027 | 2026-10-09 10:38:28.966872 | 2026-10-09 10:38:29.939589 | MEDIUM_MERGE | 1791513252679171000 | 17008606 | 0.02 | 9 | table_cnt=4,[MAJOR]snapshot_version=1791503932606725000;[MINI]start_scn=1791500421995745000,end_scn=1791513388231574000; | extra_info=“time_guard=total=985.70ms;”; |
|172.29.16.2 | 1027 | 2026-10-09 10:38:28.966482 | 2026-10-09 10:38:29.949603 | MEDIUM_MERGE | 1791513252679171000 | 16952597 | 0.02 | 9 | table_cnt=4,[MAJOR]snapshot_version=1791509208693262001;[MINI]start_scn=1791506659900569000,end_scn=1791513388207400001; | extra_info=“time_guard=total=998.35ms;”; |
查看这个tabletid 是否有major merge发现没有
MySQL [oceanbase]> SELECT ls_id, start_time, finish_time, type, compaction_scn, occupy_size, ROUND(occupy_size / 1024 / 1024 / 1024, 2) AS occupy_gb,
→ macro_block_count, participant_table, comments FROM oceanbase.GV$OB_TABLET_COMPACTION_HISTORY
→ WHERE tenant_id = 1002
→ and tablet_id=49402
→ AND type IN (‘MAJOR_MERGE’, ‘MEDIUM_MERGE’)
→ AND start_time >= ‘2026-10-09 06:50:00’ ORDER BY start_time DESC limit 20;
Empty set (0.27 sec)
是不是因为这个49402类型的 本来就不需要major merge