ocp集群从单副本增加一个zone副本后,ocp_meta和ocp_monitor的ALTER_TENANT_LOCALITY一直running

【 使用环境 】测试环境
【 OB or 其他组件 】ob,ocp
【 使用版本 】4.2.1.8
【问题描述】
1、先安装单节点ocp环境,成功
2、增加一个zone,sys租户成功完成副本同步,ocp_meta和ocp_monitor租户都报ALTER_TENANT_LOCALITY 失败。重试该步骤仍然失败。
3、选择跳过这个步骤,ocp_meta和ocp_monitor都提示如下:
image
4、obclient(root@sys)[oceanbase]> SELECT * FROM oceanbase.DBA_OB_TENANT_JOBS WHERE JOB_TYPE=‘ALTER_TENANT_LOCALITY’\G
*************************** 1. row ***************************
JOB_ID: 1
JOB_TYPE: ALTER_TENANT_LOCALITY
JOB_STATUS: INPROGRESS
RESULT_CODE: NULL
PROGRESS: 0
START_TIME: 2025-07-06 11:32:30.758491
MODIFY_TIME: 2025-07-06 11:32:30.758491
TENANT_ID: 1004
SQL_TEXT: ALTER TENANT ocp_monitor LOCALITY = ‘FULL{1}@zone1,FULL@zone_1’
EXTRA_INFO: FROM: ‘FULL{1}@zone1’, TO: ‘FULL{1}@zone1, FULL{1}@zone_1
RS_SVR_IP: 192.168.2.111
RS_SVR_PORT: 2882
*************************** 2. row ***************************
JOB_ID: 2
JOB_TYPE: ALTER_TENANT_LOCALITY
JOB_STATUS: INPROGRESS
RESULT_CODE: NULL
PROGRESS: 0
START_TIME: 2025-07-06 11:32:33.874029
MODIFY_TIME: 2025-07-06 11:32:33.874029
TENANT_ID: 1002
SQL_TEXT: ALTER TENANT ocp_meta LOCALITY = ‘FULL{1}@zone1,FULL@zone_1’
EXTRA_INFO: FROM: ‘FULL{1}@zone1’, TO: ‘FULL{1}@zone1, FULL{1}@zone_1
RS_SVR_IP: 192.168.2.111
RS_SVR_PORT: 2882
*************************** 3. row ***************************
JOB_ID: 3
JOB_TYPE: ALTER_TENANT_LOCALITY
JOB_STATUS: SUCCESS
RESULT_CODE: 0
PROGRESS: 100
START_TIME: 2025-07-06 11:32:34.091456
MODIFY_TIME: 2025-07-06 11:51:07.341756
TENANT_ID: 1
SQL_TEXT: ALTER TENANT sys LOCALITY = ‘FULL{1}@zone1,FULL@zone_1’
EXTRA_INFO: FROM: ‘FULL{1}@zone1’, TO: ‘FULL{1}@zone1, FULL{1}@zone_1
RS_SVR_IP: 192.168.2.111
RS_SVR_PORT: 2882
3 rows in set (0.003 sec)

obclient(root@sys)[oceanbase]> SELECT * FROM cdb_ob_transfer_tasks where tenant_id=1002\G;
*************************** 1. row ***************************
TENANT_ID: 1002
TASK_ID: 105
CREATE_TIME: 2025-07-06 13:36:36.384623
MODIFY_TIME: 2025-07-06 13:36:36.431844
SRC_LS: 1001
DEST_LS: 1002
PART_LIST: 500023:500613,500023:500616,500023:500619,500023:500622,500023:500625,500301:500301,500302:500302,500304:500304,500306:500306,500307:500307,500309:500309,500310:500310,500311:500311,500313:500313,500315:500315,500317:500317,500319:500319,500321:500321,500327:500327,500329:500329,500332:500332,500334:500334,500336:500336,500338:500338,500339:500339,500342:500342,500344:500344,500346:500346,500348:500348,500349:500349,500350:500350,500351:500351,500352:500352,500354:500354,500355:500355,500357:500357,500361:500361,500363:500363,500364:500364,500365:500365,500367:500367,500370:500370,500376:500376,500378:500378,500380:500380,500383:500383,500392:500392,500394:500394,500396:500396,500398:500398,500400:500400,500402:500402,500404:500404,500406:500406,500410:500410,500412:500412,500416:500416,500418:500418,500420:500420,500421:500421,500423:500423,500425:500425,500427:500427,500430:500430,500433:500433,500435:500435,500437:500437,500439:500439,500441:500441,500443:500443,500445:500445,500447:500447,500449:500449,500452:500452,500453:500453,500455:500455,500457:500457,500459:500459,500462:500462,500464:500464,500466:500466,500468:500468,500470:500470,500472:500472,500474:500474,500476:500476,500478:500478,500480:500480,500482:500482,500485:500485,500487:500487,500489:500489,500491:500491,500494:500494,500495:500495,500496:500496,500497:500497,500499:500499,500500:500500,500501:500501
PART_COUNT: 100
NOT_EXIST_PART_LIST: NULL
LOCK_CONFLICT_PART_LIST: NULL
TABLE_LOCK_TABLET_LIST: NULL
TABLET_LIST: NULL
TABLET_COUNT: 0
START_SCN: 0
FINISH_SCN: 0
STATUS: INIT
TRACE_ID: YB42C0A8026F-00063938E1540F42-0-0
RESULT: -1
BALANCE_TASK_ID: 67838
TABLE_LOCK_OWNER_ID: -1
COMMENT: Wait to retry due to the last failure
1 row in set (0.025 sec)

obclient(root@sys)[oceanbase]> SELECT * FROM cdb_ob_transfer_tasks where tenant_id=1004\G;
*************************** 1. row ***************************
TENANT_ID: 1004
TASK_ID: 96
CREATE_TIME: 2025-07-06 13:53:09.358556
MODIFY_TIME: 2025-07-06 13:53:09.398414
SRC_LS: 1001
DEST_LS: 1002
PART_LIST: 500179:503563,500179:503564,500179:503565,500179:503566,500179:503567,500179:503568,500069:502582,500069:502583,500069:502584,500069:502585,500069:502586,500069:502587,500069:502588,500069:502589,500069:502590,500069:502591,500069:502592,500069:502593,500069:502594,500069:502595,500069:502596,500069:502117,500069:502118,500069:502119,500069:502120,500069:502121,500069:502122,500069:502123,500069:502124,500069:502125,500069:502126,500069:502127,500069:502128,500069:502129,500069:502130,500069:502131,500005:503295,500005:503296,500005:503297,500005:503298,500005:503299,500005:503300,500005:503301,500005:503302,500005:503303,500005:503304,500005:503305,500005:503306,500005:503307,500005:503308,500005:503309,500069:502551,500069:502552,500069:502553,500069:502554,500069:502555,500069:502556,500069:502557,500069:502558,500069:502559,500069:502560,500069:502561,500069:502562,500069:502563,500069:502564,500069:502565,500197:501453,500197:501454,500069:502861,500069:502862,500069:502863,500069:502864,500069:502865,500069:502866,500069:502867,500069:502868,500069:502869,500069:502870,500069:502871,500069:502872,500069:502873,500069:502874,500069:502875,500316:501413,500316:501414,500069:502737,500069:502738,500069:502739,500069:502740,500069:502741,500069:502742,500069:502743,500069:502744,500069:502745,500069:502746,500069:502747,500069:502748,500069:502749,500069:502750,500069:502751
PART_COUNT: 100
NOT_EXIST_PART_LIST: NULL
LOCK_CONFLICT_PART_LIST: NULL
TABLE_LOCK_TABLET_LIST: NULL
TABLET_LIST: NULL
TABLET_COUNT: 0
START_SCN: 0
FINISH_SCN: 0
STATUS: INIT
TRACE_ID: YB42C0A8026F-00063938E6741062-0-0
RESULT: -1
BALANCE_TASK_ID: 55578
TABLE_LOCK_OWNER_ID: -1
COMMENT: Wait to retry due to the last failure
1 row in set (0.028 sec)

ERROR: No query specified

2 个赞

这个时间跟你的租户数据量大小有关系了。等副本在新的zone建完就好了。sys租户数据量比较少所以很快

1 个赞

查看租户历史Transfer task执行情况,这两个租户分别查下,balance_task_id是根据cdb_ob_transfer_tasks查出来的

select create_time,finish_time,task_id,status,result,balance_task_id,comment from CDB_OB_TRANSFER_TASK_HISTORY where tenant_id = xxx and balance_task_id = xxxx order by finish_time desc limit 5;