给表添加字段报SQL 错误 [4109] [HY000]: Server state or role not the same as expected

【 使用环境 】生产环境
【 OB or 其他组件 】OB
【 使用版本 】4.2.1.3
【问题描述】
给表添加字段报SQL 错误 [4109] [HY000]: Server state or role not the same as expected
开启
【附件及日志】
SET ob_enable_show_trace=‘ON’;
select last_trace_id();
追溯以后获取到ID:YB420A0A0A92-000610AD904FC53E-0-0
关联的日志:
[2024-02-18 11:27:14.851057] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“sql_compile”,“id”:“0006119f-8daf-e4ce-9942-6106625bac03”,“start_ts”:1708226834851022,“end_ts”:0,“parent_id”:“0006119f-8daf-e4c7-4459-6cce1afebbb9”,“is_follow”:false,“tags”:[{“sql_text”:“SHOW STATUS”}]}
[2024-02-18 11:27:14.851066] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“com_query_process”,“id”:“0006119f-8daf-e4c4-3b94-beaa923b4aff”,“start_ts”:1708226834851012,“end_ts”:0,“parent_id”:“0006119f-8daf-e395-eebb-662172854467”,“is_follow”:false,“tags”:[{“sess_id”:3221789081},{“action_name”:""},{“module_name”:""},{“client_info”:""},{“receive_ts”:1708226834850959},{“log_trace_id”:“YB420A0A0A92-000610AD904FC53E-0-0”}]}
[2024-02-18 11:27:14.851084] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“parse”,“id”:“0006119f-8daf-e500-84ca-c5da4c5d3660”,“start_ts”:1708226834851072,“end_ts”:1708226834851083,“parent_id”:“0006119f-8daf-e500-34f1-c42b3810949f”,“is_follow”:false}
[2024-02-18 11:27:14.851090] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“sql_compile”,“id”:“0006119f-8daf-e4ce-9942-6106625bac03”,“start_ts”:1708226834851022,“end_ts”:0,“parent_id”:“0006119f-8daf-e4c7-4459-6cce1afebbb9”,“is_follow”:false,“tags”:[{“hit_plan”:false}]}
[2024-02-18 11:27:14.851334] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“resolve”,“id”:“0006119f-8daf-e518-269d-876940334e0e”,“start_ts”:1708226834851096,“end_ts”:1708226834851334,“parent_id”:“0006119f-8daf-e500-34f1-c42b3810949f”,“is_follow”:false}
[2024-02-18 11:27:14.851814] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“rewrite”,“id”:“0006119f-8daf-e625-22d2-cee25d4b887f”,“start_ts”:1708226834851365,“end_ts”:1708226834851813,“parent_id”:“0006119f-8daf-e500-34f1-c42b3810949f”,“is_follow”:false}
[2024-02-18 11:27:14.852061] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“optimize”,“id”:“0006119f-8daf-e7f8-e5cb-a7086621e2f7”,“start_ts”:1708226834851832,“end_ts”:1708226834852060,“parent_id”:“0006119f-8daf-e500-34f1-c42b3810949f”,“is_follow”:false}
[2024-02-18 11:27:14.852124] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“code_generate”,“id”:“0006119f-8daf-e8eb-058e-5334dc4481aa”,“start_ts”:1708226834852075,“end_ts”:1708226834852124,“parent_id”:“0006119f-8daf-e500-34f1-c42b3810949f”,“is_follow”:false}
[2024-02-18 11:27:14.852178] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“hard_parse”,“id”:“0006119f-8daf-e500-34f1-c42b3810949f”,“start_ts”:1708226834851072,“end_ts”:1708226834852178,“parent_id”:“0006119f-8daf-e4ce-9942-6106625bac03”,“is_follow”:false}
[2024-02-18 11:27:14.852196] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“sql_compile”,“id”:“0006119f-8daf-e4ce-9942-6106625bac03”,“start_ts”:1708226834851022,“end_ts”:1708226834852195,“parent_id”:“0006119f-8daf-e4c7-4459-6cce1afebbb9”,“is_follow”:false,“tags”:[{“sql_id”:“8482C19F04EF03269F8D83A80B7DC877”},{“database_id”:500002},{“plan_hash”:11819834400086817700}]}
[2024-02-18 11:27:14.852210] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“open”,“id”:“0006119f-8daf-e96d-e028-cb5f83198f2c”,“start_ts”:1708226834852205,“end_ts”:1708226834852210,“parent_id”:“0006119f-8daf-e96d-703c-beb5c1295be2”,“is_follow”:false}
[2024-02-18 11:27:14.852238] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“do_local_das_task”,“id”:“0006119f-8daf-e97b-97be-f1eb785272f1”,“start_ts”:1708226834852219,“end_ts”:1708226834852238,“parent_id”:“0006119f-8daf-e977-1f59-51b28a43b3a4”,“is_follow”:false}
[2024-02-18 11:27:14.852274] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“response_result”,“id”:“0006119f-8daf-e977-1f59-51b28a43b3a4”,“start_ts”:1708226834852215,“end_ts”:1708226834852274,“parent_id”:“0006119f-8daf-e96d-703c-beb5c1295be2”,“is_follow”:false}
[2024-02-18 11:27:14.852291] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“close_das_task”,“id”:“0006119f-8daf-e9bd-6121-d4cfd1581689”,“start_ts”:1708226834852285,“end_ts”:1708226834852290,“parent_id”:“0006119f-8daf-e9ba-6a1b-937dc97002de”,“is_follow”:false}
[2024-02-18 11:27:14.852312] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“end_transaction”,“id”:“0006119f-8daf-e9d7-5775-74246749def5”,“start_ts”:1708226834852311,“end_ts”:1708226834852311,“parent_id”:“0006119f-8daf-e9ba-6a1b-937dc97002de”,“is_follow”:false,“tags”:[{“trans_id”:0}]}
[2024-02-18 11:27:14.852328] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“close”,“id”:“0006119f-8daf-e9ba-6a1b-937dc97002de”,“start_ts”:1708226834852282,“end_ts”:1708226834852328,“parent_id”:“0006119f-8daf-e96d-703c-beb5c1295be2”,“is_follow”:false}
[2024-02-18 11:27:14.852347] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“sql_execute”,“id”:“0006119f-8daf-e96d-703c-beb5c1295be2”,“start_ts”:1708226834852205,“end_ts”:1708226834852346,“parent_id”:“0006119f-8daf-e4c7-4459-6cce1afebbb9”,“is_follow”:false}
[2024-02-18 11:27:14.852391] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“mpquery_single_stmt”,“id”:“0006119f-8daf-e4c7-4459-6cce1afebbb9”,“start_ts”:1708226834851015,“end_ts”:1708226834852391,“parent_id”:“0006119f-8daf-e4c4-3b94-beaa923b4aff”,“is_follow”:false}
[2024-02-18 11:27:14.852407] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53E-0-0] {“trace_id”:“0006119f-8daf-e394-73cf-ece3241097b2”,“name”:“com_query_process”,“id”:“0006119f-8daf-e4c4-3b94-beaa923b4aff”,“start_ts”:1708226834851012,“end_ts”:1708226834852407,“parent_id”:“0006119f-8daf-e395-eebb-662172854467”,“is_follow”:false}
[2024-02-18 11:27:14.858846] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53F-0-0] {“trace_id”:“0006119f-8daf-fd3c-75e6-e07237e8a29d”,“name”:“pc_get_plan”,“id”:“0006119f-8db0-0359-01e8-0692c9bb3140”,“start_ts”:1708226834858841,“end_ts”:1708226834858845,“parent_id”:“0006119f-8db0-0358-0357-23695c6a7c26”,“is_follow”:false}
[2024-02-18 11:27:14.858864] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53F-0-0] {“trace_id”:“0006119f-8daf-fd3c-75e6-e07237e8a29d”,“name”:“sql_compile”,“id”:“0006119f-8db0-0358-0357-23695c6a7c26”,“start_ts”:1708226834858840,“end_ts”:0,“parent_id”:“0006119f-8db0-0353-9d60-47a975e8bee2”,“is_follow”:false,“tags”:[{“sql_text”:“select last_trace_id()”}]}
[2024-02-18 11:27:14.858872] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53F-0-0] {“trace_id”:“0006119f-8daf-fd3c-75e6-e07237e8a29d”,“name”:“com_query_process”,“id”:“0006119f-8db0-034f-07f9-cfa2c10e3f16”,“start_ts”:1708226834858831,“end_ts”:0,“parent_id”:“0006119f-8daf-fd3d-d1f0-52dfa9f88f3b”,“is_follow”:false,“tags”:[{“sess_id”:3221789081},{“action_name”:""},{“module_name”:""},{“client_info”:""},{“receive_ts”:1708226834858756},{“log_trace_id”:“YB420A0A0A92-000610AD904FC53F-0-0”}]}
[2024-02-18 11:27:14.858909] [98748][T1002_L0_G0][T1002][YB420A0A0A92-000610AD904FC53F-0-0] {“trace_id”:“0006119f-8daf-fd3c-75e6-e07237e8a29d”,“name”:“parse”,“id”:“0006119f-8db0-0383-453d-29cdf600672c”,“start_ts”:1708226834858883,“end_ts”:1708226834858908,“parent_id”:“0006119f-8db0-0382-736d-8984a8313b47”,“is_follow”:false}

【SOP系列 22 】——故障诊断第一步(系统巡检和诊断信息收集)

这个日志不像是rootservice.log或者observer.log过滤的。可以去146节点过滤下这2个日志看下。

贴的日志看着像是trace.log的日志,建议用obdiag 将报错时候的日志捞回来。OceanBase分布式数据库-海量数据 笔笔算数

提供一个排查这种问题的思路:

  1. 执行一下给表添加字段的操作;用obdiag analyze log 10m 分析一下最近10m中的日志看有哪些ERROR级别的报错信息;贴出来;
  2. 第一步操作完成后没有发现,用obdiag gather log --since 10m捞一下日志,或者更精确一点的捞日志obdiag gather log --since 10m --grep TRACE_ID

[2024-02-18 12:20:26.615119] WDIAG [SQL.SESSION] get_optimizer_features_enable_version (ob_basic_session_info.cpp:5279) [98758][T1002_L0_G0][T1002][YB420A0A0A92-000610AD41EF19B4-0-0] [lt=9][errcode=0] fail invalid optimizer features version(ret=0, version_str=, tmp_version=0)

[2024-02-18 12:20:26.616455] INFO [SQL.JO] compute_table_location (ob_join_order.cpp:245) [98758][T1002_L0_G0][T1002][YB420A0A0A92-000610AD41EF19B4-0-0] [lt=24] succeed to calculate base table sharding info(table_id=18446744073709551614, ref_table_id=12008, is_global_index=false)

[2024-02-18 12:20:26.616497] INFO [SQL.JO] init_est_sel_info_for_access_path (ob_join_order.cpp:12035) [98758][T1002_L0_G0][T1002][YB420A0A0A92-000610AD41EF19B4-0-0] [lt=28] total rowcount, use statistics(table_meta_info_.table_row_count_=2, table_meta_info_.average_row_size_=4.000000000000000000e+01, table_meta_info_.micro_block_count_=0, table_meta_info_.part_size_=8.000000000000000000e+01)

[2024-02-18 12:20:26.617461] WDIAG [SQL.PC] common_free (ob_lib_cache_object_manager.cpp:141) [98758][T1002_L0_G0][T1002][YB420A0A0A92-000610AD41EF19B4-0-0] [lt=7][errcode=0] set logical del time(cache_obj->get_logical_del_time()=6465763291612, cache_obj->added_lc()=false, cache_obj->get_object_id()=7402, cache_obj->get_tenant_id()=1002, lbt()=“0x117ed57c 0x9ecca40 0x49a0d71 0x9e44d2a 0x499dbdd 0x4999c83 0x49942fd 0x498a995 0x4985d2e 0x95a7f04 0x11ab9452 0x7fe37b81fea7 0x7fe37b73da2f”)

找到了日志,您看看

这个日志是你手动从某一个机器上grep出来的吧,看着日志不全。

OB_LOG_obcluster1_1_10.10.10.146_20240218150000_20240218150132.zip (2.3 MB)

YB420A0A0A92-000610AD92EDC1A4-0-0

包含在里面的observer的日志

是通过ocp部署的集群吗?最近集群有做过什么运维操作吗,比如升级observer。

从4.2.1.3-103000032023122818升级到4.2.1.3-103010052024011916

  1. 看版本号是hotfix的,看看从ocp的升级任务中有失败的吗,或者有失败你们手动点跳过的操作吗?
  2. 贴一下业务t_consume_order_commission表的schema,用show create table t_consume_order_commission

没有失败,没有跳过
CREATE TABLE t_consume_order_commission (
id bigint(20) NOT NULL COMMENT ‘id’,
employee_id int(10) DEFAULT NULL,
employee_name varchar(100) DEFAULT NULL,
employee_work_no int(10) DEFAULT NULL,
brand_id int(10) DEFAULT NULL,
brand_name varchar(100) DEFAULT NULL,
shop_id int(10) DEFAULT NULL,
shop_name varchar(100) DEFAULT NULL,
member_id bigint(20) DEFAULT NULL,
member_name varchar(100) DEFAULT NULL,
member_mobile varchar(20) DEFAULT NULL,
order_id bigint(20) NOT NULL,
order_name varchar(100) DEFAULT NULL,
order_amount bigint(20) DEFAULT NULL,
commission_amount bigint(20) DEFAULT NULL,
order_time datetime DEFAULT NULL,
subject_id int(10) DEFAULT NULL,
subject_name varchar(100) DEFAULT NULL,
subject_count int(10) DEFAULT NULL,
subject_type tinyint(1) DEFAULT NULL,
subject_unit_price bigint(20) DEFAULT NULL,
work_type tinyint(1) DEFAULT NULL,
room_number varchar(10) DEFAULT NULL,
account_date date DEFAULT NULL,
PRIMARY KEY (id, order_id),
KEY idx_order (order_id) BLOCK_SIZE 16384 LOCAL,
KEY idx_order_time (order_time) BLOCK_SIZE 16384 LOCAL
) DEFAULT CHARSET = utf8mb4 ROW_FORMAT = COMPACT COMPRESSION = ‘zstd_1.3.8’ REPLICA_NUM = 3 BLOCK_SIZE = 16384 USE_BLOOM_FILTER = FALSE TABLET_SIZE = 134217728 PCTFREE = 0
partition by range(order_id)
(partition p2210 values less than (154377230745600000),
partition p2211 values less than (165248866713600000),
partition p2212 values less than (176482890547200000),
partition p2301 values less than (187716914380800000),
partition p2302 values less than (197863774617600000),
partition p2303 values less than (209097798451200000),
partition p2304 values less than (219969434419200000),
partition p2305 values less than (231203458252800000),
partition p2306 values less than (242075094220800000),
partition p2307 values less than (253309118054400000),
partition p2308 values less than (264543141888000000),
partition p2309 values less than (275414777856000000),
partition p2310 values less than (286648801689600000),
partition p2311 values less than (297520437657600000),
partition p2312 values less than (308754461491200000),
partition p2401 values less than (319988485324800000),
partition p2402 values less than (330497733427200000),
partition p2403 values less than (341731757260800000),
partition p2404 values less than (352603393228800000),
partition p2405 values less than (363837417062400000),
partition p2406 values less than (374709053030400000),
partition p2407 values less than (385943076864000000),
partition p2408 values less than (397177100697600000),
partition p2409 values less than (408048736665600000),
partition p2410 values less than (419282760499200000),
partition p2411 values less than (430154396467200000),
partition p2412 values less than (441388420300800000),
partition p2501 values less than (452622444134400000),
partition p2502 values less than (462769304371200000),
partition p2503 values less than (474003328204800000),
partition p2504 values less than (484874964172800000),
partition p2505 values less than (496108988006400000),
partition p2506 values less than (506980623974400000),
partition p2507 values less than (518214647808000000),
partition p2508 values less than (529448671641600000),
partition p2509 values less than (540320307609600000),
partition p2510 values less than (551554331443200000),
partition p2511 values less than (562425967411200000),
partition p2512 values less than (573659991244800000),
partition p2601 values less than (584894015078400000),
partition p2602 values less than (595040875315200000),
partition p2603 values less than (606274899148800000),
partition p2604 values less than (617146535116800000),
partition p2605 values less than (628380558950400000),
partition p2606 values less than (639252194918400000),
partition p2607 values less than (650486218752000000),
partition p2608 values less than (661720242585600000),
partition p2609 values less than (672591878553600000),
partition p2610 values less than (683825902387200000),
partition p2611 values less than (694697538355200000),
partition p2612 values less than (705931562188800000),
partition p2701 values less than (717165586022400000),
partition p2702 values less than (727312446259200000),
partition p2703 values less than (738546470092800000),
partition p2704 values less than (749418106060800000),
partition p2705 values less than (760652129894400000),
partition p2706 values less than (771523765862400000),
partition p2707 values less than (782757789696000000),
partition p2708 values less than (793991813529600000),
partition p2709 values less than (804863449497600000),
partition p2710 values less than (816097473331200000),
partition p2711 values less than (826969109299200000),
partition p2712 values less than (838203133132800000),
partition p2801 values less than (849437156966400000),
partition p2802 values less than (859946405068800000),
partition p2803 values less than (871180428902400000),
partition p2804 values less than (882052064870400000),
partition p2805 values less than (893286088704000000),
partition p2806 values less than (904157724672000000),
partition p2807 values less than (915391748505600000),
partition p2808 values less than (926625772339200000),
partition p2809 values less than (937497408307200000),
partition p2810 values less than (948731432140800000),
partition p2811 values less than (959603068108800000),
partition p2812 values less than (970837091942400000),
partition p2901 values less than (982071115776000000),
partition p2902 values less than (992217976012800000),
partition p2903 values less than (1003451999846400000),
partition p2904 values less than (1014323635814400000),
partition p2905 values less than (1025557659648000000),
partition p2906 values less than (1036429295616000000),
partition p2907 values less than (1047663319449600000),
partition p2908 values less than (1058897343283200000),
partition p2909 values less than (1069768979251200000),
partition p2910 values less than (1081003003084800000),
partition p2911 values less than (1091874639052800000),
partition p2912 values less than (1103108662886400000),
partition p3001 values less than (1114342686720000000),
partition p3002 values less than (1124489546956800000),
partition p3003 values less than (1135723570790400000),
partition p3004 values less than (1146595206758400000),
partition p3005 values less than (1157829230592000000),
partition p3006 values less than (1168700866560000000),
partition p3007 values less than (1179934890393600000),
partition p3008 values less than (1191168914227200000),
partition p3009 values less than (1202040550195200000),
partition p3010 values less than (1213274574028800000),
partition p3011 values less than (1224146209996800000),
partition p3012 values less than (1235380233830400000),
partition p3101 values less than (1246614257664000000),
partition p3102 values less than (1256761117900800000),
partition p3103 values less than (1267995141734400000),
partition p3104 values less than (1278866777702400000),
partition p3105 values less than (1290100801536000000),
partition p3106 values less than (1300972437504000000),
partition p3107 values less than (1312206461337600000),
partition p3108 values less than (1323440485171200000),
partition p3109 values less than (1334312121139200000),
partition p3110 values less than (1345546144972800000),
partition p3111 values less than (1356417780940800000),
partition p3112 values less than (1367651804774400000),
partition p3201 values less than (1378885828608000000),
partition p3202 values less than (1389395076710400000),
partition p3203 values less than (1400629100544000000),
partition p3204 values less than (1411500736512000000),
partition p3205 values less than (1422734760345600000),
partition p3206 values less than (1433606396313600000),
partition p3207 values less than (1444840420147200000),
partition p3208 values less than (1456074443980800000),
partition p3209 values less than (1466946079948800000),
partition p3210 values less than (1478180103782400000),
partition p3211 values less than (1489051739750400000),
partition p3212 values less than (1500285763584000000),
partition p3301 values less than (1511519787417600000),
partition p3302 values less than (1521666647654400000),
partition p3303 values less than (1532900671488000000),
partition p3304 values less than (1543772307456000000),
partition p3305 values less than (1555006331289600000),
partition p3306 values less than (1565877967257600000),
partition p3307 values less than (1577111991091200000),
partition p3308 values less than (1588346014924800000),
partition p3309 values less than (1599217650892800000),
partition p3310 values less than (1610451674726400000),
partition p3311 values less than (1621323310694400000),
partition p3312 values less than (1632557334528000000),
partition p3401 values less than (1643791358361600000),
partition p3402 values less than (1653938218598400000),
partition p3403 values less than (1665172242432000000),
partition p3404 values less than (1676043878400000000),
partition p3405 values less than (1687277902233600000),
partition p3406 values less than (1698149538201600000),
partition p3407 values less than (1709383562035200000),
partition p3408 values less than (1720617585868800000),
partition p3409 values less than (1731489221836800000),
partition p3410 values less than (1742723245670400000),
partition p3411 values less than (1753594881638400000),
partition p3412 values less than (1764828905472000000),
partition p3501 values less than (1776062929305600000),
partition p3502 values less than (1786209789542400000),
partition p3503 values less than (1797443813376000000),
partition p3504 values less than (1808315449344000000),
partition p3505 values less than (1819549473177600000),
partition p3506 values less than (1830421109145600000),
partition p3507 values less than (1841655132979200000),
partition p3508 values less than (1852889156812800000),
partition p3509 values less than (1863760792780800000),
partition p3510 values less than (1874994816614400000),
partition p3511 values less than (1885866452582400000),
partition p3512 values less than (1897100476416000000),
partition p3601 values less than (1908334500249600000),
partition p3602 values less than (1918843748352000000),
partition p3603 values less than (1930077772185600000),
partition p3604 values less than (1940949408153600000),
partition p3605 values less than (1952183431987200000),
partition p3606 values less than (1963055067955200000),
partition p3607 values less than (1974289091788800000),
partition p3608 values less than (1985523115622400000),
partition p3609 values less than (1996394751590400000),
partition p3610 values less than (2007628775424000000),
partition p3611 values less than (2018500411392000000),
partition p3612 values less than (2029734435225600000),
partition p3701 values less than (2040968459059200000),
partition p3702 values less than (2051115319296000000),
partition p3703 values less than (2062349343129600000),
partition p3704 values less than (2073220979097600000),
partition p3705 values less than (2084455002931200000),
partition p3706 values less than (2095326638899200000),
partition p3707 values less than (2106560662732800000),
partition p3708 values less than (2117794686566400000),
partition p3709 values less than (2128666322534400000),
partition p3710 values less than (2139900346368000000),
partition p3711 values less than (2150771982336000000),
partition p3712 values less than (2162006006169600000),
partition p3801 values less than (2173240030003200000),
partition p3802 values less than (2183386890240000000),
partition p3803 values less than (2194620914073600000),
partition p3804 values less than (2205492550041600000),
partition p3805 values less than (2216726573875200000),
partition p3806 values less than (2227598209843200000),
partition p3807 values less than (2238832233676800000),
partition p3808 values less than (2250066257510400000),
partition p3809 values less than (2260937893478400000),
partition p3810 values less than (2272171917312000000),
partition p3811 values less than (2283043553280000000),
partition p3812 values less than (2294277577113600000),
partition p3901 values less than (2305511600947200000),
partition p3902 values less than (2315658461184000000),
partition p3903 values less than (2326892485017600000),
partition p3904 values less than (2337764120985600000),
partition p3905 values less than (2348998144819200000),
partition p3906 values less than (2359869780787200000),
partition p3907 values less than (2371103804620800000),
partition p3908 values less than (2382337828454400000),
partition p3909 values less than (2393209464422400000),
partition p3910 values less than (2404443488256000000),
partition p3911 values less than (2415315124224000000),
partition p3912 values less than (2426549148057600000),
partition p4001 values less than (2437783171891200000),
partition p4002 values less than (2448292419993600000),
partition p4003 values less than (2459526443827200000),
partition p4004 values less than (2470398079795200000),
partition p4005 values less than (2481632103628800000),
partition p4006 values less than (2492503739596800000),
partition p4007 values less than (2503737763430400000),
partition p4008 values less than (2514971787264000000),
partition p4009 values less than (2525843423232000000),
partition p4010 values less than (2537077447065600000),
partition p4011 values less than (2547949083033600000),
partition p4012 values less than (2559183106867200000),
partition p4101 values less than (2570417130700800000),
partition p4102 values less than (2580563990937600000),
partition p4103 values less than (2591798014771200000),
partition p4104 values less than (2602669650739200000),
partition p4105 values less than (2613903674572800000),
partition p4106 values less than (2624775310540800000),
partition p4107 values less than (2636009334374400000),
partition p4108 values less than (2647243358208000000),
partition p4109 values less than (2658114994176000000),
partition p4110 values less than (2669349018009600000),
partition p4111 values less than (2680220653977600000),
partition p4112 values less than (2691454677811200000),
partition p4201 values less than (2702688701644800000),
partition p4202 values less than (2712835561881600000),
partition p4203 values less than (2724069585715200000),
partition p4204 values less than (2734941221683200000),
partition p4205 values less than (2746175245516800000),
partition p4206 values less than (2757046881484800000),
partition p4207 values less than (2768280905318400000),
partition p4208 values less than (2779514929152000000),
partition p4209 values less than (2790386565120000000),
partition p4210 values less than (2801620588953600000),
partition p4211 values less than (2812492224921600000),
partition p4212 values less than (2823726248755200000),
partition p4301 values less than (2834960272588800000),
partition p4302 values less than (2845107132825600000),
partition p4303 values less than (2856341156659200000),
partition p4304 values less than (2867212792627200000),
partition p4305 values less than (2878446816460800000),
partition p4306 values less than (2889318452428800000),
partition p4307 values less than (2900552476262400000),
partition p4308 values less than (2911786500096000000),
partition p4309 values less than (2922658136064000000),
partition p4310 values less than (2933892159897600000),
partition p4311 values less than (2944763795865600000),
partition p4312 values less than (2955997819699200000))

从你提供的rootservice.log的日志中发现一条可疑的日志:[2024-02-18 15:00:50.776669] WDIAG [SHARE.SCHEMA] check_table_history_matched_ (ob_table_sql_service.cpp:2008) [97900][DDLQueueTh0][T0][YB420A0A0A92-000610AD92EDC1A1-0-0] [lt=49][errcode=-4109] __all_table_history’s row not match with __all_table’s(ret=-4109, ret=“OB_STATE_NOT_MATCH”, tenant_id=1002, table_id=501085, schema_version=1708239650768592)

我内部提工单找人确认一下,请稍后。

提供一下ob版本号,带commit_id的

拿一下表t_consume_order_commission的下面的信息

select * from __all_virtual_table where table_id = XXX and tenant_id = XXX;
select * from __all_virtual_table_history where table_id = XXX and tenant_id = XXX;

已解决;

问题原因:内部的__all_table_history和__all_table中表t_consume_order_commission中的数据不一样,在__all_table中表t_consume_order_commission的transition_point、b_transition_point、interval_range和b_interval_range字段为空字符串,而在__all_table_history中这几个字段为NULL,4.2版本加了all_table_history和all table内容的校验,所以如果插入数据不一致的情况下会报错。

排查路径:

  1. obdiag gather log 收集了一下日志

  2. 通过收集的日志分析rootservice.log,找到了4109的日志,发现日志提示__all_table_history’s row not match with __all_table’s(ret=-4109, ret=“OB_STATE_NOT_MATCH”, tenant_id=1002, table_id=501085, schema_version=1708239650768592)

  3. 通过查询__all_table和__all_table_history表找出了不同

处理措施:(手动更新__all_table表中不一致的字段)
1、sys租户登陆
2、alter system change tenant tenant_id = 1002; # 切到业务租户
3、update __all_table set transition_point = NULL, b_transition_point = NULL, interval_range = NULL, b_interval_range = NULL where table_id = xxxx;(更新字段)

总结了一个SOP的文档:【SOP系列 23】—— OB 升级到 4.2.1 版本后遇到部分分区表执行ddl报错-4109问题排查及处理方法