obkv 压测失败

dazhuangge · 2025 年4 月 18 日 10:19

【使用环境】生产环境
【 OB or 其他组件】OBKV
【使用版本】
【问题描述】用get key 压测时出现errCode:-4016, errCodeName:ObErrUnexpected, errMsg:Internal error, server:10.38.48.37:2885, trace:YA8480A2624F2-0000000000000012-0-0
【复现路径】问题出现前后相关操作
【附件及日志】推荐使用OceanBase敏捷诊断工具obdiag收集诊断信息,详情参见链接（右键跳转查看）：

【SOP系列 22 】——故障诊断第一步(自助诊断和诊断信息收集)

【备注】基于 LLM 和开源文档 RAG 的论坛小助手已开放测试，在发帖时输入 [@论坛小助手] 即可召唤小助手，欢迎试用！

第一步使用
/usr/local/bin/memtier_benchmark
-s $host
-p $port
-a $passwd
-t 10
-c 50
-n 30000
–data-size=1024
–key-maximum=15000000
–key-minimum=1
–pipeline=1
–distinct-client-seed
–random-data
–key-prefix=kv_
–command=‘set key data’
–command-key-pattern=P 生成压测数据

第二步执行/usr/local/bin/memtier_benchmark
-s $host
-p $port
-a $passwd
-t 10
-c 50
-n 31457
–pipeline=1
–distinct-client-seed
–random-data
–data-size=1024
–key-maximum=15728640
–key-minimum=1
–command=get key
–command-key-pattern=R
出现 ver 10.38.48.37:2885 handle error response: -ERR errCode:-4016, errCodeName:ObErrUnexpected, errMsg:Internal error, server:10.3**7:2885, trace:YA7320A2624F2-0000000000000011-0-0

论坛小助手 · 2025 年4 月 18 日 10:20

根因分析场景: 事务其他错误
根因分析命令: obdiag rca run --scene=transaction_other_error
日志采集场景: SQL 执行出错
日志采集命令: obdiag gather scene run --scene=observer.sql_err --env “{db_connect=’-h127.0.0.1 -P2881 -utest@test -p****** -Dtest’, trace_id=‘Yxx’}”

推荐使用 obdiag 工具进行根因分析和日志采集，以便确认问题所在。上述命令将帮助您定位 errCode:-4016（ObErrUnexpected 和 Internal error）的原因，并收集与 SQL 执行相关的日志信息。

请您执行上述命令并将采集到的日志压缩包提供给我们，以便进一步分析问题。另外，为了更好地帮助您解决问题，请您提供以下信息：

压测过程中使用的具体 OBKV 版本号是多少？
出现错误时的压测负载情况如何？例如 QPS、并发数等。
是否有其他相关错误日志或堆栈信息可以提供？
在出现此问题之前，系统是否发生过任何变更，例如配置修改、数据导入或升级操作？

附上敏捷诊断工具 obdiag 使用帮助链接

dazhuangge · 2025 年4 月 18 日 10:23

[2025-04-18 10:15:05.036772] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038794][Y0-00007F8AB6FAF980] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA8CA0A2624F2-00
0000000000001A-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036771] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038778][Y0-00007F8AC0AA7A30] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036772] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038810][Y0-00007F8AC0AA8470] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036775] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038778][Y0-00007F8AC0AA7A30] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036774] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038786][Y0-00007F8AC08470F0] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA7780A2624F2-00
0000000000001B-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036775] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038810][Y0-00007F8AC0AA8470] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036777] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038778][Y0-00007F8AC0AA7A30] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA76A0A2624F2-00
0000000000001B-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036780] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038810][Y0-00007F8AC0AA8470] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA8980A2624F2-00
0000000000001D-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036784] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038798][Y0-00007F8AC08462B0] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036785] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038801][Y0-00007F8AB70479B0] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036787] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038805][Y0-00007F8AB841D8B0] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036787] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038803][Y0-00007F8AB841E570] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036787] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038772][Y0-00007F8AB7047930] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036788] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038798][Y0-00007F8AC08462B0] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036788] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038796][Y0-00007F8AB7045EB0] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036790] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038805][Y0-00007F8AB841D8B0] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036788] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038801][Y0-00007F8AB70479B0] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036790] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038803][Y0-00007F8AB841E570] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036791] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038800][Y0-00007F8AB70471F0] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036791] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038796][Y0-00007F8AB7045EB0] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036790] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038772][Y0-00007F8AB7047930] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036793] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038800][Y0-00007F8AB70471F0] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036790] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038798][Y0-00007F8AC08462B0] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA7920A2624F2-00
0000000000001F-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036791] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038805][Y0-00007F8AB841D8B0] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA88E0A2624F2-00
0000000000001C-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036792] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038801][Y0-00007F8AB70479B0] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA7E60A2624F2-00
0000000000001C-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036792] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038815][Y0-00007F8AB8476DB0] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036794] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038796][Y0-00007F8AB7045EB0] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA78A0A2624F2-00
0000000000001A-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036794] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038772][Y0-00007F8AB7047930] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA8A60A2624F2-00
00000000000017-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036795] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038800][Y0-00007F8AB70471F0] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA8840A2624F2-00
0000000000001B-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036792] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038803][Y0-00007F8AB841E570] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA79A0A2624F2-00
0000000000001F-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036797] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038815][Y0-00007F8AB8476DB0] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036797] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038792][Y0-00007F8AB70473F0] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036800] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038792][Y0-00007F8AB70473F0] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036799] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038781][Y0-00007F8AC0846530] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036799] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038815][Y0-00007F8AB8476DB0] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA7620A2624F2-00
0000000000001B-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036800] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038801][Y0-00007F8AC0AA88B0] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036800] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038805][Y0-00007F8AB7047970] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036801] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038781][Y0-00007F8AC0846530] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036797] WDIAG [PROXY] gen_redis_request (ob_rpc_redis_command_factory.cpp:342) [2038789][Y0-00007F8AC08767F0] [lt=0] [dc=0] failed to init rowkey for redis command(ret=-4016, command=get)
[2025-04-18 10:15:05.036801] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038792][Y0-00007F8AB70473F0] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA7860A2624F2-00
0000000000001B-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036802] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038801][Y0-00007F8AC0AA88B0] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036803] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038805][Y0-00007F8AB7047970] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036803] WDIAG [PROXY] analyze_redis_request (ob_rpc_redis_analyzer.cpp:46) [2038789][Y0-00007F8AC08767F0] [lt=0] [dc=0] fail to analyze redis request(ret=-4016)
[2025-04-18 10:15:05.036803] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038781][Y0-00007F8AC0846530] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA76C0A2624F2-00
00000000000020-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036803] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038801][Y0-00007F8AC0AA88B0] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA7960A2624F2-00
0000000000001E-0-0”, sub_index_id:0})
[2025-04-18 10:15:05.036804] WDIAG [PROXY.SM] setup_process_redis_request (ob_rpc_request_sm.cpp:5784) [2038805][Y0-00007F8AB7047970] [lt=0] [dc=0] fail to call ObRpcRedisAnalyzer::analyze_redis_request(ret=-4016, rpc_trace_id={trace_id:“YA7A00A2624F2-00
0000000000001A-0-0”, sub_index_id:0})
@

论坛小助手 · 2025 年4 月 18 日 10:34

根据您提供的日志信息，errCode:-4016（ObErrUnexpected 和 Internal error）表明 OBKV 在处理 Redis 请求时遇到了内部错误。具体来说，问题出现在初始化 Rowkey 或解析 Redis 请求的过程中。

以下是可能的原因及解决方法：

1. Rowkey 初始化失败

日志中多次出现以下错误：

failed to init rowkey for redis command(ret=-4016, command=get)

这表明 OBKV 在解析 Redis 命令时无法正确初始化 Rowkey。通常，Rowkey 的生成依赖于 Redis Key 的格式和分区规则。如果 Redis Key 不符合预期的格式或分区规则配置不正确，可能会导致此问题。

解决方法：

检查 Redis Key 格式：确保压测工具生成的 Key 符合 OBKV 的分区规则。例如，Key 是否包含正确的前缀或分隔符。
验证分区规则：确认 OBKV 表的分区规则是否正确配置。可以通过以下 SQL 查询表的分区信息：
```
SHOW CREATE TABLE your_table_name;
```
确保分区键与 Redis Key 的生成逻辑一致。

2. Redis 请求解析失败

日志中还多次出现以下错误：

fail to analyze redis request(ret=-4016)

这表明 OBKV 在解析 Redis 请求时遇到了问题。可能是由于请求格式不正确或协议版本不匹配。

解决方法：

检查协议版本：确保压测工具使用的协议版本与 OBKV 兼容。OBKV 支持 Redis 协议，但某些特定命令或参数可能不受支持。
验证请求格式：确认压测工具生成的命令格式是否正确。例如，get __key__ 中的 __key__ 是否被正确替换为实际值。

3. 系统资源不足

errCode:-4016 也可能由系统资源不足引起，例如内存或线程池耗尽。

解决方法：

检查系统资源：通过以下命令查看 OBKV 节点的资源使用情况：
```
obdiag check sysinfo --env "{observer_ip='10.38.48.37', observer_port='2885'}"
```
调整资源配置：如果发现资源瓶颈，可以适当增加内存或调整线程池大小。例如，修改 memstore_limit_percentage 参数以优化内存使用。

4. 收集诊断信息

为了进一步定位问题，请使用 obdiag 工具收集诊断信息：

obdiag gather scene run --scene=observer.sql_err --env "{db_connect='-h10.38.48.37 -P2885 -utest -p****** -Dtest', trace_id='YA8CA0A2624F2-000000000000001A-0-0'}"

将生成的日志压缩包提供给我们，以便更深入地分析问题。

5. 其他建议

升级版本：如果当前 OBKV 版本较旧，建议升级到最新稳定版本，以修复已知问题。
联系技术支持：如果问题仍未解决，可联系 OceanBase 技术支持团队，提供完整的日志和配置信息。

希望以上建议能帮助您解决问题！如有进一步疑问，请随时联系我们。

检索到的文档如下:

(小助手的答复已结束，如未能解决您的问题，请继续提问并等待其他同学的回复，谢谢！)

辞霜 · 2025 年4 月 18 日 10:53

observer版本是多少

dazhuangge · 2025 年4 月 18 日 10:53

压测前,已进行过合并

dazhuangge · 2025 年4 月 18 日 10:54

OceanBase 版本号
4.2.5.2

辞霜 · 2025 年4 月 18 日 11:15

麻烦提供一份覆盖操作的observe日志。

dazhuangge · 2025 年4 月 18 日 11:27

[2025-04-18 10:47:00.006766] INFO [LIB] stat (utility.h:1212) [3747537][T1002_L0_G1][T1002][YB420A263026-000632691F3F07E0-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, thi
s=0x7f5ac1315470, extra_info="")
[2025-04-18 10:47:00.006786] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3747537][T1002_L0_G1][T1002][YB420A263026-000632691F3F07E0-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1002, self=“10.38.48.37:2882”, role=“FOLLOWER”, to
tal_group_log_cnt=10, avg_log_batch_cnt=0, total_group_log_size=1211, avg_group_log_size=121)
[2025-04-18 10:47:00.006797] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1761) [3747537][T1002_L0_G1][T1002][YB420A263026-000632691F3F07E0-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1002, self=“10.38.48.37:2882”, committed siz
e=1211)
[2025-04-18 10:47:00.007572] INFO [LIB] stat (utility.h:1212) [1028206][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=3] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=775, this=0x7f5ac1320eb0, extra_inf
o=“palf_id:1002”)
[2025-04-18 10:47:00.007587] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2238) [1028206][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1002, self:“10.38.48.37:2882”, has_set_deleted:false}
, accum_size=1211)
[2025-04-18 10:47:00.007598] INFO [LIB] stat (utility.h:1229) [1028206][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=5] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=11, this=0x7f5b1f1ad3b0, extra_in
fo="")
[2025-04-18 10:47:00.007625] INFO [LIB] stat (utility.h:1212) [1028203][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17, this=0x7f5ac13210e0, extra_info=
“palf_id:1002”)
[2025-04-18 10:47:00.007785] INFO [LIB] stat (utility.h:1212) [3747537][T1002_L0_G1][T1002][YB420A263025-0006326A76BD0390-0-0] [lt=7] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x7f5ac13156a0, e
xtra_info=“palf_id:1002”)
[2025-04-18 10:47:00.007793] INFO [LIB] stat (utility.h:1212) [3747537][T1002_L0_G1][T1002][YB420A263025-0006326A76BD0390-0-0] [lt=6] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=992, this=0x7f5ac13158d0, extra_in
fo=“palf_id:1002”)
[2025-04-18 10:47:00.007800] INFO [PALF] read (log_cache.cpp:100) [1029175][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=11] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1211, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00)
[2025-04-18 10:47:00.009557] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [1028257][T1002_HAService][T1002][YB420A263025-00063268DF8C1315-0-0] [lt=10] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1003}, {id:1002
}])
[2025-04-18 10:47:00.025959] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [1028431][T1001_HAService][T1001][YB420A263025-00063268DC1C1314-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}])
[2025-04-18 10:47:00.028606] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2238) [1028207][T1002_IOWorker][T1002][Y0-0000000000000000-0-0] [lt=4] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1003, self:“10.38.48.37:2882”, has_set_deleted:false},
accum_size=1220)
[2025-04-18 10:47:00.029539] INFO pn_print_diag_info (group.c:732) [1002661][pnio1][T0][Y0-0000000000000000-0-0] [lt=12] PNIO client:0x7f5b745ef4a8_"10.38.48.37:2882""10.38.48.37:2882"288_1744275626859889_1, write_queue=0/0, write=1700228/1472932472,
read=1700228/398119305, doing=0, done=1700228, write_time=1723556954, read_time=112629, process_time=178620
[2025-04-18 10:47:00.029556] INFO pn_print_diag_info (group.c:732) [1002661][pnio1][T0][Y0-0000000000000000-0-0] [lt=14] PNIO client:0x7f5b74550048"10.38.48.39:2882"“10.38.48.39:2882"269_1744275626650495_1, write_queue=0/0, write=7843088/6180556272,
read=7843088/1689325806, doing=0, done=7843088, write_time=7906386606, read_time=556654, process_time=918634
[2025-04-18 10:47:00.029562] INFO pn_print_diag_info (group.c:732) [1002661][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO client:0x7f5bb99594a8"10.38.48.38:2882”_“10.38.48.38:2882"250_1744275624842447_1, write_queue=0/0, write=7839201/6170806468, r
ead=7839201/1687349892, doing=0, done=7839201, write_time=7898653710, read_time=561741, process_time=903136
[2025-04-18 10:47:00.029567] INFO pn_print_diag_info (group.c:745) [1002661][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x7f5b745fe048"10.38.48.39:47524"295_1744275627562792, write_queue=0/0, write=12776867/4179799604, read=12776867/852363
6644, doing=0, done=12776867, write_time=12877946689, read_time=2749807, process_time=3375671
[2025-04-18 10:47:00.029577] INFO pn_print_diag_info (group.c:745) [1002661][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO server:0x7f5b745f1468"10.38.48.38:52982"293_1744275627273246, write_queue=0/0, write=12781017/4210860964, read=12781017/853605
0899, doing=0, done=12781017, write_time=12877620115, read_time=2628855, process_time=3185548
[2025-04-18 10:47:00.029581] INFO pn_print_diag_info (group.c:745) [1002661][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x7f5b745f0048"10.38.48.37:48230”_289_1744275626860035, write_queue=0/0, write=1700228/398119305, read=1700228/147293247
2, doing=0, done=1700228, write_time=1724571627, read_time=152834, process_time=218788
[2025-04-18 10:47:00.029589] INFO pn_print_diag_info (group.c:752) [1002661][pnio1][T0][Y0-0000000000000000-0-0] [lt=6] PNIO client_send:17382517/13824295212, client_queue_time=172187079, cnt=3, server_send:27258112/8788779873, server_queue_time=2539601
16, cnt=3
[2025-04-18 10:47:00.041891] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002818][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:47:00.042168] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002812][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:47:00.042214] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:677) [1002823][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=11] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:47:00.042250] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002819][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:47:00.042408] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:478) [1002755][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=15] try inc thread count(*this={name:TSWorker, this:0x7f5be635fb70, min_thread_cnt:1, max_thread_cnt:3,
running_thread_cnt:0, threads_idle_time:1563583471466}, cur_thread_count=2, cnt=-1, new_thread_count=1)
[2025-04-18 10:47:00.042421] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002817][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:47:00.042438] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002815][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:47:00.042517] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002816][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:47:00.042517] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [1002825][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:47:00.042517] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002811][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:47:00.042604] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002820][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:47:00.042719] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002813][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:47:00.042738] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002821][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)

dazhuangge · 2025 年4 月 18 日 11:28

[2025-04-18 10:47:00.842960] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2762) [1002938][T1_PlanCacheEvi][T1][Y0-0000000000000000-0-0] [lt=669] schedule next cache evict task(evict_interval=5000000)
[2025-04-18 10:47:00.844045] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [1028239][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret=“OB_NEED_WAIT”)
[2025-04-18 10:47:00.861915] INFO pn_print_diag_info (group.c:732) [1002659][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO client:0x7f5b745eea58_"10.38.48.38:2882""10.38.48.38:2882"284_1744275626859380_1, write_queue=0/0, write=7837859/6176652921, r
ead=7837858/1686970708, doing=0, done=7837859, write_time=7897711807, read_time=560492, process_time=942635
[2025-04-18 10:47:00.861934] INFO pn_print_diag_info (group.c:732) [1002659][pnio1][T0][Y0-0000000000000000-0-0] [lt=14] PNIO client:0x7f5b744ff468"10.38.48.37:2882"“10.38.48.37:2882"271_1744275626650481_1, write_queue=0/0, write=1700467/1471436397,
read=1700467/398191228, doing=0, done=1700467, write_time=1723928271, read_time=113195, process_time=178817
[2025-04-18 10:47:00.861940] INFO pn_print_diag_info (group.c:732) [1002659][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO client:0x7f5bb9958048"10.38.48.39:2882”_"10.38.48.39:2882"248_1744275624835244_1, write_queue=0/0, write=7844217/6173407532, r
ead=7844217/1688007999, doing=0, done=7844217, write_time=7907359275, read_time=554626, process_time=922579
[2025-04-18 10:47:00.861944] INFO pn_print_diag_info (group.c:745) [1002659][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x7f5b74550a98"10.38.48.37:48210"272_1744275626650730, write_queue=0/0, write=1700467/398191228, read=1700467/147143639
7, doing=0, done=1700467, write_time=1725108072, read_time=152341, process_time=218326
[2025-04-18 10:47:00.861949] INFO pn_print_diag_info (group.c:745) [1002659][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x7f5b744fea58"10.38.48.39:47500"268_1744275626560404, write_queue=0/0, write=12775888/4174942813, read=12775888/854800
2439, doing=0, done=12775888, write_time=12875495591, read_time=2649572, process_time=3206266
[2025-04-18 10:47:00.861952] INFO pn_print_diag_info (group.c:745) [1002659][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x7f5b744fe048"10.38.48.38:52958"267_1744275626467418, write_queue=0/0, write=12780695/4197366416, read=12780695/854716
0652, doing=0, done=12780695, write_time=12875715610, read_time=2640769, process_time=3260488
[2025-04-18 10:47:00.861955] INFO pn_print_diag_info (group.c:752) [1002659][pnio1][T0][Y0-0000000000000000-0-0] [lt=2] PNIO client_send:17382543/13821496850, client_queue_time=174255054, cnt=3, server_send:27257050/8770500457, server_queue_time=2546401
09, cnt=3
[2025-04-18 10:47:00.865760] INFO [STORAGE.TRANS] check_all_readonly_tx_clean_up (ob_ls_tx_service.cpp:370) [1028258][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=4] wait_all_readonly_tx_cleaned_up cleaned up success(ls_id={id:1})
[2025-04-18 10:47:00.865799] INFO [STORAGE.TRANS] check_all_readonly_tx_clean_up (ob_ls_tx_service.cpp:370) [1028258][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=11] wait_all_readonly_tx_cleaned_up cleaned up success(ls_id={id:1001})
[2025-04-18 10:47:00.865807] INFO [STORAGE.TRANS] check_all_readonly_tx_clean_up (ob_ls_tx_service.cpp:370) [1028258][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=6] wait_all_readonly_tx_cleaned_up cleaned up success(ls_id={id:1002})
[2025-04-18 10:47:00.865811] INFO [STORAGE.TRANS] check_all_readonly_tx_clean_up (ob_ls_tx_service.cpp:370) [1028258][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=2] wait_all_readonly_tx_cleaned_up cleaned up success(ls_id={id:1003})
[2025-04-18 10:47:00.879374] INFO [LIB] stat (utility.h:1212) [1028415][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x7f5af854d210, extra_info="
id:1")
[2025-04-18 10:47:00.880390] INFO [LIB] stat (utility.h:1212) [3747197][T1001_L0_G1][T1001][YB420A263026-000632691F1F61E3-0-0] [lt=15] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=4, this=0x7f5af858f6a0,
extra_info=“palf_id:1”)
[2025-04-18 10:47:00.880406] INFO [LIB] stat (utility.h:1212) [3747197][T1001_L0_G1][T1001][YB420A263026-000632691F1F61E3-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=1104, this=0x7f5af858f8d0, extra
info=“palf_id:1”)
[2025-04-18 10:47:00.884977] INFO [DETECT.CACHE.NODE]periodic_tasks (ob_row_hash_holder_map.cpp:373) [1028223][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=3] dump RowHolderMapper nodes info
[DETECT.CACHE][T1002][ 0] total_cnt= 146,800 history_fetch_cnt= 2,863,388 history_revert_cnt= 2,863,388 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 1] total_cnt= 146,800 history_fetch_cnt= 2,867,348 history_revert_cnt= 2,867,348 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 2] total_cnt= 146,800 history_fetch_cnt= 2,866,870 history_revert_cnt= 2,866,870 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 3] total_cnt= 146,800 history_fetch_cnt= 2,868,304 history_revert_cnt= 2,868,304 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 4] total_cnt= 146,800 history_fetch_cnt= 2,866,905 history_revert_cnt= 2,866,905 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 5] total_cnt= 146,800 history_fetch_cnt= 2,865,044 history_revert_cnt= 2,865,044 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 6] total_cnt= 146,800 history_fetch_cnt= 2,865,364 history_revert_cnt= 2,865,364 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 7] total_cnt= 146,800 history_fetch_cnt= 3,266,392 history_revert_cnt= 3,266,392 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 8] total_cnt= 146,800 history_fetch_cnt= 2,866,786 history_revert_cnt= 2,866,786 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 9] total_cnt= 146,800 history_fetch_cnt= 2,868,524 history_revert_cnt= 2,868,524 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 10] total_cnt= 146,800 history_fetch_cnt= 2,997,958 history_revert_cnt= 2,997,958 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 11] total_cnt= 146,800 history_fetch_cnt= 2,865,641 history_revert_cnt= 2,865,641 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 12] total_cnt= 146,800 history_fetch_cnt= 2,869,660 history_revert_cnt= 2,869,660 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 13] total_cnt= 146,800 history_fetch_cnt= 2,865,142 history_revert_cnt= 2,865,142 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 14] total_cnt= 146,800 history_fetch_cnt= 2,870,151 history_revert_cnt= 2,870,151 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 15] total_cnt= 146,800 history_fetch_cnt= 2,871,274 history_revert_cnt= 2,871,274 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[2025-04-18 10:47:00.885012] INFO [DETECT.CACHE.LIST]periodic_tasks (ob_row_hash_holder_map.cpp:379) [1028223][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=12] dump RowHolderMapper lists info
[DETECT.CACHE][T1002][ 0] total_cnt= 146,800 history_fetch_cnt= 2,862,765 history_revert_cnt= 2,862,765 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 1] total_cnt= 146,800 history_fetch_cnt= 2,866,564 history_revert_cnt= 2,866,564 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 2] total_cnt= 146,800 history_fetch_cnt= 2,865,366 history_revert_cnt= 2,865,366 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 3] total_cnt= 146,800 history_fetch_cnt= 2,867,082 history_revert_cnt= 2,867,082 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 4] total_cnt= 146,800 history_fetch_cnt= 2,865,958 history_revert_cnt= 2,865,958 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 5] total_cnt= 146,800 history_fetch_cnt= 2,864,272 history_revert_cnt= 2,864,272 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 6] total_cnt= 146,800 history_fetch_cnt= 2,864,149 history_revert_cnt= 2,864,149 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 7] total_cnt= 146,800 history_fetch_cnt= 3,265,683 history_revert_cnt= 3,265,683 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 8] total_cnt= 146,800 history_fetch_cnt= 2,865,629 history_revert_cnt= 2,865,629 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 9] total_cnt= 146,800 history_fetch_cnt= 2,867,679 history_revert_cnt= 2,867,679 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 10] total_cnt= 146,800 history_fetch_cnt= 2,930,479 history_revert_cnt= 2,930,479 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 11] total_cnt= 146,800 history_fetch_cnt= 2,864,272 history_revert_cnt= 2,864,272 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 12] total_cnt= 146,800 history_fetch_cnt= 2,867,626 history_revert_cnt= 2,867,626 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 13] total_cnt= 146,800 history_fetch_cnt= 2,863,672 history_revert_cnt= 2,863,672 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 14] total_cnt= 146,800 history_fetch_cnt= 2,866,456 history_revert_cnt= 2,866,456 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB

d_cnt:1, running_thread_cnt:1, threads_idle_time:665904468178}, cur_thread_count=1, cnt=1, new_thread_count=1)
[2025-04-18 10:48:00.000305] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1761) [1028203][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self=“10.38.48.37:2882”, committed size=1098
)
[2025-04-18 10:48:00.009123] INFO pn_print_diag_info (group.c:732) [1002665][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO client:0x7f5b745ff4a8_"10.38.48.38:2882""10.38.48.38:2882"300_1744275627649109_1, write_queue=0/0, write=7537083/5936607656, r
ead=7537083/1623597747, doing=0, done=7537083, write_time=7593978667, read_time=544501, process_time=832218
[2025-04-18 10:48:00.009140] INFO pn_print_diag_info (group.c:732) [1002665][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO client:0x7f5b745eca98"10.38.48.37:2882""10.38.48.37:2882"280_1744275626858845_1, write_queue=0/0, write=1635614/1414203747,
read=1635614/383065155, doing=0, done=1635614, write_time=1659042415, read_time=108559, process_time=187424
[2025-04-18 10:48:00.009146] INFO pn_print_diag_info (group.c:732) [1002665][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO client:0x7f5bb99fca98"10.38.48.39:2882""10.38.48.39:2882"256_1744275624843024_1, write_queue=0/0, write=7542720/5949667891, r
ead=7542720/1629283730, doing=0, done=7542720, write_time=7605042515, read_time=543373, process_time=893574
[2025-04-18 10:48:00.009150] INFO pn_print_diag_info (group.c:745) [1002665][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x7f5b745f0a58"10.38.48.38:52972"291_1744275627267507, write_queue=0/0, write=12289496/4033762976, read=12289496/821782
4506, doing=0, done=12289496, write_time=12385014181, read_time=2597838, process_time=3138453
[2025-04-18 10:48:00.009157] INFO pn_print_diag_info (group.c:745) [1002665][pnio1][T0][Y0-0000000000000000-0-0] [lt=6] PNIO server:0x7f5b745ee048"10.38.48.37:48220"283_1744275626859031, write_queue=0/0, write=1635614/383065155, read=1635614/141420374
7, doing=0, done=1635614, write_time=1660086942, read_time=148121, process_time=212840
[2025-04-18 10:48:00.009161] INFO pn_print_diag_info (group.c:745) [1002665][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x7f5b745514a8"10.38.48.39:47512"274_1744275626767818, write_queue=0/0, write=12284509/4014706258, read=12284509/821021
3149, doing=0, done=12284509, write_time=12382203687, read_time=2547874, process_time=3080274
[2025-04-18 10:48:00.009167] INFO pn_print_diag_info (group.c:752) [1002665][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO client_send:16715417/13300479294, client_queue_time=166749012, cnt=3, server_send:26209619/8431534389, server_queue_time=2444799
72, cnt=3
[2025-04-18 10:48:00.011510] INFO eloop_run (eloop.c:145) [1002656][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO [ratelimit] time: 1744944480011509, bytes: 219830077631, bw: 0.213287 MB/s, add_ts: 1007616, add_bytes: 225351
[2025-04-18 10:48:00.011530] INFO pn_print_diag_info (group.c:732) [1002656][pnio1][T0][Y0-0000000000000000-0-0] [lt=15] PNIO client:0x7f5b6e2f94e8"10.38.48.39:2882""10.38.48.39:2882"301_1744275627649115_1, write_queue=0/0, write=7837677/6182898938,
read=7837677/1685387032, doing=0, done=7837677, write_time=7904630412, read_time=553468, process_time=851114
[2025-04-18 10:48:00.011537] INFO pn_print_diag_info (group.c:732) [1002656][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO client:0x7f5b755b6a58"10.38.48.38:2882""10.38.48.38:2882"281_1744275626858852_1, write_queue=0/0, write=7844964/6183920723, r
ead=7844964/1687685415, doing=0, done=7844964, write_time=7904125675, read_time=561162, process_time=923694
[2025-04-18 10:48:00.011543] INFO pn_print_diag_info (group.c:732) [1002656][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO client:0x7f5bb98ee048"10.38.48.37:2882""10.38.48.37:2882"243_1744275624831513_1, write_queue=0/0, write=1701393/1469448513, r
ead=1701393/398474497, doing=0, done=1701393, write_time=1724884854, read_time=111642, process_time=196775
[2025-04-18 10:48:00.011550] INFO pn_print_diag_info (group.c:745) [1002656][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO server:0x7f5bb98eea98"10.38.48.37:48178"244_1744275624834439, write_queue=0/0, write=1701393/398474497, read=1701393/146944851
3, doing=0, done=1701393, write_time=1725662199, read_time=152613, process_time=218184
[2025-04-18 10:48:00.011555] INFO pn_print_diag_info (group.c:745) [1002656][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO server:0x7f5bb9805218"10.38.48.39:47470"239_1744275624758774, write_queue=0/0, write=12777238/4179634131, read=12777238/855134
8145, doing=0, done=12777238, write_time=12879136546, read_time=2730629, process_time=3303859
[2025-04-18 10:48:00.011562] INFO pn_print_diag_info (group.c:745) [1002656][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO server:0x7f5bb9804808"10.38.48.38:52928"234_1744275624465709, write_queue=0/0, write=12781964/4194914426, read=12781964/854545
9758, doing=0, done=12781964, write_time=12878181940, read_time=2648206, process_time=3273516
[2025-04-18 10:48:00.011565] INFO pn_print_diag_info (group.c:752) [1002656][pnio1][T0][Y0-0000000000000000-0-0] [lt=3] PNIO client_send:17384034/13836268174, client_queue_time=176652205, cnt=3, server_send:27260595/8773023054, server_queue_time=2583472
68, cnt=3
[2025-04-18 10:48:00.012428] INFO [LIB] stat (utility.h:1212) [1028415][T1001_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x7f5af854d210, extra_info=“i
d:1”)
[2025-04-18 10:48:00.013425] INFO [LIB] stat (utility.h:1212) [3747950][T1001_L0_G1][T1001][YB420A263027-00063269484C2B84-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=4, this=0x7f5af858f6a0,
extra_info=“palf_id:1”)
[2025-04-18 10:48:00.013442] INFO [LIB] stat (utility.h:1212) [3747950][T1001_L0_G1][T1001][YB420A263027-00063269484C2B84-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=1217, this=0x7f5af858f8d0, extra
info=“palf_id:1”)
[2025-04-18 10:48:00.015455] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [1028257][T1002_HAService][T1002][YB420A263025-00063268DF8C1314-0-0] [lt=11] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}, {id:1003}, {id:1002
}])
[2025-04-18 10:48:00.029242] INFO [SERVER] get_srs_cnt (ob_srs_importer.cpp:70) [1028377][T1001_TntShared][T1001][Y0-0000000000000000-0-0] [lt=11] old srs rows(srs_cnt=1)
[2025-04-18 10:48:00.031197] INFO [STORAGE.TRANS] check_all_readonly_tx_clean_up (ob_ls_tx_service.cpp:370) [1028432][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=10] wait_all_readonly_tx_cleaned_up cleaned up success(ls_id={id:1})
[2025-04-18 10:48:00.031441] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [1028431][T1001_HAService][T1001][YB420A263025-00063268DC1C1314-0-0] [lt=14] start do ls ha handler(ls_id_array_=[{id:1}])
[2025-04-18 10:48:00.032255] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [1003064][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self=“10.38.48.37:2882”, role=“LEADER”, total_group_log_c
nt=10, avg_log_batch_cnt=1, total_group_log_size=3563, avg_group_log_size=356)
[2025-04-18 10:48:00.032270] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [1003064][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:“10.38.48.37:2882”, has_set_deleted:false}, append size=356
3)
[2025-04-18 10:48:00.032279] INFO [STORAGE.TRANS] check_all_readonly_tx_clean_up (ob_ls_tx_service.cpp:370) [1003064][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=6] wait_all_readonly_tx_cleaned_up cleaned up success(ls_id={id:1})
[2025-04-18 10:48:00.033262] INFO [PALF] sliding_cb (log_sliding_window.cpp:2280) [3748241][T1_L0_G1][T1][YB420A263027-00063269254D638E-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self=“10.38.48.37:2882”, total_slide_log_cnt=10, avg_log_gen_to_free
ze_time=3, avg_log_gen_to_submit_time=30, avg_log_submit_to_flush_time=805, avg_log_submit_to_slide_time=1003)
[2025-04-18 10:48:00.033286] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:478) [1002755][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=17] try inc thread count(*this={name:TSWorker, this:0x7f5be635fb70, min_thread_cnt:1, max_thread_cnt:3,
running_thread_cnt:0, threads_idle_time:1563729961051}, cur_thread_count=2, cnt=-1, new_thread_count=1)
[2025-04-18 10:48:00.042205] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002821][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:00.042219] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002819][BatchIO][T0][Y0-0000000000000000-0-0] [lt=6] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:00.042302] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002812][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:00.042347] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002817][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:00.042508] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:677) [1002823][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:00.042601] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002814][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:00.042638] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002820][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:00.042709] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002816][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:00.042819] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [1002825][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=12] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:00.042914] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002818][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)

dazhuangge · 2025 年4 月 18 日 11:30

741dee60, block_ptr_.control_ptr:null, block_ptr_.data_ptr:null}, retry_wait_event_no:0, retry_plan_line_id:-1}})
[2025-04-18 10:48:00.660110] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1002620, task_cnt=298, avg_time=1067)
[2025-04-18 10:48:00.660127] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=13] TIMER THREAD STAT: (thread_id=1002621, task_cnt=300, avg_time=387)
[2025-04-18 10:48:00.660131] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1003005, task_cnt=2998, avg_time=0)
[2025-04-18 10:48:00.660135] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1003015, task_cnt=1996, avg_time=9)
[2025-04-18 10:48:00.660142] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=1002580, task_cnt=171, avg_time=833)
[2025-04-18 10:48:00.660145] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1002752, task_cnt=60, avg_time=1)
[2025-04-18 10:48:00.660148] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1002739, task_cnt=60, avg_time=18)
[2025-04-18 10:48:00.660157] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=8] TIMER THREAD STAT: (thread_id=1002939, task_cnt=60, avg_time=1)
[2025-04-18 10:48:00.660161] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1003018, task_cnt=120, avg_time=3)
[2025-04-18 10:48:00.660164] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1003004, task_cnt=94, avg_time=1114)
[2025-04-18 10:48:00.660168] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1003033, task_cnt=60, avg_time=1389)
[2025-04-18 10:48:00.660176] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=7] TIMER THREAD STAT: (thread_id=1003067, task_cnt=248, avg_time=2)
[2025-04-18 10:48:00.660179] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1002582, task_cnt=20, avg_time=212)
[2025-04-18 10:48:00.660182] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1002839, task_cnt=20, avg_time=15)
[2025-04-18 10:48:00.660185] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1003050, task_cnt=30, avg_time=40)
[2025-04-18 10:48:00.660188] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1002995, task_cnt=26, avg_time=5400)
[2025-04-18 10:48:00.660192] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1002583, task_cnt=12, avg_time=23)
[2025-04-18 10:48:00.660199] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=5] TIMER THREAD STAT: (thread_id=1002924, task_cnt=12, avg_time=1)
[2025-04-18 10:48:00.660202] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1002938, task_cnt=12, avg_time=4652)
[2025-04-18 10:48:00.660205] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1003000, task_cnt=12, avg_time=2729)
[2025-04-18 10:48:00.660212] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=1003048, task_cnt=12, avg_time=1141)
[2025-04-18 10:48:00.660215] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1003049, task_cnt=12, avg_time=392)
[2025-04-18 10:48:00.660218] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1003051, task_cnt=12, avg_time=42)
[2025-04-18 10:48:00.660222] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1003052, task_cnt=12, avg_time=36)
[2025-04-18 10:48:00.660230] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=1003021, task_cnt=6, avg_time=91)
[2025-04-18 10:48:00.660233] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1002703, task_cnt=0, avg_time=0)
[2025-04-18 10:48:00.660236] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1002678, task_cnt=30, avg_time=1)
[2025-04-18 10:48:00.660240] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=1002585, task_cnt=0, avg_time=0)
[2025-04-18 10:48:00.660248] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=1003213, task_cnt=36, avg_time=1304)
[2025-04-18 10:48:00.660250] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1003206, task_cnt=60, avg_time=10)
[2025-04-18 10:48:00.660254] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1003212, task_cnt=60, avg_time=2120)
[2025-04-18 10:48:00.660257] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1002730, task_cnt=60, avg_time=257)
[2025-04-18 10:48:00.660262] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=1003129, task_cnt=30, avg_time=225)
[2025-04-18 10:48:00.660269] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=1002717, task_cnt=36, avg_time=2)
[2025-04-18 10:48:00.660272] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1002729, task_cnt=12, avg_time=2777)
[2025-04-18 10:48:00.660275] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1002742, task_cnt=6, avg_time=22)
[2025-04-18 10:48:00.660279] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=4] TIMER THREAD STAT: (thread_id=1002753, task_cnt=6, avg_time=105)
[2025-04-18 10:48:00.660284] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1002731, task_cnt=6, avg_time=151)
[2025-04-18 10:48:00.660291] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=1003062, task_cnt=2, avg_time=23)
[2025-04-18 10:48:00.660294] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1003202, task_cnt=2, avg_time=102)
[2025-04-18 10:48:00.660297] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1002747, task_cnt=1, avg_time=2)
[2025-04-18 10:48:00.660300] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1002708, task_cnt=1, avg_time=18)
[2025-04-18 10:48:00.660308] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=1003201, task_cnt=1, avg_time=10262)
[2025-04-18 10:48:00.660311] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1003059, task_cnt=1, avg_time=12832)
[2025-04-18 10:48:00.660314] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1002740, task_cnt=1, avg_time=54197)
[2025-04-18 10:48:00.660317] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1003034, task_cnt=0, avg_time=0)
[2025-04-18 10:48:00.660326] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=7] TIMER THREAD STAT: (thread_id=1003207, task_cnt=0, avg_time=0)
[2025-04-18 10:48:00.660328] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1028196, task_cnt=2997, avg_time=0)
[2025-04-18 10:48:00.660332] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1028195, task_cnt=124, avg_time=933)
[2025-04-18 10:48:00.660340] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=1028209, task_cnt=1995, avg_time=10)
[2025-04-18 10:48:00.660342] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1028377, task_cnt=92, avg_time=1072)
[2025-04-18 10:48:00.660346] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1028378, task_cnt=2998, avg_time=0)
[2025-04-18 10:48:00.660349] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1028388, task_cnt=1995, avg_time=8)
[2025-04-18 10:48:00.660357] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=6] TIMER THREAD STAT: (thread_id=1028212, task_cnt=120, avg_time=57)
[2025-04-18 10:48:00.660360] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=3] TIMER THREAD STAT: (thread_id=1028391, task_cnt=120, avg_time=3)
[2025-04-18 10:48:00.660364] INFO dump (ob_timer_monitor.cpp:200) [1002753][ObTimer][T0][Y0-0000000000000000-0-0] [lt=2] TIMER THREAD STAT: (thread_id=1028132, task_cnt=60, avg_time=1)
2025-04-18 10:48:00.940348] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:998) [1002837][sql_nio8][T0][Y0-0000000000000000-0-0] [lt=4] can close safely, do destroy(*s={this:0x7f5b7b6c6db0, session_id:3221580695, trace_id:Y0-00000000000
00000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, reader:{fd:452}, err:5, last_decode_time:1744944480933081, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:446, pending_flag:1, may_handling_flag:false, ha
ndler_close_flag:false})
[2025-04-18 10:48:00.940390] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:160) [1002837][sql_nio8][T0][Y0-0000000000000000-0-0] [lt=16] unlock session of tenant(conn.sessid_=3221580695, proxy_sessid=0, conn.tenant_id_=1)
[2025-04-18 10:48:00.940405] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:249) [1002837][sql_nio8][T0][Y0-0000000000000000-0-0] [lt=11] connection close(sessid=3221580695, proxy_sessid=0, tenant_id=1, server_id=1, from_proxy=false, from_java_clien
t=false, c/s protocol=“OB_MYSQL_CS_TYPE”, is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB420A263025-0006326A74DC1ADC-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:8, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pk
t_name:“PKT_OKP”, obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:336, rec_:336, seq_:0}, pkt_name:“COM_QUERY”, obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_na
me:“PKT_RESHEAD”, obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:8, seq_:10}, pkt_name:“PKT_FIELD”, obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:11}, pkt_name:“PKT_EOF”, obp_mysql_header_.is_s
end_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:14}, pkt_name:“PKT_ROW”, obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:15}, pkt_name:“PKT_EOF”, obp_mysql_header_.is_send_:1}, pkt_rec[7]:{receive:obp_mysql_hea
der_{len_:1, rec_:1, seq_:0}, pkt_name:“COM_QUIT”, obp_mysql_header_.type_:1, obp_mysql_header_.is_send_:0}], disconnect_state=3)
[2025-04-18 10:48:00.940448] INFO [SERVER] run (obmp_disconnect.cpp:84) [1003101][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=2] free session successfully(sessid=3221580695)
[2025-04-18 10:48:00.940463] INFO [SERVER] run (obmp_disconnect.cpp:91) [1003101][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=12] mark session id unused(sessid=3221580695)
[2025-04-18 10:48:00.944279] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=10][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944296] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=13][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, m
eta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944319] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=8][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944327] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=6][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944340] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=4][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944348] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=6][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944354] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944357] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944363] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944371] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=6][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944376] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=2][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944384] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=7][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944390] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944397] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=5][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944403] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944412] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=7][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944418] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944425] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=6][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944432] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944440] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=7][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944447] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=4][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944450] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=2][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944461] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=7][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944464] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=3][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944471] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944478] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=6][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944484] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944492] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=6][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944506] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=3][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944514] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=6][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944521] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=4][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.944527] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=5][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.944538] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=7][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.945210] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=8][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11)
[2025-04-18 10:48:00.945218] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11921) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=6][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, me
ta={type:“VARCHAR”, collation:“utf8mb4_general_ci”, coercibility:“INVALID”})
[2025-04-18 10:48:00.945224] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:11908) [1003402][T1_L0_G10000][T1][YB420A263025-0006326A722C1DFB-0-0] [lt=2][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth]
[2025-04-18 10:48:00.946102] WDIAG [SERVER] process_curr_tenant (ob_all_virtual_sys_stat.cpp:311) [1003401][T1_L0_G10000][T1001][YB420A263025-0006326A125E0FB3-0-0] [lt=10][errcode=-4008] get diag info fail(ret=-4008, tenant_id_=1001)
[2025-04-18 10:48:00.946124] WDIAG [SERVER] process_curr_tenant (ob_all_virtual_sys_stat.cpp:311) [1003401][T1_L0_G10000][T1002][YB420A263025-0006326A125E0FB3-0-0] [lt=16][errcode=-4008] get diag info fail(ret=-4008, tenant_id_=1002)
[2025-04-18 10:48:00.946129] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:291) [1003401][T1_L0_G10000][T1][YB420A263025-0006326A125E0FB3-0-0] [lt=4][errcode=-4008] execute fail(ret=-4008)
[2025-04-18 10:48:00.946222] WDIAG [SERVER] process_curr_tenant (ob_all_virtual_sys_stat.cpp:311) [1003401][T1_L0_G10000][T1][YB420A263025-0006326A125E0FB3-0-0] [lt=7][errcode=-4008] get diag info fail(ret=-4008, tenant_id_=1)
[2025-04-18 10:48:00.946231] WDIAG [SERVER] process_curr_tenant (ob_all_virtual_sys_stat.cpp:311) [1003401][T1_L0_G10000][T1001][YB420A263025-0006326A125E0FB3-0-0] [lt=6][errcode=-4008] get diag info fail(ret=-4008, tenant_id_=1001)
[2025-04-18 10:48:00.949433] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:291) [1003401][T1_L0_G10000][T1][YB420A263025-0006326A125E0FB3-0-0] [lt=2][errcode=-4008] execute fail(ret=-4008)
[2025-04-18 10:48:00.952997] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_table_mgr.cpp:80) [1003607][T1_PX_G10000][T1][YB420A263025-00063269FF7EA433-0-0] [lt=6][errcode=-4008] fail to execute(ret=-4008)
[2025-04-18 10:48:00.953038] WDIAG [SQL.ENG] inner_get_next_row (ob_px_dist_transmit_op.cpp:73) [1003607][T1_PX_G10000][T1][YB420A263025-00063269FF7EA433-0-0] [lt=18][errcode=-4008] fail to exec ObPxTransmitOp::inner_get_next_row()(ret=-4008)
[2025-04-18 10:48:00.954739] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1021) [1002834][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=3][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth]
[2025-04-18 10:48:00.954764] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:867) [1003403][T1_L0_G10000][T1][YB420A263025-00063269FF7EA434-0-0] [lt=6][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth]
[2025-04-18 10:48:00.954771] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:274) [1002834][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=20] kill and revert session(conn.sessid_=3221580557, proxy_sessid=0, server_id=1, ret=0)
[2025-04-18 10:48:00.954784] INFO [SERVER] process (obmp_quit.h:60) [1003403][T1_L0_G10000][T1][YB420A263025-00063269FF7EA434-0-0] [lt=15] quit
[2025-04-18 10:48:00.954790] WDIAG [RPC.OBMYSQL] async_write_data (ob_sql_sock_session.cpp:163) [1003403][T1_L0_G10000][T1][YB420A263025-00063269FF7EA434-0-0] [lt=5][errcode=-4009] sock has error(ret=-4009)
[2025-04-18 10:48:00.954800] WDIAG [SERVER] flush_buffer (obmp_packet_sender.cpp:1069) [1003403][T1_L0_G10000][T1][YB420A263025-00063269FF7EA434-0-0] [lt=8][errcode=-4009] write response fail(ret=-4009)
[2025-04-18 10:48:00.954803] WDIAG [SERVER] response (obmp_base.cpp:87) [1003403][T1_L0_G10000][T1][YB420A263025-00063269FF7EA434-0-0] [lt=3][errcode=-4009] failed to flush_buffer(ret=-4009)
[2025-04-18 10:48:00.954812] WDIAG [RPC.FRAME] run (ob_sql_processor.cpp:48) [1003403][T1_L0_G10000][T1][YB420A263025-00063269FF7EA434-0-0] [lt=7][errcode=-4009] response rpc result fail(ret=-4009)
[2025-04-18 10:48:00.954940] INFO [DETECT.CACHE.NODE]periodic_tasks (ob_row_hash_holder_map.cpp:373) [1028223][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=5] dump RowHolderMapper nodes info
[DETECT.CACHE][T1002][ 0] total_cnt= 146,800 history_fetch_cnt= 2,863,389 history_revert_cnt= 2,863,389 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 1] total_cnt= 146,800 history_fetch_cnt= 2,867,348 history_revert_cnt= 2,867,348 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 2] total_cnt= 146,800 history_fetch_cnt= 2,866,870 history_revert_cnt= 2,866,870 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 3] total_cnt= 146,800 history_fetch_cnt= 2,868,305 history_revert_cnt= 2,868,305 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 4] total_cnt= 146,800 history_fetch_cnt= 2,866,905 history_revert_cnt= 2,866,905 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 5] total_cnt= 146,800 history_fetch_cnt= 2,865,044 history_revert_cnt= 2,865,044 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 6] total_cnt= 146,800 history_fetch_cnt= 2,865,364 history_revert_cnt= 2,865,364 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 7] total_cnt= 146,800 history_fetch_cnt= 3,266,428 history_revert_cnt= 3,266,428 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 8] total_cnt= 146,800 history_fetch_cnt= 2,866,786 history_revert_cnt= 2,866,786 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 9] total_cnt= 146,800 history_fetch_cnt= 2,868,524 history_revert_cnt= 2,868,524 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 10] total_cnt= 146,800 history_fetch_cnt= 2,997,970 history_revert_cnt= 2,997,970 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 11] total_cnt= 146,800 history_fetch_cnt= 2,865,641 history_revert_cnt= 2,865,641 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 12] total_cnt= 146,800 history_fetch_cnt= 2,869,660 history_revert_cnt= 2,869,660 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 13] total_cnt= 146,800 history_fetch_cnt= 2,865,142 history_revert_cnt= 2,865,142 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 14] total_cnt= 146,800 history_fetch_cnt= 2,870,151 history_revert_cnt= 2,870,151 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[DETECT.CACHE][T1002][ 15] total_cnt= 146,800 history_fetch_cnt= 2,871,274 history_revert_cnt= 2,871,274 free_cnt= 146,800, total_size=4.48_MB, free_size=4.48_MB
[2025-04-18 10:48:00.954975] INFO [DETECT.CACHE.LIST]periodic_tasks (ob_row_hash_holder_map.cpp:379) [1028223][T1002_LockWaitM][T1002][Y0-0000000000000000-0-0] [lt=12] dump RowHolderMapper lists info

辞霜 · 2025 年4 月 18 日 11:31

麻烦弄个附件发一份完整的observer日志。这样不方便查看

dazhuangge · 2025 年4 月 18 日 11:32

[2025-04-18 10:48:00.974391] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [1003401][T1_L0_G10000][T1][YB420A263025-0006326A74DC1ADF-0-0] [lt=6] add_update_task succeed(ret=0, ret=“OB_SUCCESS”, task={tenant_id:1, table
id:12070, add_timestamp:1744944480974383})
[2025-04-18 10:48:00.974416] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [1003401][T1_L0_G10000][T1][YB420A263025-0006326A74DC1ADF-0-0] [lt=12] add_update_task succeed(ret=0, ret=“OB_SUCCESS”, task={tenant_id:1, tabl
e_id:12241, add_timestamp:1744944480974414})
[2025-04-18 10:48:00.974429] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [1003401][T1_L0_G10000][T1][YB420A263025-0006326A74DC1ADF-0-0] [lt=6] add_update_task succeed(ret=0, ret=“OB_SUCCESS”, task={tenant_id:1, table
id:12076, add_timestamp:1744944480974428})
[2025-04-18 10:48:00.974420] INFO [SHARE.LOCATION] renew_vtable_location (ob_vtable_location_service.cpp:207) [1002737][VTblLocAsyncUp0][T0][YB420A263025-0006326A5D2D4F61-0-0] [lt=13] renew vtable location success(ret=0, ret=“OB_SUCCESS”, tenant_id=1,
table_id=12070, locations=[{table_id:12070, partition_id:12070, partition_cnt:0, replica_locations:[{server:“10.38.48.37:2882”, role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent:100}}], renew_time:1744944480974406, sql_renew_
time:1744944480974406, is_mark_fail:false}])
[2025-04-18 10:48:00.974440] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [1003401][T1_L0_G10000][T1][YB420A263025-0006326A74DC1ADF-0-0] [lt=6] add_update_task succeed(ret=0, ret=“OB_SUCCESS”, task={tenant_id:1, table
id:12082, add_timestamp:1744944480974439})
[2025-04-18 10:48:00.974451] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [1003401][T1_L0_G10000][T1][YB420A263025-0006326A74DC1ADF-0-0] [lt=4] add_update_task succeed(ret=0, ret=“OB_SUCCESS”, task={tenant_id:1, table
id:12240, add_timestamp:1744944480974450})
[2025-04-18 10:48:00.974445] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [1002737][VTblLocAsyncUp0][T0][YB420A263025-0006326A5D2D4F61-0-0] [lt=20] success to process renew task(task={tenant_id:1, table_id:12070, add_ti
mestamp:1744944480974383}, locations=[{table_id:12070, partition_id:12070, partition_cnt:0, replica_locations:[{server:“10.38.48.37:2882”, role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent:100}}], renew_time:1744944480974406,
sql_renew_time:1744944480974406, is_mark_fail:false}])
[2025-04-18 10:48:00.974464] INFO [SHARE.LOCATION] renew_vtable_location (ob_vtable_location_service.cpp:207) [1002737][VTblLocAsyncUp0][T0][YB420A263025-0006326A5D2D4F62-0-0] [lt=10] renew vtable location success(ret=0, ret=“OB_SUCCESS”, tenant_id=1,
table_id=12241, locations=[{table_id:12241, partition_id:12241, partition_cnt:0, replica_locations:[{server:“10.38.48.37:2882”, role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1744944480974461, sql_renew_
time:1744944480974461, is_mark_fail:false}])
[2025-04-18 10:48:00.974474] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [1002737][VTblLocAsyncUp0][T0][YB420A263025-0006326A5D2D4F62-0-0] [lt=8] success to process renew task(task={tenant_id:1, table_id:12241, add_tim
estamp:1744944480974414}, locations=[{table_id:12241, partition_id:12241, partition_cnt:0, replica_locations:[{server:“10.38.48.37:2882”, role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1744944480974461,
sql_renew_time:1744944480974461, is_mark_fail:false}])
[2025-04-18 10:48:00.974488] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [1002737][VTblLocAsyncUp0][T0][YB420A263025-0006326A5D2D4F63-0-0] [lt=7] renew vtable location success(ret=0, ret=“OB_SUCCESS”, tenant_id=1, t
able_id=12076, locations=[{table_id:12076, partition_id:12076, partition_cnt:0, replica_locations:[{server:“10.38.48.37:2882”, role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1744944480974486, sql_renew_t
ime:1744944480974486, is_mark_fail:false}])
[2025-04-18 10:48:00.974497] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [1002737][VTblLocAsyncUp0][T0][YB420A263025-0006326A5D2D4F63-0-0] [lt=7] success to process renew task(task={tenant_id:1, table_id:12076, add_tim
estamp:1744944480974428}, locations=[{table_id:12076, partition_id:12076, partition_cnt:0, replica_locations:[{server:“10.38.48.37:2882”, role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1744944480974486,
sql_renew_time:1744944480974486, is_mark_fail:false}])
[2025-04-18 10:48:02.034066] INFO [STORAGE.TRANS] check_all_readonly_tx_clean_up (ob_ls_tx_service.cpp:370) [1003064][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=39] wait_all_readonly_tx_cleaned_up cleaned up success(ls_id={id:1})
[2025-04-18 10:48:02.034833] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2238) [1003013][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:“10.38.48.37:2882”, has_set_deleted:false}, accum_si
ze=4227)
[2025-04-18 10:48:02.035161] INFO [LIB] stat (utility.h:1212) [3748610][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=0] [APPLY STAT CB APPEND COST TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=42, this=0x7f5b807fc280, extra_info="")
[2025-04-18 10:48:02.035182] INFO [LIB] stat (utility.h:1212) [3748610][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=17] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=1031, this=0x7f5b807fc4b0, extra_info="")
[2025-04-18 10:48:02.035191] INFO [LIB] stat (utility.h:1212) [3748610][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=7] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=0, this=0x7f5b807fc6e0, extra_info="")
[2025-04-18 10:48:02.035210] INFO [LIB] stat (utility.h:1212) [3748610][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=7] [APPLY STAT CB EXECUTE TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=-532, this=0x7f5b807fc910, extra_info="")
[2025-04-18 10:48:02.035216] INFO [LIB] stat (utility.h:1212) [3748610][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB TOTAL TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=541, this=0x7f5b807fcb40, extra_info="")
[2025-04-18 10:48:02.041645] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002815][BatchIO][T0][Y0-0000000000000000-0-0] [lt=7] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:02.042298] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002813][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:02.042404] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002817][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:02.042431] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002811][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:02.042518] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002816][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:02.042521] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002819][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:02.042552] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [1002825][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=7] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:02.042563] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:677) [1002823][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=12] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:02.042580] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002812][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:02.042579] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002818][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:02.042662] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002814][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:02.042712] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002820][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:02.042795] INFO [PALF] read (log_cache.cpp:100) [1029175][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=16] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=1098, hit_cnt=9, read_cnt=9, hit rate=1.000000000000000000e+00)
[2025-04-18 10:48:02.042872] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002821][BatchIO][T0][Y0-0000000000000000-0-0] [lt=12] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:02.051312] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:239) [1028292][T1001_Occam][T1001][YB420A263025-0006326A68EC5BDD-0-0] [lt=10] for each ls(succ_num=1, ret=0, ret=“OB_SUCCESS”)
[2025-04-18 10:48:02.053653] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:239) [1028103][T1002_Occam][T1002][YB420A263025-0006326A68FC5BDD-0-0] [lt=10] for each ls(succ_num=4, ret=0, ret=“OB_SUCCESS”)
[2025-04-18 10:48:02.056818] INFO [COMMON] replace_map (ob_kv_storecache.cpp:746) [1002621][KVCacheRep][T0][Y0-0000000000000000-0-0] [lt=14] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=5)
[2025-04-18 10:48:02.066169] INFO [SERVER] get_srs_cnt (ob_srs_importer.cpp:70) [1028195][T1002_TntShared][T1002][Y0-0000000000000000-0-0] [lt=12] old srs rows(srs_cnt=1)
[2025-04-18 10:48:02.069761] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:346) [1028244][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=7] ====== check clog disk timer task ======
[2025-04-18 10:48:02.069781] INFO [PALF] get_disk_usage (palf_env_impl.cpp:912) [1028244][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=12] get_disk_usage(ret=0, capacity(MB):=22118, used(MB):=17649)
[2025-04-18 10:48:02.069802] INFO [STORAGE] cannot_recycle_log_over_threshold_ (ob_checkpoint_service.cpp:260) [1028244][T1002_CKClogDis][T1002][Y0-0000000000000000-0-0] [lt=4] cannot_recycle_log_size statistics(cannot_recycle_log_size=118992797, thresh
old=6957847019, need_update_checkpoint_scn=false)
[2025-04-18 10:48:02.076180] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [1003047][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1, self=“10.38.48.37:2882”, role=“LEADER”, total_group_log_
cnt=10, avg_log_batch_cnt=1, total_group_log_size=3556, avg_group_log_size=355)
[2025-04-18 10:48:02.076207] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [1003047][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:“10.38.48.37:2882”, has_set_deleted:false}, append size=355
5)
[2025-04-18 10:48:02.077224] INFO [PALF] sliding_cb (log_sliding_window.cpp:2280) [3748241][T1_L0_G1][T1][YB420A263027-00063269254D63A5-0-0] [lt=3] [PALF STAT LOG TASK TIME](palf_id=1, self=“10.38.48.37:2882”, total_slide_log_cnt=10, avg_log_gen_to_free
ze_time=3, avg_log_gen_to_submit_time=33, avg_log_submit_to_flush_time=810, avg_log_submit_to_slide_time=1003)
[2025-04-18 10:48:02.077289] INFO [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1692) [1003047][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=7] execute write sql(ret=0, tenant_id=1, affected_rows=1, sql= update all_weak_read_servi
ce set min_version=1744944481875530000, max_version=1744944481875530000 where tenant_id = 1 and level_id = 0 and level_value = ‘’ and min_version = 1744944481674893000 and max_version = 1744944481674893000 , get_session().get_sessid()=3221580375)
[2025-04-18 10:48:02.077402] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:833) [1003047][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret=“O
B_SUCCESS”, tenant_id=1, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=2, start_service_tstamp=1744275639272643, error_count_for_change_leader=0, last_error_
tstamp_for_change_leader_=0)
[2025-04-18 10:48:02.077928] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [1028239][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret=“OB_NEED_WAIT”)
[2025-04-18 10:48:02.100624] INFO [STORAGE.TRANS] check_all_readonly_tx_clean_up (ob_ls_tx_service.cpp:370) [1028258][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=3] wait_all_readonly_tx_cleaned_up cleaned up success(ls_id={id:1})
[2025-04-18 10:48:02.100665] INFO [STORAGE.TRANS] check_all_readonly_tx_clean_up (ob_ls_tx_service.cpp:370) [1028258][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=10] wait_all_readonly_tx_cleaned_up cleaned up success(ls_id={id:1001})
[2025-04-18 10:48:02.100674] INFO [STORAGE.TRANS] check_all_readonly_tx_clean_up (ob_ls_tx_service.cpp:370) [1028258][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=5] wait_all_readonly_tx_cleaned_up cleaned up success(ls_id={id:1002})
[2025-04-18 10:48:02.100684] INFO [STORAGE.TRANS] check_all_readonly_tx_clean_up (ob_ls_tx_service.cpp:370) [1028258][T1002_TxLoopWor][T1002][Y0-0000000000000000-0-0] [lt=8] wait_all_readonly_tx_cleaned_up cleaned up success(ls_id={id:1003})
[2025-04-18 10:48:02.100687] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2361) [3748024][T1_L0_G19][T1][YB420A263027-00063268D3BC0ABE-0-0] [lt=9] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:“10.38.48.39:2882”, ls_array_:
[{id:1}]})
[2025-04-18 10:48:02.100710] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1565) [3748024][T1_L0_G19][T1][YB420A263027-00063268D3BC0ABE-0-0] [lt=16] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:“10.38.48.37:2882”, timestamp
:1, flag:0}{server:“10.38.48.38:2882”, timestamp:1, flag:0}{server:“10.38.48.39:2882”, timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr=“10.38.48.39:2882”, this={role:1, proposal_id:2, palf_env_:0x7f5b925ce030, is_in_stop_state_
:false, is_inited_:true, id_:1})
[2025-04-18 10:48:02.100753] INFO [COMMON] process (ob_rpc_processor_simple.cpp:2361) [3748024][T1_L0_G19][T1][YB420A263026-00063268D3BA4F37-0-0] [lt=16] handle ObQueryLSIsValidMember requeset(arg_={tenant_id_:1, self_addr_:“10.38.48.38:2882”, ls_array_
:[{id:1}]})
[2025-04-18 10:48:02.100763] INFO [CLOG] get_member_gc_stat (ob_log_handler.cpp:1565) [3748024][T1_L0_G19][T1][YB420A263026-00063268D3BA4F37-0-0] [lt=7] get_member_gc_stat(is_valid_member=true, stat=1, member_list=3{server:“10.38.48.37:2882”, timestamp:
1, flag:0}{server:“10.38.48.38:2882”, timestamp:1, flag:0}{server:“10.38.48.39:2882”, timestamp:1, flag:0}, learner_list={learner_num:0, learner_array:[]}, addr=“10.38.48.38:2882”, this={role:1, proposal_id:2, palf_env_:0x7f5b925ce030, is_in_stop_state_:
false, is_inited_:true, id_:1})
[2025-04-18 10:48:02.101481] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:478) [1028203][T1002_LogIOCb0][T1002][Y0-0000000000000000-0-0] [lt=9] try inc thread count(*this={name:LogIOCb, this:0x7f5b1feec410, min_thread_cnt:1, max_thread
cnt:1, running_thread_cnt:1, threads_idle_time:665906568012}, cur_thread_count=1, cnt=1, new_thread_count=1)
[2025-04-18 10:48:02.112281] INFO [COMMON] print_io_status (ob_io_manager.cpp:1436) [1002589][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO STATUS SYS](tenant_id=1, io_status_sys_log=sys_group_name: UNKNOWN, mode: write, size: 4505.60, iops:
9.99, rt: 750.00; )
[2025-04-18 10:48:02.112304] INFO [COMMON] print_io_status (ob_io_manager.cpp:1452) [1002589][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS CONFIG](tenant_id=1, ref_cnt=2, io_config={group_num:1, memory_limit_:5368709120, callback_thread_
count_:8, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:4}, enable_io_tracer:false, group_configs:[group_id = 10000:{deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, other_groups:{deleted
:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=4333344, pre_allocated_count=2, callback_queues=0)

dazhuangge · 2025 年4 月 18 日 11:32

安全管控最近没法上传文件

dazhuangge · 2025 年4 月 18 日 12:27

[2025-04-18 10:48:03.042522] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002814][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:03.042543] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002816][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:03.042549] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002819][BatchIO][T0][Y0-0000000000000000-0-0] [lt=10] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:03.042584] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002812][BatchIO][T0][Y0-0000000000000000-0-0] [lt=8] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:03.042587] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:677) [1002823][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=9] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:03.042610] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [1002825][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:03.042903] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [1002821][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0)
[2025-04-18 10:48:03.043356] INFO [PALF] read (log_cache.cpp:100) [1029173][T1002_ReplaySrv][T1002][Y0-0000000000000000-0-0] [lt=13] [PALF STAT HOT CACHE HIT RATE](palf_id=1002, read_size=976, hit_cnt=8, read_cnt=8, hit rate=1.000000000000000000e+00)
[2025-04-18 10:48:03.050169] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1979) [1028131][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=2] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90,
ob_plan_cache_evict_low_percentage=50, tenant_id=1002)
[2025-04-18 10:48:03.050188] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1481) [1028131][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=13] start lib cache evict(tenant_id=1002, mem_hold=277258240, mem_limit=375809635, cache_obj_num=811, cache_no
de_num=781)
[2025-04-18 10:48:03.050194] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1498) [1028131][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=5] end lib cache evict(tenant_id=1002, cache_evict_num=0, mem_hold=277258240, mem_limit=375809635, cache_obj_n
um=811, cache_node_num=781)
[2025-04-18 10:48:03.050203] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2754) [1028131][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=6] schedule next cache evict task(evict_interval=5000000)
[2025-04-18 10:48:03.051932] WDIAG [SQL.PC] run_free_cache_obj_task (ob_plan_cache.cpp:2799) [1028131][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=3][errcode=0] Cache Object Memory Leaked Much!!!(deleted_objs=[{obj_id:2350, tenant_id:1002, log_d
el_time:2042144597446, real_del_time:2480586580709, ref_count:1, added_to_lc:true, mem_used:19864}, {obj_id:2430, tenant_id:1002, log_del_time:2042144597497, real_del_time:2480586580709, ref_count:1, added_to_lc:true, mem_used:19112}, {obj_id:2477, tenan
t_id:1002, log_del_time:2042144594220, real_del_time:2480586580709, ref_count:1, added_to_lc:true, mem_used:19832}, {obj_id:2478, tenant_id:1002, log_del_time:2042144594704, real_del_time:2480586580709, ref_count:1, added_to_lc:true, mem_used:20136}, {ob
j_id:2479, tenant_id:1002, log_del_time:2042144598157, real_del_time:2480586580709, ref_count:1, added_to_lc:true, mem_used:19496}, {obj_id:3608, tenant_id:1002, log_del_time:2478364598559, real_del_time:2480586580709, ref_count:9, added_to_lc:true, mem_
used:20136}, {obj_id:3895, tenant_id:1002, log_del_time:2478364598607, real_del_time:2480586580709, ref_count:9, added_to_lc:true, mem_used:19496}, {obj_id:4260, tenant_id:1002, log_del_time:2478364598441, real_del_time:2480586580709, ref_count:9, added_
to_lc:true, mem_used:19864}, {obj_id:4380, tenant_id:1002, log_del_time:2478364598611, real_del_time:2480586580709, ref_count:9, added_to_lc:true, mem_used:19832}], safe_timestamp=2480586580709, plan_cache_->get_mem_limit()=375809635)
[2025-04-18 10:48:03.053782] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2472) [1028131][T1002_PlanCache][T1002][Y0-0000000000000000-0-0] [lt=23] Dumping All Cache Objs(alloc_obj_list.count()=820, alloc_obj_list=[{obj_id:61, tenant_id:1002, log_del_t
ime:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177184}, {obj_id:62, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {
obj_id:68, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:71, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, a
dded_to_lc:true, mem_used:1752}, {obj_id:257, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175440}, {obj_id:283, tenant_id:1002, log_del_time:9223372036854775807, real_del_ti
me:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183629}, {obj_id:333, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179716}, {obj_id:335, tenant_id:1002, log_d
el_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181377}, {obj_id:336, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175
2}, {obj_id:338, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195701}, {obj_id:339, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_c
ount:2, added_to_lc:true, mem_used:187003}, {obj_id:342, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181856}, {obj_id:344, tenant_id:1002, log_del_time:9223372036854775807,
real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174672}, {obj_id:345, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177408}, {obj_id:347, tenant_id:
1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218280}, {obj_id:349, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, m
em_used:175731}, {obj_id:350, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177288}, {obj_id:353, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854
775807, ref_count:2, added_to_lc:true, mem_used:177612}, {obj_id:361, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179419}, {obj_id:364, tenant_id:1002, log_del_time:92233720
36854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179628}, {obj_id:383, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174272}, {obj_id:38
4, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195225}, {obj_id:385, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_
to_lc:true, mem_used:187414}, {obj_id:386, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186585}, {obj_id:389, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:
9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189989}, {obj_id:390, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:394, tenant_id:1002, log_del_ti
me:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178822}, {obj_id:395, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210664},
{obj_id:396, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:397, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:
2, added_to_lc:true, mem_used:175728}, {obj_id:398, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:399, tenant_id:1002, log_del_time:9223372036854775807, real_de
l_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179497}, {obj_id:400, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:402, tenant_id:1002, log
del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:403, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218
232}, {obj_id:404, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:405, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_c
ount:2, added_to_lc:true, mem_used:1752}, {obj_id:406, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197951}, {obj_id:407, tenant_id:1002, log_del_time:9223372036854775807, re
al_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197951}, {obj_id:408, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186224}, {obj_id:409, tenant_id:10
02, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186224}, {obj_id:410, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem
used:177538}, {obj_id:415, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160307}, {obj_id:416, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:922337203685477
5807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:417, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:418, tenant_id:1002, log_del_time:92233720368547
75807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:167972}, {obj_id:419, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164279}, {obj_id:420, ten
ant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:159787}, {obj_id:421, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:
true, mem_used:1752}, {obj_id:433, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175888}, {obj_id:435, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:92233720
36854775807, ref_count:2, added_to_lc:true, mem_used:179704}, {obj_id:438, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:4497}, {obj_id:441, tenant_id:1002, log_del_time:92233
72036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:78505}, {obj_id:442, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:44
3, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184732}, {obj_id:451, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added
to_lc:true, mem_used:1752}, {obj_id:467, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175312}, {obj_id:486, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:92
23372036854775807, ref_count:2, added_to_lc:true, mem_used:234013}, {obj_id:496, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195720}, {obj_id:542, tenant_id:1002, log_del_ti
me:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197749}, {obj_id:546, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177971},
{obj_id:547, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174640}, {obj_id:548, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, reft:2, added_to_lc:true, mem_used:174560}, {obj_id:577, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:229005}, {obj_id:580, tenant_id:1002, log_del_time:9223372036854775807, rea
l_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:222776}, {obj_id:589, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201704}, {obj_id:605, tenant_id:100
2, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173821}, {obj_id:606, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem
used:175544}, {obj_id:607, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257612}, {obj_id:610, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775
807, ref_count:2, added_to_lc:true, mem_used:196282}, {obj_id:613, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234377}, {obj_id:641, tenant_id:1002, log_del_time:92233720368
54775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:649, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:657, tena
nt_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:658, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:tru
e, mem_used:182090}, {obj_id:659, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:224278}, {obj_id:660, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:922337203
6854775807, ref_count:2, added_to_lc:true, mem_used:184552}, {obj_id:661, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182042}, {obj_id:662, tenant_id:1002, log_del_time:9223
372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180769}, {obj_id:664, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178539}, {obj_i
d:665, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175888}, {obj_id:667, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, ad
ded_to_lc:true, mem_used:175888}, {obj_id:668, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177104}, {obj_id:676, tenant_id:1002, log_del_time:9223372036854775807, real_del_t
ime:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178038}, {obj_id:682, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214873}, {obj_id:698, tenant_id:1002, log_
del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:262521}, {obj_id:700, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:17
6413}, {obj_id:748, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177152}, {obj_id:751, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, re
f_count:2, added_to_lc:true, mem_used:180850}, {obj_id:752, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178180}, {obj_id:753, tenant_id:1002, log_del_time:922337203685477580
7, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178395}, {obj_id:754, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182009}, {obj_id:756, tenant_
id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180311}, {obj_id:757, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true
, mem_used:183243}, {obj_id:759, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180103}, {obj_id:760, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036
854775807, ref_count:2, added_to_lc:true, mem_used:182333}, {obj_id:764, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:12921}, {obj_id:765, tenant_id:1002, log_del_time:922337
2036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1168101}, {obj_id:766, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182996}, {obj_id
:776, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201976}, {obj_id:777, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, add
ed_to_lc:true, mem_used:1752}, {obj_id:780, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230685}, {obj_id:781, tenant_id:1002, log_del_time:9223372036854775807, real_del_time
:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220768}, {obj_id:893, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185180}, {obj_id:895, tenant_id:1002, log_del
_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186944}, {obj_id:941, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}
, {obj_id:1109, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1110, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_cou
nt:2, added_to_lc:true, mem_used:1752}, {obj_id:1111, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1112, tenant_id:1002, log_del_time:9223372036854775807, real
_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1113, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1114, tenant_id:1002,
log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1115, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used
:1752}, {obj_id:1116, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1117, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, r
ef_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1118, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:227345}, {obj_id:1119, tenant_id:1002, log_del_time:92233720368547758
07, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1182, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179578}, {obj_id:1183, tenant
_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182042}, {obj_id:1184, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:tr
ue, mem_used:178867}, {obj_id:1185, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179057}, {obj_id:1187, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:922337
2036854775807, ref_count:2, added_to_lc:true, mem_used:178347}, {obj_id:1188, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179057}, {obj_id:1200, tenant_id:1002, log_del_time
:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180312}, {obj_id:1202, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {o
bj_id:1203, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176880}, {obj_id:1210, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count
:2, added_to_lc:true, mem_used:223924}, {obj_id:1216, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1261, tenant_id:1002, log_del_time:9223372036854775807, real
del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180770}, {obj_id:1262, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180850}, {obj_id:1263, tenant_id:10
02, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182010}, {obj_id:1264, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, me
m_used:1752}, {obj_id:1273, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1274, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775
807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1275, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1276, tenant_id:1002, log_del_time:9223372036854
775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1279, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1281, tena
nt_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:232355}, {obj_id:1286, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:
true, mem_used:202808}, {obj_id:1287, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178376}, {obj_id:1288, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223
372036854775807, ref_count:2, added_to_lc:true, mem_used:175056}, {obj_id:1289, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176856}, {obj_id:1290, tenant_id:1002, log_del_ti
me:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192135}, {obj_id:1291, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188896}
, {obj_id:1305, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1315, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_cou
nt:2, added_to_lc:true, mem_used:1752}, {obj_id:1332, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1355, tenant_id:1002, log_del_time:9223372036854775807, real
del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176992}, {obj_id:1362, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182004}, {obj_id:1369, tenant_id:10
02, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176992}, {obj_id:1370, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, me
m_used:175776}, {obj_id:1373, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175840}, {obj_id:1382, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:922337203685
4775807, ref_count:2, added_to_lc:true, mem_used:175776}, {obj_id:1442, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228925}, {obj_id:1463, tenant_id:1002, log_del_time:92233
72036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:270312}, {obj_id:1464, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:242751}, {obj_i
d:1496, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:316393}, {obj_id:1524, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2,
added_to_lc:true, mem_used:280489}, {obj_id:1533, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237920}, {obj_id:1535, tenant_id:1002, log_del_time:9223372036854775807, real_d
el_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174848}, {obj_id:1536, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182569}, {obj_id:1538, tenant_id:1002
, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174192}, {obj_id:1556, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem
used:181851}, {obj_id:1558, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:173936}, {obj_id:1560, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:92233720368547
75807, ref_count:2, added_to_lc:true, mem_used:174368}, {obj_id:1655, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238128}, {obj_id:1860, tenant_id:1002, log_del_time:9223372
036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1861, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:18823372036854775807, ref_count:2, added_to_lc:true, mem_used:186329}, {obj_id:2119, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185299}, {obj_id:2120, tenant_id:1002, log_del
time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194004}, {obj_id:2121, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:20637
9}, {obj_id:2123, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205710}, {obj_id:2125, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref
_count:2, added_to_lc:true, mem_used:223016}, {obj_id:2134, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2135, tenant_id:1002, log_del_time:9223372036854775807
, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2136, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2139, tenant_id:
1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175840}, {obj_id:2140, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true,
mem_used:177136}, {obj_id:2155, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2159, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:922337203685
4775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2179, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2187, tenant_id:1002, log_del_time:922337203
6854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2192, tenant_id:1002, log_del_time:9223372036854775807, e,

dazhuangge · 2025 年4 月 18 日 12:27

9, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:292977}, {obj_id:4880, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, adde
d_to_lc:true, mem_used:264188}, {obj_id:4882, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:312128}, {obj_id:4883, tenant_id:1002, log_del_time:9223372036854775807, real_del_t
ime:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4884, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4885, tenant_id:1002, log_de
l_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4886, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}
, {obj_id:4887, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4888, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_cou
nt:2, added_to_lc:true, mem_used:1752}, {obj_id:4889, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4890, tenant_id:1002, log_del_time:9223372036854775807, real
_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182099}, {obj_id:4891, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179797}, {obj_id:4893, tenant_id:10
02, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:787291}, {obj_id:4894, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, me
m_used:780076}, {obj_id:4895, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:2783441}, {obj_id:4896, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:92233720368
54775807, ref_count:2, added_to_lc:true, mem_used:175549}, {obj_id:4897, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177202}, {obj_id:4898, tenant_id:1002, log_del_time:9223
372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228243}, {obj_id:4899, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id
:4900, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4901, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, add
ed_to_lc:true, mem_used:1752}, {obj_id:4902, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182522}, {obj_id:4903, tenant_id:1002, log_del_time:9223372036854775807, real_del_ti
me:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185128}, {obj_id:4904, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231681}, {obj_id:4905, tenant_id:1002, log
del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233640}, {obj_id:4906, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:
389736}, {obj_id:4907, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206141}, {obj_id:4908, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807
, ref_count:2, added_to_lc:true, mem_used:280583}, {obj_id:4909, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:247441}, {obj_id:4911, tenant_id:1002, log_del_time:922337203685
4775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:378998}, {obj_id:4912, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:16776}, {obj_id:4913,
tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:16776}, {obj_id:4914, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to
lc:true, mem_used:194079}, {obj_id:4915, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4916, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:922
3372036854775807, ref_count:2, added_to_lc:true, mem_used:258473}, {obj_id:4917, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:307444}, {obj_id:4919, tenant_id:1002, log_del_t
ime:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:410451}, {obj_id:4920, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:404089
}, {obj_id:4921, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4922, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_co
unt:2, added_to_lc:true, mem_used:1752}, {obj_id:4923, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4924, tenant_id:1002, log_del_time:9223372036854775807, rea
l_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4925, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4926, tenant_id:1002,
log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:299668}, {obj_id:4927, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_u
sed:265414}, {obj_id:4930, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200949}, {obj_id:4933, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:922337203685477
5807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4934, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4935, tenant_id:1002, log_del_time:922337203685
4775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4936, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4937, ten
ant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4938, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:t
rue, mem_used:1752}, {obj_id:4939, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4940, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:922337203
6854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4941, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4942, tenant_id:1002, log_del_time:922337
2036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:285421}, {obj_id:4943, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195332}, {obj_id
:4947, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4948, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, add
ed_to_lc:true, mem_used:1752}, {obj_id:4949, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4950, tenant_id:1002, log_del_time:9223372036854775807, real_del_time
:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:231126}, {obj_id:4951, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4952, tenant_id:1002, log_del
_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4953, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752},
{obj_id:4954, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4955, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_coun
t:2, added_to_lc:true, mem_used:1752}, {obj_id:4956, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213152}, {obj_id:4959, tenant_id:1002, log_del_time:9223372036854775807, rea
l_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4960, tenant_id:1002, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:4961, tenant_id:1002

dazhuangge · 2025 年4 月 18 日 12:28

[2025-04-18 10:48:03.102162] INFO [LIB] stat (utility.h:1212) [3748642][T1002_ApplySrv1][T1002][Y0-0000000000000000-0-0] [lt=0] [APPLY STAT CB APPEND COST TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=26, this=0x7f5ac1278280, extra_in
fo="")
[2025-04-18 10:48:03.102187] INFO [LIB] stat (utility.h:1212) [3748642][T1002_ApplySrv1][T1002][Y0-0000000000000000-0-0] [lt=19] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=1006, this=0x7f5ac12784b0, extra_in
fo="")
[2025-04-18 10:48:03.102192] INFO [LIB] stat (utility.h:1212) [3748642][T1002_ApplySrv1][T1002][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=0, this=0x7f5ac12786e0, extra_inf
o="")
[2025-04-18 10:48:03.102196] INFO [LIB] stat (utility.h:1212) [3748642][T1002_ApplySrv1][T1002][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB EXECUTE TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=-473, this=0x7f5ac1278910, extra_info
="")
[2025-04-18 10:48:03.102199] INFO [LIB] stat (utility.h:1212) [3748642][T1002_ApplySrv1][T1002][Y0-0000000000000000-0-0] [lt=3] [APPLY STAT CB TOTAL TIME](cur_stat_count=13, stat_interval=5000000, avg (count/cost)=559, this=0x7f5ac1278b40, extra_info=""
)
[2025-04-18 10:48:03.113312] INFO [COMMON] print_io_status (ob_io_manager.cpp:1436) [1002589][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SYS](tenant_id=1, io_status_sys_log=sys_group_name: UNKNOWN, mode: write, size: 4468.36, iops:
10.99, rt: 785.18; )
[2025-04-18 10:48:03.113332] INFO [COMMON] print_io_status (ob_io_manager.cpp:1452) [1002589][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS CONFIG](tenant_id=1, ref_cnt=2, io_config={group_num_:1, memory_limit_:5368709120, callback_thread_
count_:8, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:4}, enable_io_tracer:false, group_configs:[group_id = 10000:{deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, other_groups:{deleted
:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=4333344, pre_allocated_count=2, callback_queues=0)
[2025-04-18 10:48:03.113358] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:1515) [1002589][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4468,
avg_iops:11, avg_bw:49101, [delay/us]: prepare:1, schedule:11, submit:0, device:785, total:799)
[2025-04-18 10:48:03.113379] INFO [COMMON] print_io_status (ob_io_manager.cpp:1436) [1002589][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS SYS](tenant_id=1001, io_status_sys_log=sys_group_name: UNKNOWN, mode: write, size: 4642.13, iops:
14.98, rt: 928.47; )
[2025-04-18 10:48:03.113387] INFO [COMMON] print_io_status (ob_io_manager.cpp:1452) [1002589][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CONFIG](tenant_id=1001, ref_cnt=2, io_config={group_num_:0, memory_limit_:1073741824, callback_threa
d_count_:8, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:4}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=
4268064, pre_allocated_count=2, callback_queues=0)
[2025-04-18 10:48:03.113404] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:1515) [1002589][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS FUNCTION](tenant_id=1001, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:464
2, avg_iops:15, avg_bw:69560, [delay/us]: prepare:1, schedule:10, submit:1, device:928, total:941)
[2025-04-18 10:48:03.113423] INFO [COMMON] print_io_status (ob_io_manager.cpp:1436) [1002589][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS SYS](tenant_id=1002, io_status_sys_log=sys_group_name: SLOG_IO, mode: write, size: 4096.00, iops:
4.00, rt: 745.50; sys_group_name: UNKNOWN, mode: write, size: 4096.00, iops: 34.96, rt: 757.63; )
[2025-04-18 10:48:03.113434] INFO [COMMON] print_io_status (ob_io_manager.cpp:1452) [1002589][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CONFIG](tenant_id=1002, ref_cnt=2, io_config={group_num_:0, memory_limit_:7516192768, callback_threa
d_count_:8, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:4}, enable_io_tracer:false, group_configs:[other_groups:{deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, allocated_memory=
4268064, pre_allocated_count=2, callback_queues=0)
[2025-04-18 10:48:03.113442] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:1515) [1002589][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS FUNCTION](tenant_id=1002, io_status=function_name:OTHER_GROUPS, mode:LOCAL WRITE, avg_size:
4096, avg_iops:4, avg_bw:16367, [delay/us]: prepare:2, schedule:7, submit:2, device:745, total:757)
[2025-04-18 10:48:03.113452] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:1515) [1002589][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS FUNCTION](tenant_id=1002, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:409
6, avg_iops:35, avg_bw:143214, [delay/us]: prepare:2, schedule:11, submit:1, device:757, total:772)
[2025-04-18 10:48:03.113458] INFO [COMMON] print_sender_status (ob_io_struct.cpp:993) [1002589][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS SENDER](*this=send_index: 1, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_t
s: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 2, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807,
proportion_ts: 9223372036854775807; send_index: 3, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 4, req_count: 0, re
servation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 5, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 92233720368
54775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 6, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts:
9223372036854775807; send_index: 7, req_count: 0, reservation_ts: 9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; send_index: 8, req_count: 0, reservation_ts:
9223372036854775807, group_limitation_ts: 9223372036854775807, tenant_limitation_ts: 9223372036854775807, proportion_ts: 9223372036854775807; )
[2025-04-18 10:48:03.113628] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:131) [1028320][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=6] ====== tenant freeze timer task ======
[2025-04-18 10:48:03.113642] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1711) [1028320][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=8] [TenantFreezer] tenant have inactive memstores(ctx.freezable_active_memstore_used_=4
4040192, ctx.total_memstore_used_=77594624, ctx.total_memstore_hold_=77594624, memstore_freeze_trigger_limit_=58804200, tenant_id=1001)
[2025-04-18 10:48:03.113655] INFO [STORAGE] log_frozen_memstore_info_if_need_ (ob_tenant_freezer.cpp:1715) [1028320][T1001_Occam][T1001][Y0-0000000000000000-0-0] [lt=8] [TenantFreezer] oldest frozen memtable(list="")
[2025-04-18 10:48:03.123006] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1155) [1002620][KVCacheWash][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-251474305843, global_cache_size=
7297253376, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[500, 508, 1, 1001, 1002])
[2025-04-18 10:48:03.132742] INFO do_work (ob_rl_mgr.cpp:709) [1002677][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=11] swc wakeup.(stat_period_=1000000, ready=false)
[2025-04-18 10:48:03.132964] INFO [STORAGE.TRANS] check_all_readonly_tx_clean_up (ob_ls_tx_service.cpp:370) [1028432][T1001_TxLoopWor][T1001][Y0-0000000000000000-0-0] [lt=13] wait_all_readonly_tx_cleaned_up cleaned up success(ls_id={id:1})
[2025-04-18 10:48:03.133907] INFO [STORAGE.TRANS] check_all_readonly_tx_clean_up (ob_ls_tx_service.cpp:370) [1003064][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=7] wait_all_readonly_tx_cleaned_up cleaned up success(ls_id={id:1})
[2025-04-18 10:48:03.134360] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [1003046][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=5] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=323147918, evict_high_si
ze_level=589824, evict_batch_count=0, elapse_time=0, size_used=131072, mem_used=222532608)
[2025-04-18 10:48:03.134376] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [1003046][T1_ReqMemEvict][T1][Y0-0000000000000000-0-0] [lt=12] Construct Queue Num(construct_num=0, get_push_idx()=710, get_cur_idx()=702, get_pop_idx()=700)
[2025-04-18 10:48:03.164754] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:441) [1003129][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3] schedule fetch new schema task(ret=0, ret=“OB_SUCCESS”, schema_info={schema_version:1744941199085264,
tenant_id:1002, sequence_id:2199023257741})
[2025-04-18 10:48:03.164767] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [1003129][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=10] try reload schema success(schema_version=1744680052057480, refresh_schema_info={schema_version:1744941199085264,
tenant_id:1002, sequence_id:2199023257741}, schema_ret=0)
[2025-04-18 10:48:03.164778] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:587) [1003129][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4201] No tenant config found(tenant_id=1003, ret=-4201)
[2025-04-18 10:48:03.164771] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:256) [1002679][SerScheQueue0][T0][YB420A263025-00063268D48ECADE-0-0] [lt=8] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret=“OB_SUCCESS”, sche
ma_info={schema_version:1744941199085264, tenant_id:1002, sequence_id:2199023257741})
[2025-04-18 10:48:03.164787] WDIAG [SERVER.OMT] got_versions (ob_tenant_config_mgr.cpp:575) [1003129][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4201] fail got version(tenant_id=1003, version=2, ret=-4201)
[2025-04-18 10:48:03.164791] WDIAG [SERVER.OMT] got_version (ob_tenant_config_mgr.cpp:587) [1003129][LeaseHB][T0][Y0-0000000000000000-0-0] [lt=3][errcode=-4201] No tenant config found(tenant_id=1004, ret=-4201)
2025-04-18 10:48:03.877845] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [1028241][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=14] [PALF STAT GROUP LOG INFO](palf_id=1, self=“10.38.48.37:2882”, role=“LEADER”, total_group_l
og_cnt=16, avg_log_batch_cnt=1, total_group_log_size=10424, avg_group_log_size=651)
[2025-04-18 10:48:03.877867] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [1028241][T1002_TenantWea][T1001][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:“10.38.48.37:2882”, has_set_deleted:false}, append size=
10424)
[2025-04-18 10:48:03.878830] INFO [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1692) [1003047][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=14] execute write sql(ret=0, tenant_id=1, affected_rows=1, sql= update __all_weak_read_serv
ice set min_version=1744944483676194000, max_version=1744944483676194000 where tenant_id = 1 and level_id = 0 and level_value = ‘’ and min_version = 1744944483476046000 and max_version = 1744944483476046000 , get_session().get_sessid()=3221558894)
[2025-04-18 10:48:03.878921] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [1028239][T1002_STSWorker][T1002][Y0-0000000000000000-0-0] [lt=14][errcode=-4076] query and update last id fail(ret=-4076, ret=“OB_NEED_WAIT”)
[2025-04-18 10:48:03.879241] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2238) [1028386][T1001_IOWorker][T1001][Y0-0000000000000000-0-0] [lt=6] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:“10.38.48.37:2882”, has_set_deleted:false}, ac
cum_size=10424)
[2025-04-18 10:48:03.879402] INFO [PALF] sliding_cb (log_sliding_window.cpp:2280) [3747950][T1001_L0_G1][T1001][YB420A263027-00063269484C2BBE-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self=“10.38.48.37:2882”, total_slide_log_cnt=16, avg_log_gen_t
o_freeze_time=12, avg_log_gen_to_submit_time=39, avg_log_submit_to_flush_time=932, avg_log_submit_to_slide_time=1162)
[2025-04-18 10:48:03.884174] INFO handle (memory_dump.cpp:670) [1002619][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=8] statistics:
tenant_cnt: 5, max_chunk_cnt: 524288
tenant_id ctx_id chunk_cnt label_cnt segv_cnt
1 0 336 245 0
1 2 117 1 0
1 4 1 1 0
1 5 255 6 0
1 7 4 2 0
1 8 78 1 0
1 12 11 3 0
1 13 7 1 0
1 14 1 1 0
1 18 19 3 0
1 20 1 1 0
500 0 173 252 0
500 7 16 5 0
500 8 69 1 0
500 9 4 1 0
500 10 2 1 0
500 16 4 1 0
500 17 20 7 0
500 22 75 62 0
500 23 29 10 0
508 0 4 11 0
508 8 15 1 0
1001 0 132 230 0
1001 2 37 1 0
1001 4 1 1 0
1001 5 23 5 0
1001 8 45 1 0
1001 12 9 3 0
1001 13 3 1 0
1001 14 1 1 0
1001 18 14 3 0
1001 20 1 1 0
1002 0 264 259 0
1002 2 64 1 0
1002 4 1 1 0
1002 5 170 8 0
1002 7 16 2 0
1002 8 62 1 0
1002 12 14 3 0
1002 13 6 1 0
1002 14 1 1 0
1002 18 30 3 0
1002 20 3 1 0
cost_time: 54036
[2025-04-18 10:48:03.884207] INFO [LIB] operator() (ob_malloc_allocator.cpp:529) [1002619][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=10] [MEMORY] tenant: 1, limit: 13,464,496,603 hold: 3,870,916,608 cache_hold: 1,849,688,064 cache_used: 1,849,688,064
cache_item_count: 882
[MEMORY] ctx_id= DEFAULT_CTX_ID hold_bytes= 941,166,592 limit= 9,223,372,036,854,775,807

dazhuangge · 2025 年4 月 18 日 12:30

[2025-04-18 10:48:03.884416] INFO [MEMORY] tenant_id= 1 ctx_id= [MEMORY] idle_size= 0 free_size= [MEMORY] wash_related_chunks= [MEMORY] request_cached_chunk_cnt= [MEMORY] hold= 239,951,872 used= [MEMORY] hold= 67,129,344 used= [MEMORY] hold= 41,963,520 used= [MEMORY] hold= 33,292,288 used= [MEMORY] hold= 31,477,760 used= [MEMORY] hold= 27,967,488 used= [MEMORY] hold= 24,199,168 used= [MEMORY] hold= 21,619,440 used= [MEMORY] hold= 19,689,472 used= [MEMORY] hold= 18,894,848 used= [MEMORY] hold= 18,874,368 used= [MEMORY] hold= 17,252,352 used= [MEMORY] hold= 14,956,848 used= [MEMORY] hold= 12,771,328 used= [MEMORY] hold= 11,624,448 used= [MEMORY] hold= 11,331,392 used= [MEMORY] hold= 8,921,088 used= [MEMORY] hold= 8,839,680 used= [MEMORY] hold= 8,777,728 used= [MEMORY] hold= 8,540,160 used= [MEMORY] hold= 8,540,160 used= [MEMORY] hold= 8,449,424 used= [MEMORY] hold= 8,380,416 used= [MEMORY] hold= 7,776,832 used= [MEMORY] hold= 7,503,872 used= [MEMORY] hold= 6,814,288 used= [MEMORY] hold= 6,549,504 used= [MEMORY] hold= 6,537,216 used= [MEMORY] hold= 5,464,064 used= [MEMORY] hold= 5,337,088 used= [MEMORY] hold= 4,947,968 used= [MEMORY] hold= 4,739,072 used= [MEMORY] hold= 4,354,048 used= [MEMORY] hold= 3,865,600 used= [MEMORY] hold= 3,865,600 used= [MEMORY] hold= 3,723,264 used= [MEMORY] hold= 3,706,880 used= [MEMORY] hold= 3,145,728 used= [MEMORY] hold= 2,756,608 used= [MEMORY] hold= 2,736,128 used= [MEMORY] hold= 2,379,776 used= [MEMORY] hold= 2,375,680 used= [MEMORY] hold= 2,248,704 used= [MEMORY] hold= 2,169,120 used= [MEMORY] hold= 2,162,688 used= [MEMORY] hold= 1,736,704 used= [MEMORY] hold= 1,670,976 used= [MEMORY] hold= 1,581,056 used= [MEMORY] hold= 21,397,504 used= [2025-04-18 10:48:03.884550] INFO [MEMORY] tenant_id= 1 ctx_id= [MEMORY] idle_size= 0 free_size= [MEMORY] wash_related_chunks= [MEMORY] request_cached_chunk_cnt= [MEMORY] hold= 7,672,832 used= [MEMORY] hold= 7,672,832 used= [2025-04-18 10:48:03.884561] INFO [MEMORY] tenant_id= 1 ctx_id= [MEMORY] idle_size= 0 free_size= [MEMORY] wash_related_chunks= [MEMORY] request_cached_chunk_cnt= [MEMORY] hold= 8,128 used= [MEMORY] hold= 8,128 used= [2025-04-18 10:48:03.884583] INFO [MEMORY] tenant_id= 1 ctx_id= [MEMORY] idle_size= 0 free_size= [MEMORY] wash_related_chunks= [MEMORY] request_cached_chunk_cnt= [MEMORY] hold= 32,365,696 used= [MEMORY] hold= 5,279,680 used= [MEMORY] hold= 8,128 used= [MEMORY] hold= 37,653,504 used= [2025-04-18 10:48:03.884597] INFO [MEMORY] tenant_id= 1 ctx_id= [MEMORY] idle_size= 0 free_size= [MEMORY] wash_related_chunks= [MEMORY] request_cached_chunk_cnt= [MEMORY] hold= 2,007,616 used= [MEMORY] hold= 2,007,616 used= [2025-04-18 10:48:03.884620] INFO count: 0
[MEMORY] ctx_id= [MEMORY] ctx_id= [MEMORY] ctx_id= [MEMORY] ctx_id= [MEMORY] ctx_id= [MEMORY] ctx_id= [MEMORY] ctx_id= [MEMORY] ctx_id= [MEMORY] ctx_id= UNEXPECTED_IN_500 [2025-04-18 10:48:03.884827] INFO [MEMORY] tenant_id= 500 ctx_id= [MEMORY] idle_size= 0 free_size= [MEMORY] wash_related_chunks= [MEMORY] request_cached_chunk_cnt= [MEMORY] hold= 208,596,992 used= [MEMORY] hold= 66,584,576 used= [MEMORY] hold= 52,517,184 used= [MEMORY] hold= 33,574,912 used= [MEMORY] hold= 12,779,520 used= [MEMORY] hold= 10,747,904 used= [MEMORY] hold= 9,457,664 used= [MEMORY] hold= 7,553,024 used= [MEMORY] hold= 7,397,376 used= [MEMORY] hold= 7,321,936 used= [MEMORY] hold= 6,870,160 used= [MEMORY] hold= 6,549,504 used= [MEMORY] hold= 5,849,088 used= [MEMORY] hold= 4,747,072 used= [LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [1002619][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=7]
DEFAULT_CTX_ID hold= 941,166,592 used= 810,562,880 limit= 9,223,372,036,854,775,807
0
0 washed_blocks= 0 washed_size= 0
32
239,668,416 count= 129 avg_used= 1,857,894 block_cnt= 129 chunk_cnt= 123 mod=MysqlRequesReco
67,108,864 count= 1 avg_used= 67,108,864 block_cnt= 1 chunk_cnt= 1 mod=HHBucket
41,943,040 count= 1 avg_used= 41,943,040 block_cnt= 1 chunk_cnt= 1 mod=LogGroupBuffer
33,275,904 count= 16 avg_used= 2,079,744 block_cnt= 16 chunk_cnt= 16 mod=CACHE_MAP_NODE
31,458,304 count= 1 avg_used= 31,458,304 block_cnt= 1 chunk_cnt= 1 mod=ASHListBuffer
27,885,568 count= 4 avg_used= 6,971,392 block_cnt= 4 chunk_cnt= 4 mod=ServerObjecPool
22,820,096 count= 169 avg_used= 135,030 block_cnt= 169 chunk_cnt= 61 mod=SqlPlanMon
6,557,424 count= 75,282 avg_used= 87 block_cnt= 332 chunk_cnt= 91 mod=TLDecoderCtx
19,660,928 count= 2 avg_used= 9,830,464 block_cnt= 2 chunk_cnt= 2 mod=HashBuckDmId
18,874,392 count= 1 avg_used= 18,874,392 block_cnt= 1 chunk_cnt= 1 mod=HashBuckDmChe
16,914,176 count= 256 avg_used= 66,071 block_cnt= 256 chunk_cnt= 58 mod=SqlDtlBuf
15,335,424 count= 234 avg_used= 65,536 block_cnt= 234 chunk_cnt= 90 mod=MinorMergeMgr
14,829,049 count= 91 avg_used= 162,956 block_cnt= 31 chunk_cnt= 8 mod=OmtTenant
10,821,632 count= 238 avg_used= 45,469 block_cnt= 238 chunk_cnt= 60 mod=[T]char
11,534,688 count= 11 avg_used= 1,048,608 block_cnt= 11 chunk_cnt= 11 mod=DiagnosticInfos
10,850,928 count= 2,406 avg_used= 4,509 block_cnt= 2,314 chunk_cnt= 104 mod=WorkerMap
7,929,856 count= 121 avg_used= 65,536 block_cnt= 121 chunk_cnt= 67 mod=MajorMergeMgr
8,437,920 count= 180 avg_used= 46,877 block_cnt= 180 chunk_cnt= 57 mod=CommSysVarFac
8,760,064 count= 2 avg_used= 4,380,032 block_cnt= 2 chunk_cnt= 2 mod=FreeTbltStream
8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=RCSrv
8,519,680 count= 1 avg_used= 8,519,680 block_cnt= 1 chunk_cnt= 1 mod=ArcFetchQueue
8,244,752 count= 1,066 avg_used= 7,734 block_cnt= 1,066 chunk_cnt= 107 mod=TLDecoderAlloc
8,295,936 count= 11 avg_used= 754,176 block_cnt= 11 chunk_cnt= 11 mod=ResourceGroup
7,578,880 count= 755 avg_used= 10,038 block_cnt= 755 chunk_cnt= 66 mod=ParamCharsType
6,104,224 count= 229 avg_used= 26,656 block_cnt= 229 chunk_cnt= 59 mod=DiagnosticInfo
6,781,912 count= 63 avg_used= 107,649 block_cnt= 63 chunk_cnt= 4 mod=TabletMap
6,531,072 count= 1 avg_used= 6,531,072 block_cnt= 1 chunk_cnt= 1 mod=DiskCB
6,435,840 count= 16 avg_used= 402,240 block_cnt= 16 chunk_cnt= 16 mod=DiagnosticSum
5,394,144 count= 23 avg_used= 234,528 block_cnt= 23 chunk_cnt= 19 mod=SQLSessionInfo
5,320,192 count= 1 avg_used= 5,320,192 block_cnt= 1 chunk_cnt= 1 mod=RespTimeColl
4,931,072 count= 1 avg_used= 4,931,072 block_cnt= 1 chunk_cnt= 1 mod=TransService
4,718,712 count= 1 avg_used= 4,718,712 block_cnt= 1 chunk_cnt= 1 mod=SqlPlanMonMap
4,333,344 count= 3 avg_used= 1,444,448 block_cnt= 3 chunk_cnt= 3 mod=IoControl
3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 13 mod=CkptDgnMemCU
3,705,600 count= 800 avg_used= 4,632 block_cnt= 800 chunk_cnt= 13 mod=CkptDgnMem
3,698,704 count= 2 avg_used= 1,849,352 block_cnt= 2 chunk_cnt= 2 mod=LogDIOAligned
3,686,408 count= 1 avg_used= 3,686,408 block_cnt= 1 chunk_cnt= 1 mod=DiskIteratorSto
2,228,224 count= 128 avg_used= 17,408 block_cnt= 128 chunk_cnt= 6 mod=SqlDtlQueue
2,720,005 count= 3 avg_used= 906,668 block_cnt= 3 chunk_cnt= 2 mod=SqlDtl1stBuf
2,717,248 count= 1 avg_used= 2,717,248 block_cnt= 1 chunk_cnt= 1 mod=RpcStatInfo
2,359,608 count= 1 avg_used= 2,359,608 block_cnt= 1 chunk_cnt= 1 mod=MediumTabletMap
2,359,536 count= 2 avg_used= 1,179,768 block_cnt= 2 chunk_cnt= 2 mod=HashBuckLCSta
2,228,224 count= 1 avg_used= 2,228,224 block_cnt= 1 chunk_cnt= 1 mod=LogIOCb
408,600 count= 24,952 avg_used= 16 block_cnt= 266 chunk_cnt= 3 mod=Number
2,154,240 count= 33 avg_used= 65,280 block_cnt= 33 chunk_cnt= 7 mod=MemTblObj
853,088 count= 106 avg_used= 8,048 block_cnt= 106 chunk_cnt= 44 mod=[T]ObPerfEventR
1,663,936 count= 7 avg_used= 237,705 block_cnt= 7 chunk_cnt= 1 mod=PoolFreeList
1,572,904 count= 1 avg_used= 1,572,904 block_cnt= 1 chunk_cnt= 1 mod=MFTatCompactMap
19,532,421 count= 294 avg_used= 66,436 mod=SUMMARY
[LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [1002619][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5]
TX_CALLBACK_CTX_ID hold= 14,680,064 used= 7,672,832 limit= 9,223,372,036,854,775,807
0
0 washed_blocks= 0 washed_size= 0
0
7,491,584 count= 944 avg_used= 7,936 block_cnt= 944 chunk_cnt= 7 mod=MemtableCallbac
7,491,584 count= 944 avg_used= 7,936 mod=SUMMARY
[LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [1002619][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5]
LOB_CTX_ID hold= 2,097,152 used= 8,128 limit= 9,223,372,036,854,775,807
0
0 washed_blocks= 0 washed_size= 0
0
7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LobPersist
7,936 count= 1 avg_used= 7,936 mod=SUMMARY
[LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [1002619][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6]
TX_DATA_TABLE hold= 41,963,520 used= 37,653,504 limit= 9,223,372,036,854,775,807
0
0 washed_blocks= 0 washed_size= 0
0
31,601,152 count= 3,982 avg_used= 7,936 block_cnt= 3,982 chunk_cnt= 18 mod=TX_DATA_SLICE
5,250,880 count= 3 avg_used= 1,750,293 block_cnt= 3 chunk_cnt= 3 mod=MEMTABLE_ARENA
7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=TX_DATA_ARENA
36,859,968 count= 3,986 avg_used= 9,247 mod=SUMMARY
[LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [1002619][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5]
MDS_DATA_ID hold= 2,097,152 used= 2,007,616 limit= 9,223,372,036,854,775,807
0
0 washed_blocks= 0 washed_size= 0
0
1,960,192 count= 247 avg_used= 7,936 block_cnt= 247 chunk_cnt= 1 mod=MdsTable
1,960,192 count= 247 avg_used= 7,936 mod=SUMMARY
[LIB] operator() (ob_malloc_allocator.cpp:529) [1002619][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=6] [MEMORY] tenant: 500, limit: 9,223,372,036,854,775,807 hold: 1,861,873,664 cache_hold: 0 cache_used: 0 cache_item_
DEFAULT_CTX_ID hold_bytes= 642,666,496 limit= 9,223,372,036,854,775,807
GLIBC hold_bytes= 33,554,432 limit= 9,223,372,036,854,775,807
CO_STACK hold_bytes= 144,703,488 limit= 9,223,372,036,854,775,807
LIBEASY hold_bytes= 8,388,608 limit= 9,223,372,036,854,775,807
LOGGER_CTX_ID hold_bytes= 4,194,304 limit= 9,223,372,036,854,775,807
RPC_CTX_ID hold_bytes= 8,388,608 limit= 9,223,372,036,854,775,807
PKT_NIO hold_bytes= 54,870,016 limit= 9,223,372,036,854,775,807
SCHEMA_SERVICE hold_bytes= 162,287,616 limit= 9,223,372,036,854,775,807
hold_bytes= 802,820,096 limit= 9,223,372,036,854,775,807
[LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [1002619][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=5]
DEFAULT_CTX_ID hold= 642,666,496 used= 553,532,512 limit= 9,223,372,036,854,775,807
0
0 washed_blocks= 0 washed_size= 0
0
208,560,276 count= 2 avg_used= 104,280,138 block_cnt= 2 chunk_cnt= 2 mod=LDBlockBitMap
66,551,808 count= 32 avg_used= 2,079,744 block_cnt= 32 chunk_cnt= 32 mod=ObTxDesc
50,996,736 count= 3,317 avg_used= 15,374 block_cnt= 3,317 chunk_cnt= 29 mod=TZInfoArray
33,554,464 count= 1 avg_used= 33,554,464 block_cnt= 1 chunk_cnt= 1 mod=BloomFilter
12,760,352 count= 1 avg_used= 12,760,352 block_cnt= 1 chunk_cnt= 1 mod=MemDumpContext
10,473,472 count= 64 avg_used= 163,648 block_cnt= 64 chunk_cnt= 23 mod=Iterator<BtreeI
9,437,784 count= 1 avg_used= 9,437,784 block_cnt= 1 chunk_cnt= 1 mod=HashBuckInteChe
7,492,784 count= 6 avg_used= 1,248,797 block_cnt= 6 chunk_cnt= 4 mod=KvstCachWashStr
7,340,256 count= 7 avg_used= 1,048,608 block_cnt= 7 chunk_cnt= 7 mod=ConcurHashMap
6,365,510 count= 1,032 avg_used= 6,168 block_cnt= 542 chunk_cnt= 5 mod=CharsetInit
6,785,296 count= 48 avg_used= 141,360 block_cnt= 46 chunk_cnt= 15 mod=PartitTableTask
6,531,072 count= 1 avg_used= 6,531,072 block_cnt= 1 chunk_cnt= 1 mod=DiskCB
5,128,192 count= 88 avg_used= 58,274 block_cnt= 88 chunk_cnt= 43 mod=[T]char
4,726,520 count= 2 avg_used= 2,363,260 block_cnt= 2 chunk_cnt= 2 mod=HashBucTenComMo

AntTech_TUPCMI · 2025 年4 月 18 日 15:16

这个报错是因为odp获取get后的参数失败了，可以贴一下failed to init rowkey for redis command这个报错附近的日志吗，以及odp版本和内存占用情况

mittens · 2025 年4 月 18 日 15:33

学习下案例