2024-11-29T09:49:05.04154+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:49:05.15705+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:49:05.15718+08:00 INFO [21697,66ab39d72a904e1d] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:49:05.45008+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274980736, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274980736, max_request_id: 274980767 2024-11-29T09:49:06.20081+08:00 INFO [21697,494f485f70a62a5e] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=494f485f70a62a5e, body=] 2024-11-29T09:49:06.41291+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:49:06.41301+08:00 INFO [21697,e30230a679bb7dd9] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:49:06.4526+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274980768, result count: 60 auditMsg number: 0, diagMsg number: 0, min_request_id: 274980768, max_request_id: 274980827 2024-11-29T09:49:07.09496+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:49:07.31351+08:00 INFO [21697,2be0a81537a3f387] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=2be0a81537a3f387, body=] 2024-11-29T09:49:07.31385+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:49:07.31416+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:49:07.4543+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274980828, result count: 56 auditMsg number: 0, diagMsg number: 0, min_request_id: 274980828, max_request_id: 274980883 2024-11-29T09:49:07.78239+08:00 INFO [21697,7062d645d3da6bbc] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 1 result 2024-11-29T09:49:07.78247+08:00 INFO [21697,7062d645d3da6bbc] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:49:07.765508802 +0800 CST m=+10834912.882118334, end time:2024-11-29 09:49:07 +0800 CST, Metrics number 1 2024-11-29T09:49:07.78256+08:00 INFO [21697,7062d645d3da6bbc] caller=engine/pipeline.go:145:pipelinePush: push pipeline ob_sql_transaction metrics count 1 2024-11-29T09:49:07.87558+08:00 INFO [21697,f3aa5b9d3cef4508] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=f3aa5b9d3cef4508, body=] 2024-11-29T09:49:07.8758+08:00 WARN [21697,f3aa5b9d3cef4508] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:07.87583+08:00 WARN [21697,f3aa5b9d3cef4508] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:07.87585+08:00 WARN [21697,f3aa5b9d3cef4508] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:07.87586+08:00 WARN [21697,f3aa5b9d3cef4508] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:07.87587+08:00 WARN [21697,f3aa5b9d3cef4508] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:07.8813+08:00 INFO [21697,f3aa5b9d3cef4508] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=f3aa5b9d3cef4508, body=] 2024-11-29T09:49:08.45676+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274980884, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274980884, max_request_id: 274980918 2024-11-29T09:49:08.86366+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163131187, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163131562, max_request_id: 163131564 2024-11-29T09:49:08.86372+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163131562 is larger than startRequestId 163131187, maybe lost 375 data 2024-11-29T09:49:09.45933+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274980919, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274980919, max_request_id: 274980956 2024-11-29T09:49:10.03084+08:00 INFO [21697,9347fc1babd054dd] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=9347fc1babd054dd, body=] 2024-11-29T09:49:10.03109+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:49:10.04161+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:49:10.14835+08:00 INFO [21697,238fb0cbdc16d8b1] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=238fb0cbdc16d8b1, body=] 2024-11-29T09:49:10.14873+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:10.14876+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:10.14878+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:10.14879+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:10.1488+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:10.14916+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:49:10.46107+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274980957, result count: 27 auditMsg number: 0, diagMsg number: 0, min_request_id: 274980957, max_request_id: 274980983 2024-11-29T09:49:10.78911+08:00 INFO [21697,6c9a06ff97b13789] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=6c9a06ff97b13789, body=] 2024-11-29T09:49:10.87568+08:00 INFO [21697,321b24ac7db64435] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=321b24ac7db64435, body=] 2024-11-29T09:49:10.87599+08:00 WARN [21697,321b24ac7db64435] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:10.87604+08:00 WARN [21697,321b24ac7db64435] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:10.87606+08:00 WARN [21697,321b24ac7db64435] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:10.87607+08:00 WARN [21697,321b24ac7db64435] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:10.87609+08:00 WARN [21697,321b24ac7db64435] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:10.88052+08:00 INFO [21697,321b24ac7db64435] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=321b24ac7db64435, body=] 2024-11-29T09:49:11.46429+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274980984, result count: 76 auditMsg number: 0, diagMsg number: 0, min_request_id: 274980984, max_request_id: 274981059 2024-11-29T09:49:11.74864+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:49:12.1145+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:49:12.46621+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981060, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981060, max_request_id: 274981094 2024-11-29T09:49:12.78288+08:00 INFO [21697,26322b2bc5481dc2] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:49:12.78295+08:00 INFO [21697,26322b2bc5481dc2] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:49:12.765941233 +0800 CST m=+10834917.882550766, end time:2024-11-29 09:49:12 +0800 CST, Metrics number 0 2024-11-29T09:49:12.78303+08:00 WARN [21697,26322b2bc5481dc2] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:49:13.46813+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981095, result count: 26 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981095, max_request_id: 274981120 2024-11-29T09:49:13.87576+08:00 INFO [21697,4168e49388d64409] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=4168e49388d64409, body=] 2024-11-29T09:49:13.87593+08:00 WARN [21697,4168e49388d64409] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:13.87596+08:00 WARN [21697,4168e49388d64409] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:13.87597+08:00 WARN [21697,4168e49388d64409] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:13.87599+08:00 WARN [21697,4168e49388d64409] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:13.876+08:00 WARN [21697,4168e49388d64409] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:13.88009+08:00 INFO [21697,4168e49388d64409] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=4168e49388d64409, body=] 2024-11-29T09:49:14.47048+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981121, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981121, max_request_id: 274981161 2024-11-29T09:49:15.04132+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:49:15.09862+08:00 INFO [21697,6f883e6d8797416b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=6f883e6d8797416b, body=] 2024-11-29T09:49:15.09905+08:00 WARN [21697,6f883e6d8797416b] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:15.09909+08:00 WARN [21697,6f883e6d8797416b] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:15.09911+08:00 WARN [21697,6f883e6d8797416b] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:15.09913+08:00 WARN [21697,6f883e6d8797416b] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:15.09915+08:00 WARN [21697,6f883e6d8797416b] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:15.09957+08:00 INFO [21697,6f883e6d8797416b] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:49:15.1+08:00 INFO [21697,6f883e6d8797416b] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=6f883e6d8797416b, body=] 2024-11-29T09:49:15.12729+08:00 INFO [21697,6f883e6d8797416b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=6f883e6d8797416b, body=] 2024-11-29T09:49:15.13808+08:00 INFO [21697,6f883e6d8797416b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=6f883e6d8797416b, body=] 2024-11-29T09:49:15.13828+08:00 INFO [21697,6f883e6d8797416b] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=6f883e6d8797416b, body=] 2024-11-29T09:49:15.13843+08:00 WARN [21697,6f883e6d8797416b] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:49:15.14989+08:00 WARN [21697,6f883e6d8797416b] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:49:15.15592+08:00 INFO [21697,6f883e6d8797416b] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:49:15.47231+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981162, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981162, max_request_id: 274981196 2024-11-29T09:49:16.47485+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981197, result count: 55 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981197, max_request_id: 274981251 2024-11-29T09:49:16.8766+08:00 INFO [21697,80e98c555e294c40] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=80e98c555e294c40, body=] 2024-11-29T09:49:16.87684+08:00 WARN [21697,80e98c555e294c40] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:16.87687+08:00 WARN [21697,80e98c555e294c40] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:16.87689+08:00 WARN [21697,80e98c555e294c40] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:16.8769+08:00 WARN [21697,80e98c555e294c40] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:16.87691+08:00 WARN [21697,80e98c555e294c40] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:16.88065+08:00 INFO [21697,80e98c555e294c40] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=80e98c555e294c40, body=] 2024-11-29T09:49:17.09663+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:49:17.47702+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981252, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981252, max_request_id: 274981292 2024-11-29T09:49:17.78207+08:00 INFO [21697,2413ade57b3050dd] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:49:17.78214+08:00 INFO [21697,2413ade57b3050dd] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:49:17.765539192 +0800 CST m=+10834922.882148725, end time:2024-11-29 09:49:17 +0800 CST, Metrics number 0 2024-11-29T09:49:17.78223+08:00 WARN [21697,2413ade57b3050dd] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:49:18.47901+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981293, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981293, max_request_id: 274981325 2024-11-29T09:49:19.48081+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981326, result count: 30 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981326, max_request_id: 274981355 2024-11-29T09:49:19.4809+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:49:19.48096+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:49:19.55602+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:49:19.87515+08:00 INFO [21697,922d452b96d94eff] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=922d452b96d94eff, body=] 2024-11-29T09:49:19.87534+08:00 WARN [21697,922d452b96d94eff] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:19.87537+08:00 WARN [21697,922d452b96d94eff] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:19.87539+08:00 WARN [21697,922d452b96d94eff] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:19.8754+08:00 WARN [21697,922d452b96d94eff] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:19.87541+08:00 WARN [21697,922d452b96d94eff] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:19.8791+08:00 INFO [21697,922d452b96d94eff] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=922d452b96d94eff, body=] 2024-11-29T09:49:20.04104+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:49:20.48247+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981356, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981356, max_request_id: 274981391 2024-11-29T09:49:21.2005+08:00 INFO [21697,8274bfbe2b5383b5] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=8274bfbe2b5383b5, body=] 2024-11-29T09:49:21.48463+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981392, result count: 71 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981392, max_request_id: 274981462 2024-11-29T09:49:21.74877+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:49:22.10864+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:49:22.31271+08:00 INFO [21697,ebb83255c0862e48] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=ebb83255c0862e48, body=] 2024-11-29T09:49:22.31305+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:49:22.31339+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:49:22.48658+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981463, result count: 30 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981463, max_request_id: 274981492 2024-11-29T09:49:22.78138+08:00 INFO [21697,9b7eab4eb126e14c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:49:22.78146+08:00 INFO [21697,9b7eab4eb126e14c] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:49:22.765179076 +0800 CST m=+10834927.881788608, end time:2024-11-29 09:49:22 +0800 CST, Metrics number 0 2024-11-29T09:49:22.78158+08:00 WARN [21697,9b7eab4eb126e14c] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:49:22.87483+08:00 INFO [21697,acb84ea90c8442e5] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=acb84ea90c8442e5, body=] 2024-11-29T09:49:22.87516+08:00 WARN [21697,acb84ea90c8442e5] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:22.87519+08:00 WARN [21697,acb84ea90c8442e5] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:22.87521+08:00 WARN [21697,acb84ea90c8442e5] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:22.87523+08:00 WARN [21697,acb84ea90c8442e5] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:22.87525+08:00 WARN [21697,acb84ea90c8442e5] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:22.87987+08:00 INFO [21697,acb84ea90c8442e5] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=acb84ea90c8442e5, body=] 2024-11-29T09:49:23.48869+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981493, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981493, max_request_id: 274981528 2024-11-29T09:49:24.49035+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981529, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981529, max_request_id: 274981559 2024-11-29T09:49:24.87207+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163131565, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163131901, max_request_id: 163131903 2024-11-29T09:49:24.87213+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163131901 is larger than startRequestId 163131565, maybe lost 336 data 2024-11-29T09:49:25.03034+08:00 INFO [21697,20235f2bb2c9efcc] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=20235f2bb2c9efcc, body=] 2024-11-29T09:49:25.03063+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:49:25.0416+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:49:25.1478+08:00 INFO [21697,1bb39774e72309e5] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=1bb39774e72309e5, body=] 2024-11-29T09:49:25.1483+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:25.14835+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:25.14839+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:25.1484+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:25.14842+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:25.14901+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:49:25.49217+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981560, result count: 30 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981560, max_request_id: 274981589 2024-11-29T09:49:25.78915+08:00 INFO [21697,c51388f735145341] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=c51388f735145341, body=] 2024-11-29T09:49:25.87505+08:00 INFO [21697,3ed32fa037234b49] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=3ed32fa037234b49, body=] 2024-11-29T09:49:25.87525+08:00 WARN [21697,3ed32fa037234b49] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:25.87529+08:00 WARN [21697,3ed32fa037234b49] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:25.8753+08:00 WARN [21697,3ed32fa037234b49] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:25.87531+08:00 WARN [21697,3ed32fa037234b49] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:25.87533+08:00 WARN [21697,3ed32fa037234b49] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:25.8791+08:00 INFO [21697,3ed32fa037234b49] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=3ed32fa037234b49, body=] 2024-11-29T09:49:26.49423+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981590, result count: 54 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981590, max_request_id: 274981643 2024-11-29T09:49:27.0956+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:49:27.49702+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981644, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981644, max_request_id: 274981682 2024-11-29T09:49:27.78326+08:00 INFO [21697,349db7f901d1eead] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:49:27.78333+08:00 INFO [21697,349db7f901d1eead] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:49:27.765953688 +0800 CST m=+10834932.882563226, end time:2024-11-29 09:49:27 +0800 CST, Metrics number 0 2024-11-29T09:49:27.78353+08:00 WARN [21697,349db7f901d1eead] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:49:28.49927+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981683, result count: 27 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981683, max_request_id: 274981709 2024-11-29T09:49:28.87653+08:00 INFO [21697,2ccc09d5bf454f78] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=2ccc09d5bf454f78, body=] 2024-11-29T09:49:28.87679+08:00 WARN [21697,2ccc09d5bf454f78] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:28.87684+08:00 WARN [21697,2ccc09d5bf454f78] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:28.87688+08:00 WARN [21697,2ccc09d5bf454f78] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:28.87689+08:00 WARN [21697,2ccc09d5bf454f78] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:28.87691+08:00 WARN [21697,2ccc09d5bf454f78] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:28.88086+08:00 INFO [21697,2ccc09d5bf454f78] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=2ccc09d5bf454f78, body=] 2024-11-29T09:49:29.50134+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981710, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981710, max_request_id: 274981748 2024-11-29T09:49:30.04187+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:49:30.09824+08:00 INFO [21697,3e283f1bce5042a8] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=3e283f1bce5042a8, body=] 2024-11-29T09:49:30.09867+08:00 WARN [21697,3e283f1bce5042a8] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:30.0987+08:00 WARN [21697,3e283f1bce5042a8] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:30.09872+08:00 WARN [21697,3e283f1bce5042a8] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:30.09873+08:00 WARN [21697,3e283f1bce5042a8] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:30.09874+08:00 WARN [21697,3e283f1bce5042a8] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:30.09909+08:00 INFO [21697,3e283f1bce5042a8] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:49:30.09946+08:00 INFO [21697,3e283f1bce5042a8] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=3e283f1bce5042a8, body=] 2024-11-29T09:49:30.11242+08:00 INFO [21697,3e283f1bce5042a8] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=3e283f1bce5042a8, body=] 2024-11-29T09:49:30.11547+08:00 INFO [21697,3e283f1bce5042a8] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=3e283f1bce5042a8, body=] 2024-11-29T09:49:30.1156+08:00 INFO [21697,3e283f1bce5042a8] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=3e283f1bce5042a8, body=] 2024-11-29T09:49:30.11571+08:00 WARN [21697,3e283f1bce5042a8] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:49:30.11721+08:00 WARN [21697,3e283f1bce5042a8] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:49:30.11911+08:00 INFO [21697,3e283f1bce5042a8] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:49:30.50419+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981749, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981749, max_request_id: 274981786 2024-11-29T09:49:30.87841+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163131904, result count: 46 auditMsg number: 0, diagMsg number: 0, min_request_id: 163132079, max_request_id: 163132141 2024-11-29T09:49:30.87846+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163132079 is larger than startRequestId 163131904, maybe lost 175 data 2024-11-29T09:49:31.33535+08:00 INFO [21697,5fe43397858fee4a] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.98.86, ocpServerIp=, traceId=5fe43397858fee4a, body=] 2024-11-29T09:49:31.33556+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:31.3356+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:31.33561+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:31.33563+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:31.33564+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:31.50778+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981787, result count: 167 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981787, max_request_id: 274981953 2024-11-29T09:49:31.74365+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:49:31.87458+08:00 INFO [21697,727931a6b811479b] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=727931a6b811479b, body=] 2024-11-29T09:49:31.8748+08:00 WARN [21697,727931a6b811479b] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:31.87484+08:00 WARN [21697,727931a6b811479b] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:31.87486+08:00 WARN [21697,727931a6b811479b] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:31.87487+08:00 WARN [21697,727931a6b811479b] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:31.87488+08:00 WARN [21697,727931a6b811479b] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:31.88267+08:00 INFO [21697,727931a6b811479b] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=727931a6b811479b, body=] 2024-11-29T09:49:32.09444+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:49:32.5102+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274981954, result count: 72 auditMsg number: 0, diagMsg number: 0, min_request_id: 274981954, max_request_id: 274982025 2024-11-29T09:49:32.78195+08:00 INFO [21697,b4c3cb5156e218b8] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 1 result 2024-11-29T09:49:32.78202+08:00 INFO [21697,b4c3cb5156e218b8] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:49:32.765403037 +0800 CST m=+10834937.882012569, end time:2024-11-29 09:49:32 +0800 CST, Metrics number 1 2024-11-29T09:49:32.78211+08:00 INFO [21697,b4c3cb5156e218b8] caller=engine/pipeline.go:145:pipelinePush: push pipeline ob_sql_transaction metrics count 1 2024-11-29T09:49:32.87069+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 1 result 2024-11-29T09:49:32.87204+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:49:32.8721+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:49:32.87213+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:49:32.87215+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:49:32.87218+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:49:32.87221+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:49:32.87223+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:49:32.87225+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:49:32.87227+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:49:32.87229+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 1 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:49:32.8723+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:49:32.87582+08:00 INFO [21697,950b8b41ca2b98a3] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:49:32 +0800 CST, start_request_id:0 2024-11-29T09:49:32.87587+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:49:32.87602+08:00 INFO [21697,950b8b41ca2b98a3] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:49:32 +0800 CST, start_request_id:0 2024-11-29T09:49:32.87604+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:49:32.87618+08:00 INFO [21697,950b8b41ca2b98a3] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:49:32 +0800 CST, start_request_id:544411 2024-11-29T09:49:32.87621+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544411, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:49:32.8871+08:00 INFO [21697,950b8b41ca2b98a3] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:49:32 +0800 CST, start_request_id:0 2024-11-29T09:49:32.88717+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:49:32.88728+08:00 INFO [21697,950b8b41ca2b98a3] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:49:32 +0800 CST, start_request_id:7154 2024-11-29T09:49:32.8873+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:49:32.88742+08:00 INFO [21697,950b8b41ca2b98a3] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:49:32 +0800 CST, start_request_id:0 2024-11-29T09:49:32.88744+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:49:32.89556+08:00 INFO [21697,950b8b41ca2b98a3] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:49:32 +0800 CST, start_request_id:0 2024-11-29T09:49:32.89563+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:49:32.89574+08:00 INFO [21697,950b8b41ca2b98a3] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:49:32 +0800 CST, start_request_id:387 2024-11-29T09:49:32.89576+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:49:32.90199+08:00 INFO [21697,950b8b41ca2b98a3] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:49:32 +0800 CST, start_request_id:0 2024-11-29T09:49:32.90203+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:49:32.91343+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:49:32.91349+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:49:32.91351+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:49:32.914+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907606, got 4 result 2024-11-29T09:49:32.91437+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:49:32.91439+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:49:32.91441+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 4, planExplainMetrics number 0, planMetaMetrics number 4 2024-11-29T09:49:32.91477+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:49:32.9148+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:49:32.91482+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 4, planExplainMetrics number 0, planMetaMetrics number 4 2024-11-29T09:49:32.91514+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:49:32.91517+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:49:32.91519+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 4, planExplainMetrics number 0, planMetaMetrics number 4 2024-11-29T09:49:32.91551+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:49:32.91554+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:49:32.91556+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 4, planExplainMetrics number 0, planMetaMetrics number 4 2024-11-29T09:49:32.92367+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907606, got 4 result, planStatMetrics number 4, planMetaMetrics number 4, planExplainMetrics number 0 2024-11-29T09:49:32.92373+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 4, planExplainMetrics number 0, planMetaMetrics number 4 2024-11-29T09:49:32.94246+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807471, got 1 result 2024-11-29T09:49:32.9429+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:49:32.94293+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:49:32.94295+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:49:32.94621+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807471, got 1 result, planStatMetrics number 1, planMetaMetrics number 1, planExplainMetrics number 0 2024-11-29T09:49:32.94625+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:49:32.98677+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:49:32.98682+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:49:32.98686+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:49:32.9869+08:00 INFO [21697,db46c8c3445464b0] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:49:32 +0800 CST, end time:2024-11-29 09:49:32 +0800 CST, planMetaMetrics number 5, planExplainMetrics number 0, planStatMetrics number 5 2024-11-29T09:49:33.04849+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:49:33.0486+08:00 INFO [21697,fb9e1f5aa7b6af4e] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:49:33.51089+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 47 2024-11-29T09:49:33.51096+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:49:33.511+08:00 INFO [21697,672754d8c9418fe7] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1433, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 47, sql_audit_progress Metrics number 1 2024-11-29T09:49:33.513+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982026, result count: 74 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982026, max_request_id: 274982099 2024-11-29T09:49:34.48127+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:49:34.48134+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:49:34.51461+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982100, result count: 28 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982100, max_request_id: 274982127 2024-11-29T09:49:34.55142+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:49:34.87622+08:00 INFO [21697,9584af50045a43a8] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=9584af50045a43a8, body=] 2024-11-29T09:49:34.87643+08:00 WARN [21697,9584af50045a43a8] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:34.87646+08:00 WARN [21697,9584af50045a43a8] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:34.87648+08:00 WARN [21697,9584af50045a43a8] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:34.87649+08:00 WARN [21697,9584af50045a43a8] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:34.8765+08:00 WARN [21697,9584af50045a43a8] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:34.88018+08:00 INFO [21697,9584af50045a43a8] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=9584af50045a43a8, body=] 2024-11-29T09:49:35.04203+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:49:35.24579+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:49:35.2459+08:00 INFO [21697,b80f8632f84f62f0] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:49:35.517+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982128, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982128, max_request_id: 274982166 2024-11-29T09:49:36.20041+08:00 INFO [21697,75df65bdbb3ac61c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=75df65bdbb3ac61c, body=] 2024-11-29T09:49:36.45756+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:49:36.45766+08:00 INFO [21697,21e7a07488c12a97] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:49:36.5189+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982167, result count: 68 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982167, max_request_id: 274982234 2024-11-29T09:49:36.88237+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 12 2024-11-29T09:49:36.88243+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:49:36.88247+08:00 INFO [21697,6322a0ffe8904dca] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 52, lost data number: 886, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 12, sql_audit_progress Metrics number 1 2024-11-29T09:49:36.88775+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163132142, result count: 265 auditMsg number: 0, diagMsg number: 0, min_request_id: 163132206, max_request_id: 163132543 2024-11-29T09:49:36.88779+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163132206 is larger than startRequestId 163132142, maybe lost 64 data 2024-11-29T09:49:37.0955+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:49:37.3125+08:00 INFO [21697,462a6311d13bd369] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=462a6311d13bd369, body=] 2024-11-29T09:49:37.31286+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:49:37.31319+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:49:37.52418+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982235, result count: 55 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982235, max_request_id: 274982289 2024-11-29T09:49:37.7837+08:00 INFO [21697,9491b58db91ac0a3] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 1 result 2024-11-29T09:49:37.78377+08:00 INFO [21697,9491b58db91ac0a3] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:49:37.76540209 +0800 CST m=+10834942.882011626, end time:2024-11-29 09:49:37 +0800 CST, Metrics number 1 2024-11-29T09:49:37.78385+08:00 INFO [21697,9491b58db91ac0a3] caller=engine/pipeline.go:145:pipelinePush: push pipeline ob_sql_transaction metrics count 1 2024-11-29T09:49:37.87461+08:00 INFO [21697,b611cddbeb6049f8] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=b611cddbeb6049f8, body=] 2024-11-29T09:49:37.87481+08:00 WARN [21697,b611cddbeb6049f8] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:37.87485+08:00 WARN [21697,b611cddbeb6049f8] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:37.87486+08:00 WARN [21697,b611cddbeb6049f8] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:37.87487+08:00 WARN [21697,b611cddbeb6049f8] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:37.87488+08:00 WARN [21697,b611cddbeb6049f8] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:37.87874+08:00 INFO [21697,b611cddbeb6049f8] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=b611cddbeb6049f8, body=] 2024-11-29T09:49:38.52641+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982290, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982290, max_request_id: 274982325 2024-11-29T09:49:39.52896+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982326, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982326, max_request_id: 274982362 2024-11-29T09:49:40.03059+08:00 INFO [21697,8e7cd42dc4b1da70] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=8e7cd42dc4b1da70, body=] 2024-11-29T09:49:40.03088+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:49:40.04141+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:49:40.14675+08:00 INFO [21697,0228a6ea0f5e48a6] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=0228a6ea0f5e48a6, body=] 2024-11-29T09:49:40.14713+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:40.14717+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:40.14719+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:40.1472+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:40.14721+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:40.14759+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:49:40.53039+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982363, result count: 27 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982363, max_request_id: 274982389 2024-11-29T09:49:40.78872+08:00 INFO [21697,ffd5c0117c2842db] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=ffd5c0117c2842db, body=] 2024-11-29T09:49:40.87596+08:00 INFO [21697,9e65300f6b8f4cd0] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=9e65300f6b8f4cd0, body=] 2024-11-29T09:49:40.87618+08:00 WARN [21697,9e65300f6b8f4cd0] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:40.87622+08:00 WARN [21697,9e65300f6b8f4cd0] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:40.87623+08:00 WARN [21697,9e65300f6b8f4cd0] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:40.87625+08:00 WARN [21697,9e65300f6b8f4cd0] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:40.87626+08:00 WARN [21697,9e65300f6b8f4cd0] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:40.87828+08:00 INFO [21697,9e65300f6b8f4cd0] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=9e65300f6b8f4cd0, body=] 2024-11-29T09:49:41.53292+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982390, result count: 72 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982390, max_request_id: 274982461 2024-11-29T09:49:41.74401+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:49:42.09429+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:49:42.53536+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982462, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982462, max_request_id: 274982497 2024-11-29T09:49:42.78189+08:00 INFO [21697,130ebeeb938aa08e] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:49:42.78195+08:00 INFO [21697,130ebeeb938aa08e] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:49:42.765627279 +0800 CST m=+10834947.882236811, end time:2024-11-29 09:49:42 +0800 CST, Metrics number 0 2024-11-29T09:49:42.78204+08:00 WARN [21697,130ebeeb938aa08e] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:49:43.53838+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982498, result count: 28 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982498, max_request_id: 274982525 2024-11-29T09:49:43.87415+08:00 INFO [21697,da04bbd46dc345c3] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=da04bbd46dc345c3, body=] 2024-11-29T09:49:43.87432+08:00 WARN [21697,da04bbd46dc345c3] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:43.87436+08:00 WARN [21697,da04bbd46dc345c3] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:43.87437+08:00 WARN [21697,da04bbd46dc345c3] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:43.87438+08:00 WARN [21697,da04bbd46dc345c3] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:43.87439+08:00 WARN [21697,da04bbd46dc345c3] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:43.87809+08:00 INFO [21697,da04bbd46dc345c3] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=da04bbd46dc345c3, body=] 2024-11-29T09:49:44.54056+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982526, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982526, max_request_id: 274982562 2024-11-29T09:49:45.04171+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:49:45.09776+08:00 INFO [21697,1cef435f4861457c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=1cef435f4861457c, body=] 2024-11-29T09:49:45.09798+08:00 INFO [21697,1cef435f4861457c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=1cef435f4861457c, body=] 2024-11-29T09:49:45.10725+08:00 WARN [21697,1cef435f4861457c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:45.10731+08:00 WARN [21697,1cef435f4861457c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:45.10733+08:00 WARN [21697,1cef435f4861457c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:45.10734+08:00 WARN [21697,1cef435f4861457c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:45.10735+08:00 WARN [21697,1cef435f4861457c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:45.10983+08:00 INFO [21697,1cef435f4861457c] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:49:45.11332+08:00 INFO [21697,1cef435f4861457c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=1cef435f4861457c, body=] 2024-11-29T09:49:45.11498+08:00 INFO [21697,1cef435f4861457c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=1cef435f4861457c, body=] 2024-11-29T09:49:45.11511+08:00 INFO [21697,1cef435f4861457c] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=1cef435f4861457c, body=] 2024-11-29T09:49:45.11522+08:00 WARN [21697,1cef435f4861457c] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:49:45.11733+08:00 WARN [21697,1cef435f4861457c] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:49:45.12484+08:00 INFO [21697,1cef435f4861457c] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:49:45.54272+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982563, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982563, max_request_id: 274982599 2024-11-29T09:49:46.54541+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982600, result count: 54 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982600, max_request_id: 274982653 2024-11-29T09:49:46.87459+08:00 INFO [21697,16efcb9279e64910] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=16efcb9279e64910, body=] 2024-11-29T09:49:46.87498+08:00 WARN [21697,16efcb9279e64910] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:46.87502+08:00 WARN [21697,16efcb9279e64910] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:46.87504+08:00 WARN [21697,16efcb9279e64910] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:46.87505+08:00 WARN [21697,16efcb9279e64910] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:46.87508+08:00 WARN [21697,16efcb9279e64910] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:46.87975+08:00 INFO [21697,16efcb9279e64910] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=16efcb9279e64910, body=] 2024-11-29T09:49:47.09499+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:49:47.54771+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982654, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982654, max_request_id: 274982694 2024-11-29T09:49:47.78173+08:00 INFO [21697,5c59396f31d21410] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:49:47.78179+08:00 INFO [21697,5c59396f31d21410] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:49:47.765547618 +0800 CST m=+10834952.882157153, end time:2024-11-29 09:49:47 +0800 CST, Metrics number 0 2024-11-29T09:49:47.78187+08:00 WARN [21697,5c59396f31d21410] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:49:48.31223+08:00 INFO [21697,372ed7e9a8f5091d] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.98.86, ocpServerIp=, traceId=372ed7e9a8f5091d, body=] 2024-11-29T09:49:48.54935+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982695, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982695, max_request_id: 274982733 2024-11-29T09:49:49.48161+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:49:49.48171+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:49:49.55136+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982734, result count: 30 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982734, max_request_id: 274982763 2024-11-29T09:49:49.55275+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields: error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:49:49.87589+08:00 INFO [21697,cefa8188cfad424b] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=cefa8188cfad424b, body=] 2024-11-29T09:49:49.87611+08:00 WARN [21697,cefa8188cfad424b] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:49.87614+08:00 WARN [21697,cefa8188cfad424b] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:49.87617+08:00 WARN [21697,cefa8188cfad424b] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:49.87618+08:00 WARN [21697,cefa8188cfad424b] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:49.8762+08:00 WARN [21697,cefa8188cfad424b] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:49.88027+08:00 INFO [21697,cefa8188cfad424b] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=cefa8188cfad424b, body=] 2024-11-29T09:49:50.04205+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:49:50.55401+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982764, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982764, max_request_id: 274982803 2024-11-29T09:49:51.2001+08:00 INFO [21697,1eb813da5fdd9b4a] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=1eb813da5fdd9b4a, body=] 2024-11-29T09:49:51.55689+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982804, result count: 70 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982804, max_request_id: 274982873 2024-11-29T09:49:51.74744+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:49:52.10924+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:49:52.31194+08:00 INFO [21697,a08723f5603a28da] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=a08723f5603a28da, body=] 2024-11-29T09:49:52.31243+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:49:52.31293+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:49:52.38738+08:00 INFO [21697,ed85e7e5ed336283] caller=host/custom.go:178:func10: collect observer dir exists info 2024-11-29T09:49:52.55894+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982874, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982874, max_request_id: 274982908 2024-11-29T09:49:52.78153+08:00 INFO [21697,ec5b385036d75322] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:49:52.78159+08:00 INFO [21697,ec5b385036d75322] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:49:52.765319324 +0800 CST m=+10834957.881928859, end time:2024-11-29 09:49:52 +0800 CST, Metrics number 0 2024-11-29T09:49:52.78167+08:00 WARN [21697,ec5b385036d75322] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:49:52.87865+08:00 INFO [21697,faff37b6af044fda] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=faff37b6af044fda, body=] 2024-11-29T09:49:52.87998+08:00 INFO [21697,faff37b6af044fda] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=faff37b6af044fda, body=] 2024-11-29T09:49:52.88014+08:00 WARN [21697,faff37b6af044fda] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:52.88017+08:00 WARN [21697,faff37b6af044fda] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:52.88018+08:00 WARN [21697,faff37b6af044fda] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:52.88019+08:00 WARN [21697,faff37b6af044fda] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:52.8802+08:00 WARN [21697,faff37b6af044fda] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:52.89573+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163132544, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163132872, max_request_id: 163132874 2024-11-29T09:49:52.89576+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163132872 is larger than startRequestId 163132544, maybe lost 328 data 2024-11-29T09:49:53.56188+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982909, result count: 61 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982909, max_request_id: 274982969 2024-11-29T09:49:54.56369+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274982970, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274982970, max_request_id: 274983006 2024-11-29T09:49:54.97048+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:255:WatchFile: get file change events fields:, start="2024-11-29 09:49:54 +0800 CST", end="2024-11-29 09:49:54.970474561 +0800 CST m=+10834960.087084112", queueTailFileStat.ino=1074285550 2024-11-29T09:49:54.97082+08:00 INFO [21697,] caller=logtailer/log_tailer_utils.go:210:getLogsWithinTime: invoke FindFilesByRegexAndMTime fields:, dir=/opt/myoceanbase/oceanbase/log, fileName=observer.log, start="2024-11-29 09:49:54 +0800 CST", end="2024-11-29 09:49:54.970474561 +0800 CST m=+10834960.087084112", matchedFiles="[0xc000c8c1a0 0xc006392b60]" 2024-11-29T09:49:54.97085+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:336:getWatchedNewLogs: getLogsWithinTime match file fields:, matchedFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log.20241129094954700 2024-11-29T09:49:54.97088+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:336:getWatchedNewLogs: getLogsWithinTime match file fields:, matchedFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log 2024-11-29T09:49:55.03038+08:00 INFO [21697,7e6248fab2f94e7e] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=7e6248fab2f94e7e, body=] 2024-11-29T09:49:55.03064+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:49:55.04106+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:49:55.14677+08:00 INFO [21697,7687bc15cd31a6b0] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=7687bc15cd31a6b0, body=] 2024-11-29T09:49:55.14712+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:55.14716+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:55.14717+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:55.14718+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:55.14719+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:55.14753+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:49:55.21771+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:255:WatchFile: get file change events fields: end="2024-11-29 09:49:55.217707022 +0800 CST m=+10834960.334316579", queueTailFileStat.ino=1074285550, start="2024-11-29 09:49:54 +0800 CST" 2024-11-29T09:49:55.21806+08:00 INFO [21697,] caller=logtailer/log_tailer_utils.go:210:getLogsWithinTime: invoke FindFilesByRegexAndMTime fields:, end="2024-11-29 09:49:55.217707022 +0800 CST m=+10834960.334316579", dir=/opt/myoceanbase/oceanbase/log, fileName=observer.log, start="2024-11-29 09:49:54 +0800 CST", matchedFiles="[0xc00b4a9e10 0xc000b64ea0]" 2024-11-29T09:49:55.21809+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:336:getWatchedNewLogs: getLogsWithinTime match file fields:, matchedFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log.20241129094954700 2024-11-29T09:49:55.21811+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:336:getWatchedNewLogs: getLogsWithinTime match file fields: matchedFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log 2024-11-29T09:49:55.21929+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:255:WatchFile: get file change events fields: end="2024-11-29 09:49:55.219289673 +0800 CST m=+10834960.335899206", queueTailFileStat.ino=1074285550, start="2024-11-29 09:49:54 +0800 CST" 2024-11-29T09:49:55.21953+08:00 INFO [21697,] caller=logtailer/log_tailer_utils.go:210:getLogsWithinTime: invoke FindFilesByRegexAndMTime fields: end="2024-11-29 09:49:55.219289673 +0800 CST m=+10834960.335899206", matchedFiles="[0xc0039fe340 0xc002cec270]", dir=/opt/myoceanbase/oceanbase/log, fileName=observer.log, start="2024-11-29 09:49:54 +0800 CST" 2024-11-29T09:49:55.21956+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:336:getWatchedNewLogs: getLogsWithinTime match file fields:, matchedFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log.20241129094954700 2024-11-29T09:49:55.21958+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:336:getWatchedNewLogs: getLogsWithinTime match file fields:, matchedFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log 2024-11-29T09:49:55.39067+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:384:func1: pop head from fileProcessQueue fields:, tailConf="{/opt/myoceanbase/oceanbase/log observer.log 500ms observer ob_light}", logFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log, fileName=/opt/myoceanbase/oceanbase/log/observer.log 2024-11-29T09:49:55.39289+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:384:func1: pop head from fileProcessQueue fields: fileName=/opt/myoceanbase/oceanbase/log/observer.log, tailConf="{/opt/myoceanbase/oceanbase/log observer.log 500ms observer ob}", logFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log 2024-11-29T09:49:55.4216+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:384:func1: pop head from fileProcessQueue fields:, tailConf="{/opt/myoceanbase/oceanbase/log observer.log 500ms observer ob_light}", logFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log, fileName=/opt/myoceanbase/oceanbase/log/observer.log 2024-11-29T09:49:55.56574+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983007, result count: 24 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983007, max_request_id: 274983030 2024-11-29T09:49:55.78798+08:00 INFO [21697,b345909ada723efa] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=b345909ada723efa, body=] 2024-11-29T09:49:55.87408+08:00 INFO [21697,37caa926eb4544f1] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=37caa926eb4544f1, body=] 2024-11-29T09:49:55.87432+08:00 WARN [21697,37caa926eb4544f1] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:55.87436+08:00 WARN [21697,37caa926eb4544f1] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:55.87437+08:00 WARN [21697,37caa926eb4544f1] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:55.87438+08:00 WARN [21697,37caa926eb4544f1] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:55.87439+08:00 WARN [21697,37caa926eb4544f1] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:55.87812+08:00 INFO [21697,37caa926eb4544f1] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=37caa926eb4544f1, body=] 2024-11-29T09:49:56.56842+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983031, result count: 53 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983031, max_request_id: 274983083 2024-11-29T09:49:57.10658+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:49:57.57064+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983084, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983084, max_request_id: 274983122 2024-11-29T09:49:57.78229+08:00 INFO [21697,50f2a38868ef8312] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:49:57.78236+08:00 INFO [21697,50f2a38868ef8312] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:49:57.76594757 +0800 CST m=+10834962.882557138, end time:2024-11-29 09:49:57 +0800 CST, Metrics number 0 2024-11-29T09:49:57.78245+08:00 WARN [21697,50f2a38868ef8312] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:49:58.57214+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983123, result count: 26 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983123, max_request_id: 274983148 2024-11-29T09:49:58.87392+08:00 INFO [21697,9e6b693eddd842a3] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=9e6b693eddd842a3, body=] 2024-11-29T09:49:58.87415+08:00 WARN [21697,9e6b693eddd842a3] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:49:58.87419+08:00 WARN [21697,9e6b693eddd842a3] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:49:58.87422+08:00 WARN [21697,9e6b693eddd842a3] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:49:58.87423+08:00 WARN [21697,9e6b693eddd842a3] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:58.87425+08:00 WARN [21697,9e6b693eddd842a3] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:49:58.87806+08:00 INFO [21697,9e6b693eddd842a3] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=9e6b693eddd842a3, body=] 2024-11-29T09:49:59.57468+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983149, result count: 45 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983149, max_request_id: 274983193 2024-11-29T09:50:00.041+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:50:00.09739+08:00 INFO [21697,e838e619a7ea4b4c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=e838e619a7ea4b4c, body=] 2024-11-29T09:50:00.0976+08:00 INFO [21697,e838e619a7ea4b4c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=e838e619a7ea4b4c, body=] 2024-11-29T09:50:00.10622+08:00 WARN [21697,e838e619a7ea4b4c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:00.10628+08:00 WARN [21697,e838e619a7ea4b4c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:00.10631+08:00 WARN [21697,e838e619a7ea4b4c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:00.10632+08:00 WARN [21697,e838e619a7ea4b4c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:00.10633+08:00 WARN [21697,e838e619a7ea4b4c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:00.10872+08:00 INFO [21697,e838e619a7ea4b4c] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:50:00.11233+08:00 INFO [21697,e838e619a7ea4b4c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=e838e619a7ea4b4c, body=] 2024-11-29T09:50:00.11543+08:00 INFO [21697,e838e619a7ea4b4c] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=e838e619a7ea4b4c, body=] 2024-11-29T09:50:00.11558+08:00 WARN [21697,e838e619a7ea4b4c] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:50:00.11569+08:00 INFO [21697,e838e619a7ea4b4c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=e838e619a7ea4b4c, body=] 2024-11-29T09:50:00.11711+08:00 WARN [21697,e838e619a7ea4b4c] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:50:00.11893+08:00 INFO [21697,e838e619a7ea4b4c] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:50:00.57635+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983194, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983194, max_request_id: 274983232 2024-11-29T09:50:00.70462+08:00 INFO [21697,85caf90796940c1b] caller=common/middleware.go:59:func1: API request: [GET /api/v1/log/alarms, client=1.1.1.98, ocpServerIp=1.1.1.98, traceId=85caf90796940c1b, body=] 2024-11-29T09:50:00.70476+08:00 INFO [21697,] caller=alarmscache/alarm_handler.go:73:getPipelieGroupReader: get alarms, result size:0, cache list size:4 fields:, uri=/api/v1/log/alarms 2024-11-29T09:50:01.57967+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983233, result count: 142 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983233, max_request_id: 274983374 2024-11-29T09:50:01.74367+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:50:01.87419+08:00 INFO [21697,20c919d6341647ee] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=20c919d6341647ee, body=] 2024-11-29T09:50:01.87447+08:00 WARN [21697,20c919d6341647ee] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:01.87451+08:00 WARN [21697,20c919d6341647ee] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:01.87453+08:00 WARN [21697,20c919d6341647ee] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:01.87454+08:00 WARN [21697,20c919d6341647ee] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:01.87455+08:00 WARN [21697,20c919d6341647ee] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:01.87973+08:00 INFO [21697,20c919d6341647ee] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=20c919d6341647ee, body=] 2024-11-29T09:50:02.10831+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:50:02.58259+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983375, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983375, max_request_id: 274983415 2024-11-29T09:50:02.7424+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:245:updateMetaInfo: successfully init sqlAudit tenant config 2024-11-29T09:50:02.78238+08:00 INFO [21697,f04481e909d10a30] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:50:02.78244+08:00 INFO [21697,f04481e909d10a30] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:50:02.765821058 +0800 CST m=+10834967.882430591, end time:2024-11-29 09:50:02 +0800 CST, Metrics number 0 2024-11-29T09:50:02.78253+08:00 WARN [21697,f04481e909d10a30] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:50:02.85567+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:50:02.85709+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:50:02.85716+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:50:02.8572+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:50:02.85725+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:50:02.85728+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:50:02.85731+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:50:02.85734+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:50:02.85737+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:50:02.85741+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:50:02.85744+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:50:02.85746+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:50:02.86796+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:796:refreshTenants: configured tenants: map[1:sys 1002:ocp 1004:dbamonitor 1006:dbmanager 1008:datatransfer] 2024-11-29T09:50:02.87245+08:00 INFO [21697,915d51ae2def2c11] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:50:02 +0800 CST, start_request_id:0 2024-11-29T09:50:02.87251+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:02.8726+08:00 INFO [21697,915d51ae2def2c11] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:50:02 +0800 CST, start_request_id:0 2024-11-29T09:50:02.87262+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:02.87281+08:00 INFO [21697,915d51ae2def2c11] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:50:02 +0800 CST, start_request_id:0 2024-11-29T09:50:02.87283+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:02.88781+08:00 INFO [21697,915d51ae2def2c11] caller=oceanbase/sql_plan_monitor.go:385:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1, collect time:2024-11-29 09:50:02 +0800 CST, start_request_id:544411, min_request_id:544412, max_request_id:544422 2024-11-29T09:50:02.88794+08:00 INFO [21697,915d51ae2def2c11] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:50:02 +0800 CST, start_request_id:0 2024-11-29T09:50:02.88798+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:02.88808+08:00 INFO [21697,915d51ae2def2c11] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:50:02 +0800 CST, start_request_id:7154 2024-11-29T09:50:02.88811+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:02.88819+08:00 INFO [21697,915d51ae2def2c11] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:50:02 +0800 CST, start_request_id:387 2024-11-29T09:50:02.88821+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:02.88878+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544411, min_request_id: 544412, max_request_id: 544422, collected metrics number: 11 2024-11-29T09:50:02.89191+08:00 INFO [21697,915d51ae2def2c11] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 2 complete, collect time:2024-11-29 09:50:02 +0800 CST, start_request_id:544422 2024-11-29T09:50:02.89195+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 2, start_request_id: 544422, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:02.89203+08:00 INFO [21697,915d51ae2def2c11] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:50:02 +0800 CST, start_request_id:0 2024-11-29T09:50:02.89206+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:02.90214+08:00 INFO [21697,915d51ae2def2c11] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:50:02 +0800 CST, start_request_id:0 2024-11-29T09:50:02.90218+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:02.90974+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:50:02.90979+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:02.90982+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:50:02.90997+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:50:02.91001+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:02.91013+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:50:02.91016+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:02.91027+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807471, got 0 result 2024-11-29T09:50:02.9103+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807471, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:02.91046+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907607, got 1 result 2024-11-29T09:50:02.91061+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:50:02.91067+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:50:02.9107+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:50:02.91078+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:50:02.91082+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:02.91084+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:50:02.91096+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:50:02.91099+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:02.91104+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:50:02.91142+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907607, got 1 result, planStatMetrics number 1, planMetaMetrics number 1, planExplainMetrics number 0 2024-11-29T09:50:02.91145+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:50:02.9144+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:50:02.91445+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:02.91447+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:50:02.99126+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:50:02.99133+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:02.99136+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:50:02.99142+08:00 INFO [21697,82d72edcf3cb5e37] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:50:02 +0800 CST, end time:2024-11-29 09:50:02 +0800 CST, planMetaMetrics number 1, planExplainMetrics number 0, planStatMetrics number 1 2024-11-29T09:50:03.11421+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:50:03.11432+08:00 INFO [21697,630b88031fa1afb0] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:50:03.58336+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 48 2024-11-29T09:50:03.58343+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:50:03.58348+08:00 INFO [21697,0835cefcfd648299] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1390, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 48, sql_audit_progress Metrics number 1 2024-11-29T09:50:03.59596+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983416, result count: 88 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983416, max_request_id: 274983503 2024-11-29T09:50:03.89964+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 5 2024-11-29T09:50:03.89972+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:50:03.89977+08:00 INFO [21697,e27739bd7c616d8b] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 268, lost data number: 392, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 5, sql_audit_progress Metrics number 1 2024-11-29T09:50:04.48128+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields: coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:50:04.48138+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:50:04.55287+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields: error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:50:04.59838+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983504, result count: 46 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983504, max_request_id: 274983549 2024-11-29T09:50:04.87542+08:00 INFO [21697,5de72510308448bb] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=5de72510308448bb, body=] 2024-11-29T09:50:04.87564+08:00 WARN [21697,5de72510308448bb] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:04.87568+08:00 WARN [21697,5de72510308448bb] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:04.87569+08:00 WARN [21697,5de72510308448bb] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:04.8757+08:00 WARN [21697,5de72510308448bb] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:04.87571+08:00 WARN [21697,5de72510308448bb] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:04.8796+08:00 INFO [21697,5de72510308448bb] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=5de72510308448bb, body=] 2024-11-29T09:50:05.04148+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:50:05.34961+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:50:05.34972+08:00 INFO [21697,ab4d8abbab4e26ff] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:50:05.60171+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983550, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983550, max_request_id: 274983586 2024-11-29T09:50:06.20048+08:00 INFO [21697,06088292e043a674] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=06088292e043a674, body=] 2024-11-29T09:50:06.517+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:50:06.51711+08:00 INFO [21697,83cd86559d6f400a] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:50:06.60498+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983587, result count: 87 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983587, max_request_id: 274983673 2024-11-29T09:50:07.09495+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:50:07.31111+08:00 INFO [21697,4ebf9abd5d3d92fc] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=4ebf9abd5d3d92fc, body=] 2024-11-29T09:50:07.31147+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:50:07.31183+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 610, err: 2024-11-29T09:50:07.60829+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983674, result count: 103 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983674, max_request_id: 274983776 2024-11-29T09:50:07.78144+08:00 INFO [21697,7cdf5c523782dbbb] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:50:07.78151+08:00 INFO [21697,7cdf5c523782dbbb] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:50:07.765125849 +0800 CST m=+10834972.881735407, end time:2024-11-29 09:50:07 +0800 CST, Metrics number 0 2024-11-29T09:50:07.7816+08:00 WARN [21697,7cdf5c523782dbbb] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:50:07.87376+08:00 INFO [21697,b5763086f6344d79] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=b5763086f6344d79, body=] 2024-11-29T09:50:07.87397+08:00 WARN [21697,b5763086f6344d79] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:07.874+08:00 WARN [21697,b5763086f6344d79] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:07.87402+08:00 WARN [21697,b5763086f6344d79] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:07.87403+08:00 WARN [21697,b5763086f6344d79] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:07.87404+08:00 WARN [21697,b5763086f6344d79] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:07.87782+08:00 INFO [21697,b5763086f6344d79] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=b5763086f6344d79, body=] 2024-11-29T09:50:08.60985+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983777, result count: 29 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983777, max_request_id: 274983805 2024-11-29T09:50:08.90742+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163132875, result count: 6 auditMsg number: 0, diagMsg number: 0, min_request_id: 163133242, max_request_id: 163133257 2024-11-29T09:50:08.90748+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163133242 is larger than startRequestId 163132875, maybe lost 367 data 2024-11-29T09:50:09.61201+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983806, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983806, max_request_id: 274983843 2024-11-29T09:50:10.02949+08:00 INFO [21697,f186bf85d4300b2c] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=f186bf85d4300b2c, body=] 2024-11-29T09:50:10.02974+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:50:10.04122+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:50:10.14603+08:00 INFO [21697,3e1183fcb423306f] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=3e1183fcb423306f, body=] 2024-11-29T09:50:10.14641+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:10.14645+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:10.14647+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:10.14648+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:10.14649+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:10.1469+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:50:10.61432+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983844, result count: 34 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983844, max_request_id: 274983877 2024-11-29T09:50:10.78775+08:00 INFO [21697,cb58a3c6e6049dba] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=cb58a3c6e6049dba, body=] 2024-11-29T09:50:10.8826+08:00 INFO [21697,112f625f3a334cf2] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=112f625f3a334cf2, body=] 2024-11-29T09:50:10.88284+08:00 WARN [21697,112f625f3a334cf2] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:10.88288+08:00 WARN [21697,112f625f3a334cf2] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:10.88289+08:00 WARN [21697,112f625f3a334cf2] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:10.8829+08:00 WARN [21697,112f625f3a334cf2] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:10.88291+08:00 WARN [21697,112f625f3a334cf2] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:10.88661+08:00 INFO [21697,112f625f3a334cf2] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=112f625f3a334cf2, body=] 2024-11-29T09:50:11.61652+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983878, result count: 65 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983878, max_request_id: 274983942 2024-11-29T09:50:11.74404+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:50:12.10856+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:50:12.6182+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983943, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983943, max_request_id: 274983978 2024-11-29T09:50:12.78243+08:00 INFO [21697,b774cc3d43537ab6] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:50:12.7825+08:00 INFO [21697,b774cc3d43537ab6] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:50:12.76551161 +0800 CST m=+10834977.882121143, end time:2024-11-29 09:50:12 +0800 CST, Metrics number 0 2024-11-29T09:50:12.78261+08:00 WARN [21697,b774cc3d43537ab6] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:50:13.62063+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274983979, result count: 34 auditMsg number: 0, diagMsg number: 0, min_request_id: 274983979, max_request_id: 274984012 2024-11-29T09:50:13.87358+08:00 INFO [21697,7a66bbee614048c5] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=7a66bbee614048c5, body=] 2024-11-29T09:50:13.8738+08:00 WARN [21697,7a66bbee614048c5] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:13.87384+08:00 WARN [21697,7a66bbee614048c5] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:13.87385+08:00 WARN [21697,7a66bbee614048c5] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:13.87386+08:00 WARN [21697,7a66bbee614048c5] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:13.87388+08:00 WARN [21697,7a66bbee614048c5] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:13.87806+08:00 INFO [21697,7a66bbee614048c5] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=7a66bbee614048c5, body=] 2024-11-29T09:50:14.62409+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984013, result count: 30 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984013, max_request_id: 274984042 2024-11-29T09:50:15.04145+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:50:15.09744+08:00 INFO [21697,9edb40256f0c4f9e] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=9edb40256f0c4f9e, body=] 2024-11-29T09:50:15.09774+08:00 INFO [21697,9edb40256f0c4f9e] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=9edb40256f0c4f9e, body=] 2024-11-29T09:50:15.09832+08:00 WARN [21697,9edb40256f0c4f9e] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:15.09836+08:00 WARN [21697,9edb40256f0c4f9e] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:15.09838+08:00 WARN [21697,9edb40256f0c4f9e] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:15.09839+08:00 WARN [21697,9edb40256f0c4f9e] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:15.09841+08:00 WARN [21697,9edb40256f0c4f9e] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:15.10772+08:00 INFO [21697,9edb40256f0c4f9e] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1506, err: 2024-11-29T09:50:15.11765+08:00 INFO [21697,9edb40256f0c4f9e] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=9edb40256f0c4f9e, body=] 2024-11-29T09:50:15.1217+08:00 INFO [21697,9edb40256f0c4f9e] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=9edb40256f0c4f9e, body=] 2024-11-29T09:50:15.12184+08:00 INFO [21697,9edb40256f0c4f9e] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=9edb40256f0c4f9e, body=] 2024-11-29T09:50:15.12199+08:00 WARN [21697,9edb40256f0c4f9e] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:50:15.12383+08:00 WARN [21697,9edb40256f0c4f9e] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:50:15.12604+08:00 INFO [21697,9edb40256f0c4f9e] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:50:15.62602+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984043, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984043, max_request_id: 274984080 2024-11-29T09:50:16.62892+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984081, result count: 63 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984081, max_request_id: 274984143 2024-11-29T09:50:16.87383+08:00 INFO [21697,5e227ac782624172] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=5e227ac782624172, body=] 2024-11-29T09:50:16.87407+08:00 WARN [21697,5e227ac782624172] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:16.87411+08:00 WARN [21697,5e227ac782624172] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:16.87413+08:00 WARN [21697,5e227ac782624172] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:16.87414+08:00 WARN [21697,5e227ac782624172] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:16.87416+08:00 WARN [21697,5e227ac782624172] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:16.87873+08:00 INFO [21697,5e227ac782624172] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=5e227ac782624172, body=] 2024-11-29T09:50:17.09552+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:50:17.63191+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984144, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984144, max_request_id: 274984178 2024-11-29T09:50:17.78725+08:00 INFO [21697,b4f615972c5f8c9f] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:50:17.78734+08:00 INFO [21697,b4f615972c5f8c9f] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:50:17.765900818 +0800 CST m=+10834982.882510357, end time:2024-11-29 09:50:17 +0800 CST, Metrics number 0 2024-11-29T09:50:17.78746+08:00 WARN [21697,b4f615972c5f8c9f] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:50:18.63558+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984179, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984179, max_request_id: 274984210 2024-11-29T09:50:19.48088+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields: coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:50:19.48096+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:50:19.55215+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:50:19.63817+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984211, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984211, max_request_id: 274984247 2024-11-29T09:50:19.87315+08:00 INFO [21697,3921cb18a89545db] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=3921cb18a89545db, body=] 2024-11-29T09:50:19.87337+08:00 WARN [21697,3921cb18a89545db] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:19.8734+08:00 WARN [21697,3921cb18a89545db] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:19.87342+08:00 WARN [21697,3921cb18a89545db] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:19.87343+08:00 WARN [21697,3921cb18a89545db] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:19.87344+08:00 WARN [21697,3921cb18a89545db] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:19.87739+08:00 INFO [21697,3921cb18a89545db] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=3921cb18a89545db, body=] 2024-11-29T09:50:20.04114+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:50:20.63969+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984248, result count: 29 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984248, max_request_id: 274984276 2024-11-29T09:50:21.19868+08:00 INFO [21697,e1a721872ebfccc3] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=e1a721872ebfccc3, body=] 2024-11-29T09:50:21.64243+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984277, result count: 69 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984277, max_request_id: 274984345 2024-11-29T09:50:21.74368+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:50:22.10888+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:50:22.31074+08:00 INFO [21697,2881523b3c66c0fa] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=2881523b3c66c0fa, body=] 2024-11-29T09:50:22.31107+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:50:22.3114+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 610, err: 2024-11-29T09:50:22.64564+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984346, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984346, max_request_id: 274984384 2024-11-29T09:50:22.783+08:00 INFO [21697,55f9a2b48bbe4f7b] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:50:22.78307+08:00 INFO [21697,55f9a2b48bbe4f7b] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:50:22.764937726 +0800 CST m=+10834987.881547259, end time:2024-11-29 09:50:22 +0800 CST, Metrics number 0 2024-11-29T09:50:22.78315+08:00 WARN [21697,55f9a2b48bbe4f7b] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:50:22.87503+08:00 INFO [21697,7064be8d3bc34710] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=7064be8d3bc34710, body=] 2024-11-29T09:50:22.8753+08:00 WARN [21697,7064be8d3bc34710] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:22.87534+08:00 WARN [21697,7064be8d3bc34710] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:22.87537+08:00 WARN [21697,7064be8d3bc34710] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:22.87538+08:00 WARN [21697,7064be8d3bc34710] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:22.8754+08:00 WARN [21697,7064be8d3bc34710] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:22.8792+08:00 INFO [21697,7064be8d3bc34710] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=7064be8d3bc34710, body=] 2024-11-29T09:50:23.64818+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984385, result count: 29 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984385, max_request_id: 274984413 2024-11-29T09:50:24.65093+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984414, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984414, max_request_id: 274984448 2024-11-29T09:50:24.91782+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163133258, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163133580, max_request_id: 163133582 2024-11-29T09:50:24.91788+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163133580 is larger than startRequestId 163133258, maybe lost 322 data 2024-11-29T09:50:25.02862+08:00 INFO [21697,cddd23918d43fe26] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=cddd23918d43fe26, body=] 2024-11-29T09:50:25.02884+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:50:25.04138+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:50:25.14607+08:00 INFO [21697,52ef51508b98c55b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=52ef51508b98c55b, body=] 2024-11-29T09:50:25.14643+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:25.14647+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:25.14648+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:25.14649+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:25.14651+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:25.1469+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1506, err: 2024-11-29T09:50:25.65353+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984449, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984449, max_request_id: 274984483 2024-11-29T09:50:25.78738+08:00 INFO [21697,f00ce025a46f231d] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=f00ce025a46f231d, body=] 2024-11-29T09:50:25.87328+08:00 INFO [21697,2d634bfdd324475a] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=2d634bfdd324475a, body=] 2024-11-29T09:50:25.87351+08:00 WARN [21697,2d634bfdd324475a] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:25.87356+08:00 WARN [21697,2d634bfdd324475a] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:25.87358+08:00 WARN [21697,2d634bfdd324475a] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:25.87359+08:00 WARN [21697,2d634bfdd324475a] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:25.8736+08:00 WARN [21697,2d634bfdd324475a] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:25.87749+08:00 INFO [21697,2d634bfdd324475a] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=2d634bfdd324475a, body=] 2024-11-29T09:50:26.65759+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984484, result count: 49 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984484, max_request_id: 274984532 2024-11-29T09:50:27.09489+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:50:27.65937+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984533, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984533, max_request_id: 274984573 2024-11-29T09:50:27.78155+08:00 INFO [21697,e0ef83b10b396ca9] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:50:27.78164+08:00 INFO [21697,e0ef83b10b396ca9] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:50:27.765221086 +0800 CST m=+10834992.881830625, end time:2024-11-29 09:50:27 +0800 CST, Metrics number 0 2024-11-29T09:50:27.78173+08:00 WARN [21697,e0ef83b10b396ca9] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:50:28.66117+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984574, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984574, max_request_id: 274984605 2024-11-29T09:50:28.87336+08:00 INFO [21697,a4045c0e4f53482c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=a4045c0e4f53482c, body=] 2024-11-29T09:50:28.87361+08:00 WARN [21697,a4045c0e4f53482c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:28.87366+08:00 WARN [21697,a4045c0e4f53482c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:28.87368+08:00 WARN [21697,a4045c0e4f53482c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:28.8737+08:00 WARN [21697,a4045c0e4f53482c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:28.87372+08:00 WARN [21697,a4045c0e4f53482c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:28.87776+08:00 INFO [21697,a4045c0e4f53482c] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=a4045c0e4f53482c, body=] 2024-11-29T09:50:29.66296+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984606, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984606, max_request_id: 274984637 2024-11-29T09:50:30.04138+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:50:30.09847+08:00 INFO [21697,1ea34bbb15704c93] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=1ea34bbb15704c93, body=] 2024-11-29T09:50:30.10514+08:00 INFO [21697,1ea34bbb15704c93] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=1ea34bbb15704c93, body=] 2024-11-29T09:50:30.1054+08:00 WARN [21697,1ea34bbb15704c93] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:50:30.10572+08:00 INFO [21697,1ea34bbb15704c93] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:50:30.1062+08:00 INFO [21697,1ea34bbb15704c93] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=1ea34bbb15704c93, body=] 2024-11-29T09:50:30.10633+08:00 WARN [21697,1ea34bbb15704c93] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:50:30.10659+08:00 INFO [21697,1ea34bbb15704c93] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=1ea34bbb15704c93, body=] 2024-11-29T09:50:30.145+08:00 INFO [21697,1ea34bbb15704c93] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=1ea34bbb15704c93, body=] 2024-11-29T09:50:30.14539+08:00 WARN [21697,1ea34bbb15704c93] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:30.14543+08:00 WARN [21697,1ea34bbb15704c93] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:30.14545+08:00 WARN [21697,1ea34bbb15704c93] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:30.14547+08:00 WARN [21697,1ea34bbb15704c93] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:30.14548+08:00 WARN [21697,1ea34bbb15704c93] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:30.14602+08:00 INFO [21697,1ea34bbb15704c93] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:50:30.66587+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984638, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984638, max_request_id: 274984678 2024-11-29T09:50:30.92257+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163133583, result count: 66 auditMsg number: 0, diagMsg number: 0, min_request_id: 163133767, max_request_id: 163133856 2024-11-29T09:50:30.92262+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163133767 is larger than startRequestId 163133583, maybe lost 184 data 2024-11-29T09:50:31.33366+08:00 INFO [21697,ccc9614ff887a875] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.98.86, ocpServerIp=, traceId=ccc9614ff887a875, body=] 2024-11-29T09:50:31.33387+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:31.3339+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:31.33391+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:31.33392+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:31.33394+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:31.66871+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984679, result count: 82 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984679, max_request_id: 274984760 2024-11-29T09:50:31.74351+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:50:31.87302+08:00 INFO [21697,70da614290e94481] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=70da614290e94481, body=] 2024-11-29T09:50:31.87325+08:00 WARN [21697,70da614290e94481] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:31.87329+08:00 WARN [21697,70da614290e94481] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:31.8733+08:00 WARN [21697,70da614290e94481] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:31.87331+08:00 WARN [21697,70da614290e94481] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:31.87332+08:00 WARN [21697,70da614290e94481] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:31.87708+08:00 INFO [21697,70da614290e94481] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=70da614290e94481, body=] 2024-11-29T09:50:32.10886+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:50:32.67099+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984761, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984761, max_request_id: 274984792 2024-11-29T09:50:32.78157+08:00 INFO [21697,ccb90985f8f1c2f6] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:50:32.78165+08:00 INFO [21697,ccb90985f8f1c2f6] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:50:32.76518344 +0800 CST m=+10834997.881792972, end time:2024-11-29 09:50:32 +0800 CST, Metrics number 0 2024-11-29T09:50:32.78176+08:00 WARN [21697,ccb90985f8f1c2f6] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:50:32.85665+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:50:32.85819+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:50:32.85826+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:50:32.85829+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:50:32.85831+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:50:32.85834+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:50:32.85836+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:50:32.85839+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:50:32.85841+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:50:32.85846+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:50:32.85849+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:50:32.85851+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:50:32.8699+08:00 INFO [21697,c4164d6cc8ec2212] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:50:32 +0800 CST, start_request_id:0 2024-11-29T09:50:32.86999+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:32.87015+08:00 INFO [21697,c4164d6cc8ec2212] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:50:32 +0800 CST, start_request_id:544422 2024-11-29T09:50:32.87018+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544422, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:32.8704+08:00 INFO [21697,c4164d6cc8ec2212] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:50:32 +0800 CST, start_request_id:0 2024-11-29T09:50:32.87044+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:32.87102+08:00 INFO [21697,c4164d6cc8ec2212] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:50:32 +0800 CST, start_request_id:0 2024-11-29T09:50:32.87105+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:32.87623+08:00 INFO [21697,c4164d6cc8ec2212] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:50:32 +0800 CST, start_request_id:0 2024-11-29T09:50:32.87629+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:32.87637+08:00 INFO [21697,c4164d6cc8ec2212] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:50:32 +0800 CST, start_request_id:0 2024-11-29T09:50:32.87639+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:32.87814+08:00 INFO [21697,c4164d6cc8ec2212] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:50:32 +0800 CST, start_request_id:7154 2024-11-29T09:50:32.87818+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:32.87873+08:00 INFO [21697,c4164d6cc8ec2212] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:50:32 +0800 CST, start_request_id:387 2024-11-29T09:50:32.87876+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:32.88559+08:00 INFO [21697,c4164d6cc8ec2212] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:50:32 +0800 CST, start_request_id:0 2024-11-29T09:50:32.88566+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:50:32.90604+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:50:32.90611+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:32.90614+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:50:32.90625+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:50:32.90627+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:32.90631+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:50:32.9064+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:50:32.90643+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:32.90645+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:50:32.90653+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:50:32.90657+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:32.90661+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:50:32.90804+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:50:32.90807+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:32.9081+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:50:32.9082+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:50:32.90822+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:32.90823+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:50:32.91433+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907608, got 1 result 2024-11-29T09:50:32.91498+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807497, got 26 result 2024-11-29T09:50:32.91548+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907608, got 1 result, planStatMetrics number 1, planMetaMetrics number 1, planExplainMetrics number 0 2024-11-29T09:50:32.91552+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 27, planExplainMetrics number 0, planMetaMetrics number 27 2024-11-29T09:50:32.92888+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807497, got 26 result, planStatMetrics number 26, planMetaMetrics number 26, planExplainMetrics number 0 2024-11-29T09:50:32.92893+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 27, planExplainMetrics number 0, planMetaMetrics number 27 2024-11-29T09:50:32.95948+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:50:32.95956+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:50:32.95959+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 27, planExplainMetrics number 0, planMetaMetrics number 27 2024-11-29T09:50:32.95964+08:00 INFO [21697,a586a89ce6b46a75] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:50:32 +0800 CST, end time:2024-11-29 09:50:32 +0800 CST, planMetaMetrics number 27, planExplainMetrics number 0, planStatMetrics number 27 2024-11-29T09:50:33.22261+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:50:33.22273+08:00 INFO [21697,1ebbc1b6b719cdd1] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:50:33.67206+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 56 2024-11-29T09:50:33.67215+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:50:33.67219+08:00 INFO [21697,d23cb23343e2f88c] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1377, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 56, sql_audit_progress Metrics number 1 2024-11-29T09:50:33.67515+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984793, result count: 118 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984793, max_request_id: 274984910 2024-11-29T09:50:34.48183+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields: coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:50:34.48192+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:50:34.55192+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:50:34.67707+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984911, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984911, max_request_id: 274984949 2024-11-29T09:50:34.87302+08:00 INFO [21697,e8a5351d29bd410b] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=e8a5351d29bd410b, body=] 2024-11-29T09:50:34.87324+08:00 WARN [21697,e8a5351d29bd410b] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:34.87328+08:00 WARN [21697,e8a5351d29bd410b] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:34.8733+08:00 WARN [21697,e8a5351d29bd410b] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:34.87331+08:00 WARN [21697,e8a5351d29bd410b] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:34.87332+08:00 WARN [21697,e8a5351d29bd410b] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:34.87718+08:00 INFO [21697,e8a5351d29bd410b] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=e8a5351d29bd410b, body=] 2024-11-29T09:50:35.04144+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:50:35.51782+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:50:35.51793+08:00 INFO [21697,b36a1108d8261f6f] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:50:35.68578+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984950, result count: 29 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984950, max_request_id: 274984978 2024-11-29T09:50:36.19844+08:00 INFO [21697,f6b853e40d40e97e] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=f6b853e40d40e97e, body=] 2024-11-29T09:50:36.6091+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:50:36.6092+08:00 INFO [21697,2628e3516fce21e9] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:50:36.68863+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274984979, result count: 74 auditMsg number: 0, diagMsg number: 0, min_request_id: 274984979, max_request_id: 274985052 2024-11-29T09:50:36.92437+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 17 2024-11-29T09:50:36.92444+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:50:36.92452+08:00 INFO [21697,15ad2cbd675dcb1c] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 75, lost data number: 873, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 17, sql_audit_progress Metrics number 1 2024-11-29T09:50:36.93392+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163133857, result count: 265 auditMsg number: 0, diagMsg number: 0, min_request_id: 163133921, max_request_id: 163134249 2024-11-29T09:50:36.93397+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163133921 is larger than startRequestId 163133857, maybe lost 64 data 2024-11-29T09:50:37.09474+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:50:37.31058+08:00 INFO [21697,33e3f5c479a00e1c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=33e3f5c479a00e1c, body=] 2024-11-29T09:50:37.31095+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:50:37.31127+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 609, err: 2024-11-29T09:50:37.69197+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985053, result count: 53 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985053, max_request_id: 274985105 2024-11-29T09:50:37.79964+08:00 INFO [21697,9f6379feada6ae89] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 1 result 2024-11-29T09:50:37.79971+08:00 INFO [21697,9f6379feada6ae89] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:50:37.765611323 +0800 CST m=+10835002.882220857, end time:2024-11-29 09:50:37 +0800 CST, Metrics number 1 2024-11-29T09:50:37.79979+08:00 INFO [21697,9f6379feada6ae89] caller=engine/pipeline.go:145:pipelinePush: push pipeline ob_sql_transaction metrics count 1 2024-11-29T09:50:37.87293+08:00 INFO [21697,f5c922633fec4403] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=f5c922633fec4403, body=] 2024-11-29T09:50:37.87315+08:00 WARN [21697,f5c922633fec4403] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:37.87318+08:00 WARN [21697,f5c922633fec4403] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:37.8732+08:00 WARN [21697,f5c922633fec4403] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:37.87322+08:00 WARN [21697,f5c922633fec4403] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:37.87323+08:00 WARN [21697,f5c922633fec4403] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:37.87725+08:00 INFO [21697,f5c922633fec4403] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=f5c922633fec4403, body=] 2024-11-29T09:50:38.69438+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985106, result count: 28 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985106, max_request_id: 274985133 2024-11-29T09:50:39.6962+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985134, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985134, max_request_id: 274985171 2024-11-29T09:50:40.02856+08:00 INFO [21697,37e159fc73f02a7b] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=37e159fc73f02a7b, body=] 2024-11-29T09:50:40.02882+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:50:40.04142+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:50:40.14524+08:00 INFO [21697,c868ae6d681cc7a1] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=c868ae6d681cc7a1, body=] 2024-11-29T09:50:40.14562+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:40.14566+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:40.14568+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:40.1457+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:40.14571+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:40.14607+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:50:40.69794+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985172, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985172, max_request_id: 274985203 2024-11-29T09:50:40.78623+08:00 INFO [21697,2b9881fdeca3b67e] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=2b9881fdeca3b67e, body=] 2024-11-29T09:50:40.87414+08:00 INFO [21697,c426db0815a8406d] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=c426db0815a8406d, body=] 2024-11-29T09:50:40.87438+08:00 WARN [21697,c426db0815a8406d] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:40.87444+08:00 WARN [21697,c426db0815a8406d] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:40.87447+08:00 WARN [21697,c426db0815a8406d] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:40.87448+08:00 WARN [21697,c426db0815a8406d] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:40.8745+08:00 WARN [21697,c426db0815a8406d] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:40.87939+08:00 INFO [21697,c426db0815a8406d] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=c426db0815a8406d, body=] 2024-11-29T09:50:41.701+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985204, result count: 65 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985204, max_request_id: 274985268 2024-11-29T09:50:41.74366+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:50:42.10848+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:50:42.70293+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985269, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985269, max_request_id: 274985306 2024-11-29T09:50:42.78142+08:00 INFO [21697,8224189423268e30] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:50:42.78148+08:00 INFO [21697,8224189423268e30] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:50:42.765074365 +0800 CST m=+10835007.881683898, end time:2024-11-29 09:50:42 +0800 CST, Metrics number 0 2024-11-29T09:50:42.78156+08:00 WARN [21697,8224189423268e30] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:50:43.70554+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985307, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985307, max_request_id: 274985341 2024-11-29T09:50:43.8726+08:00 INFO [21697,414b93f70843406e] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=414b93f70843406e, body=] 2024-11-29T09:50:43.87283+08:00 WARN [21697,414b93f70843406e] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:43.87287+08:00 WARN [21697,414b93f70843406e] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:43.87289+08:00 WARN [21697,414b93f70843406e] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:43.8729+08:00 WARN [21697,414b93f70843406e] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:43.87291+08:00 WARN [21697,414b93f70843406e] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:43.87663+08:00 INFO [21697,414b93f70843406e] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=414b93f70843406e, body=] 2024-11-29T09:50:44.70727+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985342, result count: 30 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985342, max_request_id: 274985371 2024-11-29T09:50:45.04202+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:50:45.09613+08:00 INFO [21697,1fd13759cac042c0] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=1fd13759cac042c0, body=] 2024-11-29T09:50:45.09644+08:00 INFO [21697,1fd13759cac042c0] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=1fd13759cac042c0, body=] 2024-11-29T09:50:45.12736+08:00 WARN [21697,1fd13759cac042c0] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:45.12743+08:00 WARN [21697,1fd13759cac042c0] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:45.12745+08:00 WARN [21697,1fd13759cac042c0] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:45.12748+08:00 WARN [21697,1fd13759cac042c0] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:45.12749+08:00 WARN [21697,1fd13759cac042c0] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:45.12981+08:00 INFO [21697,1fd13759cac042c0] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=1fd13759cac042c0, body=] 2024-11-29T09:50:45.1437+08:00 INFO [21697,1fd13759cac042c0] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=1fd13759cac042c0, body=] 2024-11-29T09:50:45.14388+08:00 INFO [21697,1fd13759cac042c0] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=1fd13759cac042c0, body=] 2024-11-29T09:50:45.14464+08:00 INFO [21697,1fd13759cac042c0] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:50:45.14683+08:00 WARN [21697,1fd13759cac042c0] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:50:45.14697+08:00 WARN [21697,1fd13759cac042c0] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:50:45.15231+08:00 INFO [21697,1fd13759cac042c0] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:50:45.71027+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985372, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985372, max_request_id: 274985410 2024-11-29T09:50:46.71259+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985411, result count: 59 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985411, max_request_id: 274985469 2024-11-29T09:50:46.87264+08:00 INFO [21697,41b4856c954e432b] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=41b4856c954e432b, body=] 2024-11-29T09:50:46.87286+08:00 WARN [21697,41b4856c954e432b] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:46.87289+08:00 WARN [21697,41b4856c954e432b] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:46.87291+08:00 WARN [21697,41b4856c954e432b] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:46.87292+08:00 WARN [21697,41b4856c954e432b] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:46.87293+08:00 WARN [21697,41b4856c954e432b] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:46.87663+08:00 INFO [21697,41b4856c954e432b] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=41b4856c954e432b, body=] 2024-11-29T09:50:47.09536+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:50:47.71811+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985470, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985470, max_request_id: 274985505 2024-11-29T09:50:47.795+08:00 INFO [21697,7f9ab8d9c20cd13b] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:50:47.79506+08:00 INFO [21697,7f9ab8d9c20cd13b] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:50:47.765240032 +0800 CST m=+10835012.881849565, end time:2024-11-29 09:50:47 +0800 CST, Metrics number 0 2024-11-29T09:50:47.79514+08:00 WARN [21697,7f9ab8d9c20cd13b] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:50:47.94231+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163134250, result count: 45 auditMsg number: 0, diagMsg number: 0, min_request_id: 163134487, max_request_id: 163134576 2024-11-29T09:50:47.94236+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163134487 is larger than startRequestId 163134250, maybe lost 237 data 2024-11-29T09:50:48.31071+08:00 INFO [21697,6fb7506688133a62] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.98.86, ocpServerIp=, traceId=6fb7506688133a62, body=] 2024-11-29T09:50:48.72123+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985506, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985506, max_request_id: 274985543 2024-11-29T09:50:48.94444+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163134577, result count: 46 auditMsg number: 0, diagMsg number: 0, min_request_id: 163134579, max_request_id: 163134640 2024-11-29T09:50:48.94449+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163134579 is larger than startRequestId 163134577, maybe lost 2 data 2024-11-29T09:50:49.48093+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:50:49.48103+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:50:49.55138+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields: error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:50:49.72331+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985544, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985544, max_request_id: 274985581 2024-11-29T09:50:49.87283+08:00 INFO [21697,402097867f35432f] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=402097867f35432f, body=] 2024-11-29T09:50:49.87305+08:00 WARN [21697,402097867f35432f] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:49.8731+08:00 WARN [21697,402097867f35432f] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:49.87312+08:00 WARN [21697,402097867f35432f] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:49.87314+08:00 WARN [21697,402097867f35432f] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:49.87315+08:00 WARN [21697,402097867f35432f] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:49.87702+08:00 INFO [21697,402097867f35432f] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=402097867f35432f, body=] 2024-11-29T09:50:49.94597+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163134641, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 163134643, max_request_id: 163134695 2024-11-29T09:50:49.94601+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163134643 is larger than startRequestId 163134641, maybe lost 2 data 2024-11-29T09:50:50.04183+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:50:50.72589+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985582, result count: 34 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985582, max_request_id: 274985615 2024-11-29T09:50:50.95577+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163134696, result count: 45 auditMsg number: 0, diagMsg number: 0, min_request_id: 163134703, max_request_id: 163134766 2024-11-29T09:50:50.95582+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163134703 is larger than startRequestId 163134696, maybe lost 7 data 2024-11-29T09:50:51.19795+08:00 INFO [21697,a89cf8b5ad06d2be] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=a89cf8b5ad06d2be, body=] 2024-11-29T09:50:51.72853+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985616, result count: 71 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985616, max_request_id: 274985686 2024-11-29T09:50:51.74378+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:50:51.95757+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163134767, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 163134768, max_request_id: 163134846 2024-11-29T09:50:51.95762+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163134768 is larger than startRequestId 163134767, maybe lost 1 data 2024-11-29T09:50:52.09449+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:50:52.31038+08:00 INFO [21697,2d11502120067573] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=2d11502120067573, body=] 2024-11-29T09:50:52.31073+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:50:52.31105+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 609, err: 2024-11-29T09:50:52.37897+08:00 INFO [21697,ed85e7e5ed336283] caller=host/custom.go:178:func10: collect observer dir exists info 2024-11-29T09:50:52.7307+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985687, result count: 51 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985687, max_request_id: 274985737 2024-11-29T09:50:52.78219+08:00 INFO [21697,27f0a24447cacc7b] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:50:52.78226+08:00 INFO [21697,27f0a24447cacc7b] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:50:52.765765075 +0800 CST m=+10835017.882374608, end time:2024-11-29 09:50:52 +0800 CST, Metrics number 0 2024-11-29T09:50:52.78234+08:00 WARN [21697,27f0a24447cacc7b] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:50:52.87414+08:00 INFO [21697,3a695a2c76c24cc5] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=3a695a2c76c24cc5, body=] 2024-11-29T09:50:52.87436+08:00 WARN [21697,3a695a2c76c24cc5] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:52.87439+08:00 WARN [21697,3a695a2c76c24cc5] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:52.87442+08:00 WARN [21697,3a695a2c76c24cc5] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:52.87443+08:00 WARN [21697,3a695a2c76c24cc5] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:52.87444+08:00 WARN [21697,3a695a2c76c24cc5] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:52.8781+08:00 INFO [21697,3a695a2c76c24cc5] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=3a695a2c76c24cc5, body=] 2024-11-29T09:50:52.95947+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163134847, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 163134849, max_request_id: 163134903 2024-11-29T09:50:52.95952+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163134849 is larger than startRequestId 163134847, maybe lost 2 data 2024-11-29T09:50:53.73242+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985738, result count: 34 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985738, max_request_id: 274985771 2024-11-29T09:50:53.9613+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163134904, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 163134906, max_request_id: 163134966 2024-11-29T09:50:53.96136+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163134906 is larger than startRequestId 163134904, maybe lost 2 data 2024-11-29T09:50:54.734+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985772, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985772, max_request_id: 274985807 2024-11-29T09:50:54.96294+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163134967, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 163134968, max_request_id: 163135026 2024-11-29T09:50:54.963+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163134968 is larger than startRequestId 163134967, maybe lost 1 data 2024-11-29T09:50:55.03242+08:00 INFO [21697,1746e27b49169841] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=1746e27b49169841, body=] 2024-11-29T09:50:55.03264+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:50:55.04106+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:50:55.14567+08:00 INFO [21697,1ea563b683b00f82] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=1ea563b683b00f82, body=] 2024-11-29T09:50:55.14603+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:55.14607+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:55.14608+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:55.1461+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:55.14611+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:55.14646+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:50:55.74454+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985808, result count: 30 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985808, max_request_id: 274985837 2024-11-29T09:50:55.78639+08:00 INFO [21697,b57a7fa7908fc6e7] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=b57a7fa7908fc6e7, body=] 2024-11-29T09:50:55.87248+08:00 INFO [21697,01bc44878d4949f5] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=01bc44878d4949f5, body=] 2024-11-29T09:50:55.87272+08:00 WARN [21697,01bc44878d4949f5] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:55.87275+08:00 WARN [21697,01bc44878d4949f5] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:55.87277+08:00 WARN [21697,01bc44878d4949f5] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:55.87278+08:00 WARN [21697,01bc44878d4949f5] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:55.8728+08:00 WARN [21697,01bc44878d4949f5] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:55.87671+08:00 INFO [21697,01bc44878d4949f5] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=01bc44878d4949f5, body=] 2024-11-29T09:50:55.96493+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163135027, result count: 48 auditMsg number: 0, diagMsg number: 0, min_request_id: 163135029, max_request_id: 163135088 2024-11-29T09:50:55.96499+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163135029 is larger than startRequestId 163135027, maybe lost 2 data 2024-11-29T09:50:56.75361+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985838, result count: 48 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985838, max_request_id: 274985885 2024-11-29T09:50:56.96721+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163135089, result count: 54 auditMsg number: 0, diagMsg number: 0, min_request_id: 163135091, max_request_id: 163135172 2024-11-29T09:50:56.96726+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163135091 is larger than startRequestId 163135089, maybe lost 2 data 2024-11-29T09:50:57.09472+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:50:57.75571+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985886, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985886, max_request_id: 274985927 2024-11-29T09:50:57.7851+08:00 INFO [21697,731cbc97d84547c4] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:50:57.78516+08:00 INFO [21697,731cbc97d84547c4] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:50:57.765590491 +0800 CST m=+10835022.882200024, end time:2024-11-29 09:50:57 +0800 CST, Metrics number 0 2024-11-29T09:50:57.78537+08:00 WARN [21697,731cbc97d84547c4] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:50:57.96944+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163135173, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 163135174, max_request_id: 163135235 2024-11-29T09:50:57.96951+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163135174 is larger than startRequestId 163135173, maybe lost 1 data 2024-11-29T09:50:58.75761+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985928, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985928, max_request_id: 274985960 2024-11-29T09:50:58.87225+08:00 INFO [21697,02a41c8ae72749ad] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=02a41c8ae72749ad, body=] 2024-11-29T09:50:58.87252+08:00 WARN [21697,02a41c8ae72749ad] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:50:58.87258+08:00 WARN [21697,02a41c8ae72749ad] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:50:58.8726+08:00 WARN [21697,02a41c8ae72749ad] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:50:58.87261+08:00 WARN [21697,02a41c8ae72749ad] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:58.87263+08:00 WARN [21697,02a41c8ae72749ad] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:50:58.87667+08:00 INFO [21697,02a41c8ae72749ad] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=02a41c8ae72749ad, body=] 2024-11-29T09:50:58.97203+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163135236, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 163135239, max_request_id: 163135287 2024-11-29T09:50:58.97209+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163135239 is larger than startRequestId 163135236, maybe lost 3 data 2024-11-29T09:50:59.75947+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985961, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985961, max_request_id: 274985993 2024-11-29T09:50:59.97364+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163135288, result count: 19 auditMsg number: 0, diagMsg number: 0, min_request_id: 163135290, max_request_id: 163135313 2024-11-29T09:50:59.97369+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163135290 is larger than startRequestId 163135288, maybe lost 2 data 2024-11-29T09:51:00.0417+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:51:00.09572+08:00 INFO [21697,6e1fc587eb3b4f59] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=6e1fc587eb3b4f59, body=] 2024-11-29T09:51:00.09594+08:00 INFO [21697,6e1fc587eb3b4f59] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=6e1fc587eb3b4f59, body=] 2024-11-29T09:51:00.10544+08:00 WARN [21697,6e1fc587eb3b4f59] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:00.10551+08:00 WARN [21697,6e1fc587eb3b4f59] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:00.10553+08:00 WARN [21697,6e1fc587eb3b4f59] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:00.10555+08:00 WARN [21697,6e1fc587eb3b4f59] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:00.10557+08:00 WARN [21697,6e1fc587eb3b4f59] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:00.10807+08:00 INFO [21697,6e1fc587eb3b4f59] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1506, err: 2024-11-29T09:51:00.11164+08:00 INFO [21697,6e1fc587eb3b4f59] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=6e1fc587eb3b4f59, body=] 2024-11-29T09:51:00.11531+08:00 INFO [21697,6e1fc587eb3b4f59] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=6e1fc587eb3b4f59, body=] 2024-11-29T09:51:00.11546+08:00 INFO [21697,6e1fc587eb3b4f59] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=6e1fc587eb3b4f59, body=] 2024-11-29T09:51:00.11562+08:00 WARN [21697,6e1fc587eb3b4f59] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:51:00.11722+08:00 WARN [21697,6e1fc587eb3b4f59] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:51:00.11945+08:00 INFO [21697,6e1fc587eb3b4f59] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:51:00.76222+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274985994, result count: 56 auditMsg number: 0, diagMsg number: 0, min_request_id: 274985994, max_request_id: 274986049 2024-11-29T09:51:01.16603+08:00 INFO [21697,d939b47c17b832e6] caller=common/middleware.go:59:func1: API request: [GET /api/v1/log/alarms, client=1.1.1.98, ocpServerIp=1.1.1.98, traceId=d939b47c17b832e6, body=] 2024-11-29T09:51:01.16628+08:00 INFO [21697,] caller=alarmscache/alarm_handler.go:73:getPipelieGroupReader: get alarms, result size:0, cache list size:4 fields: uri=/api/v1/log/alarms 2024-11-29T09:51:01.74391+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:51:01.76633+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986050, result count: 122 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986050, max_request_id: 274986171 2024-11-29T09:51:01.87213+08:00 INFO [21697,8155640559b64873] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=8155640559b64873, body=] 2024-11-29T09:51:01.87241+08:00 WARN [21697,8155640559b64873] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:01.87245+08:00 WARN [21697,8155640559b64873] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:01.87248+08:00 WARN [21697,8155640559b64873] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:01.8725+08:00 WARN [21697,8155640559b64873] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:01.87252+08:00 WARN [21697,8155640559b64873] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:01.87519+08:00 INFO [21697,8155640559b64873] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=8155640559b64873, body=] 2024-11-29T09:51:02.10912+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:51:02.74269+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:245:updateMetaInfo: successfully init sqlAudit tenant config 2024-11-29T09:51:02.7698+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986172, result count: 34 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986172, max_request_id: 274986205 2024-11-29T09:51:02.78204+08:00 INFO [21697,0fbf274fb3499045] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:51:02.7821+08:00 INFO [21697,0fbf274fb3499045] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:51:02.765115756 +0800 CST m=+10835027.881725289, end time:2024-11-29 09:51:02 +0800 CST, Metrics number 0 2024-11-29T09:51:02.78218+08:00 WARN [21697,0fbf274fb3499045] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:51:02.85649+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:51:02.85805+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:51:02.85814+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:51:02.85817+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:51:02.85821+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:51:02.85824+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:51:02.85827+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:51:02.8583+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:51:02.85833+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:51:02.85837+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:51:02.8584+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:51:02.85842+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:51:02.86547+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:796:refreshTenants: configured tenants: map[1:sys 1002:ocp 1004:dbamonitor 1006:dbmanager 1008:datatransfer] 2024-11-29T09:51:02.87223+08:00 INFO [21697,cd31b76e92df6429] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:51:02 +0800 CST, start_request_id:0 2024-11-29T09:51:02.87229+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:02.87237+08:00 INFO [21697,cd31b76e92df6429] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:51:02 +0800 CST, start_request_id:0 2024-11-29T09:51:02.8724+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:02.87246+08:00 INFO [21697,cd31b76e92df6429] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:51:02 +0800 CST, start_request_id:0 2024-11-29T09:51:02.87248+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:02.88816+08:00 INFO [21697,cd31b76e92df6429] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:51:02 +0800 CST, start_request_id:387 2024-11-29T09:51:02.88823+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:02.8883+08:00 INFO [21697,cd31b76e92df6429] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:51:02 +0800 CST, start_request_id:544422 2024-11-29T09:51:02.88832+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544422, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:02.88837+08:00 INFO [21697,cd31b76e92df6429] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:51:02 +0800 CST, start_request_id:7154 2024-11-29T09:51:02.88839+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:02.88847+08:00 INFO [21697,cd31b76e92df6429] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:51:02 +0800 CST, start_request_id:0 2024-11-29T09:51:02.88849+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:02.89219+08:00 INFO [21697,cd31b76e92df6429] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:51:02 +0800 CST, start_request_id:0 2024-11-29T09:51:02.89226+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:02.89565+08:00 INFO [21697,cd31b76e92df6429] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:51:02 +0800 CST, start_request_id:0 2024-11-29T09:51:02.8957+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:02.91937+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:51:02.91943+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:02.91946+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:51:02.91957+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:51:02.91959+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:02.91966+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:51:02.91968+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:02.91974+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:51:02.91976+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:02.91982+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:51:02.91983+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:02.91987+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:51:02.91989+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:51:02.91991+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:51:02.91992+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:51:02.91999+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907608, got 0 result 2024-11-29T09:51:02.92+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907608, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:02.92002+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:51:02.92007+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:51:02.92009+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:02.92106+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807564, got 66 result 2024-11-29T09:51:02.92114+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 66, planExplainMetrics number 0, planMetaMetrics number 66 2024-11-29T09:51:02.94867+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807564, got 66 result, planStatMetrics number 66, planMetaMetrics number 66, planExplainMetrics number 0 2024-11-29T09:51:02.94878+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 66, planExplainMetrics number 0, planMetaMetrics number 66 2024-11-29T09:51:02.9753+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:51:02.97537+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:02.97539+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 66, planExplainMetrics number 0, planMetaMetrics number 66 2024-11-29T09:51:02.97545+08:00 INFO [21697,15cd2d1ca3b27ac8] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:51:02 +0800 CST, end time:2024-11-29 09:51:02 +0800 CST, planMetaMetrics number 66, planExplainMetrics number 0, planStatMetrics number 66 2024-11-29T09:51:03.3588+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:51:03.35898+08:00 INFO [21697,e59613ff43f3652c] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:51:03.77105+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 48 2024-11-29T09:51:03.77111+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:51:03.77115+08:00 INFO [21697,1da39327a02092df] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1413, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 48, sql_audit_progress Metrics number 1 2024-11-29T09:51:03.77451+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986206, result count: 198 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986206, max_request_id: 274986403 2024-11-29T09:51:04.48167+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:51:04.48177+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:51:04.5538+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:51:04.782+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986404, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986404, max_request_id: 274986438 2024-11-29T09:51:04.87383+08:00 INFO [21697,91b767c54bbf4fb7] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=91b767c54bbf4fb7, body=] 2024-11-29T09:51:04.87409+08:00 WARN [21697,91b767c54bbf4fb7] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:04.87414+08:00 WARN [21697,91b767c54bbf4fb7] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:04.87416+08:00 WARN [21697,91b767c54bbf4fb7] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:04.87418+08:00 WARN [21697,91b767c54bbf4fb7] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:04.87419+08:00 WARN [21697,91b767c54bbf4fb7] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:04.87808+08:00 INFO [21697,91b767c54bbf4fb7] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=91b767c54bbf4fb7, body=] 2024-11-29T09:51:05.04138+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:51:05.69644+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:51:05.69654+08:00 INFO [21697,c82c0398daf4a0ba] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:51:05.78849+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986439, result count: 25 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986439, max_request_id: 274986463 2024-11-29T09:51:05.97631+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 11 2024-11-29T09:51:05.97637+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:51:05.97641+08:00 INFO [21697,d0ee6cb5c2fbbef1] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 806, lost data number: 328, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 11, sql_audit_progress Metrics number 1 2024-11-29T09:51:05.97836+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163135314, result count: 8 auditMsg number: 0, diagMsg number: 0, min_request_id: 163135488, max_request_id: 163135504 2024-11-29T09:51:05.9784+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163135488 is larger than startRequestId 163135314, maybe lost 174 data 2024-11-29T09:51:06.19779+08:00 INFO [21697,692c79048b2e1cae] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=692c79048b2e1cae, body=] 2024-11-29T09:51:06.71502+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:51:06.71515+08:00 INFO [21697,d3f7aadd5229adbe] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:51:06.72943+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1008 from request_id 117575976, result count: 5 auditMsg number: 0, diagMsg number: 0, min_request_id: 117579590, max_request_id: 117579594 2024-11-29T09:51:06.72948+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1008 minRequestId 117579590 is larger than startRequestId 117575976, maybe lost 3614 data 2024-11-29T09:51:06.79079+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986464, result count: 76 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986464, max_request_id: 274986539 2024-11-29T09:51:07.09499+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:51:07.31112+08:00 INFO [21697,d0a26cc3c5a913f2] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=d0a26cc3c5a913f2, body=] 2024-11-29T09:51:07.314+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:51:07.31602+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:51:07.78153+08:00 INFO [21697,e70b36e73b9253c1] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:51:07.78161+08:00 INFO [21697,e70b36e73b9253c1] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:51:07.765208728 +0800 CST m=+10835032.881818260, end time:2024-11-29 09:51:07 +0800 CST, Metrics number 0 2024-11-29T09:51:07.7817+08:00 WARN [21697,e70b36e73b9253c1] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:51:07.79442+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986540, result count: 55 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986540, max_request_id: 274986594 2024-11-29T09:51:07.87218+08:00 INFO [21697,de506a5cd0574f28] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=de506a5cd0574f28, body=] 2024-11-29T09:51:07.87241+08:00 WARN [21697,de506a5cd0574f28] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:07.87245+08:00 WARN [21697,de506a5cd0574f28] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:07.87247+08:00 WARN [21697,de506a5cd0574f28] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:07.87248+08:00 WARN [21697,de506a5cd0574f28] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:07.87249+08:00 WARN [21697,de506a5cd0574f28] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:07.87634+08:00 INFO [21697,de506a5cd0574f28] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=de506a5cd0574f28, body=] 2024-11-29T09:51:08.39519+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1006 from request_id 136520497, result count: 5 auditMsg number: 0, diagMsg number: 0, min_request_id: 136524667, max_request_id: 136524671 2024-11-29T09:51:08.39526+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1006 minRequestId 136524667 is larger than startRequestId 136520497, maybe lost 4170 data 2024-11-29T09:51:08.79606+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986595, result count: 28 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986595, max_request_id: 274986622 2024-11-29T09:51:09.7984+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986623, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986623, max_request_id: 274986660 2024-11-29T09:51:10.02768+08:00 INFO [21697,794989099d5ff912] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=794989099d5ff912, body=] 2024-11-29T09:51:10.02791+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:51:10.04136+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:51:10.14428+08:00 INFO [21697,3b3f04e02b20c014] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=3b3f04e02b20c014, body=] 2024-11-29T09:51:10.14466+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:10.1447+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:10.14472+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:10.14473+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:10.14474+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:10.14509+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:51:10.78577+08:00 INFO [21697,3e9b028492eb76f7] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=3e9b028492eb76f7, body=] 2024-11-29T09:51:10.82829+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986661, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986661, max_request_id: 274986696 2024-11-29T09:51:10.8734+08:00 INFO [21697,50c7c1c2263b4718] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=50c7c1c2263b4718, body=] 2024-11-29T09:51:10.87363+08:00 WARN [21697,50c7c1c2263b4718] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:10.87367+08:00 WARN [21697,50c7c1c2263b4718] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:10.87368+08:00 WARN [21697,50c7c1c2263b4718] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:10.87369+08:00 WARN [21697,50c7c1c2263b4718] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:10.8737+08:00 WARN [21697,50c7c1c2263b4718] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:10.87741+08:00 INFO [21697,50c7c1c2263b4718] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=50c7c1c2263b4718, body=] 2024-11-29T09:51:11.744+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:51:11.83129+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986697, result count: 62 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986697, max_request_id: 274986758 2024-11-29T09:51:12.1155+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:51:12.78161+08:00 INFO [21697,c320957eb97f4f9d] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:51:12.78168+08:00 INFO [21697,c320957eb97f4f9d] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:51:12.765295606 +0800 CST m=+10835037.881905145, end time:2024-11-29 09:51:12 +0800 CST, Metrics number 0 2024-11-29T09:51:12.78175+08:00 WARN [21697,c320957eb97f4f9d] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:51:12.83354+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986759, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986759, max_request_id: 274986799 2024-11-29T09:51:13.83551+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986800, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986800, max_request_id: 274986830 2024-11-29T09:51:13.87213+08:00 INFO [21697,d56ab5e2164a4119] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=d56ab5e2164a4119, body=] 2024-11-29T09:51:13.87235+08:00 WARN [21697,d56ab5e2164a4119] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:13.87239+08:00 WARN [21697,d56ab5e2164a4119] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:13.8724+08:00 WARN [21697,d56ab5e2164a4119] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:13.87241+08:00 WARN [21697,d56ab5e2164a4119] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:13.87242+08:00 WARN [21697,d56ab5e2164a4119] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:13.87769+08:00 INFO [21697,d56ab5e2164a4119] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=d56ab5e2164a4119, body=] 2024-11-29T09:51:14.83722+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986831, result count: 30 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986831, max_request_id: 274986860 2024-11-29T09:51:15.04176+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:51:15.09527+08:00 INFO [21697,58fd1562f9ff4743] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=58fd1562f9ff4743, body=] 2024-11-29T09:51:15.09547+08:00 INFO [21697,58fd1562f9ff4743] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=58fd1562f9ff4743, body=] 2024-11-29T09:51:15.11947+08:00 WARN [21697,58fd1562f9ff4743] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:15.11954+08:00 WARN [21697,58fd1562f9ff4743] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:15.11957+08:00 WARN [21697,58fd1562f9ff4743] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:15.11959+08:00 WARN [21697,58fd1562f9ff4743] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:15.1196+08:00 WARN [21697,58fd1562f9ff4743] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:15.12581+08:00 INFO [21697,58fd1562f9ff4743] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:51:15.1306+08:00 INFO [21697,58fd1562f9ff4743] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=58fd1562f9ff4743, body=] 2024-11-29T09:51:15.14475+08:00 INFO [21697,58fd1562f9ff4743] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=58fd1562f9ff4743, body=] 2024-11-29T09:51:15.14499+08:00 INFO [21697,58fd1562f9ff4743] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=58fd1562f9ff4743, body=] 2024-11-29T09:51:15.14518+08:00 WARN [21697,58fd1562f9ff4743] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:51:15.1458+08:00 WARN [21697,58fd1562f9ff4743] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:51:15.1491+08:00 INFO [21697,58fd1562f9ff4743] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:51:15.83972+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986861, result count: 49 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986861, max_request_id: 274986909 2024-11-29T09:51:16.84184+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986910, result count: 50 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986910, max_request_id: 274986959 2024-11-29T09:51:16.87174+08:00 INFO [21697,6c425f2e424f43d3] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=6c425f2e424f43d3, body=] 2024-11-29T09:51:16.87197+08:00 WARN [21697,6c425f2e424f43d3] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:16.87201+08:00 WARN [21697,6c425f2e424f43d3] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:16.87202+08:00 WARN [21697,6c425f2e424f43d3] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:16.87203+08:00 WARN [21697,6c425f2e424f43d3] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:16.87204+08:00 WARN [21697,6c425f2e424f43d3] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:16.87594+08:00 INFO [21697,6c425f2e424f43d3] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=6c425f2e424f43d3, body=] 2024-11-29T09:51:17.09994+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:51:17.79847+08:00 INFO [21697,3ad54cc04ea0117b] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:51:17.79854+08:00 INFO [21697,3ad54cc04ea0117b] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:51:17.765635456 +0800 CST m=+10835042.882244988, end time:2024-11-29 09:51:17 +0800 CST, Metrics number 0 2024-11-29T09:51:17.79864+08:00 WARN [21697,3ad54cc04ea0117b] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:51:17.84446+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986960, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986960, max_request_id: 274986996 2024-11-29T09:51:18.84679+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274986997, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 274986997, max_request_id: 274987027 2024-11-29T09:51:19.4816+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:51:19.48171+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:51:19.56079+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields: error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:51:19.84898+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987028, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987028, max_request_id: 274987066 2024-11-29T09:51:19.87179+08:00 INFO [21697,3c5d9b0ab5c64056] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=3c5d9b0ab5c64056, body=] 2024-11-29T09:51:19.87207+08:00 WARN [21697,3c5d9b0ab5c64056] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:19.87213+08:00 WARN [21697,3c5d9b0ab5c64056] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:19.87216+08:00 WARN [21697,3c5d9b0ab5c64056] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:19.87218+08:00 WARN [21697,3c5d9b0ab5c64056] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:19.87221+08:00 WARN [21697,3c5d9b0ab5c64056] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:19.8767+08:00 INFO [21697,3c5d9b0ab5c64056] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=3c5d9b0ab5c64056, body=] 2024-11-29T09:51:20.04171+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:51:20.85061+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987067, result count: 28 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987067, max_request_id: 274987094 2024-11-29T09:51:21.20029+08:00 INFO [21697,dfc078defaae766e] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=dfc078defaae766e, body=] 2024-11-29T09:51:21.74401+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:51:21.85319+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987095, result count: 69 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987095, max_request_id: 274987163 2024-11-29T09:51:21.9867+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163135505, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163135837, max_request_id: 163135839 2024-11-29T09:51:21.98676+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163135837 is larger than startRequestId 163135505, maybe lost 332 data 2024-11-29T09:51:22.10596+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:51:22.30982+08:00 INFO [21697,a63e18ce12b555f1] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=a63e18ce12b555f1, body=] 2024-11-29T09:51:22.31027+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:51:22.31065+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:51:22.78125+08:00 INFO [21697,b6829a774b40c993] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:51:22.78131+08:00 INFO [21697,b6829a774b40c993] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:51:22.764927094 +0800 CST m=+10835047.881536627, end time:2024-11-29 09:51:22 +0800 CST, Metrics number 0 2024-11-29T09:51:22.7814+08:00 WARN [21697,b6829a774b40c993] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:51:22.85519+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987164, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987164, max_request_id: 274987206 2024-11-29T09:51:22.87182+08:00 INFO [21697,3e4a2819ad384112] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=3e4a2819ad384112, body=] 2024-11-29T09:51:22.87209+08:00 WARN [21697,3e4a2819ad384112] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:22.87214+08:00 WARN [21697,3e4a2819ad384112] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:22.87216+08:00 WARN [21697,3e4a2819ad384112] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:22.87218+08:00 WARN [21697,3e4a2819ad384112] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:22.87219+08:00 WARN [21697,3e4a2819ad384112] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:22.87612+08:00 INFO [21697,3e4a2819ad384112] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=3e4a2819ad384112, body=] 2024-11-29T09:51:23.85703+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987207, result count: 25 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987207, max_request_id: 274987231 2024-11-29T09:51:24.85947+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987232, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987232, max_request_id: 274987267 2024-11-29T09:51:25.02718+08:00 INFO [21697,47c3b7f41ae9b248] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=47c3b7f41ae9b248, body=] 2024-11-29T09:51:25.02742+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:51:25.04194+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:51:25.14485+08:00 INFO [21697,f41629754d6f4739] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=f41629754d6f4739, body=] 2024-11-29T09:51:25.14521+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:25.14525+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:25.14527+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:25.14528+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:25.14529+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:25.14568+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:51:25.78563+08:00 INFO [21697,29457a41e9769e6c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=29457a41e9769e6c, body=] 2024-11-29T09:51:25.86236+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987268, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987268, max_request_id: 274987304 2024-11-29T09:51:25.87157+08:00 INFO [21697,292d35a568684c60] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=292d35a568684c60, body=] 2024-11-29T09:51:25.87178+08:00 WARN [21697,292d35a568684c60] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:25.87182+08:00 WARN [21697,292d35a568684c60] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:25.87183+08:00 WARN [21697,292d35a568684c60] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:25.87184+08:00 WARN [21697,292d35a568684c60] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:25.87185+08:00 WARN [21697,292d35a568684c60] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:25.87574+08:00 INFO [21697,292d35a568684c60] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=292d35a568684c60, body=] 2024-11-29T09:51:26.8644+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987305, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987305, max_request_id: 274987347 2024-11-29T09:51:27.09498+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:51:27.78132+08:00 INFO [21697,95512345fd3374ee] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:51:27.78139+08:00 INFO [21697,95512345fd3374ee] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:51:27.76513938 +0800 CST m=+10835052.881748917, end time:2024-11-29 09:51:27 +0800 CST, Metrics number 0 2024-11-29T09:51:27.78164+08:00 WARN [21697,95512345fd3374ee] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:51:27.86625+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987348, result count: 45 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987348, max_request_id: 274987392 2024-11-29T09:51:28.86784+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987393, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987393, max_request_id: 274987424 2024-11-29T09:51:28.87163+08:00 INFO [21697,305b6109c0734c8c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=305b6109c0734c8c, body=] 2024-11-29T09:51:28.87182+08:00 WARN [21697,305b6109c0734c8c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:28.87185+08:00 WARN [21697,305b6109c0734c8c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:28.87187+08:00 WARN [21697,305b6109c0734c8c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:28.87188+08:00 WARN [21697,305b6109c0734c8c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:28.87189+08:00 WARN [21697,305b6109c0734c8c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:28.87588+08:00 INFO [21697,305b6109c0734c8c] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=305b6109c0734c8c, body=] 2024-11-29T09:51:29.87019+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987425, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987425, max_request_id: 274987456 2024-11-29T09:51:30.04121+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:51:30.09495+08:00 INFO [21697,c996f2ad50dc4732] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=c996f2ad50dc4732, body=] 2024-11-29T09:51:30.09517+08:00 INFO [21697,c996f2ad50dc4732] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=c996f2ad50dc4732, body=] 2024-11-29T09:51:30.10466+08:00 WARN [21697,c996f2ad50dc4732] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:30.10472+08:00 WARN [21697,c996f2ad50dc4732] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:30.10474+08:00 WARN [21697,c996f2ad50dc4732] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:30.10475+08:00 WARN [21697,c996f2ad50dc4732] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:30.10476+08:00 WARN [21697,c996f2ad50dc4732] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:30.10735+08:00 INFO [21697,c996f2ad50dc4732] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:51:30.11069+08:00 INFO [21697,c996f2ad50dc4732] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=c996f2ad50dc4732, body=] 2024-11-29T09:51:30.11397+08:00 INFO [21697,c996f2ad50dc4732] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=c996f2ad50dc4732, body=] 2024-11-29T09:51:30.11409+08:00 INFO [21697,c996f2ad50dc4732] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=c996f2ad50dc4732, body=] 2024-11-29T09:51:30.11421+08:00 WARN [21697,c996f2ad50dc4732] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:51:30.11582+08:00 WARN [21697,c996f2ad50dc4732] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:51:30.11789+08:00 INFO [21697,c996f2ad50dc4732] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:51:30.8718+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987457, result count: 50 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987457, max_request_id: 274987506 2024-11-29T09:51:31.33276+08:00 INFO [21697,08ecd1b7fba23fe5] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.98.86, ocpServerIp=, traceId=08ecd1b7fba23fe5, body=] 2024-11-29T09:51:31.33299+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:31.33303+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:31.33305+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:31.33306+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:31.33307+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:31.74361+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:51:31.88498+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987507, result count: 157 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987507, max_request_id: 274987663 2024-11-29T09:51:31.88555+08:00 INFO [21697,74689b5cd8894a68] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=74689b5cd8894a68, body=] 2024-11-29T09:51:31.88698+08:00 INFO [21697,74689b5cd8894a68] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=74689b5cd8894a68, body=] 2024-11-29T09:51:31.88713+08:00 WARN [21697,74689b5cd8894a68] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:31.88716+08:00 WARN [21697,74689b5cd8894a68] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:31.88717+08:00 WARN [21697,74689b5cd8894a68] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:31.88718+08:00 WARN [21697,74689b5cd8894a68] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:31.88719+08:00 WARN [21697,74689b5cd8894a68] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:32.09506+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:51:32.78143+08:00 INFO [21697,459f886330971b98] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:51:32.7815+08:00 INFO [21697,459f886330971b98] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:51:32.765070671 +0800 CST m=+10835057.881680203, end time:2024-11-29 09:51:32 +0800 CST, Metrics number 0 2024-11-29T09:51:32.78159+08:00 WARN [21697,459f886330971b98] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:51:32.85635+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:51:32.85796+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:51:32.85803+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:51:32.85804+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:51:32.85808+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:51:32.8581+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:51:32.85812+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:51:32.85815+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:51:32.85817+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:51:32.85822+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:51:32.85824+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:51:32.85825+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:51:32.87091+08:00 INFO [21697,816daf5496e38d9e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:51:32 +0800 CST, start_request_id:0 2024-11-29T09:51:32.87096+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:32.87107+08:00 INFO [21697,816daf5496e38d9e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:51:32 +0800 CST, start_request_id:544422 2024-11-29T09:51:32.87108+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544422, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:32.87122+08:00 INFO [21697,816daf5496e38d9e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:51:32 +0800 CST, start_request_id:0 2024-11-29T09:51:32.87123+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:32.88071+08:00 INFO [21697,816daf5496e38d9e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:51:32 +0800 CST, start_request_id:0 2024-11-29T09:51:32.88074+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:32.88083+08:00 INFO [21697,816daf5496e38d9e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:51:32 +0800 CST, start_request_id:387 2024-11-29T09:51:32.88085+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:32.88089+08:00 INFO [21697,816daf5496e38d9e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:51:32 +0800 CST, start_request_id:0 2024-11-29T09:51:32.88091+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:32.88097+08:00 INFO [21697,816daf5496e38d9e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:51:32 +0800 CST, start_request_id:7154 2024-11-29T09:51:32.88098+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:32.88502+08:00 INFO [21697,816daf5496e38d9e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:51:32 +0800 CST, start_request_id:0 2024-11-29T09:51:32.88504+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:32.88556+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 47 2024-11-29T09:51:32.88559+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:51:32.88562+08:00 INFO [21697,2a89c8b5ecac7a59] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1458, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 47, sql_audit_progress Metrics number 1 2024-11-29T09:51:32.88821+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987664, result count: 86 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987664, max_request_id: 274987749 2024-11-29T09:51:32.89719+08:00 INFO [21697,816daf5496e38d9e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:51:32 +0800 CST, start_request_id:0 2024-11-29T09:51:32.89722+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:51:32.91019+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:51:32.91022+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:32.9103+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907608, got 0 result 2024-11-29T09:51:32.91033+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907608, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:32.91041+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:51:32.91042+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:32.9105+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:51:32.91052+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:32.91054+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:51:32.91062+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:51:32.91065+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:51:32.91066+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:51:32.91081+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807569, got 5 result 2024-11-29T09:51:32.9109+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:51:32.91092+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:32.91093+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:51:32.911+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:51:32.91102+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:32.91109+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:51:32.91111+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:32.91114+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:51:32.91115+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:51:32.91297+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807569, got 5 result, planStatMetrics number 5, planMetaMetrics number 5, planExplainMetrics number 0 2024-11-29T09:51:32.913+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:51:32.96377+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:51:32.96383+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:51:32.96385+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:51:32.96391+08:00 INFO [21697,04c9fd97692d423b] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:51:32 +0800 CST, end time:2024-11-29 09:51:32 +0800 CST, planMetaMetrics number 5, planExplainMetrics number 0, planStatMetrics number 5 2024-11-29T09:51:32.99142+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 3 2024-11-29T09:51:32.99147+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:51:32.99151+08:00 INFO [21697,88a19b989dd3fe3c] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 11, lost data number: 506, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 3, sql_audit_progress Metrics number 1 2024-11-29T09:51:33.00129+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163135840, result count: 305 auditMsg number: 0, diagMsg number: 0, min_request_id: 163136024, max_request_id: 163136423 2024-11-29T09:51:33.00133+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163136024 is larger than startRequestId 163135840, maybe lost 184 data 2024-11-29T09:51:33.89122+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987750, result count: 53 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987750, max_request_id: 274987802 2024-11-29T09:51:34.40943+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:51:34.40961+08:00 INFO [21697,7fbc65d46dbce5cc] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 5, lost data number: 4170, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:51:34.48136+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:51:34.48144+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:51:34.56385+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:51:34.87127+08:00 INFO [21697,ffa043a2d05f4b35] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=ffa043a2d05f4b35, body=] 2024-11-29T09:51:34.8716+08:00 WARN [21697,ffa043a2d05f4b35] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:34.87165+08:00 WARN [21697,ffa043a2d05f4b35] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:34.87168+08:00 WARN [21697,ffa043a2d05f4b35] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:34.8717+08:00 WARN [21697,ffa043a2d05f4b35] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:34.87172+08:00 WARN [21697,ffa043a2d05f4b35] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:34.88029+08:00 INFO [21697,ffa043a2d05f4b35] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=ffa043a2d05f4b35, body=] 2024-11-29T09:51:34.89048+08:00 WARN [21697,] caller=process/process.go:177:doCollect: failed to get process obproxyd.sh createTime fields:, error="open /proc/163483/stat: no such file or directory" 2024-11-29T09:51:34.89377+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987803, result count: 45 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987803, max_request_id: 274987847 2024-11-29T09:51:35.04135+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:51:35.86422+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:51:35.8644+08:00 INFO [21697,ff60cab8a6275ab7] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:51:35.89627+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987848, result count: 26 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987848, max_request_id: 274987873 2024-11-29T09:51:36.19739+08:00 INFO [21697,7b723c913c4b66c5] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=7b723c913c4b66c5, body=] 2024-11-29T09:51:36.89885+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987874, result count: 82 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987874, max_request_id: 274987955 2024-11-29T09:51:37.09515+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:51:37.30852+08:00 INFO [21697,dfff1c0fd096b5fa] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=dfff1c0fd096b5fa, body=] 2024-11-29T09:51:37.3089+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:51:37.30924+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:51:37.74563+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:51:37.74575+08:00 INFO [21697,48323308a1c70062] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 5, lost data number: 3614, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:51:37.78194+08:00 INFO [21697,25caa92363cf3a12] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 3 result 2024-11-29T09:51:37.782+08:00 INFO [21697,25caa92363cf3a12] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:51:37.765019471 +0800 CST m=+10835062.881629004, end time:2024-11-29 09:51:37 +0800 CST, Metrics number 3 2024-11-29T09:51:37.78208+08:00 INFO [21697,25caa92363cf3a12] caller=engine/pipeline.go:145:pipelinePush: push pipeline ob_sql_transaction metrics count 3 2024-11-29T09:51:37.87136+08:00 INFO [21697,dfb38bcac77747c7] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=dfb38bcac77747c7, body=] 2024-11-29T09:51:37.8716+08:00 WARN [21697,dfb38bcac77747c7] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:37.87164+08:00 WARN [21697,dfb38bcac77747c7] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:37.87166+08:00 WARN [21697,dfb38bcac77747c7] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:37.87167+08:00 WARN [21697,dfb38bcac77747c7] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:37.87168+08:00 WARN [21697,dfb38bcac77747c7] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:37.87564+08:00 INFO [21697,dfb38bcac77747c7] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=dfb38bcac77747c7, body=] 2024-11-29T09:51:37.90082+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987956, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987956, max_request_id: 274987998 2024-11-29T09:51:38.90236+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274987999, result count: 24 auditMsg number: 0, diagMsg number: 0, min_request_id: 274987999, max_request_id: 274988022 2024-11-29T09:51:39.00498+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163136424, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163136479, max_request_id: 163136481 2024-11-29T09:51:39.00504+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163136479 is larger than startRequestId 163136424, maybe lost 55 data 2024-11-29T09:51:39.90493+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988023, result count: 44 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988023, max_request_id: 274988066 2024-11-29T09:51:40.02739+08:00 INFO [21697,6dba19abce240399] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=6dba19abce240399, body=] 2024-11-29T09:51:40.02762+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:51:40.04124+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:51:40.14446+08:00 INFO [21697,509acc10aa49efc4] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=509acc10aa49efc4, body=] 2024-11-29T09:51:40.14491+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:40.14496+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:40.14498+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:40.14499+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:40.14501+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:40.14548+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1506, err: 2024-11-29T09:51:40.78528+08:00 INFO [21697,bac46351e3e31191] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=bac46351e3e31191, body=] 2024-11-29T09:51:40.87307+08:00 INFO [21697,c575383a11a646c8] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=c575383a11a646c8, body=] 2024-11-29T09:51:40.87337+08:00 WARN [21697,c575383a11a646c8] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:40.87343+08:00 WARN [21697,c575383a11a646c8] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:40.87346+08:00 WARN [21697,c575383a11a646c8] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:40.87347+08:00 WARN [21697,c575383a11a646c8] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:40.87349+08:00 WARN [21697,c575383a11a646c8] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:40.8776+08:00 INFO [21697,c575383a11a646c8] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=c575383a11a646c8, body=] 2024-11-29T09:51:40.90722+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988067, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988067, max_request_id: 274988103 2024-11-29T09:51:41.74379+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:51:41.90971+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988104, result count: 62 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988104, max_request_id: 274988165 2024-11-29T09:51:42.09429+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:51:42.78165+08:00 INFO [21697,7bbc87d3b6a01c2f] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 1 result 2024-11-29T09:51:42.78172+08:00 INFO [21697,7bbc87d3b6a01c2f] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:51:42.765187718 +0800 CST m=+10835067.881797251, end time:2024-11-29 09:51:42 +0800 CST, Metrics number 1 2024-11-29T09:51:42.78181+08:00 INFO [21697,7bbc87d3b6a01c2f] caller=engine/pipeline.go:145:pipelinePush: push pipeline ob_sql_transaction metrics count 1 2024-11-29T09:51:42.91144+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988166, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988166, max_request_id: 274988204 2024-11-29T09:51:43.87141+08:00 INFO [21697,147979321354498c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=147979321354498c, body=] 2024-11-29T09:51:43.87165+08:00 WARN [21697,147979321354498c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:43.87169+08:00 WARN [21697,147979321354498c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:43.8717+08:00 WARN [21697,147979321354498c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:43.87171+08:00 WARN [21697,147979321354498c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:43.87172+08:00 WARN [21697,147979321354498c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:43.87644+08:00 INFO [21697,147979321354498c] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=147979321354498c, body=] 2024-11-29T09:51:43.91331+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988205, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988205, max_request_id: 274988236 2024-11-29T09:51:44.91541+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988237, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988237, max_request_id: 274988267 2024-11-29T09:51:45.04157+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:51:45.09449+08:00 INFO [21697,250d23792454426e] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=250d23792454426e, body=] 2024-11-29T09:51:45.0949+08:00 WARN [21697,250d23792454426e] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:45.09494+08:00 WARN [21697,250d23792454426e] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:45.09496+08:00 WARN [21697,250d23792454426e] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:45.09497+08:00 WARN [21697,250d23792454426e] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:45.09498+08:00 WARN [21697,250d23792454426e] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:45.09533+08:00 INFO [21697,250d23792454426e] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:51:45.09569+08:00 INFO [21697,250d23792454426e] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=250d23792454426e, body=] 2024-11-29T09:51:45.0989+08:00 INFO [21697,250d23792454426e] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=250d23792454426e, body=] 2024-11-29T09:51:45.09904+08:00 WARN [21697,250d23792454426e] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:51:45.09915+08:00 INFO [21697,250d23792454426e] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=250d23792454426e, body=] 2024-11-29T09:51:45.09924+08:00 INFO [21697,250d23792454426e] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=250d23792454426e, body=] 2024-11-29T09:51:45.10067+08:00 WARN [21697,250d23792454426e] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:51:45.10853+08:00 INFO [21697,250d23792454426e] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:51:45.91828+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988268, result count: 46 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988268, max_request_id: 274988313 2024-11-29T09:51:46.87108+08:00 INFO [21697,58b95d952ed44403] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=58b95d952ed44403, body=] 2024-11-29T09:51:46.87132+08:00 WARN [21697,58b95d952ed44403] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:46.87136+08:00 WARN [21697,58b95d952ed44403] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:46.87137+08:00 WARN [21697,58b95d952ed44403] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:46.87138+08:00 WARN [21697,58b95d952ed44403] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:46.87139+08:00 WARN [21697,58b95d952ed44403] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:46.87543+08:00 INFO [21697,58b95d952ed44403] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=58b95d952ed44403, body=] 2024-11-29T09:51:46.9208+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988314, result count: 55 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988314, max_request_id: 274988368 2024-11-29T09:51:47.09576+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:51:47.79041+08:00 INFO [21697,175ac52d9a022642] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:51:47.79049+08:00 INFO [21697,175ac52d9a022642] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:51:47.765507257 +0800 CST m=+10835072.882116792, end time:2024-11-29 09:51:47 +0800 CST, Metrics number 0 2024-11-29T09:51:47.79059+08:00 WARN [21697,175ac52d9a022642] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:51:47.92325+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988369, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988369, max_request_id: 274988399 2024-11-29T09:51:48.3098+08:00 INFO [21697,a75373de66660196] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.98.86, ocpServerIp=, traceId=a75373de66660196, body=] 2024-11-29T09:51:48.9248+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988400, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988400, max_request_id: 274988434 2024-11-29T09:51:49.4816+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:51:49.48169+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:51:49.5528+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:51:49.87101+08:00 INFO [21697,e350903298274b7b] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=e350903298274b7b, body=] 2024-11-29T09:51:49.87124+08:00 WARN [21697,e350903298274b7b] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:49.87127+08:00 WARN [21697,e350903298274b7b] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:49.87129+08:00 WARN [21697,e350903298274b7b] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:49.8713+08:00 WARN [21697,e350903298274b7b] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:49.87131+08:00 WARN [21697,e350903298274b7b] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:49.8784+08:00 INFO [21697,e350903298274b7b] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=e350903298274b7b, body=] 2024-11-29T09:51:49.92661+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988435, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988435, max_request_id: 274988476 2024-11-29T09:51:50.04104+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:51:50.92889+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988477, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988477, max_request_id: 274988508 2024-11-29T09:51:51.1971+08:00 INFO [21697,ab9e29771e584f50] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=ab9e29771e584f50, body=] 2024-11-29T09:51:51.74409+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:51:51.93687+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988509, result count: 76 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988509, max_request_id: 274988584 2024-11-29T09:51:52.10801+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:51:52.30832+08:00 INFO [21697,8079f225385b98b3] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=8079f225385b98b3, body=] 2024-11-29T09:51:52.30869+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:51:52.309+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:51:52.37883+08:00 INFO [21697,ed85e7e5ed336283] caller=host/custom.go:178:func10: collect observer dir exists info 2024-11-29T09:51:52.78286+08:00 INFO [21697,89908c4fcf10fdf2] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:51:52.78295+08:00 INFO [21697,89908c4fcf10fdf2] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:51:52.765892381 +0800 CST m=+10835077.882501942, end time:2024-11-29 09:51:52 +0800 CST, Metrics number 0 2024-11-29T09:51:52.78306+08:00 WARN [21697,89908c4fcf10fdf2] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:51:52.87244+08:00 INFO [21697,27f449aafb2b4007] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=27f449aafb2b4007, body=] 2024-11-29T09:51:52.87273+08:00 WARN [21697,27f449aafb2b4007] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:52.87278+08:00 WARN [21697,27f449aafb2b4007] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:52.8728+08:00 WARN [21697,27f449aafb2b4007] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:52.87282+08:00 WARN [21697,27f449aafb2b4007] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:52.87283+08:00 WARN [21697,27f449aafb2b4007] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:52.87837+08:00 INFO [21697,27f449aafb2b4007] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=27f449aafb2b4007, body=] 2024-11-29T09:51:52.94854+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988585, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988585, max_request_id: 274988622 2024-11-29T09:51:53.95748+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988623, result count: 27 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988623, max_request_id: 274988649 2024-11-29T09:51:54.95985+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988650, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988650, max_request_id: 274988686 2024-11-29T09:51:55.01722+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163136482, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163136806, max_request_id: 163136808 2024-11-29T09:51:55.01729+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163136806 is larger than startRequestId 163136482, maybe lost 324 data 2024-11-29T09:51:55.02729+08:00 INFO [21697,d2fd0d8c4d3a67c1] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=d2fd0d8c4d3a67c1, body=] 2024-11-29T09:51:55.02766+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:51:55.04129+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:51:55.14339+08:00 INFO [21697,2705d343f0e4b01b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=2705d343f0e4b01b, body=] 2024-11-29T09:51:55.14391+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:55.14397+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:55.14399+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:55.14401+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:55.14402+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:55.14457+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:51:55.78494+08:00 INFO [21697,5c9d183b58a37a2b] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=5c9d183b58a37a2b, body=] 2024-11-29T09:51:55.87077+08:00 INFO [21697,9b0d349ab13e44cb] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=9b0d349ab13e44cb, body=] 2024-11-29T09:51:55.8712+08:00 WARN [21697,9b0d349ab13e44cb] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:55.87127+08:00 WARN [21697,9b0d349ab13e44cb] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:55.87131+08:00 WARN [21697,9b0d349ab13e44cb] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:55.87134+08:00 WARN [21697,9b0d349ab13e44cb] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:55.87137+08:00 WARN [21697,9b0d349ab13e44cb] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:55.87692+08:00 INFO [21697,9b0d349ab13e44cb] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=9b0d349ab13e44cb, body=] 2024-11-29T09:51:55.96206+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988687, result count: 34 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988687, max_request_id: 274988720 2024-11-29T09:51:56.96594+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988721, result count: 52 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988721, max_request_id: 274988772 2024-11-29T09:51:57.095+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:51:57.79994+08:00 INFO [21697,2d4224c3a133215b] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:51:57.80004+08:00 INFO [21697,2d4224c3a133215b] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:51:57.765647593 +0800 CST m=+10835082.882257160, end time:2024-11-29 09:51:57 +0800 CST, Metrics number 0 2024-11-29T09:51:57.80018+08:00 WARN [21697,2d4224c3a133215b] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:51:57.96858+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988773, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988773, max_request_id: 274988811 2024-11-29T09:51:58.87067+08:00 INFO [21697,773cdf6a6e6c4ec1] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=773cdf6a6e6c4ec1, body=] 2024-11-29T09:51:58.87094+08:00 WARN [21697,773cdf6a6e6c4ec1] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:51:58.87098+08:00 WARN [21697,773cdf6a6e6c4ec1] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:51:58.871+08:00 WARN [21697,773cdf6a6e6c4ec1] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:51:58.87102+08:00 WARN [21697,773cdf6a6e6c4ec1] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:58.87103+08:00 WARN [21697,773cdf6a6e6c4ec1] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:51:58.87612+08:00 INFO [21697,773cdf6a6e6c4ec1] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=773cdf6a6e6c4ec1, body=] 2024-11-29T09:51:58.9716+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988812, result count: 27 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988812, max_request_id: 274988838 2024-11-29T09:51:59.97473+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988839, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988839, max_request_id: 274988871 2024-11-29T09:52:00.04146+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:52:00.09815+08:00 INFO [21697,e66fd944825548e1] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=e66fd944825548e1, body=] 2024-11-29T09:52:00.11393+08:00 INFO [21697,e66fd944825548e1] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=e66fd944825548e1, body=] 2024-11-29T09:52:00.11414+08:00 INFO [21697,e66fd944825548e1] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=e66fd944825548e1, body=] 2024-11-29T09:52:00.11432+08:00 WARN [21697,e66fd944825548e1] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:52:00.11447+08:00 INFO [21697,e66fd944825548e1] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=e66fd944825548e1, body=] 2024-11-29T09:52:00.11464+08:00 INFO [21697,e66fd944825548e1] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=e66fd944825548e1, body=] 2024-11-29T09:52:00.11716+08:00 WARN [21697,e66fd944825548e1] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:52:00.11771+08:00 WARN [21697,e66fd944825548e1] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:00.11775+08:00 WARN [21697,e66fd944825548e1] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:00.11777+08:00 WARN [21697,e66fd944825548e1] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:00.11779+08:00 WARN [21697,e66fd944825548e1] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:00.11782+08:00 WARN [21697,e66fd944825548e1] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:00.12611+08:00 INFO [21697,e66fd944825548e1] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:52:00.12757+08:00 INFO [21697,e66fd944825548e1] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:52:00.97706+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988872, result count: 91 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988872, max_request_id: 274988962 2024-11-29T09:52:01.40837+08:00 INFO [21697,83dc6a69dc1f6b11] caller=common/middleware.go:59:func1: API request: [GET /api/v1/log/alarms, client=1.1.1.69.58, ocpServerIp=1.1.1.69.58, traceId=83dc6a69dc1f6b11, body=] 2024-11-29T09:52:01.40854+08:00 INFO [21697,] caller=alarmscache/alarm_handler.go:73:getPipelieGroupReader: get alarms, result size:0, cache list size:4 fields:, uri=/api/v1/log/alarms 2024-11-29T09:52:01.74366+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:52:01.87056+08:00 INFO [21697,cd711492075a4c00] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=cd711492075a4c00, body=] 2024-11-29T09:52:01.87077+08:00 WARN [21697,cd711492075a4c00] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:01.87081+08:00 WARN [21697,cd711492075a4c00] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:01.87082+08:00 WARN [21697,cd711492075a4c00] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:01.87083+08:00 WARN [21697,cd711492075a4c00] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:01.87084+08:00 WARN [21697,cd711492075a4c00] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:01.87479+08:00 INFO [21697,cd711492075a4c00] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=cd711492075a4c00, body=] 2024-11-29T09:52:01.97977+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274988963, result count: 80 auditMsg number: 0, diagMsg number: 0, min_request_id: 274988963, max_request_id: 274989042 2024-11-29T09:52:02.10877+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:52:02.74241+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:245:updateMetaInfo: successfully init sqlAudit tenant config 2024-11-29T09:52:02.78218+08:00 INFO [21697,361a3750d64c0936] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:52:02.78226+08:00 INFO [21697,361a3750d64c0936] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:52:02.765809435 +0800 CST m=+10835087.882418968, end time:2024-11-29 09:52:02 +0800 CST, Metrics number 0 2024-11-29T09:52:02.78235+08:00 WARN [21697,361a3750d64c0936] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:52:02.86479+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:52:02.8663+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:52:02.86639+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:52:02.86643+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:52:02.86646+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:52:02.8665+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:52:02.86654+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:52:02.86659+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:52:02.86663+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:52:02.86666+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:52:02.86671+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:52:02.86675+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:52:02.87123+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:796:refreshTenants: configured tenants: map[1:sys 1002:ocp 1004:dbamonitor 1006:dbmanager 1008:datatransfer] 2024-11-29T09:52:02.87154+08:00 INFO [21697,8395373804ba9a0f] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:52:02 +0800 CST, start_request_id:0 2024-11-29T09:52:02.87159+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:02.87172+08:00 INFO [21697,8395373804ba9a0f] caller=oceanbase/sql_plan_monitor.go:385:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1, collect time:2024-11-29 09:52:02 +0800 CST, start_request_id:544422, min_request_id:544423, max_request_id:544433 2024-11-29T09:52:02.87197+08:00 INFO [21697,8395373804ba9a0f] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:52:02 +0800 CST, start_request_id:0 2024-11-29T09:52:02.87199+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:02.87249+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544422, min_request_id: 544423, max_request_id: 544433, collected metrics number: 11 2024-11-29T09:52:02.87433+08:00 INFO [21697,8395373804ba9a0f] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 2 complete, collect time:2024-11-29 09:52:02 +0800 CST, start_request_id:544433 2024-11-29T09:52:02.87438+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 2, start_request_id: 544433, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:02.87445+08:00 INFO [21697,8395373804ba9a0f] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:52:02 +0800 CST, start_request_id:0 2024-11-29T09:52:02.87448+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:02.88058+08:00 INFO [21697,8395373804ba9a0f] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:52:02 +0800 CST, start_request_id:0 2024-11-29T09:52:02.88065+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:02.88072+08:00 INFO [21697,8395373804ba9a0f] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:52:02 +0800 CST, start_request_id:7154 2024-11-29T09:52:02.88074+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:02.88082+08:00 INFO [21697,8395373804ba9a0f] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:52:02 +0800 CST, start_request_id:0 2024-11-29T09:52:02.88084+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:02.88119+08:00 INFO [21697,8395373804ba9a0f] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:52:02 +0800 CST, start_request_id:387 2024-11-29T09:52:02.88123+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:02.89195+08:00 INFO [21697,8395373804ba9a0f] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:52:02 +0800 CST, start_request_id:0 2024-11-29T09:52:02.89201+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:02.90622+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:52:02.90629+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:52:02.90632+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:52:02.90641+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:52:02.90645+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:52:02.90652+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:52:02.90654+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:52:02.90662+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:52:02.90664+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:52:02.90668+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:52:02.9067+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:52:02.90672+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:52:02.90688+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807574, got 5 result 2024-11-29T09:52:02.90696+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:52:02.90697+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:52:02.90699+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:52:02.90706+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:52:02.90708+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:52:02.9071+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:52:02.90928+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807574, got 5 result, planStatMetrics number 5, planMetaMetrics number 5, planExplainMetrics number 0 2024-11-29T09:52:02.90931+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:52:02.91015+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907614, got 6 result 2024-11-29T09:52:02.91284+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907614, got 6 result, planStatMetrics number 6, planMetaMetrics number 6, planExplainMetrics number 2 2024-11-29T09:52:02.91288+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 11, planExplainMetrics number 2, planMetaMetrics number 11 2024-11-29T09:52:02.95869+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:52:02.95875+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:52:02.95877+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 11, planExplainMetrics number 2, planMetaMetrics number 11 2024-11-29T09:52:02.95882+08:00 INFO [21697,aba7993244fee5dc] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:52:02 +0800 CST, end time:2024-11-29 09:52:02 +0800 CST, planMetaMetrics number 11, planExplainMetrics number 2, planStatMetrics number 11 2024-11-29T09:52:02.9809+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 44 2024-11-29T09:52:02.98096+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:52:02.981+08:00 INFO [21697,6d7f5ddffb5b19aa] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1379, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 44, sql_audit_progress Metrics number 1 2024-11-29T09:52:02.98496+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989043, result count: 87 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989043, max_request_id: 274989129 2024-11-29T09:52:03.98694+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989130, result count: 25 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989130, max_request_id: 274989154 2024-11-29T09:52:04.45087+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:52:04.45101+08:00 INFO [21697,c584a40d942d20ab] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:52:04.48164+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:52:04.48172+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:52:04.55337+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields: error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:52:04.87245+08:00 INFO [21697,99d24a8cbbdb43e5] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=99d24a8cbbdb43e5, body=] 2024-11-29T09:52:04.8727+08:00 WARN [21697,99d24a8cbbdb43e5] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:04.87274+08:00 WARN [21697,99d24a8cbbdb43e5] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:04.87275+08:00 WARN [21697,99d24a8cbbdb43e5] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:04.87277+08:00 WARN [21697,99d24a8cbbdb43e5] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:04.87278+08:00 WARN [21697,99d24a8cbbdb43e5] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:04.87731+08:00 INFO [21697,99d24a8cbbdb43e5] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=99d24a8cbbdb43e5, body=] 2024-11-29T09:52:04.98893+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989155, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989155, max_request_id: 274989193 2024-11-29T09:52:05.04147+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:52:05.991+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989194, result count: 23 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989194, max_request_id: 274989216 2024-11-29T09:52:06.02193+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 12 2024-11-29T09:52:06.02199+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:52:06.02203+08:00 INFO [21697,8dd136b11dca38fd] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 311, lost data number: 563, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 12, sql_audit_progress Metrics number 1 2024-11-29T09:52:06.02465+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163136809, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163137195, max_request_id: 163137197 2024-11-29T09:52:06.02469+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163137195 is larger than startRequestId 163136809, maybe lost 386 data 2024-11-29T09:52:06.03809+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:52:06.03816+08:00 INFO [21697,e3dc1c2f7bdfdb63] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:52:06.1957+08:00 INFO [21697,6a7c770171335d0d] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=6a7c770171335d0d, body=] 2024-11-29T09:52:06.99302+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989217, result count: 90 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989217, max_request_id: 274989306 2024-11-29T09:52:07.09558+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:52:07.30788+08:00 INFO [21697,c0beb9d3adbf7372] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=c0beb9d3adbf7372, body=] 2024-11-29T09:52:07.30836+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:52:07.30888+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:52:07.78418+08:00 INFO [21697,66e67f09ab5cfd2d] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:52:07.78425+08:00 INFO [21697,66e67f09ab5cfd2d] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:52:07.765647104 +0800 CST m=+10835092.882256836, end time:2024-11-29 09:52:07 +0800 CST, Metrics number 0 2024-11-29T09:52:07.78433+08:00 WARN [21697,66e67f09ab5cfd2d] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:52:07.81814+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:52:07.81823+08:00 INFO [21697,4ac1fc12619e2bdc] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:52:07.8706+08:00 INFO [21697,76916b1b16da472c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=76916b1b16da472c, body=] 2024-11-29T09:52:07.87083+08:00 WARN [21697,76916b1b16da472c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:07.87086+08:00 WARN [21697,76916b1b16da472c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:07.87088+08:00 WARN [21697,76916b1b16da472c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:07.87089+08:00 WARN [21697,76916b1b16da472c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:07.8709+08:00 WARN [21697,76916b1b16da472c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:07.87451+08:00 INFO [21697,76916b1b16da472c] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=76916b1b16da472c, body=] 2024-11-29T09:52:07.99511+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989307, result count: 44 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989307, max_request_id: 274989350 2024-11-29T09:52:08.99683+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989351, result count: 17 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989351, max_request_id: 274989367 2024-11-29T09:52:09.99844+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989368, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989368, max_request_id: 274989407 2024-11-29T09:52:10.02617+08:00 INFO [21697,0b45e4c34bd34539] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=0b45e4c34bd34539, body=] 2024-11-29T09:52:10.02641+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:52:10.04189+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:52:10.0507+08:00 WARN [21697,] caller=common/process.go:106:refreshProcesses: failed to get process name of pid: 164228 fields: error="open /proc/164228/status: no such file or directory" 2024-11-29T09:52:10.14273+08:00 INFO [21697,243ea490284bb775] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=243ea490284bb775, body=] 2024-11-29T09:52:10.14309+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:10.14313+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:10.14314+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:10.14315+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:10.14316+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:10.1435+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:52:10.78452+08:00 INFO [21697,a1cdd3c81a62a136] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=a1cdd3c81a62a136, body=] 2024-11-29T09:52:10.87195+08:00 INFO [21697,42ae4ddf574547ee] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=42ae4ddf574547ee, body=] 2024-11-29T09:52:10.87216+08:00 WARN [21697,42ae4ddf574547ee] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:10.8722+08:00 WARN [21697,42ae4ddf574547ee] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:10.87221+08:00 WARN [21697,42ae4ddf574547ee] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:10.87222+08:00 WARN [21697,42ae4ddf574547ee] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:10.87224+08:00 WARN [21697,42ae4ddf574547ee] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:10.87609+08:00 INFO [21697,42ae4ddf574547ee] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=42ae4ddf574547ee, body=] 2024-11-29T09:52:11.00021+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989408, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989408, max_request_id: 274989443 2024-11-29T09:52:11.74367+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:52:12.00917+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989444, result count: 70 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989444, max_request_id: 274989513 2024-11-29T09:52:12.09545+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:52:12.78152+08:00 INFO [21697,449cf1e17700f26d] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:52:12.78159+08:00 INFO [21697,449cf1e17700f26d] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:52:12.765152089 +0800 CST m=+10835097.881761621, end time:2024-11-29 09:52:12 +0800 CST, Metrics number 0 2024-11-29T09:52:12.78168+08:00 WARN [21697,449cf1e17700f26d] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:52:13.01086+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989514, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989514, max_request_id: 274989553 2024-11-29T09:52:13.8706+08:00 INFO [21697,a8ebc3f724b14866] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=a8ebc3f724b14866, body=] 2024-11-29T09:52:13.87094+08:00 WARN [21697,a8ebc3f724b14866] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:13.87098+08:00 WARN [21697,a8ebc3f724b14866] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:13.871+08:00 WARN [21697,a8ebc3f724b14866] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:13.87102+08:00 WARN [21697,a8ebc3f724b14866] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:13.87103+08:00 WARN [21697,a8ebc3f724b14866] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:13.87554+08:00 INFO [21697,a8ebc3f724b14866] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=a8ebc3f724b14866, body=] 2024-11-29T09:52:14.01288+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989554, result count: 26 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989554, max_request_id: 274989579 2024-11-29T09:52:15.01528+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989580, result count: 29 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989580, max_request_id: 274989608 2024-11-29T09:52:15.04178+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:52:15.09369+08:00 INFO [21697,e27be31ac4ee41fb] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=e27be31ac4ee41fb, body=] 2024-11-29T09:52:15.09416+08:00 WARN [21697,e27be31ac4ee41fb] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:15.09421+08:00 WARN [21697,e27be31ac4ee41fb] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:15.09423+08:00 WARN [21697,e27be31ac4ee41fb] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:15.09425+08:00 WARN [21697,e27be31ac4ee41fb] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:15.09426+08:00 WARN [21697,e27be31ac4ee41fb] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:15.09473+08:00 INFO [21697,e27be31ac4ee41fb] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:52:15.09522+08:00 INFO [21697,e27be31ac4ee41fb] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=e27be31ac4ee41fb, body=] 2024-11-29T09:52:15.09732+08:00 INFO [21697,e27be31ac4ee41fb] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=e27be31ac4ee41fb, body=] 2024-11-29T09:52:15.09747+08:00 INFO [21697,e27be31ac4ee41fb] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=e27be31ac4ee41fb, body=] 2024-11-29T09:52:15.09765+08:00 WARN [21697,e27be31ac4ee41fb] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:52:15.09779+08:00 INFO [21697,e27be31ac4ee41fb] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=e27be31ac4ee41fb, body=] 2024-11-29T09:52:15.10096+08:00 WARN [21697,e27be31ac4ee41fb] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:52:15.10531+08:00 INFO [21697,e27be31ac4ee41fb] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:52:16.01865+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989609, result count: 49 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989609, max_request_id: 274989657 2024-11-29T09:52:16.87156+08:00 INFO [21697,5d547fc0f81c4a72] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=5d547fc0f81c4a72, body=] 2024-11-29T09:52:16.87179+08:00 WARN [21697,5d547fc0f81c4a72] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:16.87183+08:00 WARN [21697,5d547fc0f81c4a72] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:16.87185+08:00 WARN [21697,5d547fc0f81c4a72] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:16.87186+08:00 WARN [21697,5d547fc0f81c4a72] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:16.87187+08:00 WARN [21697,5d547fc0f81c4a72] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:16.87579+08:00 INFO [21697,5d547fc0f81c4a72] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=5d547fc0f81c4a72, body=] 2024-11-29T09:52:17.02113+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989658, result count: 59 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989658, max_request_id: 274989716 2024-11-29T09:52:17.09619+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:52:17.7838+08:00 INFO [21697,0bd8484919767336] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:52:17.78387+08:00 INFO [21697,0bd8484919767336] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:52:17.765216607 +0800 CST m=+10835102.881826139, end time:2024-11-29 09:52:17 +0800 CST, Metrics number 0 2024-11-29T09:52:17.78396+08:00 WARN [21697,0bd8484919767336] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:52:18.02284+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989717, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989717, max_request_id: 274989749 2024-11-29T09:52:19.02474+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989750, result count: 28 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989750, max_request_id: 274989777 2024-11-29T09:52:19.48116+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields: coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:52:19.48125+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:52:19.56244+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields: error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:52:19.87011+08:00 INFO [21697,9d783c1aec4a44b6] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=9d783c1aec4a44b6, body=] 2024-11-29T09:52:19.87036+08:00 WARN [21697,9d783c1aec4a44b6] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:19.8704+08:00 WARN [21697,9d783c1aec4a44b6] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:19.87042+08:00 WARN [21697,9d783c1aec4a44b6] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:19.87044+08:00 WARN [21697,9d783c1aec4a44b6] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:19.87045+08:00 WARN [21697,9d783c1aec4a44b6] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:19.87497+08:00 INFO [21697,9d783c1aec4a44b6] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=9d783c1aec4a44b6, body=] 2024-11-29T09:52:20.02646+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989778, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989778, max_request_id: 274989819 2024-11-29T09:52:20.04201+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:52:21.02823+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989820, result count: 28 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989820, max_request_id: 274989847 2024-11-29T09:52:21.19621+08:00 INFO [21697,baba33e708d353b0] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=baba33e708d353b0, body=] 2024-11-29T09:52:21.74818+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:52:22.03122+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989848, result count: 76 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989848, max_request_id: 274989923 2024-11-29T09:52:22.03331+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163137198, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163137561, max_request_id: 163137563 2024-11-29T09:52:22.03335+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163137561 is larger than startRequestId 163137198, maybe lost 363 data 2024-11-29T09:52:22.10772+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:52:22.30791+08:00 INFO [21697,6a9a1c90b2db4cff] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=6a9a1c90b2db4cff, body=] 2024-11-29T09:52:22.30832+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:52:22.30874+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:52:22.78154+08:00 INFO [21697,1d506fd8865e321a] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:52:22.78162+08:00 INFO [21697,1d506fd8865e321a] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:52:22.765215031 +0800 CST m=+10835107.881824564, end time:2024-11-29 09:52:22 +0800 CST, Metrics number 0 2024-11-29T09:52:22.7817+08:00 WARN [21697,1d506fd8865e321a] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:52:22.87149+08:00 INFO [21697,75fc9924002a4a34] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=75fc9924002a4a34, body=] 2024-11-29T09:52:22.87177+08:00 WARN [21697,75fc9924002a4a34] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:22.87184+08:00 WARN [21697,75fc9924002a4a34] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:22.87186+08:00 WARN [21697,75fc9924002a4a34] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:22.87188+08:00 WARN [21697,75fc9924002a4a34] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:22.87189+08:00 WARN [21697,75fc9924002a4a34] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:22.87772+08:00 INFO [21697,75fc9924002a4a34] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=75fc9924002a4a34, body=] 2024-11-29T09:52:23.03341+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989924, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989924, max_request_id: 274989966 2024-11-29T09:52:24.03549+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989967, result count: 18 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989967, max_request_id: 274989984 2024-11-29T09:52:25.02572+08:00 INFO [21697,849c04a7004865dd] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=849c04a7004865dd, body=] 2024-11-29T09:52:25.02597+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:52:25.03774+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274989985, result count: 44 auditMsg number: 0, diagMsg number: 0, min_request_id: 274989985, max_request_id: 274990028 2024-11-29T09:52:25.04198+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:52:25.14272+08:00 INFO [21697,a8b22ba81c980895] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=a8b22ba81c980895, body=] 2024-11-29T09:52:25.14305+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:25.1431+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:25.14311+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:25.14312+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:25.14314+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:25.14348+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:52:25.78363+08:00 INFO [21697,fa180f54d36e3ce1] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=fa180f54d36e3ce1, body=] 2024-11-29T09:52:25.86999+08:00 INFO [21697,74172d623b534e06] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=74172d623b534e06, body=] 2024-11-29T09:52:25.87021+08:00 WARN [21697,74172d623b534e06] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:25.87025+08:00 WARN [21697,74172d623b534e06] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:25.87026+08:00 WARN [21697,74172d623b534e06] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:25.87027+08:00 WARN [21697,74172d623b534e06] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:25.87028+08:00 WARN [21697,74172d623b534e06] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:25.87396+08:00 INFO [21697,74172d623b534e06] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=74172d623b534e06, body=] 2024-11-29T09:52:26.04039+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990029, result count: 68 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990029, max_request_id: 274990096 2024-11-29T09:52:27.04282+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990097, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990097, max_request_id: 274990139 2024-11-29T09:52:27.09498+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:52:27.79876+08:00 INFO [21697,a1f8f08684e77e11] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:52:27.79884+08:00 INFO [21697,a1f8f08684e77e11] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:52:27.782537171 +0800 CST m=+10835112.899146704, end time:2024-11-29 09:52:27 +0800 CST, Metrics number 0 2024-11-29T09:52:27.79893+08:00 WARN [21697,a1f8f08684e77e11] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:52:27.86218+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1008 from request_id 117579595, result count: 22 auditMsg number: 0, diagMsg number: 0, min_request_id: 117581193, max_request_id: 117581214 2024-11-29T09:52:27.86223+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1008 minRequestId 117581193 is larger than startRequestId 117579595, maybe lost 1598 data 2024-11-29T09:52:28.03847+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163137564, result count: 21 auditMsg number: 0, diagMsg number: 0, min_request_id: 163137673, max_request_id: 163137694 2024-11-29T09:52:28.03855+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163137673 is larger than startRequestId 163137564, maybe lost 109 data 2024-11-29T09:52:28.04621+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990140, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990140, max_request_id: 274990180 2024-11-29T09:52:28.86972+08:00 INFO [21697,e0478a943ab24770] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=e0478a943ab24770, body=] 2024-11-29T09:52:28.86993+08:00 WARN [21697,e0478a943ab24770] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:28.86997+08:00 WARN [21697,e0478a943ab24770] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:28.86998+08:00 WARN [21697,e0478a943ab24770] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:28.86999+08:00 WARN [21697,e0478a943ab24770] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:28.87+08:00 WARN [21697,e0478a943ab24770] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:28.8738+08:00 INFO [21697,e0478a943ab24770] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=e0478a943ab24770, body=] 2024-11-29T09:52:29.04052+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163137695, result count: 13 auditMsg number: 0, diagMsg number: 0, min_request_id: 163137768, max_request_id: 163137787 2024-11-29T09:52:29.0406+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163137768 is larger than startRequestId 163137695, maybe lost 73 data 2024-11-29T09:52:29.04785+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990181, result count: 28 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990181, max_request_id: 274990208 2024-11-29T09:52:29.5016+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1006 from request_id 136524672, result count: 21 auditMsg number: 0, diagMsg number: 0, min_request_id: 136526522, max_request_id: 136526542 2024-11-29T09:52:29.50166+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1006 minRequestId 136526522 is larger than startRequestId 136524672, maybe lost 1850 data 2024-11-29T09:52:30.04203+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:52:30.0495+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163137788, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 163137789, max_request_id: 163137826 2024-11-29T09:52:30.04956+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163137789 is larger than startRequestId 163137788, maybe lost 1 data 2024-11-29T09:52:30.05069+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990209, result count: 34 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990209, max_request_id: 274990242 2024-11-29T09:52:30.09333+08:00 INFO [21697,ea9b2036779f4cf0] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=ea9b2036779f4cf0, body=] 2024-11-29T09:52:30.09381+08:00 WARN [21697,ea9b2036779f4cf0] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:30.09385+08:00 WARN [21697,ea9b2036779f4cf0] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:30.09386+08:00 WARN [21697,ea9b2036779f4cf0] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:30.09388+08:00 WARN [21697,ea9b2036779f4cf0] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:30.0939+08:00 WARN [21697,ea9b2036779f4cf0] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:30.09431+08:00 INFO [21697,ea9b2036779f4cf0] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:52:30.09471+08:00 INFO [21697,ea9b2036779f4cf0] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=ea9b2036779f4cf0, body=] 2024-11-29T09:52:30.09644+08:00 INFO [21697,ea9b2036779f4cf0] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=ea9b2036779f4cf0, body=] 2024-11-29T09:52:30.09658+08:00 INFO [21697,ea9b2036779f4cf0] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=ea9b2036779f4cf0, body=] 2024-11-29T09:52:30.09671+08:00 WARN [21697,ea9b2036779f4cf0] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:52:30.09682+08:00 INFO [21697,ea9b2036779f4cf0] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=ea9b2036779f4cf0, body=] 2024-11-29T09:52:30.09907+08:00 WARN [21697,ea9b2036779f4cf0] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:52:30.10537+08:00 INFO [21697,ea9b2036779f4cf0] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:52:31.05578+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990243, result count: 100 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990243, max_request_id: 274990342 2024-11-29T09:52:31.33467+08:00 INFO [21697,f3756b5263c0af63] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.98.86, ocpServerIp=, traceId=f3756b5263c0af63, body=] 2024-11-29T09:52:31.33486+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:31.33489+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:31.3349+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:31.33491+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:31.33492+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:31.74984+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:52:31.8696+08:00 INFO [21697,02fa446a08054c2c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=02fa446a08054c2c, body=] 2024-11-29T09:52:31.86982+08:00 WARN [21697,02fa446a08054c2c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:31.86985+08:00 WARN [21697,02fa446a08054c2c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:31.86986+08:00 WARN [21697,02fa446a08054c2c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:31.86988+08:00 WARN [21697,02fa446a08054c2c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:31.86989+08:00 WARN [21697,02fa446a08054c2c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:31.87609+08:00 INFO [21697,02fa446a08054c2c] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=02fa446a08054c2c, body=] 2024-11-29T09:52:32.05906+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990343, result count: 149 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990343, max_request_id: 274990491 2024-11-29T09:52:32.10834+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:52:32.78204+08:00 INFO [21697,defc14afaadee0cf] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:52:32.78212+08:00 INFO [21697,defc14afaadee0cf] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:52:32.765797799 +0800 CST m=+10835117.882407332, end time:2024-11-29 09:52:32 +0800 CST, Metrics number 0 2024-11-29T09:52:32.78221+08:00 WARN [21697,defc14afaadee0cf] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:52:32.85524+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:52:32.85666+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:52:32.85675+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:52:32.8568+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:52:32.85683+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:52:32.85687+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:52:32.8569+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:52:32.85693+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:52:32.85697+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:52:32.857+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:52:32.85703+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:52:32.85705+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:52:32.87211+08:00 INFO [21697,f222ca9dcc25d21e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:52:32 +0800 CST, start_request_id:0 2024-11-29T09:52:32.87218+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:32.87949+08:00 INFO [21697,f222ca9dcc25d21e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:52:32 +0800 CST, start_request_id:0 2024-11-29T09:52:32.87957+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:32.87965+08:00 INFO [21697,f222ca9dcc25d21e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:52:32 +0800 CST, start_request_id:0 2024-11-29T09:52:32.87967+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:32.87978+08:00 INFO [21697,f222ca9dcc25d21e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:52:32 +0800 CST, start_request_id:7154 2024-11-29T09:52:32.8798+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:32.88685+08:00 INFO [21697,f222ca9dcc25d21e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:52:32 +0800 CST, start_request_id:0 2024-11-29T09:52:32.88692+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:32.88698+08:00 INFO [21697,f222ca9dcc25d21e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:52:32 +0800 CST, start_request_id:544433 2024-11-29T09:52:32.887+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544433, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:32.88719+08:00 INFO [21697,f222ca9dcc25d21e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:52:32 +0800 CST, start_request_id:0 2024-11-29T09:52:32.88721+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:32.88869+08:00 INFO [21697,f222ca9dcc25d21e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:52:32 +0800 CST, start_request_id:387 2024-11-29T09:52:32.88874+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:32.89487+08:00 INFO [21697,f222ca9dcc25d21e] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:52:32 +0800 CST, start_request_id:0 2024-11-29T09:52:32.89494+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:52:32.91477+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:52:32.91484+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:52:32.91486+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:52:32.91496+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:52:32.91498+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:52:32.91502+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:52:32.91589+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907674, got 60 result 2024-11-29T09:52:32.91599+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:52:32.91601+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:52:32.91603+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 60, planExplainMetrics number 0, planMetaMetrics number 60 2024-11-29T09:52:32.9161+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:52:32.91612+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:52:32.91614+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 60, planExplainMetrics number 0, planMetaMetrics number 60 2024-11-29T09:52:32.9288+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:52:32.92884+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:52:32.92888+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 60, planExplainMetrics number 0, planMetaMetrics number 60 2024-11-29T09:52:32.92895+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:52:32.92897+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:52:32.929+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 60, planExplainMetrics number 0, planMetaMetrics number 60 2024-11-29T09:52:32.93634+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807606, got 32 result 2024-11-29T09:52:32.93706+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907674, got 60 result, planStatMetrics number 60, planMetaMetrics number 60, planExplainMetrics number 0 2024-11-29T09:52:32.93709+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 92, planExplainMetrics number 0, planMetaMetrics number 92 2024-11-29T09:52:32.95198+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807606, got 32 result, planStatMetrics number 32, planMetaMetrics number 32, planExplainMetrics number 0 2024-11-29T09:52:32.95203+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 92, planExplainMetrics number 0, planMetaMetrics number 92 2024-11-29T09:52:32.97196+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:52:32.97204+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:52:32.97206+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 92, planExplainMetrics number 0, planMetaMetrics number 92 2024-11-29T09:52:32.97211+08:00 INFO [21697,a00af3607c4b087e] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:52:32 +0800 CST, end time:2024-11-29 09:52:32 +0800 CST, planMetaMetrics number 92, planExplainMetrics number 0, planStatMetrics number 92 2024-11-29T09:52:33.06054+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 50 2024-11-29T09:52:33.06061+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:52:33.06065+08:00 INFO [21697,40b2fd2cf34d9565] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1449, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 50, sql_audit_progress Metrics number 1 2024-11-29T09:52:33.06487+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990492, result count: 244 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990492, max_request_id: 274990735 2024-11-29T09:52:33.86504+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1008 topSqlMetrics length: 7 2024-11-29T09:52:33.8651+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:52:33.86514+08:00 INFO [21697,fdece95939d67b0d] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 22, lost data number: 1598, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 7, sql_audit_progress Metrics number 1 2024-11-29T09:52:34.06678+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990736, result count: 26 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990736, max_request_id: 274990761 2024-11-29T09:52:34.48172+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:52:34.4818+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:52:34.55418+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields: error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:52:34.87047+08:00 INFO [21697,ba5aa7e4f98d4c2b] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=ba5aa7e4f98d4c2b, body=] 2024-11-29T09:52:34.87068+08:00 WARN [21697,ba5aa7e4f98d4c2b] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:34.87072+08:00 WARN [21697,ba5aa7e4f98d4c2b] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:34.87074+08:00 WARN [21697,ba5aa7e4f98d4c2b] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:34.87076+08:00 WARN [21697,ba5aa7e4f98d4c2b] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:34.87078+08:00 WARN [21697,ba5aa7e4f98d4c2b] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:34.87441+08:00 INFO [21697,ba5aa7e4f98d4c2b] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=ba5aa7e4f98d4c2b, body=] 2024-11-29T09:52:35.04127+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:52:35.06892+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990762, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990762, max_request_id: 274990803 2024-11-29T09:52:35.50352+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1006 topSqlMetrics length: 7 2024-11-29T09:52:35.5036+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:52:35.50365+08:00 INFO [21697,34c7ff46724cfdbb] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 21, lost data number: 1850, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 7, sql_audit_progress Metrics number 1 2024-11-29T09:52:36.05485+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 19 2024-11-29T09:52:36.05491+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:52:36.05495+08:00 INFO [21697,c001ddc9c72689ac] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 71, lost data number: 932, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 19, sql_audit_progress Metrics number 1 2024-11-29T09:52:36.06071+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163137827, result count: 265 auditMsg number: 0, diagMsg number: 0, min_request_id: 163137900, max_request_id: 163138233 2024-11-29T09:52:36.06075+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163137900 is larger than startRequestId 163137827, maybe lost 73 data 2024-11-29T09:52:36.07091+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990804, result count: 34 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990804, max_request_id: 274990837 2024-11-29T09:52:36.19498+08:00 INFO [21697,46d7882f8b7ef3ec] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=46d7882f8b7ef3ec, body=] 2024-11-29T09:52:36.2455+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:52:36.24561+08:00 INFO [21697,89bc4f5ba8f30fc8] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:52:37.07858+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990838, result count: 82 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990838, max_request_id: 274990919 2024-11-29T09:52:37.09439+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:52:37.30687+08:00 INFO [21697,4454a72d0becc9a7] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=4454a72d0becc9a7, body=] 2024-11-29T09:52:37.30729+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:52:37.30774+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:52:37.78351+08:00 INFO [21697,2aecfd04beb73244] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:52:37.78359+08:00 INFO [21697,2aecfd04beb73244] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:52:37.765686041 +0800 CST m=+10835122.882295573, end time:2024-11-29 09:52:37 +0800 CST, Metrics number 0 2024-11-29T09:52:37.78367+08:00 WARN [21697,2aecfd04beb73244] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:52:37.86957+08:00 INFO [21697,cba7932d8361416f] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=cba7932d8361416f, body=] 2024-11-29T09:52:37.86981+08:00 WARN [21697,cba7932d8361416f] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:37.86985+08:00 WARN [21697,cba7932d8361416f] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:37.86987+08:00 WARN [21697,cba7932d8361416f] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:37.86988+08:00 WARN [21697,cba7932d8361416f] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:37.86989+08:00 WARN [21697,cba7932d8361416f] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:37.87461+08:00 INFO [21697,cba7932d8361416f] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=cba7932d8361416f, body=] 2024-11-29T09:52:38.08112+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990920, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990920, max_request_id: 274990962 2024-11-29T09:52:39.08293+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990963, result count: 18 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990963, max_request_id: 274990980 2024-11-29T09:52:40.02556+08:00 INFO [21697,acdd9f5911467e80] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=acdd9f5911467e80, body=] 2024-11-29T09:52:40.02578+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:52:40.04129+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:52:40.08462+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274990981, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 274990981, max_request_id: 274991020 2024-11-29T09:52:40.14274+08:00 INFO [21697,b4962911e5bb6f81] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=b4962911e5bb6f81, body=] 2024-11-29T09:52:40.1431+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:40.14314+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:40.14317+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:40.14318+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:40.14319+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:40.14358+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:52:40.78301+08:00 INFO [21697,da84ea253144b8c9] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=da84ea253144b8c9, body=] 2024-11-29T09:52:40.87115+08:00 INFO [21697,fa5365ad2a9047c1] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=fa5365ad2a9047c1, body=] 2024-11-29T09:52:40.87141+08:00 WARN [21697,fa5365ad2a9047c1] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:40.87145+08:00 WARN [21697,fa5365ad2a9047c1] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:40.87148+08:00 WARN [21697,fa5365ad2a9047c1] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:40.87149+08:00 WARN [21697,fa5365ad2a9047c1] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:40.87151+08:00 WARN [21697,fa5365ad2a9047c1] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:40.87596+08:00 INFO [21697,fa5365ad2a9047c1] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=fa5365ad2a9047c1, body=] 2024-11-29T09:52:41.0873+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991021, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991021, max_request_id: 274991062 2024-11-29T09:52:41.7439+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:52:42.08996+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991063, result count: 64 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991063, max_request_id: 274991126 2024-11-29T09:52:42.115+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:52:42.78223+08:00 INFO [21697,08e79269c7c01f47] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:52:42.7823+08:00 INFO [21697,08e79269c7c01f47] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:52:42.765331322 +0800 CST m=+10835127.881940854, end time:2024-11-29 09:52:42 +0800 CST, Metrics number 0 2024-11-29T09:52:42.78239+08:00 WARN [21697,08e79269c7c01f47] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:52:43.09246+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991127, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991127, max_request_id: 274991164 2024-11-29T09:52:43.86942+08:00 INFO [21697,9b33bab6f942476c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=9b33bab6f942476c, body=] 2024-11-29T09:52:43.86971+08:00 WARN [21697,9b33bab6f942476c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:43.86975+08:00 WARN [21697,9b33bab6f942476c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:43.86977+08:00 WARN [21697,9b33bab6f942476c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:43.86979+08:00 WARN [21697,9b33bab6f942476c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:43.86981+08:00 WARN [21697,9b33bab6f942476c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:43.87469+08:00 INFO [21697,9b33bab6f942476c] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=9b33bab6f942476c, body=] 2024-11-29T09:52:44.10741+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991165, result count: 27 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991165, max_request_id: 274991191 2024-11-29T09:52:45.04146+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:52:45.09278+08:00 INFO [21697,29c227d3ea294250] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=29c227d3ea294250, body=] 2024-11-29T09:52:45.09315+08:00 WARN [21697,29c227d3ea294250] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:45.09319+08:00 WARN [21697,29c227d3ea294250] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:45.0932+08:00 WARN [21697,29c227d3ea294250] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:45.09321+08:00 WARN [21697,29c227d3ea294250] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:45.09322+08:00 WARN [21697,29c227d3ea294250] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:45.09357+08:00 INFO [21697,29c227d3ea294250] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:52:45.09394+08:00 INFO [21697,29c227d3ea294250] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=29c227d3ea294250, body=] 2024-11-29T09:52:45.09711+08:00 INFO [21697,29c227d3ea294250] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=29c227d3ea294250, body=] 2024-11-29T09:52:45.09724+08:00 INFO [21697,29c227d3ea294250] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=29c227d3ea294250, body=] 2024-11-29T09:52:45.09737+08:00 WARN [21697,29c227d3ea294250] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:52:45.09747+08:00 INFO [21697,29c227d3ea294250] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=29c227d3ea294250, body=] 2024-11-29T09:52:45.09892+08:00 WARN [21697,29c227d3ea294250] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:52:45.10463+08:00 INFO [21697,29c227d3ea294250] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:52:45.11752+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991192, result count: 29 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991192, max_request_id: 274991220 2024-11-29T09:52:46.12009+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991221, result count: 60 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991221, max_request_id: 274991280 2024-11-29T09:52:46.87068+08:00 INFO [21697,81f31882f3034544] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=81f31882f3034544, body=] 2024-11-29T09:52:46.87093+08:00 WARN [21697,81f31882f3034544] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:46.87096+08:00 WARN [21697,81f31882f3034544] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:46.87099+08:00 WARN [21697,81f31882f3034544] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:46.871+08:00 WARN [21697,81f31882f3034544] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:46.87101+08:00 WARN [21697,81f31882f3034544] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:46.87511+08:00 INFO [21697,81f31882f3034544] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=81f31882f3034544, body=] 2024-11-29T09:52:47.09528+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:52:47.12214+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991281, result count: 50 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991281, max_request_id: 274991330 2024-11-29T09:52:47.78147+08:00 INFO [21697,fd8d56ed5489be21] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:52:47.78154+08:00 INFO [21697,fd8d56ed5489be21] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:52:47.765070596 +0800 CST m=+10835132.881680129, end time:2024-11-29 09:52:47 +0800 CST, Metrics number 0 2024-11-29T09:52:47.78163+08:00 WARN [21697,fd8d56ed5489be21] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:52:48.12506+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991331, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991331, max_request_id: 274991366 2024-11-29T09:52:48.30731+08:00 INFO [21697,d78e14564db48ed4] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.98.86, ocpServerIp=, traceId=d78e14564db48ed4, body=] 2024-11-29T09:52:49.12747+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991367, result count: 29 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991367, max_request_id: 274991395 2024-11-29T09:52:49.48077+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:52:49.48087+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:52:49.55187+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:52:49.86935+08:00 INFO [21697,425af959561f482d] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=425af959561f482d, body=] 2024-11-29T09:52:49.86962+08:00 WARN [21697,425af959561f482d] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:49.86966+08:00 WARN [21697,425af959561f482d] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:49.86968+08:00 WARN [21697,425af959561f482d] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:49.86969+08:00 WARN [21697,425af959561f482d] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:49.86972+08:00 WARN [21697,425af959561f482d] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:49.87447+08:00 INFO [21697,425af959561f482d] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=425af959561f482d, body=] 2024-11-29T09:52:50.04172+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:52:50.12957+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991396, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991396, max_request_id: 274991436 2024-11-29T09:52:51.1333+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991437, result count: 45 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991437, max_request_id: 274991481 2024-11-29T09:52:51.19496+08:00 INFO [21697,509534b80c14a3c1] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=509534b80c14a3c1, body=] 2024-11-29T09:52:51.74358+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:52:52.07139+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163138234, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163138559, max_request_id: 163138561 2024-11-29T09:52:52.07144+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163138559 is larger than startRequestId 163138234, maybe lost 325 data 2024-11-29T09:52:52.10826+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:52:52.13644+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991482, result count: 68 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991482, max_request_id: 274991549 2024-11-29T09:52:52.30703+08:00 INFO [21697,562ee70caa754565] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=562ee70caa754565, body=] 2024-11-29T09:52:52.30742+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:52:52.30773+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:52:52.37861+08:00 INFO [21697,ed85e7e5ed336283] caller=host/custom.go:178:func10: collect observer dir exists info 2024-11-29T09:52:52.78149+08:00 INFO [21697,9b736edf0eee3116] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:52:52.78157+08:00 INFO [21697,9b736edf0eee3116] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:52:52.765352926 +0800 CST m=+10835137.881962459, end time:2024-11-29 09:52:52 +0800 CST, Metrics number 0 2024-11-29T09:52:52.78167+08:00 WARN [21697,9b736edf0eee3116] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:52:52.86902+08:00 INFO [21697,a2782b7ecf2b4cdf] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=a2782b7ecf2b4cdf, body=] 2024-11-29T09:52:52.86935+08:00 WARN [21697,a2782b7ecf2b4cdf] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:52.8694+08:00 WARN [21697,a2782b7ecf2b4cdf] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:52.86942+08:00 WARN [21697,a2782b7ecf2b4cdf] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:52.86946+08:00 WARN [21697,a2782b7ecf2b4cdf] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:52.86948+08:00 WARN [21697,a2782b7ecf2b4cdf] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:52.87347+08:00 INFO [21697,a2782b7ecf2b4cdf] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=a2782b7ecf2b4cdf, body=] 2024-11-29T09:52:53.1394+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991550, result count: 73 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991550, max_request_id: 274991622 2024-11-29T09:52:54.14122+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991623, result count: 19 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991623, max_request_id: 274991641 2024-11-29T09:52:55.02465+08:00 INFO [21697,5c1b070255c9946e] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=5c1b070255c9946e, body=] 2024-11-29T09:52:55.02491+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:52:55.04157+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:52:55.14217+08:00 INFO [21697,07e156dccc339654] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=07e156dccc339654, body=] 2024-11-29T09:52:55.14254+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:55.14259+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:55.1426+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:55.14261+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:55.14263+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:55.14296+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1506, err: 2024-11-29T09:52:55.14398+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991642, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991642, max_request_id: 274991679 2024-11-29T09:52:55.78257+08:00 INFO [21697,7e0256903c30f570] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=7e0256903c30f570, body=] 2024-11-29T09:52:55.86887+08:00 INFO [21697,73689f14fb6c4b23] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=73689f14fb6c4b23, body=] 2024-11-29T09:52:55.86911+08:00 WARN [21697,73689f14fb6c4b23] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:55.86915+08:00 WARN [21697,73689f14fb6c4b23] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:55.86917+08:00 WARN [21697,73689f14fb6c4b23] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:55.86919+08:00 WARN [21697,73689f14fb6c4b23] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:55.8692+08:00 WARN [21697,73689f14fb6c4b23] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:55.87363+08:00 INFO [21697,73689f14fb6c4b23] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=73689f14fb6c4b23, body=] 2024-11-29T09:52:56.14609+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991680, result count: 45 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991680, max_request_id: 274991724 2024-11-29T09:52:57.09565+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:52:57.14751+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991725, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991725, max_request_id: 274991760 2024-11-29T09:52:57.78505+08:00 INFO [21697,f0cd286ecaa68147] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:52:57.78513+08:00 INFO [21697,f0cd286ecaa68147] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:52:57.765188375 +0800 CST m=+10835142.881797908, end time:2024-11-29 09:52:57 +0800 CST, Metrics number 0 2024-11-29T09:52:57.78522+08:00 WARN [21697,f0cd286ecaa68147] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:52:58.14968+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991761, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991761, max_request_id: 274991800 2024-11-29T09:52:58.86912+08:00 INFO [21697,f12490421cd245d0] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=f12490421cd245d0, body=] 2024-11-29T09:52:58.86937+08:00 WARN [21697,f12490421cd245d0] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:52:58.8694+08:00 WARN [21697,f12490421cd245d0] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:52:58.86942+08:00 WARN [21697,f12490421cd245d0] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:52:58.86944+08:00 WARN [21697,f12490421cd245d0] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:58.86945+08:00 WARN [21697,f12490421cd245d0] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:52:58.87385+08:00 INFO [21697,f12490421cd245d0] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=f12490421cd245d0, body=] 2024-11-29T09:52:59.15154+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991801, result count: 26 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991801, max_request_id: 274991826 2024-11-29T09:53:00.04205+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:53:00.09246+08:00 INFO [21697,1206d363168a4b57] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=1206d363168a4b57, body=] 2024-11-29T09:53:00.09301+08:00 WARN [21697,1206d363168a4b57] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:00.09305+08:00 WARN [21697,1206d363168a4b57] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:00.09306+08:00 WARN [21697,1206d363168a4b57] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:00.09308+08:00 WARN [21697,1206d363168a4b57] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:00.09309+08:00 WARN [21697,1206d363168a4b57] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:00.09366+08:00 INFO [21697,1206d363168a4b57] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:53:00.09418+08:00 INFO [21697,1206d363168a4b57] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=1206d363168a4b57, body=] 2024-11-29T09:53:00.09834+08:00 INFO [21697,1206d363168a4b57] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=1206d363168a4b57, body=] 2024-11-29T09:53:00.09849+08:00 INFO [21697,1206d363168a4b57] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=1206d363168a4b57, body=] 2024-11-29T09:53:00.09865+08:00 WARN [21697,1206d363168a4b57] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:53:00.09877+08:00 INFO [21697,1206d363168a4b57] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=1206d363168a4b57, body=] 2024-11-29T09:53:00.09925+08:00 WARN [21697,1206d363168a4b57] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:53:00.11387+08:00 INFO [21697,1206d363168a4b57] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:53:00.15367+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991827, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991827, max_request_id: 274991863 2024-11-29T09:53:00.78528+08:00 INFO [21697,e202b0bbdcc4120f] caller=common/middleware.go:59:func1: API request: [GET /api/v1/log/alarms, client=1.1.1.69.58, ocpServerIp=1.1.1.69.58, traceId=e202b0bbdcc4120f, body=] 2024-11-29T09:53:00.78543+08:00 INFO [21697,] caller=alarmscache/alarm_handler.go:73:getPipelieGroupReader: get alarms, result size:0, cache list size:4 fields:, uri=/api/v1/log/alarms 2024-11-29T09:53:01.15612+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991864, result count: 99 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991864, max_request_id: 274991962 2024-11-29T09:53:01.74364+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:53:01.87418+08:00 INFO [21697,3476d62138b14b2d] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=3476d62138b14b2d, body=] 2024-11-29T09:53:01.87448+08:00 WARN [21697,3476d62138b14b2d] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:01.87452+08:00 WARN [21697,3476d62138b14b2d] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:01.87454+08:00 WARN [21697,3476d62138b14b2d] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:01.87456+08:00 WARN [21697,3476d62138b14b2d] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:01.87458+08:00 WARN [21697,3476d62138b14b2d] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:01.87844+08:00 INFO [21697,3476d62138b14b2d] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=3476d62138b14b2d, body=] 2024-11-29T09:53:02.09464+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:53:02.16033+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274991963, result count: 101 auditMsg number: 0, diagMsg number: 0, min_request_id: 274991963, max_request_id: 274992063 2024-11-29T09:53:02.74259+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:245:updateMetaInfo: successfully init sqlAudit tenant config 2024-11-29T09:53:02.78146+08:00 INFO [21697,f9f2c4e2d2df1bf6] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:53:02.78152+08:00 INFO [21697,f9f2c4e2d2df1bf6] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:53:02.765016372 +0800 CST m=+10835147.881625905, end time:2024-11-29 09:53:02 +0800 CST, Metrics number 0 2024-11-29T09:53:02.7816+08:00 WARN [21697,f9f2c4e2d2df1bf6] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:53:02.85803+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 1 result 2024-11-29T09:53:02.8595+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:53:02.85959+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:53:02.85961+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:53:02.85964+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:53:02.85966+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:53:02.85968+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:53:02.8597+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:53:02.85972+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:53:02.85974+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:53:02.85983+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 1 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:53:02.85986+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:53:02.86589+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:796:refreshTenants: configured tenants: map[1:sys 1002:ocp 1004:dbamonitor 1006:dbmanager 1008:datatransfer] 2024-11-29T09:53:02.8731+08:00 INFO [21697,1dec224a82546c74] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:53:02 +0800 CST, start_request_id:0 2024-11-29T09:53:02.87317+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:02.87324+08:00 INFO [21697,1dec224a82546c74] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:53:02 +0800 CST, start_request_id:0 2024-11-29T09:53:02.87327+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:02.87332+08:00 INFO [21697,1dec224a82546c74] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:53:02 +0800 CST, start_request_id:544433 2024-11-29T09:53:02.87333+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544433, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:02.88035+08:00 INFO [21697,1dec224a82546c74] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:53:02 +0800 CST, start_request_id:0 2024-11-29T09:53:02.88042+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:02.88051+08:00 INFO [21697,1dec224a82546c74] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:53:02 +0800 CST, start_request_id:0 2024-11-29T09:53:02.88053+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:02.88062+08:00 INFO [21697,1dec224a82546c74] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:53:02 +0800 CST, start_request_id:7154 2024-11-29T09:53:02.88064+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:02.89029+08:00 INFO [21697,1dec224a82546c74] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:53:02 +0800 CST, start_request_id:0 2024-11-29T09:53:02.89039+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:02.89082+08:00 INFO [21697,1dec224a82546c74] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:53:02 +0800 CST, start_request_id:387 2024-11-29T09:53:02.89086+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:02.89381+08:00 INFO [21697,1dec224a82546c74] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:53:02 +0800 CST, start_request_id:0 2024-11-29T09:53:02.89386+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:02.91221+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907694, got 20 result 2024-11-29T09:53:02.91235+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:53:02.91238+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:53:02.9124+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 20, planExplainMetrics number 0, planMetaMetrics number 20 2024-11-29T09:53:02.91249+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:53:02.91251+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:53:02.91258+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 20, planExplainMetrics number 0, planMetaMetrics number 20 2024-11-29T09:53:02.91268+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:53:02.9127+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:53:02.91272+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 20, planExplainMetrics number 0, planMetaMetrics number 20 2024-11-29T09:53:02.91293+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807614, got 8 result 2024-11-29T09:53:02.91301+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:53:02.91303+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:53:02.9131+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:53:02.91312+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:53:02.91316+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 28, planExplainMetrics number 0, planMetaMetrics number 28 2024-11-29T09:53:02.91318+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 28, planExplainMetrics number 0, planMetaMetrics number 28 2024-11-29T09:53:02.91331+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:53:02.91333+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:53:02.91335+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 28, planExplainMetrics number 0, planMetaMetrics number 28 2024-11-29T09:53:02.91636+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807614, got 8 result, planStatMetrics number 8, planMetaMetrics number 8, planExplainMetrics number 0 2024-11-29T09:53:02.9164+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 28, planExplainMetrics number 0, planMetaMetrics number 28 2024-11-29T09:53:02.92105+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907694, got 20 result, planStatMetrics number 20, planMetaMetrics number 20, planExplainMetrics number 0 2024-11-29T09:53:02.92109+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 28, planExplainMetrics number 0, planMetaMetrics number 28 2024-11-29T09:53:02.97253+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:53:02.97262+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:53:02.97264+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 28, planExplainMetrics number 0, planMetaMetrics number 28 2024-11-29T09:53:02.97271+08:00 INFO [21697,8610236a2d28b38d] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:53:02 +0800 CST, end time:2024-11-29 09:53:02 +0800 CST, planMetaMetrics number 28, planExplainMetrics number 0, planStatMetrics number 28 2024-11-29T09:53:03.07691+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 5 2024-11-29T09:53:03.07697+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:53:03.07706+08:00 INFO [21697,f494f06db9cb2eaa] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 268, lost data number: 398, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 5, sql_audit_progress Metrics number 1 2024-11-29T09:53:03.07993+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163138562, result count: 21 auditMsg number: 0, diagMsg number: 0, min_request_id: 163138821, max_request_id: 163138855 2024-11-29T09:53:03.07997+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163138821 is larger than startRequestId 163138562, maybe lost 259 data 2024-11-29T09:53:03.16135+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 49 2024-11-29T09:53:03.16142+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:53:03.16146+08:00 INFO [21697,f241f213dd4b0975] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1572, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 49, sql_audit_progress Metrics number 1 2024-11-29T09:53:03.16457+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992064, result count: 125 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992064, max_request_id: 274992188 2024-11-29T09:53:03.89189+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:53:03.89199+08:00 INFO [21697,d0191a8aac857ebd] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:53:03.90036+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1008 from request_id 117581215, result count: 26 auditMsg number: 0, diagMsg number: 0, min_request_id: 117581922, max_request_id: 117581949 2024-11-29T09:53:03.9004+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1008 minRequestId 117581922 is larger than startRequestId 117581215, maybe lost 707 data 2024-11-29T09:53:04.16834+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992189, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992189, max_request_id: 274992230 2024-11-29T09:53:04.48101+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:53:04.4811+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:53:04.55162+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:53:04.8701+08:00 INFO [21697,4093f08e8705458c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=4093f08e8705458c, body=] 2024-11-29T09:53:04.87031+08:00 WARN [21697,4093f08e8705458c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:04.87035+08:00 WARN [21697,4093f08e8705458c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:04.87036+08:00 WARN [21697,4093f08e8705458c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:04.87037+08:00 WARN [21697,4093f08e8705458c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:04.87038+08:00 WARN [21697,4093f08e8705458c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:04.87415+08:00 INFO [21697,4093f08e8705458c] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=4093f08e8705458c, body=] 2024-11-29T09:53:05.0419+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:53:05.1708+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992231, result count: 50 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992231, max_request_id: 274992280 2024-11-29T09:53:05.52417+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:53:05.52427+08:00 INFO [21697,c62858c56d79551e] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:53:05.53867+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1006 from request_id 136526543, result count: 21 auditMsg number: 0, diagMsg number: 0, min_request_id: 136527360, max_request_id: 136527396 2024-11-29T09:53:05.53874+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1006 minRequestId 136527360 is larger than startRequestId 136526543, maybe lost 817 data 2024-11-29T09:53:06.17353+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992281, result count: 47 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992281, max_request_id: 274992327 2024-11-29T09:53:06.1942+08:00 INFO [21697,0fced562990178de] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=0fced562990178de, body=] 2024-11-29T09:53:06.43711+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:53:06.43722+08:00 INFO [21697,f08e24245476bca2] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:53:07.09527+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:53:07.17607+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992328, result count: 73 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992328, max_request_id: 274992400 2024-11-29T09:53:07.30661+08:00 INFO [21697,3f8746fe91400f8a] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=3f8746fe91400f8a, body=] 2024-11-29T09:53:07.30718+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:53:07.30753+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:53:07.78785+08:00 INFO [21697,9a773bb872d09b81] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:53:07.78792+08:00 INFO [21697,9a773bb872d09b81] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:53:07.765490189 +0800 CST m=+10835152.882099722, end time:2024-11-29 09:53:07 +0800 CST, Metrics number 0 2024-11-29T09:53:07.78801+08:00 WARN [21697,9a773bb872d09b81] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:53:07.86873+08:00 INFO [21697,ec08d11415924119] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=ec08d11415924119, body=] 2024-11-29T09:53:07.86897+08:00 WARN [21697,ec08d11415924119] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:07.869+08:00 WARN [21697,ec08d11415924119] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:07.86902+08:00 WARN [21697,ec08d11415924119] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:07.86903+08:00 WARN [21697,ec08d11415924119] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:07.86904+08:00 WARN [21697,ec08d11415924119] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:07.87286+08:00 INFO [21697,ec08d11415924119] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=ec08d11415924119, body=] 2024-11-29T09:53:08.17861+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992401, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992401, max_request_id: 274992441 2024-11-29T09:53:09.08829+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163138856, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163138971, max_request_id: 163138973 2024-11-29T09:53:09.08836+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163138971 is larger than startRequestId 163138856, maybe lost 115 data 2024-11-29T09:53:09.18047+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992442, result count: 16 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992442, max_request_id: 274992457 2024-11-29T09:53:10.02431+08:00 INFO [21697,bdc0e721c768c4f8] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=bdc0e721c768c4f8, body=] 2024-11-29T09:53:10.02458+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:53:10.04102+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:53:10.14594+08:00 INFO [21697,b234c4eb22e9e024] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=b234c4eb22e9e024, body=] 2024-11-29T09:53:10.14631+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:10.14634+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:10.14636+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:10.14637+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:10.14638+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:10.14673+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:53:10.18212+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992458, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992458, max_request_id: 274992498 2024-11-29T09:53:10.78298+08:00 INFO [21697,a2b7c3b536e502fd] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=a2b7c3b536e502fd, body=] 2024-11-29T09:53:10.86889+08:00 INFO [21697,1b0683ab0126436f] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=1b0683ab0126436f, body=] 2024-11-29T09:53:10.8691+08:00 WARN [21697,1b0683ab0126436f] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:10.86914+08:00 WARN [21697,1b0683ab0126436f] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:10.86916+08:00 WARN [21697,1b0683ab0126436f] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:10.86917+08:00 WARN [21697,1b0683ab0126436f] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:10.86919+08:00 WARN [21697,1b0683ab0126436f] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:10.873+08:00 INFO [21697,1b0683ab0126436f] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=1b0683ab0126436f, body=] 2024-11-29T09:53:11.18485+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992499, result count: 49 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992499, max_request_id: 274992547 2024-11-29T09:53:11.74355+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:53:12.09533+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:53:12.18897+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992548, result count: 49 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992548, max_request_id: 274992596 2024-11-29T09:53:12.7904+08:00 INFO [21697,8774e9f777fae894] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:53:12.79047+08:00 INFO [21697,8774e9f777fae894] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:53:12.765522006 +0800 CST m=+10835157.882131539, end time:2024-11-29 09:53:12 +0800 CST, Metrics number 0 2024-11-29T09:53:12.79056+08:00 WARN [21697,8774e9f777fae894] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:53:13.19199+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992597, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992597, max_request_id: 274992636 2024-11-29T09:53:13.86858+08:00 INFO [21697,f4c7873935684513] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=f4c7873935684513, body=] 2024-11-29T09:53:13.86892+08:00 WARN [21697,f4c7873935684513] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:13.86895+08:00 WARN [21697,f4c7873935684513] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:13.86896+08:00 WARN [21697,f4c7873935684513] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:13.86898+08:00 WARN [21697,f4c7873935684513] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:13.86899+08:00 WARN [21697,f4c7873935684513] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:13.87305+08:00 INFO [21697,f4c7873935684513] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=f4c7873935684513, body=] 2024-11-29T09:53:14.1945+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992637, result count: 27 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992637, max_request_id: 274992663 2024-11-29T09:53:15.04135+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:53:15.09198+08:00 INFO [21697,30fa1b6e89c84afa] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=30fa1b6e89c84afa, body=] 2024-11-29T09:53:15.09241+08:00 WARN [21697,30fa1b6e89c84afa] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:15.09245+08:00 WARN [21697,30fa1b6e89c84afa] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:15.09247+08:00 WARN [21697,30fa1b6e89c84afa] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:15.09248+08:00 WARN [21697,30fa1b6e89c84afa] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:15.0925+08:00 WARN [21697,30fa1b6e89c84afa] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:15.0929+08:00 INFO [21697,30fa1b6e89c84afa] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:53:15.09351+08:00 INFO [21697,30fa1b6e89c84afa] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=30fa1b6e89c84afa, body=] 2024-11-29T09:53:15.09709+08:00 INFO [21697,30fa1b6e89c84afa] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=30fa1b6e89c84afa, body=] 2024-11-29T09:53:15.09722+08:00 INFO [21697,30fa1b6e89c84afa] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=30fa1b6e89c84afa, body=] 2024-11-29T09:53:15.09734+08:00 WARN [21697,30fa1b6e89c84afa] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:53:15.09746+08:00 INFO [21697,30fa1b6e89c84afa] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=30fa1b6e89c84afa, body=] 2024-11-29T09:53:15.09906+08:00 WARN [21697,30fa1b6e89c84afa] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:53:15.11024+08:00 INFO [21697,30fa1b6e89c84afa] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:53:15.19589+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992664, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992664, max_request_id: 274992698 2024-11-29T09:53:16.19891+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992699, result count: 61 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992699, max_request_id: 274992759 2024-11-29T09:53:16.86862+08:00 INFO [21697,c48231c2eacf4afe] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=c48231c2eacf4afe, body=] 2024-11-29T09:53:16.8688+08:00 WARN [21697,c48231c2eacf4afe] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:16.86883+08:00 WARN [21697,c48231c2eacf4afe] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:16.86885+08:00 WARN [21697,c48231c2eacf4afe] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:16.86886+08:00 WARN [21697,c48231c2eacf4afe] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:16.86888+08:00 WARN [21697,c48231c2eacf4afe] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:16.87329+08:00 INFO [21697,c48231c2eacf4afe] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=c48231c2eacf4afe, body=] 2024-11-29T09:53:17.09598+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:53:17.20127+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992760, result count: 44 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992760, max_request_id: 274992803 2024-11-29T09:53:17.78197+08:00 INFO [21697,1e18b4e40c0e1bc8] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:53:17.78205+08:00 INFO [21697,1e18b4e40c0e1bc8] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:53:17.765357126 +0800 CST m=+10835162.881966674, end time:2024-11-29 09:53:17 +0800 CST, Metrics number 0 2024-11-29T09:53:17.78214+08:00 WARN [21697,1e18b4e40c0e1bc8] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:53:18.20348+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992804, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992804, max_request_id: 274992835 2024-11-29T09:53:19.20503+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992836, result count: 24 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992836, max_request_id: 274992859 2024-11-29T09:53:19.48132+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:53:19.48141+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:53:19.5523+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:53:19.86864+08:00 INFO [21697,911fa3c089cc41fa] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=911fa3c089cc41fa, body=] 2024-11-29T09:53:19.86885+08:00 WARN [21697,911fa3c089cc41fa] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:19.86889+08:00 WARN [21697,911fa3c089cc41fa] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:19.8689+08:00 WARN [21697,911fa3c089cc41fa] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:19.86892+08:00 WARN [21697,911fa3c089cc41fa] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:19.86893+08:00 WARN [21697,911fa3c089cc41fa] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:19.87282+08:00 INFO [21697,911fa3c089cc41fa] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=911fa3c089cc41fa, body=] 2024-11-29T09:53:20.04175+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:53:20.20667+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992860, result count: 47 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992860, max_request_id: 274992906 2024-11-29T09:53:21.19418+08:00 INFO [21697,980f91af2deccbb1] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=980f91af2deccbb1, body=] 2024-11-29T09:53:21.20898+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992907, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992907, max_request_id: 274992945 2024-11-29T09:53:21.74392+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:53:22.09474+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:53:22.2117+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274992946, result count: 57 auditMsg number: 0, diagMsg number: 0, min_request_id: 274992946, max_request_id: 274993002 2024-11-29T09:53:22.30581+08:00 INFO [21697,28ca18a9e21a90df] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=28ca18a9e21a90df, body=] 2024-11-29T09:53:22.3063+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:53:22.30681+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:53:22.78298+08:00 INFO [21697,79d1e11e83a827af] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:53:22.78305+08:00 INFO [21697,79d1e11e83a827af] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:53:22.765909496 +0800 CST m=+10835167.882519028, end time:2024-11-29 09:53:22 +0800 CST, Metrics number 0 2024-11-29T09:53:22.78313+08:00 WARN [21697,79d1e11e83a827af] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:53:22.86979+08:00 INFO [21697,c7351ac5e2c5491a] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=c7351ac5e2c5491a, body=] 2024-11-29T09:53:22.86999+08:00 WARN [21697,c7351ac5e2c5491a] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:22.87002+08:00 WARN [21697,c7351ac5e2c5491a] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:22.87004+08:00 WARN [21697,c7351ac5e2c5491a] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:22.87005+08:00 WARN [21697,c7351ac5e2c5491a] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:22.87006+08:00 WARN [21697,c7351ac5e2c5491a] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:22.8741+08:00 INFO [21697,c7351ac5e2c5491a] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=c7351ac5e2c5491a, body=] 2024-11-29T09:53:23.21412+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993003, result count: 46 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993003, max_request_id: 274993048 2024-11-29T09:53:24.21572+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993049, result count: 18 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993049, max_request_id: 274993066 2024-11-29T09:53:25.02428+08:00 INFO [21697,17facf1c8539afed] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=17facf1c8539afed, body=] 2024-11-29T09:53:25.02455+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:53:25.04206+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:53:25.09896+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163138974, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163139298, max_request_id: 163139300 2024-11-29T09:53:25.09901+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163139298 is larger than startRequestId 163138974, maybe lost 324 data 2024-11-29T09:53:25.14108+08:00 INFO [21697,ca87dc3c959d8d2b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=ca87dc3c959d8d2b, body=] 2024-11-29T09:53:25.1417+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:25.14178+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:25.14183+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:25.14186+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:25.14188+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:25.1423+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:53:25.21829+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993067, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993067, max_request_id: 274993107 2024-11-29T09:53:25.78197+08:00 INFO [21697,2402901bc0618962] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=2402901bc0618962, body=] 2024-11-29T09:53:25.8683+08:00 INFO [21697,e82aaf2815114bf8] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=e82aaf2815114bf8, body=] 2024-11-29T09:53:25.8686+08:00 WARN [21697,e82aaf2815114bf8] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:25.86864+08:00 WARN [21697,e82aaf2815114bf8] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:25.86865+08:00 WARN [21697,e82aaf2815114bf8] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:25.86866+08:00 WARN [21697,e82aaf2815114bf8] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:25.86867+08:00 WARN [21697,e82aaf2815114bf8] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:25.87326+08:00 INFO [21697,e82aaf2815114bf8] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=e82aaf2815114bf8, body=] 2024-11-29T09:53:26.2206+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993108, result count: 54 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993108, max_request_id: 274993161 2024-11-29T09:53:27.0954+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:53:27.22234+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993162, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993162, max_request_id: 274993192 2024-11-29T09:53:27.78148+08:00 INFO [21697,716d6f708e0065d1] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:53:27.78154+08:00 INFO [21697,716d6f708e0065d1] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:53:27.765168915 +0800 CST m=+10835172.881778452, end time:2024-11-29 09:53:27 +0800 CST, Metrics number 0 2024-11-29T09:53:27.78164+08:00 WARN [21697,716d6f708e0065d1] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:53:28.22434+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993193, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993193, max_request_id: 274993232 2024-11-29T09:53:28.86813+08:00 INFO [21697,bbc3c836afce441b] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=bbc3c836afce441b, body=] 2024-11-29T09:53:28.86844+08:00 WARN [21697,bbc3c836afce441b] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:28.86849+08:00 WARN [21697,bbc3c836afce441b] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:28.86853+08:00 WARN [21697,bbc3c836afce441b] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:28.86856+08:00 WARN [21697,bbc3c836afce441b] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:28.86858+08:00 WARN [21697,bbc3c836afce441b] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:28.87369+08:00 INFO [21697,bbc3c836afce441b] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=bbc3c836afce441b, body=] 2024-11-29T09:53:29.22665+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993233, result count: 27 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993233, max_request_id: 274993259 2024-11-29T09:53:30.04177+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:53:30.04789+08:00 WARN [21697,] caller=common/process.go:106:refreshProcesses: failed to get process name of pid: 165460 fields: error="open /proc/165460/status: no such file or directory" 2024-11-29T09:53:30.04795+08:00 WARN [21697,] caller=common/process.go:106:refreshProcesses: failed to get process name of pid: 165461 fields:, error="open /proc/165461/status: no such file or directory" 2024-11-29T09:53:30.04797+08:00 WARN [21697,] caller=common/process.go:106:refreshProcesses: failed to get process name of pid: 165462 fields: error="open /proc/165462/status: no such file or directory" 2024-11-29T09:53:30.04799+08:00 WARN [21697,] caller=common/process.go:106:refreshProcesses: failed to get process name of pid: 165463 fields:, error="open /proc/165463/status: no such file or directory" 2024-11-29T09:53:30.09158+08:00 INFO [21697,10f04bd264e44dff] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=10f04bd264e44dff, body=] 2024-11-29T09:53:30.09178+08:00 INFO [21697,10f04bd264e44dff] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=10f04bd264e44dff, body=] 2024-11-29T09:53:30.1005+08:00 WARN [21697,10f04bd264e44dff] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:30.10057+08:00 WARN [21697,10f04bd264e44dff] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:30.10058+08:00 WARN [21697,10f04bd264e44dff] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:30.10059+08:00 WARN [21697,10f04bd264e44dff] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:30.10061+08:00 WARN [21697,10f04bd264e44dff] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:30.10298+08:00 INFO [21697,10f04bd264e44dff] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:53:30.10742+08:00 INFO [21697,10f04bd264e44dff] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=10f04bd264e44dff, body=] 2024-11-29T09:53:30.11393+08:00 INFO [21697,10f04bd264e44dff] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=10f04bd264e44dff, body=] 2024-11-29T09:53:30.11417+08:00 WARN [21697,10f04bd264e44dff] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:53:30.11446+08:00 INFO [21697,10f04bd264e44dff] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:53:30.11496+08:00 INFO [21697,10f04bd264e44dff] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=10f04bd264e44dff, body=] 2024-11-29T09:53:30.11509+08:00 WARN [21697,10f04bd264e44dff] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:53:30.22866+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993260, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993260, max_request_id: 274993296 2024-11-29T09:53:31.1045+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163139301, result count: 52 auditMsg number: 0, diagMsg number: 0, min_request_id: 163139483, max_request_id: 163139549 2024-11-29T09:53:31.10457+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163139483 is larger than startRequestId 163139301, maybe lost 182 data 2024-11-29T09:53:31.23137+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993297, result count: 85 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993297, max_request_id: 274993381 2024-11-29T09:53:31.32905+08:00 INFO [21697,18ebdae0bbf94b0e] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.98.86, ocpServerIp=, traceId=18ebdae0bbf94b0e, body=] 2024-11-29T09:53:31.32927+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:31.3293+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:31.32932+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:31.32933+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:31.32934+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:31.74366+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:53:31.86791+08:00 INFO [21697,ef309ea2f5014681] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=ef309ea2f5014681, body=] 2024-11-29T09:53:31.86815+08:00 WARN [21697,ef309ea2f5014681] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:31.86818+08:00 WARN [21697,ef309ea2f5014681] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:31.86819+08:00 WARN [21697,ef309ea2f5014681] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:31.86821+08:00 WARN [21697,ef309ea2f5014681] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:31.86822+08:00 WARN [21697,ef309ea2f5014681] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:31.87241+08:00 INFO [21697,ef309ea2f5014681] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=ef309ea2f5014681, body=] 2024-11-29T09:53:32.10959+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163139550, result count: 112 auditMsg number: 0, diagMsg number: 0, min_request_id: 163139596, max_request_id: 163139729 2024-11-29T09:53:32.10966+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163139596 is larger than startRequestId 163139550, maybe lost 46 data 2024-11-29T09:53:32.1102+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:53:32.23612+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993382, result count: 153 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993382, max_request_id: 274993534 2024-11-29T09:53:32.79838+08:00 INFO [21697,d75e4590bbfee2bd] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:53:32.79846+08:00 INFO [21697,d75e4590bbfee2bd] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:53:32.765584866 +0800 CST m=+10835177.882194411, end time:2024-11-29 09:53:32 +0800 CST, Metrics number 0 2024-11-29T09:53:32.79858+08:00 WARN [21697,d75e4590bbfee2bd] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:53:32.8732+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:53:32.87344+08:00 INFO [21697,251b2164f94b5b10] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:53:32 +0800 CST, start_request_id:0 2024-11-29T09:53:32.87347+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:32.87357+08:00 INFO [21697,251b2164f94b5b10] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:53:32 +0800 CST, start_request_id:0 2024-11-29T09:53:32.8736+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:32.87368+08:00 INFO [21697,251b2164f94b5b10] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:53:32 +0800 CST, start_request_id:544433 2024-11-29T09:53:32.8737+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544433, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:32.87467+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:53:32.87478+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:53:32.87481+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:53:32.87485+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:53:32.87488+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:53:32.87491+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:53:32.87495+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:53:32.875+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:53:32.87504+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:53:32.87507+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:53:32.8751+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:53:32.88281+08:00 INFO [21697,251b2164f94b5b10] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:53:32 +0800 CST, start_request_id:7154 2024-11-29T09:53:32.88287+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:32.88297+08:00 INFO [21697,251b2164f94b5b10] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:53:32 +0800 CST, start_request_id:0 2024-11-29T09:53:32.88299+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:32.88751+08:00 INFO [21697,251b2164f94b5b10] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:53:32 +0800 CST, start_request_id:0 2024-11-29T09:53:32.88758+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:32.88766+08:00 INFO [21697,251b2164f94b5b10] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:53:32 +0800 CST, start_request_id:387 2024-11-29T09:53:32.88769+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:32.88796+08:00 INFO [21697,251b2164f94b5b10] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:53:32 +0800 CST, start_request_id:0 2024-11-29T09:53:32.88798+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:32.89645+08:00 INFO [21697,251b2164f94b5b10] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:53:32 +0800 CST, start_request_id:0 2024-11-29T09:53:32.8965+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:53:32.90491+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:53:32.90498+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:53:32.90502+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:53:32.90511+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:53:32.90513+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:53:32.9052+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:53:32.90522+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:53:32.90526+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:53:32.90528+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:53:32.90541+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907696, got 2 result 2024-11-29T09:53:32.90551+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:53:32.90553+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:53:32.90554+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 2, planExplainMetrics number 0, planMetaMetrics number 2 2024-11-29T09:53:32.90568+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807617, got 3 result 2024-11-29T09:53:32.90674+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907696, got 2 result, planStatMetrics number 2, planMetaMetrics number 2, planExplainMetrics number 0 2024-11-29T09:53:32.90678+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:53:32.90727+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807617, got 3 result, planStatMetrics number 3, planMetaMetrics number 3, planExplainMetrics number 0 2024-11-29T09:53:32.9073+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:53:32.9231+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:53:32.92316+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:53:32.9232+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:53:32.92332+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:53:32.92334+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:53:32.92335+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:53:32.95945+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:53:32.95952+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:53:32.95954+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:53:32.95961+08:00 INFO [21697,134e34515907e120] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:53:32 +0800 CST, end time:2024-11-29 09:53:32 +0800 CST, planMetaMetrics number 5, planExplainMetrics number 0, planStatMetrics number 5 2024-11-29T09:53:33.11012+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 21 2024-11-29T09:53:33.11019+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:53:33.11024+08:00 INFO [21697,15f9699823b61e33] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 191, lost data number: 926, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 21, sql_audit_progress Metrics number 1 2024-11-29T09:53:33.11388+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163139730, result count: 150 auditMsg number: 0, diagMsg number: 0, min_request_id: 163139730, max_request_id: 163139892 2024-11-29T09:53:33.23704+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 52 2024-11-29T09:53:33.2371+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:53:33.23714+08:00 INFO [21697,06048b96c95c604c] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1471, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 52, sql_audit_progress Metrics number 1 2024-11-29T09:53:33.23953+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993535, result count: 74 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993535, max_request_id: 274993608 2024-11-29T09:53:34.24249+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993609, result count: 26 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993609, max_request_id: 274993634 2024-11-29T09:53:34.48181+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields: coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:53:34.4819+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:53:34.55172+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields: error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:53:34.86788+08:00 INFO [21697,c45a79b2b04c4bb9] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=c45a79b2b04c4bb9, body=] 2024-11-29T09:53:34.86809+08:00 WARN [21697,c45a79b2b04c4bb9] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:34.86813+08:00 WARN [21697,c45a79b2b04c4bb9] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:34.86815+08:00 WARN [21697,c45a79b2b04c4bb9] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:34.86816+08:00 WARN [21697,c45a79b2b04c4bb9] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:34.86817+08:00 WARN [21697,c45a79b2b04c4bb9] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:34.87212+08:00 INFO [21697,c45a79b2b04c4bb9] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=c45a79b2b04c4bb9, body=] 2024-11-29T09:53:34.921+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1008 topSqlMetrics length: 11 2024-11-29T09:53:34.92107+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:53:34.92111+08:00 INFO [21697,04c303991bf430f6] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 26, lost data number: 707, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 11, sql_audit_progress Metrics number 1 2024-11-29T09:53:35.04115+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:53:35.24468+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993635, result count: 52 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993635, max_request_id: 274993686 2024-11-29T09:53:36.19363+08:00 INFO [21697,40603a3fd4651d84] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=40603a3fd4651d84, body=] 2024-11-29T09:53:36.25474+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993687, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993687, max_request_id: 274993727 2024-11-29T09:53:36.55882+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1006 topSqlMetrics length: 7 2024-11-29T09:53:36.55889+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:53:36.55893+08:00 INFO [21697,fc674afaff3cfec2] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 21, lost data number: 817, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 7, sql_audit_progress Metrics number 1 2024-11-29T09:53:36.70698+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:53:36.70708+08:00 INFO [21697,64b8e3fdfd0d6682] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:53:37.09601+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:53:37.25743+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993728, result count: 71 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993728, max_request_id: 274993798 2024-11-29T09:53:37.30514+08:00 INFO [21697,b8701c3aee76ecb3] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=b8701c3aee76ecb3, body=] 2024-11-29T09:53:37.30553+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:53:37.30592+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:53:37.7814+08:00 INFO [21697,c0d6ce5bdf38402a] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:53:37.78146+08:00 INFO [21697,c0d6ce5bdf38402a] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:53:37.765035113 +0800 CST m=+10835182.881644646, end time:2024-11-29 09:53:37 +0800 CST, Metrics number 0 2024-11-29T09:53:37.78154+08:00 WARN [21697,c0d6ce5bdf38402a] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:53:37.86887+08:00 INFO [21697,8f0739c64c5447db] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=8f0739c64c5447db, body=] 2024-11-29T09:53:37.86912+08:00 WARN [21697,8f0739c64c5447db] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:37.86916+08:00 WARN [21697,8f0739c64c5447db] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:37.86918+08:00 WARN [21697,8f0739c64c5447db] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:37.86919+08:00 WARN [21697,8f0739c64c5447db] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:37.86921+08:00 WARN [21697,8f0739c64c5447db] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:37.87327+08:00 INFO [21697,8f0739c64c5447db] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=8f0739c64c5447db, body=] 2024-11-29T09:53:38.26394+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993799, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993799, max_request_id: 274993841 2024-11-29T09:53:39.11839+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163139893, result count: 8 auditMsg number: 0, diagMsg number: 0, min_request_id: 163139954, max_request_id: 163139961 2024-11-29T09:53:39.11847+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163139954 is larger than startRequestId 163139893, maybe lost 61 data 2024-11-29T09:53:39.26615+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993842, result count: 24 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993842, max_request_id: 274993865 2024-11-29T09:53:40.02335+08:00 INFO [21697,e9754fa9f5253c3b] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=e9754fa9f5253c3b, body=] 2024-11-29T09:53:40.0236+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:53:40.0412+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:53:40.14037+08:00 INFO [21697,7976b842405c8f59] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=7976b842405c8f59, body=] 2024-11-29T09:53:40.14079+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:40.14083+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:40.14084+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:40.14086+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:40.14087+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:40.14126+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:53:40.27127+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993866, result count: 44 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993866, max_request_id: 274993909 2024-11-29T09:53:40.78213+08:00 INFO [21697,85ccab6c53b7d809] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=85ccab6c53b7d809, body=] 2024-11-29T09:53:40.86933+08:00 INFO [21697,5bc550232e9d4184] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=5bc550232e9d4184, body=] 2024-11-29T09:53:40.86957+08:00 WARN [21697,5bc550232e9d4184] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:40.8696+08:00 WARN [21697,5bc550232e9d4184] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:40.86962+08:00 WARN [21697,5bc550232e9d4184] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:40.86963+08:00 WARN [21697,5bc550232e9d4184] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:40.86965+08:00 WARN [21697,5bc550232e9d4184] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:40.87384+08:00 INFO [21697,5bc550232e9d4184] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=5bc550232e9d4184, body=] 2024-11-29T09:53:41.27318+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993910, result count: 49 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993910, max_request_id: 274993958 2024-11-29T09:53:41.74376+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:53:42.09517+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:53:42.27621+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274993959, result count: 49 auditMsg number: 0, diagMsg number: 0, min_request_id: 274993959, max_request_id: 274994007 2024-11-29T09:53:42.79789+08:00 INFO [21697,0d9d25b0c8840361] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:53:42.79796+08:00 INFO [21697,0d9d25b0c8840361] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:53:42.765649716 +0800 CST m=+10835187.882259249, end time:2024-11-29 09:53:42 +0800 CST, Metrics number 0 2024-11-29T09:53:42.79805+08:00 WARN [21697,0d9d25b0c8840361] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:53:43.27859+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994008, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994008, max_request_id: 274994045 2024-11-29T09:53:43.86779+08:00 INFO [21697,7c680b3ccb7a4677] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=7c680b3ccb7a4677, body=] 2024-11-29T09:53:43.86802+08:00 WARN [21697,7c680b3ccb7a4677] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:43.86806+08:00 WARN [21697,7c680b3ccb7a4677] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:43.86807+08:00 WARN [21697,7c680b3ccb7a4677] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:43.86809+08:00 WARN [21697,7c680b3ccb7a4677] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:43.8681+08:00 WARN [21697,7c680b3ccb7a4677] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:43.8719+08:00 INFO [21697,7c680b3ccb7a4677] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=7c680b3ccb7a4677, body=] 2024-11-29T09:53:44.28147+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994046, result count: 29 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994046, max_request_id: 274994074 2024-11-29T09:53:45.04198+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:53:45.09116+08:00 INFO [21697,0696fca9aedb4c2c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=0696fca9aedb4c2c, body=] 2024-11-29T09:53:45.09161+08:00 WARN [21697,0696fca9aedb4c2c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:45.09165+08:00 WARN [21697,0696fca9aedb4c2c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:45.09166+08:00 WARN [21697,0696fca9aedb4c2c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:45.09167+08:00 WARN [21697,0696fca9aedb4c2c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:45.09169+08:00 WARN [21697,0696fca9aedb4c2c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:45.09206+08:00 INFO [21697,0696fca9aedb4c2c] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:53:45.09245+08:00 INFO [21697,0696fca9aedb4c2c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=0696fca9aedb4c2c, body=] 2024-11-29T09:53:45.09595+08:00 INFO [21697,0696fca9aedb4c2c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=0696fca9aedb4c2c, body=] 2024-11-29T09:53:45.09608+08:00 INFO [21697,0696fca9aedb4c2c] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=0696fca9aedb4c2c, body=] 2024-11-29T09:53:45.09622+08:00 WARN [21697,0696fca9aedb4c2c] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:53:45.09633+08:00 INFO [21697,0696fca9aedb4c2c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=0696fca9aedb4c2c, body=] 2024-11-29T09:53:45.09673+08:00 WARN [21697,0696fca9aedb4c2c] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:53:45.10777+08:00 INFO [21697,0696fca9aedb4c2c] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:53:45.28363+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994075, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994075, max_request_id: 274994109 2024-11-29T09:53:46.28632+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994110, result count: 60 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994110, max_request_id: 274994169 2024-11-29T09:53:46.86762+08:00 INFO [21697,f489f048110c4c1f] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=f489f048110c4c1f, body=] 2024-11-29T09:53:46.86795+08:00 WARN [21697,f489f048110c4c1f] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:46.86801+08:00 WARN [21697,f489f048110c4c1f] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:46.86803+08:00 WARN [21697,f489f048110c4c1f] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:46.86804+08:00 WARN [21697,f489f048110c4c1f] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:46.86806+08:00 WARN [21697,f489f048110c4c1f] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:46.87331+08:00 INFO [21697,f489f048110c4c1f] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=f489f048110c4c1f, body=] 2024-11-29T09:53:47.09481+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:53:47.28868+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994170, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994170, max_request_id: 274994209 2024-11-29T09:53:47.79903+08:00 INFO [21697,38ccbf62f9ede584] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:53:47.79909+08:00 INFO [21697,38ccbf62f9ede584] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:53:47.765647038 +0800 CST m=+10835192.882256571, end time:2024-11-29 09:53:47 +0800 CST, Metrics number 0 2024-11-29T09:53:47.79916+08:00 WARN [21697,38ccbf62f9ede584] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:53:48.29096+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994210, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994210, max_request_id: 274994241 2024-11-29T09:53:48.30625+08:00 INFO [21697,49543f1ab3130f8d] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.98.86, ocpServerIp=, traceId=49543f1ab3130f8d, body=] 2024-11-29T09:53:49.29308+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994242, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994242, max_request_id: 274994274 2024-11-29T09:53:49.48081+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields: coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:53:49.48089+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:53:49.5585+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:53:49.86771+08:00 INFO [21697,8770319ae6624fa0] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=8770319ae6624fa0, body=] 2024-11-29T09:53:49.86796+08:00 WARN [21697,8770319ae6624fa0] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:49.868+08:00 WARN [21697,8770319ae6624fa0] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:49.86801+08:00 WARN [21697,8770319ae6624fa0] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:49.86802+08:00 WARN [21697,8770319ae6624fa0] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:49.86804+08:00 WARN [21697,8770319ae6624fa0] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:49.87503+08:00 INFO [21697,8770319ae6624fa0] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=8770319ae6624fa0, body=] 2024-11-29T09:53:50.0419+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:53:50.29575+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994275, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994275, max_request_id: 274994317 2024-11-29T09:53:51.19371+08:00 INFO [21697,083cac5c20b4bf2c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=083cac5c20b4bf2c, body=] 2024-11-29T09:53:51.29963+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994318, result count: 44 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994318, max_request_id: 274994361 2024-11-29T09:53:51.74402+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:53:52.10824+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:53:52.30249+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994362, result count: 59 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994362, max_request_id: 274994420 2024-11-29T09:53:52.30501+08:00 INFO [21697,2052c34215017409] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=2052c34215017409, body=] 2024-11-29T09:53:52.30534+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:53:52.30568+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:53:52.37878+08:00 INFO [21697,ed85e7e5ed336283] caller=host/custom.go:178:func10: collect observer dir exists info 2024-11-29T09:53:52.79066+08:00 INFO [21697,32b2016fa775442d] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:53:52.79072+08:00 INFO [21697,32b2016fa775442d] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:53:52.765707196 +0800 CST m=+10835197.882316729, end time:2024-11-29 09:53:52 +0800 CST, Metrics number 0 2024-11-29T09:53:52.7908+08:00 WARN [21697,32b2016fa775442d] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:53:52.86904+08:00 INFO [21697,4cc24ef5426c4851] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=4cc24ef5426c4851, body=] 2024-11-29T09:53:52.86926+08:00 WARN [21697,4cc24ef5426c4851] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:52.86929+08:00 WARN [21697,4cc24ef5426c4851] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:52.86931+08:00 WARN [21697,4cc24ef5426c4851] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:52.86932+08:00 WARN [21697,4cc24ef5426c4851] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:52.86933+08:00 WARN [21697,4cc24ef5426c4851] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:52.87306+08:00 INFO [21697,4cc24ef5426c4851] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=4cc24ef5426c4851, body=] 2024-11-29T09:53:53.30588+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994421, result count: 46 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994421, max_request_id: 274994466 2024-11-29T09:53:54.30803+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994467, result count: 28 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994467, max_request_id: 274994494 2024-11-29T09:53:55.02354+08:00 INFO [21697,6dfa305c5a110de4] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=6dfa305c5a110de4, body=] 2024-11-29T09:53:55.02379+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:53:55.04143+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:53:55.13578+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163139962, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163140286, max_request_id: 163140288 2024-11-29T09:53:55.13583+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163140286 is larger than startRequestId 163139962, maybe lost 324 data 2024-11-29T09:53:55.14042+08:00 INFO [21697,529e8228f858d239] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=529e8228f858d239, body=] 2024-11-29T09:53:55.14079+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:55.14083+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:55.14084+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:55.14086+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:55.14087+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:55.14127+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1506, err: 2024-11-29T09:53:55.31023+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994495, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994495, max_request_id: 274994533 2024-11-29T09:53:55.78073+08:00 INFO [21697,2b45a71aacbf8502] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=2b45a71aacbf8502, body=] 2024-11-29T09:53:55.86777+08:00 INFO [21697,ed97768310b64f6e] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=ed97768310b64f6e, body=] 2024-11-29T09:53:55.868+08:00 WARN [21697,ed97768310b64f6e] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:55.86803+08:00 WARN [21697,ed97768310b64f6e] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:55.86805+08:00 WARN [21697,ed97768310b64f6e] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:55.86806+08:00 WARN [21697,ed97768310b64f6e] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:55.86807+08:00 WARN [21697,ed97768310b64f6e] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:55.87185+08:00 INFO [21697,ed97768310b64f6e] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=ed97768310b64f6e, body=] 2024-11-29T09:53:56.31328+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994534, result count: 51 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994534, max_request_id: 274994584 2024-11-29T09:53:57.09459+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:53:57.31566+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994585, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994585, max_request_id: 274994616 2024-11-29T09:53:57.80104+08:00 INFO [21697,3aeddd893d0ef3d7] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:53:57.8011+08:00 INFO [21697,3aeddd893d0ef3d7] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:53:57.76541192 +0800 CST m=+10835202.882021459, end time:2024-11-29 09:53:57 +0800 CST, Metrics number 0 2024-11-29T09:53:57.80118+08:00 WARN [21697,3aeddd893d0ef3d7] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:53:58.31809+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994617, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994617, max_request_id: 274994653 2024-11-29T09:53:58.86749+08:00 INFO [21697,7afde58f89154bcc] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=7afde58f89154bcc, body=] 2024-11-29T09:53:58.86779+08:00 WARN [21697,7afde58f89154bcc] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:53:58.86783+08:00 WARN [21697,7afde58f89154bcc] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:53:58.86786+08:00 WARN [21697,7afde58f89154bcc] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:53:58.86788+08:00 WARN [21697,7afde58f89154bcc] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:58.8679+08:00 WARN [21697,7afde58f89154bcc] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:53:58.87413+08:00 INFO [21697,7afde58f89154bcc] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=7afde58f89154bcc, body=] 2024-11-29T09:53:59.32079+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994654, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994654, max_request_id: 274994688 2024-11-29T09:54:00.04182+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:54:00.09074+08:00 INFO [21697,845f8bfc23da473c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=845f8bfc23da473c, body=] 2024-11-29T09:54:00.09114+08:00 WARN [21697,845f8bfc23da473c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:00.09118+08:00 WARN [21697,845f8bfc23da473c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:00.0912+08:00 WARN [21697,845f8bfc23da473c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:00.09121+08:00 WARN [21697,845f8bfc23da473c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:00.09123+08:00 WARN [21697,845f8bfc23da473c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:00.09162+08:00 INFO [21697,845f8bfc23da473c] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:54:00.09202+08:00 INFO [21697,845f8bfc23da473c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=845f8bfc23da473c, body=] 2024-11-29T09:54:00.09604+08:00 INFO [21697,845f8bfc23da473c] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=845f8bfc23da473c, body=] 2024-11-29T09:54:00.09621+08:00 WARN [21697,845f8bfc23da473c] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:54:00.09633+08:00 INFO [21697,845f8bfc23da473c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=845f8bfc23da473c, body=] 2024-11-29T09:54:00.09644+08:00 INFO [21697,845f8bfc23da473c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=845f8bfc23da473c, body=] 2024-11-29T09:54:00.09858+08:00 WARN [21697,845f8bfc23da473c] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:54:00.10992+08:00 INFO [21697,845f8bfc23da473c] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 610, err: 2024-11-29T09:54:00.32233+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994689, result count: 34 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994689, max_request_id: 274994722 2024-11-29T09:54:00.67017+08:00 INFO [21697,498cfa9676b9a316] caller=common/middleware.go:59:func1: API request: [GET /api/v1/log/alarms, client=1.1.1.180, ocpServerIp=1.1.1.180, traceId=498cfa9676b9a316, body=] 2024-11-29T09:54:00.67047+08:00 INFO [21697,] caller=alarmscache/alarm_handler.go:73:getPipelieGroupReader: get alarms, result size:0, cache list size:4 fields:, uri=/api/v1/log/alarms 2024-11-29T09:54:01.33811+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994723, result count: 141 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994723, max_request_id: 274994863 2024-11-29T09:54:01.74375+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:54:01.86855+08:00 INFO [21697,7d99030e85604dfd] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=7d99030e85604dfd, body=] 2024-11-29T09:54:01.86873+08:00 WARN [21697,7d99030e85604dfd] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:01.86878+08:00 WARN [21697,7d99030e85604dfd] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:01.86879+08:00 WARN [21697,7d99030e85604dfd] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:01.8688+08:00 WARN [21697,7d99030e85604dfd] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:01.86882+08:00 WARN [21697,7d99030e85604dfd] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:01.87259+08:00 INFO [21697,7d99030e85604dfd] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=7d99030e85604dfd, body=] 2024-11-29T09:54:02.10815+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:54:02.34563+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994864, result count: 64 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994864, max_request_id: 274994927 2024-11-29T09:54:02.74227+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:245:updateMetaInfo: successfully init sqlAudit tenant config 2024-11-29T09:54:02.78235+08:00 INFO [21697,9356e83b5d93c014] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 1 result 2024-11-29T09:54:02.7824+08:00 INFO [21697,9356e83b5d93c014] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:54:02.765931575 +0800 CST m=+10835207.882541122, end time:2024-11-29 09:54:02 +0800 CST, Metrics number 1 2024-11-29T09:54:02.78247+08:00 INFO [21697,9356e83b5d93c014] caller=engine/pipeline.go:145:pipelinePush: push pipeline ob_sql_transaction metrics count 1 2024-11-29T09:54:02.83918+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:295:needReportPlanStat: the time need report sql plan stat data 2024-11-29T09:54:02.85784+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:54:02.85935+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:54:02.85942+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:54:02.85944+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:54:02.85947+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:54:02.8595+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:54:02.85953+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:54:02.85957+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:54:02.8596+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:54:02.85963+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:54:02.85966+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:54:02.85967+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:54:02.86562+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:796:refreshTenants: configured tenants: map[1:sys 1002:ocp 1004:dbamonitor 1006:dbmanager 1008:datatransfer] 2024-11-29T09:54:02.87176+08:00 INFO [21697,76df86be6fd163a5] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:54:02 +0800 CST, start_request_id:0 2024-11-29T09:54:02.87182+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:02.8719+08:00 INFO [21697,76df86be6fd163a5] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:54:02 +0800 CST, start_request_id:0 2024-11-29T09:54:02.87192+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:02.87201+08:00 INFO [21697,76df86be6fd163a5] caller=oceanbase/sql_plan_monitor.go:385:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1, collect time:2024-11-29 09:54:02 +0800 CST, start_request_id:544433, min_request_id:544434, max_request_id:544444 2024-11-29T09:54:02.87687+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544433, min_request_id: 544434, max_request_id: 544444, collected metrics number: 11 2024-11-29T09:54:02.87724+08:00 INFO [21697,76df86be6fd163a5] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 2 complete, collect time:2024-11-29 09:54:02 +0800 CST, start_request_id:544444 2024-11-29T09:54:02.87727+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 2, start_request_id: 544444, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:02.87733+08:00 INFO [21697,76df86be6fd163a5] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:54:02 +0800 CST, start_request_id:0 2024-11-29T09:54:02.87735+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:02.87906+08:00 INFO [21697,76df86be6fd163a5] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:54:02 +0800 CST, start_request_id:0 2024-11-29T09:54:02.8791+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:02.87917+08:00 INFO [21697,76df86be6fd163a5] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:54:02 +0800 CST, start_request_id:7154 2024-11-29T09:54:02.8792+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:02.8818+08:00 INFO [21697,76df86be6fd163a5] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:54:02 +0800 CST, start_request_id:0 2024-11-29T09:54:02.88184+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:02.88444+08:00 INFO [21697,76df86be6fd163a5] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:54:02 +0800 CST, start_request_id:387 2024-11-29T09:54:02.88453+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:02.88822+08:00 INFO [21697,76df86be6fd163a5] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:54:02 +0800 CST, start_request_id:0 2024-11-29T09:54:02.88827+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:02.90912+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807617, got 174 result 2024-11-29T09:54:02.91136+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907703, got 171 result 2024-11-29T09:54:02.91336+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 209 result 2024-11-29T09:54:02.91529+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 171 result 2024-11-29T09:54:02.91954+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 442 result 2024-11-29T09:54:02.92244+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 341 result 2024-11-29T09:54:02.92268+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 209 result, planStatMetrics number 133, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:02.9227+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 816, planExplainMetrics number 0, planMetaMetrics number 7 2024-11-29T09:54:02.92675+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 337 result 2024-11-29T09:54:02.94657+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 383 result 2024-11-29T09:54:02.95637+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 171 result, planStatMetrics number 102, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:02.95644+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 1109, planExplainMetrics number 0, planMetaMetrics number 7 2024-11-29T09:54:02.95857+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 442 result, planStatMetrics number 133, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:02.95861+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 1109, planExplainMetrics number 0, planMetaMetrics number 7 2024-11-29T09:54:02.96422+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 383 result, planStatMetrics number 107, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:02.96426+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 1109, planExplainMetrics number 0, planMetaMetrics number 7 2024-11-29T09:54:02.96828+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 341 result, planStatMetrics number 178, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:02.96832+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 1109, planExplainMetrics number 0, planMetaMetrics number 7 2024-11-29T09:54:02.96944+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 337 result, planStatMetrics number 186, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:02.96948+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 1109, planExplainMetrics number 0, planMetaMetrics number 7 2024-11-29T09:54:02.98661+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 237 result 2024-11-29T09:54:02.98935+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 237 result, planStatMetrics number 106, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:02.98939+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 1215, planExplainMetrics number 0, planMetaMetrics number 7 2024-11-29T09:54:02.99489+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807617, got 174 result, planStatMetrics number 133, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:02.99492+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 1215, planExplainMetrics number 0, planMetaMetrics number 7 2024-11-29T09:54:02.99621+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907703, got 171 result, planStatMetrics number 137, planMetaMetrics number 7, planExplainMetrics number 0 2024-11-29T09:54:02.99627+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 1215, planExplainMetrics number 0, planMetaMetrics number 7 2024-11-29T09:54:02.99632+08:00 INFO [21697,23a72d1c0b72d5bc] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:54:02 +0800 CST, end time:2024-11-29 09:54:02 +0800 CST, planMetaMetrics number 7, planExplainMetrics number 0, planStatMetrics number 1215 2024-11-29T09:54:03.34613+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 49 2024-11-29T09:54:03.34619+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:54:03.34623+08:00 INFO [21697,6000c8155ef029a0] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1393, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 49, sql_audit_progress Metrics number 1 2024-11-29T09:54:03.36024+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274994928, result count: 470 auditMsg number: 0, diagMsg number: 0, min_request_id: 274994928, max_request_id: 274995397 2024-11-29T09:54:04.36693+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995398, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995398, max_request_id: 274995429 2024-11-29T09:54:04.48148+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:54:04.48158+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:54:04.55302+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:54:04.86866+08:00 INFO [21697,effcddb1a1794baa] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=effcddb1a1794baa, body=] 2024-11-29T09:54:04.86896+08:00 WARN [21697,effcddb1a1794baa] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:04.86901+08:00 WARN [21697,effcddb1a1794baa] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:04.86904+08:00 WARN [21697,effcddb1a1794baa] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:04.86906+08:00 WARN [21697,effcddb1a1794baa] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:04.86909+08:00 WARN [21697,effcddb1a1794baa] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:04.87418+08:00 INFO [21697,effcddb1a1794baa] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=effcddb1a1794baa, body=] 2024-11-29T09:54:04.96958+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:54:04.9697+08:00 INFO [21697,802eedc92c3c660d] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:54:05.04201+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:54:05.36932+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995430, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995430, max_request_id: 274995466 2024-11-29T09:54:06.14163+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 9 2024-11-29T09:54:06.14169+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:54:06.14173+08:00 INFO [21697,089c0e43d2d02392] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 161, lost data number: 385, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 9, sql_audit_progress Metrics number 1 2024-11-29T09:54:06.14521+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163140289, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163140672, max_request_id: 163140675 2024-11-29T09:54:06.14525+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163140672 is larger than startRequestId 163140289, maybe lost 383 data 2024-11-29T09:54:06.19263+08:00 INFO [21697,92c7fb9306499664] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=92c7fb9306499664, body=] 2024-11-29T09:54:06.3717+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995467, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995467, max_request_id: 274995508 2024-11-29T09:54:06.60515+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:54:06.60525+08:00 INFO [21697,b6bc4dcc0346deef] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:54:06.81802+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:54:06.81813+08:00 INFO [21697,02088876768c8b97] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:54:07.09491+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:54:07.3051+08:00 INFO [21697,fbc52601c9c9732d] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=fbc52601c9c9732d, body=] 2024-11-29T09:54:07.30547+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:54:07.30587+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:54:07.37439+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995509, result count: 76 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995509, max_request_id: 274995584 2024-11-29T09:54:07.78368+08:00 INFO [21697,a97b0502c117b6bf] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:54:07.78376+08:00 INFO [21697,a97b0502c117b6bf] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:54:07.765254783 +0800 CST m=+10835212.881864344, end time:2024-11-29 09:54:07 +0800 CST, Metrics number 0 2024-11-29T09:54:07.78387+08:00 WARN [21697,a97b0502c117b6bf] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:54:07.86876+08:00 INFO [21697,76b2fd4c35c34311] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=76b2fd4c35c34311, body=] 2024-11-29T09:54:07.86906+08:00 WARN [21697,76b2fd4c35c34311] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:07.86911+08:00 WARN [21697,76b2fd4c35c34311] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:07.86914+08:00 WARN [21697,76b2fd4c35c34311] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:07.86916+08:00 WARN [21697,76b2fd4c35c34311] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:07.86918+08:00 WARN [21697,76b2fd4c35c34311] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:07.87327+08:00 INFO [21697,76b2fd4c35c34311] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=76b2fd4c35c34311, body=] 2024-11-29T09:54:08.37666+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995585, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995585, max_request_id: 274995619 2024-11-29T09:54:09.37919+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995620, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995620, max_request_id: 274995652 2024-11-29T09:54:10.02335+08:00 INFO [21697,c18395062d51d0f7] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=c18395062d51d0f7, body=] 2024-11-29T09:54:10.02361+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:54:10.04106+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:54:10.13994+08:00 INFO [21697,9436469f95a98779] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=9436469f95a98779, body=] 2024-11-29T09:54:10.14032+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:10.14036+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:10.14038+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:10.14039+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:10.1404+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:10.14082+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:54:10.38149+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995653, result count: 29 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995653, max_request_id: 274995681 2024-11-29T09:54:10.78046+08:00 INFO [21697,96210507fb7bdeab] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=96210507fb7bdeab, body=] 2024-11-29T09:54:10.8685+08:00 INFO [21697,8be6788344c740be] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=8be6788344c740be, body=] 2024-11-29T09:54:10.86878+08:00 WARN [21697,8be6788344c740be] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:10.86882+08:00 WARN [21697,8be6788344c740be] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:10.86883+08:00 WARN [21697,8be6788344c740be] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:10.86884+08:00 WARN [21697,8be6788344c740be] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:10.86885+08:00 WARN [21697,8be6788344c740be] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:10.87392+08:00 INFO [21697,8be6788344c740be] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=8be6788344c740be, body=] 2024-11-29T09:54:11.3844+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995682, result count: 51 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995682, max_request_id: 274995732 2024-11-29T09:54:11.75024+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:54:12.1085+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:54:12.38809+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995733, result count: 58 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995733, max_request_id: 274995790 2024-11-29T09:54:12.78176+08:00 INFO [21697,a54a05cf24397039] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:54:12.78183+08:00 INFO [21697,a54a05cf24397039] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:54:12.765376769 +0800 CST m=+10835217.881986308, end time:2024-11-29 09:54:12 +0800 CST, Metrics number 0 2024-11-29T09:54:12.78192+08:00 WARN [21697,a54a05cf24397039] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:54:13.39041+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995791, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995791, max_request_id: 274995821 2024-11-29T09:54:13.86716+08:00 INFO [21697,5369819e79c940a2] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=5369819e79c940a2, body=] 2024-11-29T09:54:13.86748+08:00 WARN [21697,5369819e79c940a2] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:13.86752+08:00 WARN [21697,5369819e79c940a2] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:13.86754+08:00 WARN [21697,5369819e79c940a2] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:13.86756+08:00 WARN [21697,5369819e79c940a2] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:13.86758+08:00 WARN [21697,5369819e79c940a2] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:13.87193+08:00 INFO [21697,5369819e79c940a2] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=5369819e79c940a2, body=] 2024-11-29T09:54:14.39208+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995822, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995822, max_request_id: 274995863 2024-11-29T09:54:15.04178+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:54:15.09182+08:00 INFO [21697,4141a03e04b849a3] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=4141a03e04b849a3, body=] 2024-11-29T09:54:15.09618+08:00 INFO [21697,4141a03e04b849a3] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=4141a03e04b849a3, body=] 2024-11-29T09:54:15.09633+08:00 INFO [21697,4141a03e04b849a3] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=4141a03e04b849a3, body=] 2024-11-29T09:54:15.09649+08:00 WARN [21697,4141a03e04b849a3] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:54:15.09664+08:00 INFO [21697,4141a03e04b849a3] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=4141a03e04b849a3, body=] 2024-11-29T09:54:15.09695+08:00 INFO [21697,4141a03e04b849a3] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=4141a03e04b849a3, body=] 2024-11-29T09:54:15.09901+08:00 WARN [21697,4141a03e04b849a3] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:54:15.09933+08:00 WARN [21697,4141a03e04b849a3] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:15.09936+08:00 WARN [21697,4141a03e04b849a3] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:15.09938+08:00 WARN [21697,4141a03e04b849a3] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:15.09939+08:00 WARN [21697,4141a03e04b849a3] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:15.0994+08:00 WARN [21697,4141a03e04b849a3] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:15.11234+08:00 INFO [21697,4141a03e04b849a3] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:54:15.11605+08:00 INFO [21697,4141a03e04b849a3] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:54:15.39486+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995864, result count: 28 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995864, max_request_id: 274995891 2024-11-29T09:54:16.39743+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995892, result count: 55 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995892, max_request_id: 274995946 2024-11-29T09:54:16.86835+08:00 INFO [21697,084d1350459543a3] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=084d1350459543a3, body=] 2024-11-29T09:54:16.86859+08:00 WARN [21697,084d1350459543a3] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:16.86863+08:00 WARN [21697,084d1350459543a3] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:16.86864+08:00 WARN [21697,084d1350459543a3] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:16.86866+08:00 WARN [21697,084d1350459543a3] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:16.86867+08:00 WARN [21697,084d1350459543a3] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:16.87286+08:00 INFO [21697,084d1350459543a3] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=084d1350459543a3, body=] 2024-11-29T09:54:17.09492+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:54:17.39927+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995947, result count: 47 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995947, max_request_id: 274995993 2024-11-29T09:54:17.78567+08:00 INFO [21697,a8189e4af370101c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:54:17.78574+08:00 INFO [21697,a8189e4af370101c] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:54:17.765885705 +0800 CST m=+10835222.882495243, end time:2024-11-29 09:54:17 +0800 CST, Metrics number 0 2024-11-29T09:54:17.78582+08:00 WARN [21697,a8189e4af370101c] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:54:18.40104+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274995994, result count: 28 auditMsg number: 0, diagMsg number: 0, min_request_id: 274995994, max_request_id: 274996021 2024-11-29T09:54:19.40278+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996022, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996022, max_request_id: 274996057 2024-11-29T09:54:19.48108+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:54:19.48117+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:54:19.55222+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields: error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:54:19.86691+08:00 INFO [21697,1b1177e1697c4367] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=1b1177e1697c4367, body=] 2024-11-29T09:54:19.86713+08:00 WARN [21697,1b1177e1697c4367] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:19.86717+08:00 WARN [21697,1b1177e1697c4367] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:19.86718+08:00 WARN [21697,1b1177e1697c4367] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:19.86719+08:00 WARN [21697,1b1177e1697c4367] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:19.86721+08:00 WARN [21697,1b1177e1697c4367] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:19.87129+08:00 INFO [21697,1b1177e1697c4367] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=1b1177e1697c4367, body=] 2024-11-29T09:54:20.04204+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:54:20.40453+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996058, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996058, max_request_id: 274996089 2024-11-29T09:54:21.19276+08:00 INFO [21697,1050a1247ddb9380] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=1050a1247ddb9380, body=] 2024-11-29T09:54:21.40705+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996090, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996090, max_request_id: 274996130 2024-11-29T09:54:21.74361+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:54:22.09622+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:54:22.15371+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163140676, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163141000, max_request_id: 163141003 2024-11-29T09:54:22.15377+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163141000 is larger than startRequestId 163140676, maybe lost 324 data 2024-11-29T09:54:22.30437+08:00 INFO [21697,39e1697c729059dc] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=39e1697c729059dc, body=] 2024-11-29T09:54:22.30474+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:54:22.3052+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:54:22.40944+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996131, result count: 62 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996131, max_request_id: 274996192 2024-11-29T09:54:22.78284+08:00 INFO [21697,77da314aff54234a] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:54:22.78292+08:00 INFO [21697,77da314aff54234a] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:54:22.765415592 +0800 CST m=+10835227.882025128, end time:2024-11-29 09:54:22 +0800 CST, Metrics number 0 2024-11-29T09:54:22.78303+08:00 WARN [21697,77da314aff54234a] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:54:22.86813+08:00 INFO [21697,9e962647d8dc452b] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=9e962647d8dc452b, body=] 2024-11-29T09:54:22.86832+08:00 WARN [21697,9e962647d8dc452b] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:22.86834+08:00 WARN [21697,9e962647d8dc452b] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:22.86836+08:00 WARN [21697,9e962647d8dc452b] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:22.86837+08:00 WARN [21697,9e962647d8dc452b] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:22.86838+08:00 WARN [21697,9e962647d8dc452b] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:22.87449+08:00 INFO [21697,9e962647d8dc452b] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=9e962647d8dc452b, body=] 2024-11-29T09:54:23.41197+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996193, result count: 34 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996193, max_request_id: 274996226 2024-11-29T09:54:24.41439+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996227, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996227, max_request_id: 274996263 2024-11-29T09:54:25.02219+08:00 INFO [21697,ecd52aca38c75afa] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=ecd52aca38c75afa, body=] 2024-11-29T09:54:25.02243+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:54:25.04183+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:54:25.13908+08:00 INFO [21697,b82c63152e3eab5c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=b82c63152e3eab5c, body=] 2024-11-29T09:54:25.13946+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:25.1395+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:25.13952+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:25.13953+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:25.13955+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:25.13992+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:54:25.41637+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996264, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996264, max_request_id: 274996296 2024-11-29T09:54:25.78318+08:00 INFO [21697,0f5a80ca7a881111] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=0f5a80ca7a881111, body=] 2024-11-29T09:54:25.86645+08:00 INFO [21697,e67f93c5a04549f0] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=e67f93c5a04549f0, body=] 2024-11-29T09:54:25.86667+08:00 WARN [21697,e67f93c5a04549f0] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:25.86671+08:00 WARN [21697,e67f93c5a04549f0] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:25.86673+08:00 WARN [21697,e67f93c5a04549f0] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:25.86674+08:00 WARN [21697,e67f93c5a04549f0] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:25.86675+08:00 WARN [21697,e67f93c5a04549f0] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:25.87074+08:00 INFO [21697,e67f93c5a04549f0] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=e67f93c5a04549f0, body=] 2024-11-29T09:54:26.41883+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996297, result count: 48 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996297, max_request_id: 274996344 2024-11-29T09:54:27.09571+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:54:27.42089+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996345, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996345, max_request_id: 274996383 2024-11-29T09:54:27.78227+08:00 INFO [21697,9a10d6e5d977c1aa] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:54:27.78234+08:00 INFO [21697,9a10d6e5d977c1aa] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:54:27.765673998 +0800 CST m=+10835232.882283532, end time:2024-11-29 09:54:27 +0800 CST, Metrics number 0 2024-11-29T09:54:27.78243+08:00 WARN [21697,9a10d6e5d977c1aa] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:54:28.42336+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996384, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996384, max_request_id: 274996416 2024-11-29T09:54:28.86661+08:00 INFO [21697,6c13a142c2f9464d] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=6c13a142c2f9464d, body=] 2024-11-29T09:54:28.86687+08:00 WARN [21697,6c13a142c2f9464d] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:28.86692+08:00 WARN [21697,6c13a142c2f9464d] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:28.86694+08:00 WARN [21697,6c13a142c2f9464d] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:28.86695+08:00 WARN [21697,6c13a142c2f9464d] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:28.86697+08:00 WARN [21697,6c13a142c2f9464d] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:28.87188+08:00 INFO [21697,6c13a142c2f9464d] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=6c13a142c2f9464d, body=] 2024-11-29T09:54:29.42594+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996417, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996417, max_request_id: 274996456 2024-11-29T09:54:30.04205+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:54:30.08994+08:00 INFO [21697,1ec4cfd258524704] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=1ec4cfd258524704, body=] 2024-11-29T09:54:30.09035+08:00 WARN [21697,1ec4cfd258524704] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:30.09039+08:00 WARN [21697,1ec4cfd258524704] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:30.0904+08:00 WARN [21697,1ec4cfd258524704] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:30.09041+08:00 WARN [21697,1ec4cfd258524704] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:30.09042+08:00 WARN [21697,1ec4cfd258524704] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:30.09078+08:00 INFO [21697,1ec4cfd258524704] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:54:30.09116+08:00 INFO [21697,1ec4cfd258524704] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=1ec4cfd258524704, body=] 2024-11-29T09:54:30.09452+08:00 INFO [21697,1ec4cfd258524704] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=1ec4cfd258524704, body=] 2024-11-29T09:54:30.09465+08:00 INFO [21697,1ec4cfd258524704] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=1ec4cfd258524704, body=] 2024-11-29T09:54:30.09478+08:00 WARN [21697,1ec4cfd258524704] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:54:30.09489+08:00 INFO [21697,1ec4cfd258524704] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=1ec4cfd258524704, body=] 2024-11-29T09:54:30.09647+08:00 WARN [21697,1ec4cfd258524704] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:54:30.10515+08:00 INFO [21697,1ec4cfd258524704] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:54:30.42775+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996457, result count: 30 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996457, max_request_id: 274996486 2024-11-29T09:54:31.32702+08:00 INFO [21697,cfd907917352614f] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.98.86, ocpServerIp=, traceId=cfd907917352614f, body=] 2024-11-29T09:54:31.32723+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:31.32726+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:31.32727+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:31.32728+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:31.3273+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:31.43138+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996487, result count: 172 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996487, max_request_id: 274996658 2024-11-29T09:54:31.74412+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:54:31.86683+08:00 INFO [21697,8652cea4641445c8] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=8652cea4641445c8, body=] 2024-11-29T09:54:31.86818+08:00 INFO [21697,8652cea4641445c8] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=8652cea4641445c8, body=] 2024-11-29T09:54:31.86835+08:00 WARN [21697,8652cea4641445c8] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:31.86838+08:00 WARN [21697,8652cea4641445c8] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:31.86839+08:00 WARN [21697,8652cea4641445c8] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:31.8684+08:00 WARN [21697,8652cea4641445c8] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:31.86842+08:00 WARN [21697,8652cea4641445c8] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:32.10885+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:54:32.43366+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996659, result count: 65 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996659, max_request_id: 274996723 2024-11-29T09:54:32.78226+08:00 INFO [21697,c66b286986450565] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:54:32.78232+08:00 INFO [21697,c66b286986450565] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:54:32.765895994 +0800 CST m=+10835237.882505527, end time:2024-11-29 09:54:32 +0800 CST, Metrics number 0 2024-11-29T09:54:32.7824+08:00 WARN [21697,c66b286986450565] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:54:32.85563+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:54:32.85712+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:54:32.85722+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:54:32.85725+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:54:32.85728+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:54:32.85732+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:54:32.85735+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:54:32.85738+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:54:32.85741+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:54:32.85745+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:54:32.85748+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:54:32.8575+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:54:32.86863+08:00 INFO [21697,ee37a389aa0f8c32] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:54:32 +0800 CST, start_request_id:0 2024-11-29T09:54:32.86871+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:32.86883+08:00 INFO [21697,ee37a389aa0f8c32] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:54:32 +0800 CST, start_request_id:0 2024-11-29T09:54:32.86885+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:32.86897+08:00 INFO [21697,ee37a389aa0f8c32] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:54:32 +0800 CST, start_request_id:0 2024-11-29T09:54:32.86899+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:32.89417+08:00 INFO [21697,ee37a389aa0f8c32] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:54:32 +0800 CST, start_request_id:544444 2024-11-29T09:54:32.89424+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544444, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:32.89436+08:00 INFO [21697,ee37a389aa0f8c32] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:54:32 +0800 CST, start_request_id:7154 2024-11-29T09:54:32.89438+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:32.89444+08:00 INFO [21697,ee37a389aa0f8c32] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:54:32 +0800 CST, start_request_id:0 2024-11-29T09:54:32.89446+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:32.89485+08:00 INFO [21697,ee37a389aa0f8c32] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:54:32 +0800 CST, start_request_id:387 2024-11-29T09:54:32.89489+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:32.89987+08:00 INFO [21697,ee37a389aa0f8c32] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:54:32 +0800 CST, start_request_id:0 2024-11-29T09:54:32.89994+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:32.90136+08:00 INFO [21697,ee37a389aa0f8c32] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:54:32 +0800 CST, start_request_id:0 2024-11-29T09:54:32.90139+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:54:32.91154+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:54:32.91158+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:32.9117+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:54:32.91172+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:32.91181+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807617, got 0 result 2024-11-29T09:54:32.91183+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807617, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:32.91191+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:54:32.91194+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:32.91202+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:54:32.91205+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:32.91208+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:54:32.91218+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:54:32.9122+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:54:32.91224+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:54:32.91226+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:54:32.91234+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:54:32.91237+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:32.91239+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:54:32.9125+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907703, got 0 result 2024-11-29T09:54:32.91252+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907703, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:32.91255+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:54:32.92393+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:54:32.92396+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:32.92398+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:54:32.96821+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:54:32.96825+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:54:32.96828+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:54:32.96832+08:00 INFO [21697,1bb6406ccd30b0a6] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:54:32 +0800 CST, end time:2024-11-29 09:54:32 +0800 CST, planMetaMetrics number 0, planExplainMetrics number 0, planStatMetrics number 0 2024-11-29T09:54:33.15835+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 3 2024-11-29T09:54:33.15844+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:54:33.1585+08:00 INFO [21697,6a991c59d2192219] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 6, lost data number: 707, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 3, sql_audit_progress Metrics number 1 2024-11-29T09:54:33.16568+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163141004, result count: 308 auditMsg number: 0, diagMsg number: 0, min_request_id: 163141185, max_request_id: 163141570 2024-11-29T09:54:33.16573+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163141185 is larger than startRequestId 163141004, maybe lost 181 data 2024-11-29T09:54:33.4343+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 47 2024-11-29T09:54:33.43438+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:54:33.43442+08:00 INFO [21697,69d1347dd10b8e23] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1796, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 47, sql_audit_progress Metrics number 1 2024-11-29T09:54:33.43679+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996724, result count: 63 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996724, max_request_id: 274996786 2024-11-29T09:54:34.43906+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996787, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996787, max_request_id: 274996823 2024-11-29T09:54:34.4808+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:54:34.48088+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:54:34.55888+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:54:34.86621+08:00 INFO [21697,cf6fb9b0a529483d] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=cf6fb9b0a529483d, body=] 2024-11-29T09:54:34.86645+08:00 WARN [21697,cf6fb9b0a529483d] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:34.86648+08:00 WARN [21697,cf6fb9b0a529483d] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:34.8665+08:00 WARN [21697,cf6fb9b0a529483d] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:34.86651+08:00 WARN [21697,cf6fb9b0a529483d] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:34.86652+08:00 WARN [21697,cf6fb9b0a529483d] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:34.87034+08:00 INFO [21697,cf6fb9b0a529483d] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=cf6fb9b0a529483d, body=] 2024-11-29T09:54:35.033+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:54:35.03311+08:00 INFO [21697,88b3541a662e5c3d] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:54:35.04201+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:54:35.44148+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996824, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996824, max_request_id: 274996861 2024-11-29T09:54:36.19174+08:00 INFO [21697,b8d56b61d2887d13] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=b8d56b61d2887d13, body=] 2024-11-29T09:54:36.44792+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996862, result count: 49 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996862, max_request_id: 274996910 2024-11-29T09:54:36.67185+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:54:36.67194+08:00 INFO [21697,f9cbd64483d9e189] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:54:36.84533+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:54:36.84542+08:00 INFO [21697,09cc591bd19e6547] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:54:37.09518+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:54:37.3035+08:00 INFO [21697,1fabcfc9f12d3dda] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=1fabcfc9f12d3dda, body=] 2024-11-29T09:54:37.30386+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:54:37.30418+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:54:37.44967+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996911, result count: 66 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996911, max_request_id: 274996976 2024-11-29T09:54:37.78151+08:00 INFO [21697,b61ae1f6ab4795d8] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:54:37.78159+08:00 INFO [21697,b61ae1f6ab4795d8] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:54:37.765171041 +0800 CST m=+10835242.881780580, end time:2024-11-29 09:54:37 +0800 CST, Metrics number 0 2024-11-29T09:54:37.78167+08:00 WARN [21697,b61ae1f6ab4795d8] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:54:37.86642+08:00 INFO [21697,996e61ac134a4ad1] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=996e61ac134a4ad1, body=] 2024-11-29T09:54:37.86664+08:00 WARN [21697,996e61ac134a4ad1] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:37.86668+08:00 WARN [21697,996e61ac134a4ad1] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:37.86669+08:00 WARN [21697,996e61ac134a4ad1] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:37.86671+08:00 WARN [21697,996e61ac134a4ad1] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:37.86672+08:00 WARN [21697,996e61ac134a4ad1] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:37.8706+08:00 INFO [21697,996e61ac134a4ad1] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=996e61ac134a4ad1, body=] 2024-11-29T09:54:38.45202+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274996977, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274996977, max_request_id: 274997011 2024-11-29T09:54:39.16882+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163141571, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163141651, max_request_id: 163141653 2024-11-29T09:54:39.16888+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163141651 is larger than startRequestId 163141571, maybe lost 80 data 2024-11-29T09:54:39.4548+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997012, result count: 34 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997012, max_request_id: 274997045 2024-11-29T09:54:40.02169+08:00 INFO [21697,a5c1999c3e1744e5] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=a5c1999c3e1744e5, body=] 2024-11-29T09:54:40.02194+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:54:40.04146+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:54:40.13931+08:00 INFO [21697,65a535095456e39d] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=65a535095456e39d, body=] 2024-11-29T09:54:40.13971+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:40.13975+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:40.13978+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:40.13979+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:40.1398+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:40.14016+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:54:40.45703+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997046, result count: 30 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997046, max_request_id: 274997075 2024-11-29T09:54:40.77961+08:00 INFO [21697,555ca352531481bd] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=555ca352531481bd, body=] 2024-11-29T09:54:40.86614+08:00 INFO [21697,482e5dc139684d70] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=482e5dc139684d70, body=] 2024-11-29T09:54:40.86637+08:00 WARN [21697,482e5dc139684d70] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:40.86641+08:00 WARN [21697,482e5dc139684d70] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:40.86642+08:00 WARN [21697,482e5dc139684d70] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:40.86644+08:00 WARN [21697,482e5dc139684d70] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:40.86645+08:00 WARN [21697,482e5dc139684d70] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:40.87038+08:00 INFO [21697,482e5dc139684d70] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=482e5dc139684d70, body=] 2024-11-29T09:54:41.45935+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997076, result count: 49 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997076, max_request_id: 274997124 2024-11-29T09:54:41.74348+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:54:42.10783+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:54:42.46169+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997125, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997125, max_request_id: 274997161 2024-11-29T09:54:42.78223+08:00 INFO [21697,ce199d664ee83b5b] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:54:42.7823+08:00 INFO [21697,ce199d664ee83b5b] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:54:42.765849455 +0800 CST m=+10835247.882458988, end time:2024-11-29 09:54:42 +0800 CST, Metrics number 0 2024-11-29T09:54:42.7824+08:00 WARN [21697,ce199d664ee83b5b] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:54:43.46429+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997162, result count: 50 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997162, max_request_id: 274997211 2024-11-29T09:54:43.86589+08:00 INFO [21697,09c572683b8f4d75] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=09c572683b8f4d75, body=] 2024-11-29T09:54:43.86611+08:00 WARN [21697,09c572683b8f4d75] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:43.86616+08:00 WARN [21697,09c572683b8f4d75] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:43.86617+08:00 WARN [21697,09c572683b8f4d75] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:43.86618+08:00 WARN [21697,09c572683b8f4d75] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:43.86619+08:00 WARN [21697,09c572683b8f4d75] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:43.86991+08:00 INFO [21697,09c572683b8f4d75] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=09c572683b8f4d75, body=] 2024-11-29T09:54:44.46789+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997212, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997212, max_request_id: 274997253 2024-11-29T09:54:45.04147+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:54:45.08952+08:00 INFO [21697,c9bb7bda394b40ff] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=c9bb7bda394b40ff, body=] 2024-11-29T09:54:45.08972+08:00 INFO [21697,c9bb7bda394b40ff] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=c9bb7bda394b40ff, body=] 2024-11-29T09:54:45.09487+08:00 WARN [21697,c9bb7bda394b40ff] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:45.09491+08:00 WARN [21697,c9bb7bda394b40ff] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:45.09493+08:00 WARN [21697,c9bb7bda394b40ff] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:45.09494+08:00 WARN [21697,c9bb7bda394b40ff] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:45.09495+08:00 WARN [21697,c9bb7bda394b40ff] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:45.10057+08:00 INFO [21697,c9bb7bda394b40ff] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:54:45.1066+08:00 INFO [21697,c9bb7bda394b40ff] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=c9bb7bda394b40ff, body=] 2024-11-29T09:54:45.11018+08:00 INFO [21697,c9bb7bda394b40ff] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=c9bb7bda394b40ff, body=] 2024-11-29T09:54:45.11031+08:00 INFO [21697,c9bb7bda394b40ff] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=c9bb7bda394b40ff, body=] 2024-11-29T09:54:45.11044+08:00 WARN [21697,c9bb7bda394b40ff] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:54:45.11227+08:00 WARN [21697,c9bb7bda394b40ff] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:54:45.11456+08:00 INFO [21697,c9bb7bda394b40ff] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:54:45.47599+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997254, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997254, max_request_id: 274997284 2024-11-29T09:54:46.47958+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997285, result count: 54 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997285, max_request_id: 274997338 2024-11-29T09:54:46.8659+08:00 INFO [21697,121b651131374d9a] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=121b651131374d9a, body=] 2024-11-29T09:54:46.86612+08:00 WARN [21697,121b651131374d9a] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:46.86616+08:00 WARN [21697,121b651131374d9a] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:46.86617+08:00 WARN [21697,121b651131374d9a] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:46.86619+08:00 WARN [21697,121b651131374d9a] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:46.8662+08:00 WARN [21697,121b651131374d9a] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:46.87028+08:00 INFO [21697,121b651131374d9a] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=121b651131374d9a, body=] 2024-11-29T09:54:47.09539+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:54:47.48151+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997339, result count: 46 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997339, max_request_id: 274997384 2024-11-29T09:54:47.78216+08:00 INFO [21697,9ad392ee228ab129] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:54:47.78223+08:00 INFO [21697,9ad392ee228ab129] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:54:47.765481826 +0800 CST m=+10835252.882091365, end time:2024-11-29 09:54:47 +0800 CST, Metrics number 0 2024-11-29T09:54:47.78232+08:00 WARN [21697,9ad392ee228ab129] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:54:48.30441+08:00 INFO [21697,7e100a419d33202b] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.98.86, ocpServerIp=, traceId=7e100a419d33202b, body=] 2024-11-29T09:54:48.48388+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997385, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997385, max_request_id: 274997417 2024-11-29T09:54:49.48095+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:54:49.48105+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:54:49.48599+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997418, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997418, max_request_id: 274997453 2024-11-29T09:54:49.55554+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields: error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:54:49.86596+08:00 INFO [21697,65492ea2906e4acf] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=65492ea2906e4acf, body=] 2024-11-29T09:54:49.86618+08:00 WARN [21697,65492ea2906e4acf] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:49.86621+08:00 WARN [21697,65492ea2906e4acf] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:49.86623+08:00 WARN [21697,65492ea2906e4acf] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:49.86625+08:00 WARN [21697,65492ea2906e4acf] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:49.86626+08:00 WARN [21697,65492ea2906e4acf] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:49.877+08:00 INFO [21697,65492ea2906e4acf] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=65492ea2906e4acf, body=] 2024-11-29T09:54:50.04207+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:54:50.48755+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997454, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997454, max_request_id: 274997491 2024-11-29T09:54:51.19133+08:00 INFO [21697,3b14de27d3e4680c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=3b14de27d3e4680c, body=] 2024-11-29T09:54:51.49012+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997492, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997492, max_request_id: 274997534 2024-11-29T09:54:51.74396+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:54:52.10879+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:54:52.3045+08:00 INFO [21697,b80d57c0b1e12395] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=b80d57c0b1e12395, body=] 2024-11-29T09:54:52.30495+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:54:52.30564+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:54:52.37891+08:00 INFO [21697,ed85e7e5ed336283] caller=host/custom.go:178:func10: collect observer dir exists info 2024-11-29T09:54:52.49198+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997535, result count: 44 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997535, max_request_id: 274997578 2024-11-29T09:54:52.78144+08:00 INFO [21697,bc86ea8d18760cbf] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:54:52.78151+08:00 INFO [21697,bc86ea8d18760cbf] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:54:52.765079808 +0800 CST m=+10835257.881689341, end time:2024-11-29 09:54:52 +0800 CST, Metrics number 0 2024-11-29T09:54:52.78159+08:00 WARN [21697,bc86ea8d18760cbf] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:54:52.86707+08:00 INFO [21697,c32f25e621a94198] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=c32f25e621a94198, body=] 2024-11-29T09:54:52.86728+08:00 WARN [21697,c32f25e621a94198] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:52.86731+08:00 WARN [21697,c32f25e621a94198] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:52.86732+08:00 WARN [21697,c32f25e621a94198] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:52.86733+08:00 WARN [21697,c32f25e621a94198] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:52.86735+08:00 WARN [21697,c32f25e621a94198] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:52.87197+08:00 INFO [21697,c32f25e621a94198] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=c32f25e621a94198, body=] 2024-11-29T09:54:53.49451+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997579, result count: 59 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997579, max_request_id: 274997637 2024-11-29T09:54:54.49621+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997638, result count: 34 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997638, max_request_id: 274997671 2024-11-29T09:54:55.0213+08:00 INFO [21697,873743d85bdc8cd8] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=873743d85bdc8cd8, body=] 2024-11-29T09:54:55.02154+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:54:55.04115+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:54:55.13884+08:00 INFO [21697,2063eb2d453ded6a] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=2063eb2d453ded6a, body=] 2024-11-29T09:54:55.13925+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:55.13929+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:55.13931+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:55.13932+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:55.13934+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:55.13978+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:54:55.17833+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163141654, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163141979, max_request_id: 163141981 2024-11-29T09:54:55.17838+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163141979 is larger than startRequestId 163141654, maybe lost 325 data 2024-11-29T09:54:55.49834+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997672, result count: 27 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997672, max_request_id: 274997698 2024-11-29T09:54:55.77954+08:00 INFO [21697,9d2bc57b7614750a] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=9d2bc57b7614750a, body=] 2024-11-29T09:54:55.86578+08:00 INFO [21697,9258f9576b864276] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=9258f9576b864276, body=] 2024-11-29T09:54:55.86598+08:00 WARN [21697,9258f9576b864276] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:55.86602+08:00 WARN [21697,9258f9576b864276] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:55.86603+08:00 WARN [21697,9258f9576b864276] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:55.86604+08:00 WARN [21697,9258f9576b864276] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:55.86605+08:00 WARN [21697,9258f9576b864276] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:55.86969+08:00 INFO [21697,9258f9576b864276] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=9258f9576b864276, body=] 2024-11-29T09:54:56.50033+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997699, result count: 51 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997699, max_request_id: 274997749 2024-11-29T09:54:57.09626+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:54:57.50229+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997750, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997750, max_request_id: 274997788 2024-11-29T09:54:57.78413+08:00 INFO [21697,29002031a530bf60] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:54:57.78421+08:00 INFO [21697,29002031a530bf60] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:54:57.765419865 +0800 CST m=+10835262.882029431, end time:2024-11-29 09:54:57 +0800 CST, Metrics number 0 2024-11-29T09:54:57.7843+08:00 WARN [21697,29002031a530bf60] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:54:58.50923+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997789, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997789, max_request_id: 274997819 2024-11-29T09:54:58.86578+08:00 INFO [21697,1b44b91d8c1441e3] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=1b44b91d8c1441e3, body=] 2024-11-29T09:54:58.86607+08:00 WARN [21697,1b44b91d8c1441e3] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:54:58.86612+08:00 WARN [21697,1b44b91d8c1441e3] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:54:58.86614+08:00 WARN [21697,1b44b91d8c1441e3] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:54:58.86616+08:00 WARN [21697,1b44b91d8c1441e3] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:58.86618+08:00 WARN [21697,1b44b91d8c1441e3] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:54:58.87278+08:00 INFO [21697,1b44b91d8c1441e3] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=1b44b91d8c1441e3, body=] 2024-11-29T09:54:59.51182+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997820, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997820, max_request_id: 274997859 2024-11-29T09:55:00.042+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:55:00.08914+08:00 INFO [21697,276c721c60cc4b58] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=276c721c60cc4b58, body=] 2024-11-29T09:55:00.08963+08:00 WARN [21697,276c721c60cc4b58] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:00.08968+08:00 WARN [21697,276c721c60cc4b58] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:00.0897+08:00 WARN [21697,276c721c60cc4b58] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:00.08972+08:00 WARN [21697,276c721c60cc4b58] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:00.08974+08:00 WARN [21697,276c721c60cc4b58] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:00.09014+08:00 INFO [21697,276c721c60cc4b58] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:55:00.09052+08:00 INFO [21697,276c721c60cc4b58] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=276c721c60cc4b58, body=] 2024-11-29T09:55:00.09367+08:00 INFO [21697,276c721c60cc4b58] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=276c721c60cc4b58, body=] 2024-11-29T09:55:00.09379+08:00 INFO [21697,276c721c60cc4b58] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=276c721c60cc4b58, body=] 2024-11-29T09:55:00.0939+08:00 INFO [21697,276c721c60cc4b58] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=276c721c60cc4b58, body=] 2024-11-29T09:55:00.09402+08:00 WARN [21697,276c721c60cc4b58] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:55:00.09553+08:00 WARN [21697,276c721c60cc4b58] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:55:00.10532+08:00 INFO [21697,276c721c60cc4b58] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:55:00.51388+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997860, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997860, max_request_id: 274997892 2024-11-29T09:55:01.27047+08:00 INFO [21697,5ffda003ee9feb75] caller=common/middleware.go:59:func1: API request: [GET /api/v1/log/alarms, client=1.1.1.180, ocpServerIp=1.1.1.180, traceId=5ffda003ee9feb75, body=] 2024-11-29T09:55:01.27071+08:00 INFO [21697,] caller=alarmscache/alarm_handler.go:73:getPipelieGroupReader: get alarms, result size:0, cache list size:4 fields:, uri=/api/v1/log/alarms 2024-11-29T09:55:01.5166+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274997893, result count: 127 auditMsg number: 0, diagMsg number: 0, min_request_id: 274997893, max_request_id: 274998019 2024-11-29T09:55:01.74787+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:55:01.86596+08:00 INFO [21697,98b557634ac04ebf] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=98b557634ac04ebf, body=] 2024-11-29T09:55:01.86617+08:00 WARN [21697,98b557634ac04ebf] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:01.8662+08:00 WARN [21697,98b557634ac04ebf] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:01.86622+08:00 WARN [21697,98b557634ac04ebf] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:01.86623+08:00 WARN [21697,98b557634ac04ebf] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:01.86624+08:00 WARN [21697,98b557634ac04ebf] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:01.87009+08:00 INFO [21697,98b557634ac04ebf] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=98b557634ac04ebf, body=] 2024-11-29T09:55:02.10824+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:55:02.51945+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998020, result count: 72 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998020, max_request_id: 274998091 2024-11-29T09:55:02.74228+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:245:updateMetaInfo: successfully init sqlAudit tenant config 2024-11-29T09:55:02.78223+08:00 INFO [21697,8e61093c76dcbc95] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:55:02.7823+08:00 INFO [21697,8e61093c76dcbc95] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:55:02.765647995 +0800 CST m=+10835267.882257533, end time:2024-11-29 09:55:02 +0800 CST, Metrics number 0 2024-11-29T09:55:02.78238+08:00 WARN [21697,8e61093c76dcbc95] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:55:02.85694+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:55:02.85832+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:55:02.85838+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:55:02.8584+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:55:02.85842+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:55:02.85844+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:55:02.85846+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:55:02.85849+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:55:02.85852+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:55:02.85853+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:55:02.85857+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:55:02.85858+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:55:02.86538+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:796:refreshTenants: configured tenants: map[1:sys 1002:ocp 1004:dbamonitor 1006:dbmanager 1008:datatransfer] 2024-11-29T09:55:02.87492+08:00 INFO [21697,0dee5d60fc61b4a2] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:55:02 +0800 CST, start_request_id:0 2024-11-29T09:55:02.87498+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:02.87505+08:00 INFO [21697,0dee5d60fc61b4a2] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:55:02 +0800 CST, start_request_id:544444 2024-11-29T09:55:02.87507+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544444, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:02.87511+08:00 INFO [21697,0dee5d60fc61b4a2] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:55:02 +0800 CST, start_request_id:0 2024-11-29T09:55:02.87512+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:02.87516+08:00 INFO [21697,0dee5d60fc61b4a2] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:55:02 +0800 CST, start_request_id:0 2024-11-29T09:55:02.87518+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:02.88457+08:00 INFO [21697,0dee5d60fc61b4a2] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:55:02 +0800 CST, start_request_id:0 2024-11-29T09:55:02.88464+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:02.88491+08:00 INFO [21697,0dee5d60fc61b4a2] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:55:02 +0800 CST, start_request_id:387 2024-11-29T09:55:02.88493+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:02.885+08:00 INFO [21697,0dee5d60fc61b4a2] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:55:02 +0800 CST, start_request_id:7154 2024-11-29T09:55:02.88502+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:02.88983+08:00 INFO [21697,0dee5d60fc61b4a2] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:55:02 +0800 CST, start_request_id:0 2024-11-29T09:55:02.88989+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:02.89425+08:00 INFO [21697,0dee5d60fc61b4a2] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:55:02 +0800 CST, start_request_id:0 2024-11-29T09:55:02.8943+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:02.90753+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807618, got 1 result 2024-11-29T09:55:02.90767+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:55:02.90769+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:55:02.90771+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:55:02.90778+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:55:02.90781+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:55:02.90788+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:55:02.9079+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:55:02.90794+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:55:02.90796+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:55:02.9087+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907750, got 47 result 2024-11-29T09:55:02.90879+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:55:02.90881+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:55:02.90882+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 48, planExplainMetrics number 0, planMetaMetrics number 48 2024-11-29T09:55:02.90889+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:55:02.9089+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:55:02.90893+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 48, planExplainMetrics number 0, planMetaMetrics number 48 2024-11-29T09:55:02.90901+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807618, got 1 result, planStatMetrics number 1, planMetaMetrics number 1, planExplainMetrics number 0 2024-11-29T09:55:02.90902+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 48, planExplainMetrics number 0, planMetaMetrics number 48 2024-11-29T09:55:02.9127+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:55:02.91274+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:55:02.91276+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 48, planExplainMetrics number 0, planMetaMetrics number 48 2024-11-29T09:55:02.92336+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907750, got 47 result, planStatMetrics number 47, planMetaMetrics number 47, planExplainMetrics number 0 2024-11-29T09:55:02.92341+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 48, planExplainMetrics number 0, planMetaMetrics number 48 2024-11-29T09:55:02.96058+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:55:02.96065+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:55:02.96067+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 48, planExplainMetrics number 0, planMetaMetrics number 48 2024-11-29T09:55:02.96073+08:00 INFO [21697,e2ad5e0a7083f577] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:55:02 +0800 CST, end time:2024-11-29 09:55:02 +0800 CST, planMetaMetrics number 48, planExplainMetrics number 0, planStatMetrics number 48 2024-11-29T09:55:03.52025+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 53 2024-11-29T09:55:03.52032+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:55:03.52037+08:00 INFO [21697,3ecb0acc1a7c043f] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1368, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 53, sql_audit_progress Metrics number 1 2024-11-29T09:55:03.52488+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998092, result count: 189 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998092, max_request_id: 274998280 2024-11-29T09:55:04.48157+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:55:04.48166+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:55:04.52871+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998281, result count: 47 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998281, max_request_id: 274998327 2024-11-29T09:55:04.55146+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:55:04.87177+08:00 INFO [21697,5d2416926da14bcd] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=5d2416926da14bcd, body=] 2024-11-29T09:55:04.87204+08:00 WARN [21697,5d2416926da14bcd] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:04.87208+08:00 WARN [21697,5d2416926da14bcd] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:04.8721+08:00 WARN [21697,5d2416926da14bcd] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:04.87212+08:00 WARN [21697,5d2416926da14bcd] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:04.87213+08:00 WARN [21697,5d2416926da14bcd] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:04.87751+08:00 INFO [21697,5d2416926da14bcd] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=5d2416926da14bcd, body=] 2024-11-29T09:55:05.0411+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:55:05.11143+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:55:05.11154+08:00 INFO [21697,f96d3ce65ade521d] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:55:05.53095+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998328, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998328, max_request_id: 274998366 2024-11-29T09:55:06.18264+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 13 2024-11-29T09:55:06.18271+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:55:06.18275+08:00 INFO [21697,3a7ab5fcb3259b05] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 314, lost data number: 586, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 13, sql_audit_progress Metrics number 1 2024-11-29T09:55:06.18563+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163141982, result count: 6 auditMsg number: 0, diagMsg number: 0, min_request_id: 163142347, max_request_id: 163142371 2024-11-29T09:55:06.18567+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163142347 is larger than startRequestId 163141982, maybe lost 365 data 2024-11-29T09:55:06.19146+08:00 INFO [21697,5e6e3e529d19bccb] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=5e6e3e529d19bccb, body=] 2024-11-29T09:55:06.53317+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998367, result count: 59 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998367, max_request_id: 274998425 2024-11-29T09:55:06.7404+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:55:06.74051+08:00 INFO [21697,c3e63076d060ba25] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:55:06.88939+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:55:06.8895+08:00 INFO [21697,e886c7be77f1bc58] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:55:07.09605+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:55:07.30355+08:00 INFO [21697,bbbcbaec94c9f0f1] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=bbbcbaec94c9f0f1, body=] 2024-11-29T09:55:07.3039+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:55:07.30422+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:55:07.53553+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998426, result count: 61 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998426, max_request_id: 274998486 2024-11-29T09:55:07.79849+08:00 INFO [21697,05261637f7e684be] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 1 result 2024-11-29T09:55:07.79859+08:00 INFO [21697,05261637f7e684be] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:55:07.764944328 +0800 CST m=+10835272.881553863, end time:2024-11-29 09:55:07 +0800 CST, Metrics number 1 2024-11-29T09:55:07.79869+08:00 INFO [21697,05261637f7e684be] caller=engine/pipeline.go:145:pipelinePush: push pipeline ob_sql_transaction metrics count 1 2024-11-29T09:55:07.86746+08:00 INFO [21697,f936340cdde746cd] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=f936340cdde746cd, body=] 2024-11-29T09:55:07.86767+08:00 WARN [21697,f936340cdde746cd] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:07.8677+08:00 WARN [21697,f936340cdde746cd] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:07.86772+08:00 WARN [21697,f936340cdde746cd] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:07.86773+08:00 WARN [21697,f936340cdde746cd] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:07.86774+08:00 WARN [21697,f936340cdde746cd] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:07.87131+08:00 INFO [21697,f936340cdde746cd] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=f936340cdde746cd, body=] 2024-11-29T09:55:07.92307+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:255:WatchFile: get file change events fields:, start="2024-11-29 09:55:07 +0800 CST", end="2024-11-29 09:55:07.923063511 +0800 CST m=+10835273.039673056", queueTailFileStat.ino=1074285534 2024-11-29T09:55:07.9234+08:00 INFO [21697,] caller=logtailer/log_tailer_utils.go:210:getLogsWithinTime: invoke FindFilesByRegexAndMTime fields: start="2024-11-29 09:55:07 +0800 CST", end="2024-11-29 09:55:07.923063511 +0800 CST m=+10835273.039673056", matchedFiles="[0xc0007cc270 0xc006392820]", dir=/opt/myoceanbase/oceanbase/log, fileName=observer.log 2024-11-29T09:55:07.92345+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:336:getWatchedNewLogs: getLogsWithinTime match file fields:, matchedFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log.20241129095507682 2024-11-29T09:55:07.92348+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:336:getWatchedNewLogs: getLogsWithinTime match file fields: matchedFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log 2024-11-29T09:55:08.11416+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:384:func1: pop head from fileProcessQueue fields: fileName=/opt/myoceanbase/oceanbase/log/observer.log, tailConf="{/opt/myoceanbase/oceanbase/log observer.log 500ms observer ob_light}", logFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log 2024-11-29T09:55:08.18139+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:255:WatchFile: get file change events fields:, start="2024-11-29 09:55:07 +0800 CST", end="2024-11-29 09:55:08.181393554 +0800 CST m=+10835273.298003098", queueTailFileStat.ino=1074285534 2024-11-29T09:55:08.18174+08:00 INFO [21697,] caller=logtailer/log_tailer_utils.go:210:getLogsWithinTime: invoke FindFilesByRegexAndMTime fields:, end="2024-11-29 09:55:08.181393554 +0800 CST m=+10835273.298003098", matchedFiles="[0xc0004d8d00 0xc0001972b0]", dir=/opt/myoceanbase/oceanbase/log, fileName=observer.log, start="2024-11-29 09:55:07 +0800 CST" 2024-11-29T09:55:08.18177+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:336:getWatchedNewLogs: getLogsWithinTime match file fields:, matchedFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log.20241129095507682 2024-11-29T09:55:08.1818+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:336:getWatchedNewLogs: getLogsWithinTime match file fields:, matchedFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log 2024-11-29T09:55:08.18185+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:255:WatchFile: get file change events fields:, start="2024-11-29 09:55:07 +0800 CST", end="2024-11-29 09:55:08.181855071 +0800 CST m=+10835273.298464603", queueTailFileStat.ino=1074285534 2024-11-29T09:55:08.18206+08:00 INFO [21697,] caller=logtailer/log_tailer_utils.go:210:getLogsWithinTime: invoke FindFilesByRegexAndMTime fields: fileName=observer.log, start="2024-11-29 09:55:07 +0800 CST", end="2024-11-29 09:55:08.181855071 +0800 CST m=+10835273.298464603", dir=/opt/myoceanbase/oceanbase/log, matchedFiles="[0xc004e35380 0xc001283040]" 2024-11-29T09:55:08.18208+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:336:getWatchedNewLogs: getLogsWithinTime match file fields:, matchedFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log.20241129095507682 2024-11-29T09:55:08.1821+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:336:getWatchedNewLogs: getLogsWithinTime match file fields:, matchedFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log 2024-11-29T09:55:08.53898+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998487, result count: 84 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998487, max_request_id: 274998570 2024-11-29T09:55:08.61597+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:384:func1: pop head from fileProcessQueue fields: logFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log, fileName=/opt/myoceanbase/oceanbase/log/observer.log, tailConf="{/opt/myoceanbase/oceanbase/log observer.log 500ms observer ob}" 2024-11-29T09:55:08.67504+08:00 INFO [21697,] caller=logtailer/log_tailer_executor.go:384:func1: pop head from fileProcessQueue fields: tailConf="{/opt/myoceanbase/oceanbase/log observer.log 500ms observer ob_light}", fileName=/opt/myoceanbase/oceanbase/log/observer.log, logFileRealPath=/opt/myoceanbase/oceanbase/log/observer.log 2024-11-29T09:55:09.54044+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998571, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998571, max_request_id: 274998603 2024-11-29T09:55:10.02177+08:00 INFO [21697,eb0bd8cd20cb1357] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=eb0bd8cd20cb1357, body=] 2024-11-29T09:55:10.02211+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:55:10.0417+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:55:10.13796+08:00 INFO [21697,328819ddea2dc2a8] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=328819ddea2dc2a8, body=] 2024-11-29T09:55:10.1383+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:10.13834+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:10.13835+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:10.13836+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:10.13838+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:10.13875+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:55:10.54282+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998604, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998604, max_request_id: 274998635 2024-11-29T09:55:10.77917+08:00 INFO [21697,434a2b2592ddda4b] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=434a2b2592ddda4b, body=] 2024-11-29T09:55:10.86678+08:00 INFO [21697,fba2841ea6cf4913] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=fba2841ea6cf4913, body=] 2024-11-29T09:55:10.86705+08:00 WARN [21697,fba2841ea6cf4913] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:10.86708+08:00 WARN [21697,fba2841ea6cf4913] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:10.8671+08:00 WARN [21697,fba2841ea6cf4913] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:10.86711+08:00 WARN [21697,fba2841ea6cf4913] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:10.86712+08:00 WARN [21697,fba2841ea6cf4913] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:10.8708+08:00 INFO [21697,fba2841ea6cf4913] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=fba2841ea6cf4913, body=] 2024-11-29T09:55:11.54536+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998636, result count: 50 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998636, max_request_id: 274998685 2024-11-29T09:55:11.74873+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:55:12.10832+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:55:12.54703+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998686, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998686, max_request_id: 274998722 2024-11-29T09:55:12.79159+08:00 INFO [21697,5915e5426d2e828c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 1 result 2024-11-29T09:55:12.79167+08:00 INFO [21697,5915e5426d2e828c] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:55:12.764875231 +0800 CST m=+10835277.881484763, end time:2024-11-29 09:55:12 +0800 CST, Metrics number 1 2024-11-29T09:55:12.79188+08:00 INFO [21697,5915e5426d2e828c] caller=engine/pipeline.go:145:pipelinePush: push pipeline ob_sql_transaction metrics count 1 2024-11-29T09:55:13.54932+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998723, result count: 49 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998723, max_request_id: 274998771 2024-11-29T09:55:13.86545+08:00 INFO [21697,1bf50690f47f4d7a] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=1bf50690f47f4d7a, body=] 2024-11-29T09:55:13.86566+08:00 WARN [21697,1bf50690f47f4d7a] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:13.86569+08:00 WARN [21697,1bf50690f47f4d7a] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:13.86571+08:00 WARN [21697,1bf50690f47f4d7a] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:13.86572+08:00 WARN [21697,1bf50690f47f4d7a] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:13.86573+08:00 WARN [21697,1bf50690f47f4d7a] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:13.86939+08:00 INFO [21697,1bf50690f47f4d7a] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=1bf50690f47f4d7a, body=] 2024-11-29T09:55:14.55224+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998772, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998772, max_request_id: 274998811 2024-11-29T09:55:15.04146+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:55:15.08871+08:00 INFO [21697,290cdd2740f540ef] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=290cdd2740f540ef, body=] 2024-11-29T09:55:15.08895+08:00 INFO [21697,290cdd2740f540ef] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=290cdd2740f540ef, body=] 2024-11-29T09:55:15.08925+08:00 WARN [21697,290cdd2740f540ef] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:15.08928+08:00 WARN [21697,290cdd2740f540ef] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:15.0893+08:00 WARN [21697,290cdd2740f540ef] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:15.08931+08:00 WARN [21697,290cdd2740f540ef] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:15.08932+08:00 WARN [21697,290cdd2740f540ef] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:15.09962+08:00 INFO [21697,290cdd2740f540ef] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:55:15.10589+08:00 INFO [21697,290cdd2740f540ef] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=290cdd2740f540ef, body=] 2024-11-29T09:55:15.1127+08:00 INFO [21697,290cdd2740f540ef] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=290cdd2740f540ef, body=] 2024-11-29T09:55:15.11285+08:00 INFO [21697,290cdd2740f540ef] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=290cdd2740f540ef, body=] 2024-11-29T09:55:15.11302+08:00 WARN [21697,290cdd2740f540ef] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:55:15.12169+08:00 WARN [21697,290cdd2740f540ef] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:55:15.12504+08:00 INFO [21697,290cdd2740f540ef] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 610, err: 2024-11-29T09:55:15.55389+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998812, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998812, max_request_id: 274998843 2024-11-29T09:55:16.55669+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998844, result count: 58 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998844, max_request_id: 274998901 2024-11-29T09:55:16.87153+08:00 INFO [21697,35d2904c90584053] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=35d2904c90584053, body=] 2024-11-29T09:55:16.87294+08:00 INFO [21697,35d2904c90584053] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=35d2904c90584053, body=] 2024-11-29T09:55:16.8731+08:00 WARN [21697,35d2904c90584053] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:16.87313+08:00 WARN [21697,35d2904c90584053] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:16.87315+08:00 WARN [21697,35d2904c90584053] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:16.87316+08:00 WARN [21697,35d2904c90584053] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:16.87317+08:00 WARN [21697,35d2904c90584053] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:17.09492+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:55:17.55864+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998902, result count: 46 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998902, max_request_id: 274998947 2024-11-29T09:55:17.78589+08:00 INFO [21697,26fe408051ac0889] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:55:17.78595+08:00 INFO [21697,26fe408051ac0889] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:55:17.765673455 +0800 CST m=+10835282.882282988, end time:2024-11-29 09:55:17 +0800 CST, Metrics number 0 2024-11-29T09:55:17.78602+08:00 WARN [21697,26fe408051ac0889] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:55:18.56054+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998948, result count: 28 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998948, max_request_id: 274998975 2024-11-29T09:55:19.48083+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields: coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:55:19.48093+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:55:19.55299+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields: error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:55:19.56286+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274998976, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274998976, max_request_id: 274999011 2024-11-29T09:55:19.8651+08:00 INFO [21697,c1cc8aafe0b84a37] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=c1cc8aafe0b84a37, body=] 2024-11-29T09:55:19.86531+08:00 WARN [21697,c1cc8aafe0b84a37] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:19.86535+08:00 WARN [21697,c1cc8aafe0b84a37] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:19.86536+08:00 WARN [21697,c1cc8aafe0b84a37] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:19.86538+08:00 WARN [21697,c1cc8aafe0b84a37] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:19.86539+08:00 WARN [21697,c1cc8aafe0b84a37] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:19.86986+08:00 INFO [21697,c1cc8aafe0b84a37] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=c1cc8aafe0b84a37, body=] 2024-11-29T09:55:20.042+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:55:20.56853+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999012, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999012, max_request_id: 274999043 2024-11-29T09:55:21.19112+08:00 INFO [21697,cf201b0c75d943f0] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=cf201b0c75d943f0, body=] 2024-11-29T09:55:21.57207+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999044, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999044, max_request_id: 274999085 2024-11-29T09:55:21.74351+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:55:22.10885+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:55:22.19411+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163142372, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163142696, max_request_id: 163142698 2024-11-29T09:55:22.19417+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163142696 is larger than startRequestId 163142372, maybe lost 324 data 2024-11-29T09:55:22.30295+08:00 INFO [21697,d3adc5843769fb61] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=d3adc5843769fb61, body=] 2024-11-29T09:55:22.30348+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:55:22.30406+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:55:22.57513+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999086, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999086, max_request_id: 274999126 2024-11-29T09:55:22.78245+08:00 INFO [21697,0944d8b73812b225] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:55:22.78253+08:00 INFO [21697,0944d8b73812b225] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:55:22.76494882 +0800 CST m=+10835287.881558354, end time:2024-11-29 09:55:22 +0800 CST, Metrics number 0 2024-11-29T09:55:22.78263+08:00 WARN [21697,0944d8b73812b225] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:55:22.86678+08:00 INFO [21697,092fa92433de4dda] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=092fa92433de4dda, body=] 2024-11-29T09:55:22.86699+08:00 WARN [21697,092fa92433de4dda] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:22.86702+08:00 WARN [21697,092fa92433de4dda] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:22.86703+08:00 WARN [21697,092fa92433de4dda] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:22.86704+08:00 WARN [21697,092fa92433de4dda] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:22.86705+08:00 WARN [21697,092fa92433de4dda] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:22.87069+08:00 INFO [21697,092fa92433de4dda] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=092fa92433de4dda, body=] 2024-11-29T09:55:23.57712+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999127, result count: 57 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999127, max_request_id: 274999183 2024-11-29T09:55:24.58384+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999184, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999184, max_request_id: 274999215 2024-11-29T09:55:25.02128+08:00 INFO [21697,109b7e1748f9ac8c] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=109b7e1748f9ac8c, body=] 2024-11-29T09:55:25.02152+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:55:25.04103+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:55:25.13827+08:00 INFO [21697,c4ee229fba25df13] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=c4ee229fba25df13, body=] 2024-11-29T09:55:25.1387+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:25.13874+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:25.13875+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:25.13876+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:25.13878+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:25.13913+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:55:25.58659+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999216, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999216, max_request_id: 274999250 2024-11-29T09:55:25.77905+08:00 INFO [21697,34b865e23f3c6584] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=34b865e23f3c6584, body=] 2024-11-29T09:55:25.86533+08:00 INFO [21697,b1b29f537e4149b6] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=b1b29f537e4149b6, body=] 2024-11-29T09:55:25.86564+08:00 WARN [21697,b1b29f537e4149b6] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:25.8657+08:00 WARN [21697,b1b29f537e4149b6] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:25.86573+08:00 WARN [21697,b1b29f537e4149b6] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:25.86576+08:00 WARN [21697,b1b29f537e4149b6] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:25.86578+08:00 WARN [21697,b1b29f537e4149b6] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:25.87035+08:00 INFO [21697,b1b29f537e4149b6] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=b1b29f537e4149b6, body=] 2024-11-29T09:55:26.58915+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999251, result count: 49 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999251, max_request_id: 274999299 2024-11-29T09:55:27.09544+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:55:27.59068+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999300, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999300, max_request_id: 274999339 2024-11-29T09:55:27.78421+08:00 INFO [21697,f7055d1f43963df4] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:55:27.78429+08:00 INFO [21697,f7055d1f43963df4] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:55:27.765358978 +0800 CST m=+10835292.881968518, end time:2024-11-29 09:55:27 +0800 CST, Metrics number 0 2024-11-29T09:55:27.78439+08:00 WARN [21697,f7055d1f43963df4] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:55:28.60655+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999340, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999340, max_request_id: 274999376 2024-11-29T09:55:28.86488+08:00 INFO [21697,6a24f8d2704a4ae9] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=6a24f8d2704a4ae9, body=] 2024-11-29T09:55:28.86514+08:00 WARN [21697,6a24f8d2704a4ae9] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:28.86518+08:00 WARN [21697,6a24f8d2704a4ae9] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:28.86519+08:00 WARN [21697,6a24f8d2704a4ae9] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:28.86521+08:00 WARN [21697,6a24f8d2704a4ae9] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:28.86522+08:00 WARN [21697,6a24f8d2704a4ae9] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:28.86988+08:00 INFO [21697,6a24f8d2704a4ae9] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=6a24f8d2704a4ae9, body=] 2024-11-29T09:55:29.60917+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999377, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999377, max_request_id: 274999416 2024-11-29T09:55:30.04158+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:55:30.08827+08:00 INFO [21697,fa5a79e4993c4d7b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=fa5a79e4993c4d7b, body=] 2024-11-29T09:55:30.0887+08:00 WARN [21697,fa5a79e4993c4d7b] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:30.08874+08:00 WARN [21697,fa5a79e4993c4d7b] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:30.08876+08:00 WARN [21697,fa5a79e4993c4d7b] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:30.08877+08:00 WARN [21697,fa5a79e4993c4d7b] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:30.08878+08:00 WARN [21697,fa5a79e4993c4d7b] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:30.08912+08:00 INFO [21697,fa5a79e4993c4d7b] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:55:30.08949+08:00 INFO [21697,fa5a79e4993c4d7b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=fa5a79e4993c4d7b, body=] 2024-11-29T09:55:30.09288+08:00 INFO [21697,fa5a79e4993c4d7b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=fa5a79e4993c4d7b, body=] 2024-11-29T09:55:30.09301+08:00 INFO [21697,fa5a79e4993c4d7b] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=fa5a79e4993c4d7b, body=] 2024-11-29T09:55:30.09313+08:00 WARN [21697,fa5a79e4993c4d7b] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:55:30.09324+08:00 INFO [21697,fa5a79e4993c4d7b] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=fa5a79e4993c4d7b, body=] 2024-11-29T09:55:30.09356+08:00 WARN [21697,fa5a79e4993c4d7b] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:55:30.10397+08:00 INFO [21697,fa5a79e4993c4d7b] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 610, err: 2024-11-29T09:55:30.61105+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999417, result count: 30 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999417, max_request_id: 274999446 2024-11-29T09:55:31.32588+08:00 INFO [21697,eb8963cfcc132429] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.98.86, ocpServerIp=, traceId=eb8963cfcc132429, body=] 2024-11-29T09:55:31.3261+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:31.32614+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:31.32615+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:31.32616+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:31.32618+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:31.61395+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999447, result count: 63 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999447, max_request_id: 274999509 2024-11-29T09:55:31.74367+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:55:31.86512+08:00 INFO [21697,bda886a88c6b45a6] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=bda886a88c6b45a6, body=] 2024-11-29T09:55:31.86534+08:00 WARN [21697,bda886a88c6b45a6] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:31.86538+08:00 WARN [21697,bda886a88c6b45a6] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:31.8654+08:00 WARN [21697,bda886a88c6b45a6] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:31.86541+08:00 WARN [21697,bda886a88c6b45a6] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:31.86542+08:00 WARN [21697,bda886a88c6b45a6] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:31.86913+08:00 INFO [21697,bda886a88c6b45a6] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=bda886a88c6b45a6, body=] 2024-11-29T09:55:32.10817+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:55:32.61602+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999510, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999510, max_request_id: 274999547 2024-11-29T09:55:32.78196+08:00 INFO [21697,68c581364e1760d1] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:55:32.78203+08:00 INFO [21697,68c581364e1760d1] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:55:32.765646563 +0800 CST m=+10835297.882256096, end time:2024-11-29 09:55:32 +0800 CST, Metrics number 0 2024-11-29T09:55:32.78211+08:00 WARN [21697,68c581364e1760d1] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:55:32.85571+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 1 result 2024-11-29T09:55:32.85716+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:55:32.85725+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:55:32.85729+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:55:32.85732+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:55:32.85736+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:55:32.8574+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:55:32.85744+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:55:32.85747+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:55:32.85751+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:55:32.85757+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 1 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:55:32.8576+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:55:32.87457+08:00 INFO [21697,3c1d0e25b06362af] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:55:32 +0800 CST, start_request_id:0 2024-11-29T09:55:32.87463+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:32.87472+08:00 INFO [21697,3c1d0e25b06362af] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:55:32 +0800 CST, start_request_id:0 2024-11-29T09:55:32.87474+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:32.87487+08:00 INFO [21697,3c1d0e25b06362af] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:55:32 +0800 CST, start_request_id:0 2024-11-29T09:55:32.87489+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:32.88402+08:00 INFO [21697,3c1d0e25b06362af] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:55:32 +0800 CST, start_request_id:544444 2024-11-29T09:55:32.88408+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544444, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:32.88416+08:00 INFO [21697,3c1d0e25b06362af] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:55:32 +0800 CST, start_request_id:0 2024-11-29T09:55:32.88417+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:32.88771+08:00 INFO [21697,3c1d0e25b06362af] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:55:32 +0800 CST, start_request_id:7154 2024-11-29T09:55:32.88782+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:32.88788+08:00 INFO [21697,3c1d0e25b06362af] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:55:32 +0800 CST, start_request_id:387 2024-11-29T09:55:32.88789+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:32.889+08:00 INFO [21697,3c1d0e25b06362af] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:55:32 +0800 CST, start_request_id:0 2024-11-29T09:55:32.88904+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:32.89239+08:00 INFO [21697,3c1d0e25b06362af] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:55:32 +0800 CST, start_request_id:0 2024-11-29T09:55:32.89244+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:55:32.90782+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907751, got 1 result 2024-11-29T09:55:32.90797+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:55:32.90799+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:55:32.90801+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:55:32.90811+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:55:32.90838+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:55:32.90886+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:55:32.90906+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:55:32.90939+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:55:32.90975+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:55:32.91017+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:55:32.91024+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:55:32.91036+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:55:32.91116+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907751, got 1 result, planStatMetrics number 1, planMetaMetrics number 1, planExplainMetrics number 0 2024-11-29T09:55:32.9112+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:55:32.92149+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807619, got 1 result 2024-11-29T09:55:32.92235+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807619, got 1 result, planStatMetrics number 1, planMetaMetrics number 1, planExplainMetrics number 0 2024-11-29T09:55:32.92239+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 2, planExplainMetrics number 0, planMetaMetrics number 2 2024-11-29T09:55:32.92341+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:55:32.92344+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:55:32.92346+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 2, planExplainMetrics number 0, planMetaMetrics number 2 2024-11-29T09:55:32.92733+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:55:32.92737+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:55:32.92739+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 2, planExplainMetrics number 0, planMetaMetrics number 2 2024-11-29T09:55:32.97012+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:55:32.97017+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:55:32.97021+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 2, planExplainMetrics number 0, planMetaMetrics number 2 2024-11-29T09:55:32.97027+08:00 INFO [21697,04a7203a3c3d9455] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:55:32 +0800 CST, end time:2024-11-29 09:55:32 +0800 CST, planMetaMetrics number 2, planExplainMetrics number 0, planStatMetrics number 2 2024-11-29T09:55:33.19906+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 5 2024-11-29T09:55:33.19912+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:55:33.19916+08:00 INFO [21697,0b9c31ac6ca3a4f9] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 9, lost data number: 689, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 5, sql_audit_progress Metrics number 1 2024-11-29T09:55:33.21217+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163142699, result count: 304 auditMsg number: 0, diagMsg number: 0, min_request_id: 163142880, max_request_id: 163143254 2024-11-29T09:55:33.21223+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163142880 is larger than startRequestId 163142699, maybe lost 181 data 2024-11-29T09:55:33.61671+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 51 2024-11-29T09:55:33.61678+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:55:33.61683+08:00 INFO [21697,f55d6932f458eb65] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1456, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 51, sql_audit_progress Metrics number 1 2024-11-29T09:55:33.61938+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999548, result count: 83 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999548, max_request_id: 274999630 2024-11-29T09:55:34.48123+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:55:34.48134+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:55:34.55286+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:55:34.62177+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999631, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999631, max_request_id: 274999672 2024-11-29T09:55:34.86622+08:00 INFO [21697,49272a4225dd4e9a] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=49272a4225dd4e9a, body=] 2024-11-29T09:55:34.86645+08:00 WARN [21697,49272a4225dd4e9a] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:34.86649+08:00 WARN [21697,49272a4225dd4e9a] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:34.86651+08:00 WARN [21697,49272a4225dd4e9a] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:34.86652+08:00 WARN [21697,49272a4225dd4e9a] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:34.86653+08:00 WARN [21697,49272a4225dd4e9a] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:34.87073+08:00 INFO [21697,49272a4225dd4e9a] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=49272a4225dd4e9a, body=] 2024-11-29T09:55:35.04186+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:55:35.20046+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:55:35.20058+08:00 INFO [21697,eb3d920479b7fab8] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:55:35.62444+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999673, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999673, max_request_id: 274999704 2024-11-29T09:55:36.1909+08:00 INFO [21697,96cb7946f7eab982] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=96cb7946f7eab982, body=] 2024-11-29T09:55:36.63373+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999705, result count: 62 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999705, max_request_id: 274999766 2024-11-29T09:55:36.86281+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:55:36.86292+08:00 INFO [21697,5e7d5b91919d8fd9] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:55:36.95399+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:55:36.9541+08:00 INFO [21697,fdd3be64a61b8bf0] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:55:37.09474+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:55:37.30184+08:00 INFO [21697,d1d0a733f51173f3] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=d1d0a733f51173f3, body=] 2024-11-29T09:55:37.30217+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:55:37.30249+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:55:37.63826+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999767, result count: 62 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999767, max_request_id: 274999828 2024-11-29T09:55:37.78697+08:00 INFO [21697,623c8442594a8c88] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:55:37.78704+08:00 INFO [21697,623c8442594a8c88] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:55:37.76577766 +0800 CST m=+10835302.882387193, end time:2024-11-29 09:55:37 +0800 CST, Metrics number 0 2024-11-29T09:55:37.78715+08:00 WARN [21697,623c8442594a8c88] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:55:37.86468+08:00 INFO [21697,3edc4f823f744958] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=3edc4f823f744958, body=] 2024-11-29T09:55:37.86493+08:00 WARN [21697,3edc4f823f744958] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:37.86497+08:00 WARN [21697,3edc4f823f744958] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:37.86498+08:00 WARN [21697,3edc4f823f744958] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:37.865+08:00 WARN [21697,3edc4f823f744958] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:37.86501+08:00 WARN [21697,3edc4f823f744958] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:37.86934+08:00 INFO [21697,3edc4f823f744958] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=3edc4f823f744958, body=] 2024-11-29T09:55:38.64052+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999829, result count: 28 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999829, max_request_id: 274999856 2024-11-29T09:55:39.21609+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163143255, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163143337, max_request_id: 163143339 2024-11-29T09:55:39.21615+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163143337 is larger than startRequestId 163143255, maybe lost 82 data 2024-11-29T09:55:39.64271+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999857, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999857, max_request_id: 274999891 2024-11-29T09:55:40.02075+08:00 INFO [21697,ee215bc6d63e4118] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=ee215bc6d63e4118, body=] 2024-11-29T09:55:40.021+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:55:40.04148+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:55:40.1372+08:00 INFO [21697,533124bed2f5411d] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=533124bed2f5411d, body=] 2024-11-29T09:55:40.13768+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:40.13773+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:40.13775+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:40.13777+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:40.13778+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:40.13826+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1506, err: 2024-11-29T09:55:40.64471+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999892, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999892, max_request_id: 274999927 2024-11-29T09:55:40.77832+08:00 INFO [21697,f3a9bc0bcb199076] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=f3a9bc0bcb199076, body=] 2024-11-29T09:55:40.86582+08:00 INFO [21697,e1dfe2e30e374e41] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=e1dfe2e30e374e41, body=] 2024-11-29T09:55:40.86609+08:00 WARN [21697,e1dfe2e30e374e41] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:40.86614+08:00 WARN [21697,e1dfe2e30e374e41] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:40.86616+08:00 WARN [21697,e1dfe2e30e374e41] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:40.86617+08:00 WARN [21697,e1dfe2e30e374e41] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:40.86619+08:00 WARN [21697,e1dfe2e30e374e41] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:40.88048+08:00 INFO [21697,e1dfe2e30e374e41] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=e1dfe2e30e374e41, body=] 2024-11-29T09:55:41.64701+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999928, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999928, max_request_id: 274999970 2024-11-29T09:55:41.74335+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:55:42.09548+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:55:42.64883+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 274999971, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 274999971, max_request_id: 275000010 2024-11-29T09:55:42.79135+08:00 INFO [21697,e06314a561b7d88c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:55:42.79142+08:00 INFO [21697,e06314a561b7d88c] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:55:42.775152446 +0800 CST m=+10835307.891761978, end time:2024-11-29 09:55:42 +0800 CST, Metrics number 0 2024-11-29T09:55:42.79149+08:00 WARN [21697,e06314a561b7d88c] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:55:43.65216+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000011, result count: 58 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000011, max_request_id: 275000068 2024-11-29T09:55:43.86458+08:00 INFO [21697,17bb15fe8aa34db4] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=17bb15fe8aa34db4, body=] 2024-11-29T09:55:43.86479+08:00 WARN [21697,17bb15fe8aa34db4] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:43.86482+08:00 WARN [21697,17bb15fe8aa34db4] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:43.86483+08:00 WARN [21697,17bb15fe8aa34db4] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:43.86484+08:00 WARN [21697,17bb15fe8aa34db4] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:43.86486+08:00 WARN [21697,17bb15fe8aa34db4] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:43.86858+08:00 INFO [21697,17bb15fe8aa34db4] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=17bb15fe8aa34db4, body=] 2024-11-29T09:55:44.65395+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000069, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000069, max_request_id: 275000103 2024-11-29T09:55:45.04169+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:55:45.0879+08:00 INFO [21697,b3b02b2bde4a48cd] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=b3b02b2bde4a48cd, body=] 2024-11-29T09:55:45.0883+08:00 WARN [21697,b3b02b2bde4a48cd] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:45.08834+08:00 WARN [21697,b3b02b2bde4a48cd] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:45.08835+08:00 WARN [21697,b3b02b2bde4a48cd] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:45.08837+08:00 WARN [21697,b3b02b2bde4a48cd] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:45.08839+08:00 WARN [21697,b3b02b2bde4a48cd] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:45.08879+08:00 INFO [21697,b3b02b2bde4a48cd] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:55:45.08919+08:00 INFO [21697,b3b02b2bde4a48cd] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=b3b02b2bde4a48cd, body=] 2024-11-29T09:55:45.09241+08:00 INFO [21697,b3b02b2bde4a48cd] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=b3b02b2bde4a48cd, body=] 2024-11-29T09:55:45.09255+08:00 INFO [21697,b3b02b2bde4a48cd] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=b3b02b2bde4a48cd, body=] 2024-11-29T09:55:45.09269+08:00 WARN [21697,b3b02b2bde4a48cd] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:55:45.09279+08:00 INFO [21697,b3b02b2bde4a48cd] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=b3b02b2bde4a48cd, body=] 2024-11-29T09:55:45.09432+08:00 WARN [21697,b3b02b2bde4a48cd] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:55:45.11147+08:00 INFO [21697,b3b02b2bde4a48cd] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:55:45.65681+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000104, result count: 29 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000104, max_request_id: 275000132 2024-11-29T09:55:46.6638+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000133, result count: 62 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000133, max_request_id: 275000194 2024-11-29T09:55:46.86448+08:00 INFO [21697,0de5b6e16da34426] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=0de5b6e16da34426, body=] 2024-11-29T09:55:46.86471+08:00 WARN [21697,0de5b6e16da34426] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:46.86475+08:00 WARN [21697,0de5b6e16da34426] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:46.86476+08:00 WARN [21697,0de5b6e16da34426] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:46.86478+08:00 WARN [21697,0de5b6e16da34426] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:46.86479+08:00 WARN [21697,0de5b6e16da34426] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:46.86917+08:00 INFO [21697,0de5b6e16da34426] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=0de5b6e16da34426, body=] 2024-11-29T09:55:47.09602+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:55:47.66647+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000195, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000195, max_request_id: 275000234 2024-11-29T09:55:47.79172+08:00 INFO [21697,b528001fd350e9ed] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:55:47.79181+08:00 INFO [21697,b528001fd350e9ed] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:55:47.765171858 +0800 CST m=+10835312.881781394, end time:2024-11-29 09:55:47 +0800 CST, Metrics number 0 2024-11-29T09:55:47.79193+08:00 WARN [21697,b528001fd350e9ed] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:55:48.30312+08:00 INFO [21697,02b1b77813075346] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.98.86, ocpServerIp=, traceId=02b1b77813075346, body=] 2024-11-29T09:55:48.66815+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000235, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000235, max_request_id: 275000267 2024-11-29T09:55:49.48135+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields: coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:55:49.48143+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:55:49.55337+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:55:49.67012+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000268, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000268, max_request_id: 275000310 2024-11-29T09:55:49.86435+08:00 INFO [21697,7cbf152568484f0c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=7cbf152568484f0c, body=] 2024-11-29T09:55:49.86459+08:00 WARN [21697,7cbf152568484f0c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:49.86462+08:00 WARN [21697,7cbf152568484f0c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:49.86464+08:00 WARN [21697,7cbf152568484f0c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:49.86465+08:00 WARN [21697,7cbf152568484f0c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:49.86466+08:00 WARN [21697,7cbf152568484f0c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:49.86843+08:00 INFO [21697,7cbf152568484f0c] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=7cbf152568484f0c, body=] 2024-11-29T09:55:50.04131+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:55:50.67169+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000311, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000311, max_request_id: 275000341 2024-11-29T09:55:51.1905+08:00 INFO [21697,28dfc765410cf06b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=28dfc765410cf06b, body=] 2024-11-29T09:55:51.67371+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000342, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000342, max_request_id: 275000384 2024-11-29T09:55:51.74383+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:55:52.11505+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:55:52.30187+08:00 INFO [21697,3d057d1ace071dc1] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=3d057d1ace071dc1, body=] 2024-11-29T09:55:52.30226+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:55:52.30262+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:55:52.37859+08:00 INFO [21697,ed85e7e5ed336283] caller=host/custom.go:178:func10: collect observer dir exists info 2024-11-29T09:55:52.67563+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000385, result count: 52 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000385, max_request_id: 275000436 2024-11-29T09:55:52.78176+08:00 INFO [21697,b6b98ec2678c8d8e] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:55:52.78183+08:00 INFO [21697,b6b98ec2678c8d8e] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:55:52.764872878 +0800 CST m=+10835317.881482410, end time:2024-11-29 09:55:52 +0800 CST, Metrics number 0 2024-11-29T09:55:52.7819+08:00 WARN [21697,b6b98ec2678c8d8e] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:55:52.86429+08:00 INFO [21697,dbe6fd1a8d7a4ec3] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=dbe6fd1a8d7a4ec3, body=] 2024-11-29T09:55:52.86454+08:00 WARN [21697,dbe6fd1a8d7a4ec3] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:52.8646+08:00 WARN [21697,dbe6fd1a8d7a4ec3] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:52.86462+08:00 WARN [21697,dbe6fd1a8d7a4ec3] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:52.86464+08:00 WARN [21697,dbe6fd1a8d7a4ec3] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:52.86465+08:00 WARN [21697,dbe6fd1a8d7a4ec3] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:52.86969+08:00 INFO [21697,dbe6fd1a8d7a4ec3] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=dbe6fd1a8d7a4ec3, body=] 2024-11-29T09:55:53.67828+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000437, result count: 73 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000437, max_request_id: 275000509 2024-11-29T09:55:54.68044+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000510, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000510, max_request_id: 275000540 2024-11-29T09:55:55.01942+08:00 INFO [21697,239c033cb9f338a3] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=239c033cb9f338a3, body=] 2024-11-29T09:55:55.01964+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:55:55.04172+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:55:55.13723+08:00 INFO [21697,e58d4c7a59557766] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=e58d4c7a59557766, body=] 2024-11-29T09:55:55.13762+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:55.13766+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:55.13768+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:55.1377+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:55.13771+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:55.13809+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:55:55.22674+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163143340, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163143662, max_request_id: 163143671 2024-11-29T09:55:55.22679+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163143662 is larger than startRequestId 163143340, maybe lost 322 data 2024-11-29T09:55:55.68184+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000541, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000541, max_request_id: 275000572 2024-11-29T09:55:55.77798+08:00 INFO [21697,f6e8863f65f1da33] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=f6e8863f65f1da33, body=] 2024-11-29T09:55:55.86442+08:00 INFO [21697,99b89c9742514b61] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=99b89c9742514b61, body=] 2024-11-29T09:55:55.86467+08:00 WARN [21697,99b89c9742514b61] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:55.86471+08:00 WARN [21697,99b89c9742514b61] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:55.86472+08:00 WARN [21697,99b89c9742514b61] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:55.86473+08:00 WARN [21697,99b89c9742514b61] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:55.86475+08:00 WARN [21697,99b89c9742514b61] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:55.86897+08:00 INFO [21697,99b89c9742514b61] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=99b89c9742514b61, body=] 2024-11-29T09:55:56.68638+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000573, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000573, max_request_id: 275000615 2024-11-29T09:55:57.09521+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:55:57.68809+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000616, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000616, max_request_id: 275000653 2024-11-29T09:55:57.78201+08:00 INFO [21697,3c67e0acfee2c441] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:55:57.78208+08:00 INFO [21697,3c67e0acfee2c441] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:55:57.765668946 +0800 CST m=+10835322.882278479, end time:2024-11-29 09:55:57 +0800 CST, Metrics number 0 2024-11-29T09:55:57.78216+08:00 WARN [21697,3c67e0acfee2c441] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:55:58.6897+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000654, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000654, max_request_id: 275000693 2024-11-29T09:55:58.86561+08:00 INFO [21697,865b4b9ab7664d68] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=865b4b9ab7664d68, body=] 2024-11-29T09:55:58.86582+08:00 WARN [21697,865b4b9ab7664d68] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:55:58.86585+08:00 WARN [21697,865b4b9ab7664d68] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:55:58.86587+08:00 WARN [21697,865b4b9ab7664d68] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:55:58.86588+08:00 WARN [21697,865b4b9ab7664d68] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:58.86589+08:00 WARN [21697,865b4b9ab7664d68] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:55:58.86984+08:00 INFO [21697,865b4b9ab7664d68] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=865b4b9ab7664d68, body=] 2024-11-29T09:55:59.69174+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000694, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000694, max_request_id: 275000731 2024-11-29T09:56:00.04204+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:56:00.09024+08:00 INFO [21697,fbdfe3c9b57547d1] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=fbdfe3c9b57547d1, body=] 2024-11-29T09:56:00.09403+08:00 INFO [21697,fbdfe3c9b57547d1] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=fbdfe3c9b57547d1, body=] 2024-11-29T09:56:00.09417+08:00 INFO [21697,fbdfe3c9b57547d1] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=fbdfe3c9b57547d1, body=] 2024-11-29T09:56:00.09432+08:00 WARN [21697,fbdfe3c9b57547d1] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:56:00.09444+08:00 INFO [21697,fbdfe3c9b57547d1] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=fbdfe3c9b57547d1, body=] 2024-11-29T09:56:00.09457+08:00 INFO [21697,fbdfe3c9b57547d1] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=fbdfe3c9b57547d1, body=] 2024-11-29T09:56:00.09625+08:00 WARN [21697,fbdfe3c9b57547d1] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:56:00.0965+08:00 WARN [21697,fbdfe3c9b57547d1] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:00.09652+08:00 WARN [21697,fbdfe3c9b57547d1] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:00.09654+08:00 WARN [21697,fbdfe3c9b57547d1] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:00.09656+08:00 WARN [21697,fbdfe3c9b57547d1] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:00.09658+08:00 WARN [21697,fbdfe3c9b57547d1] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:00.10578+08:00 INFO [21697,fbdfe3c9b57547d1] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:56:00.10614+08:00 INFO [21697,fbdfe3c9b57547d1] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:56:00.69454+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000732, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000732, max_request_id: 275000767 2024-11-29T09:56:01.18622+08:00 INFO [21697,388171c92900a462] caller=common/middleware.go:59:func1: API request: [GET /api/v1/log/alarms, client=1.1.1.98, ocpServerIp=1.1.1.98, traceId=388171c92900a462, body=] 2024-11-29T09:56:01.18642+08:00 INFO [21697,] caller=alarmscache/alarm_handler.go:73:getPipelieGroupReader: get alarms, result size:0, cache list size:4 fields: uri=/api/v1/log/alarms 2024-11-29T09:56:01.70443+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000768, result count: 126 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000768, max_request_id: 275000893 2024-11-29T09:56:01.74396+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:56:01.86418+08:00 INFO [21697,64bd5779d50c4773] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=64bd5779d50c4773, body=] 2024-11-29T09:56:01.8644+08:00 WARN [21697,64bd5779d50c4773] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:01.86443+08:00 WARN [21697,64bd5779d50c4773] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:01.86445+08:00 WARN [21697,64bd5779d50c4773] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:01.86446+08:00 WARN [21697,64bd5779d50c4773] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:01.86447+08:00 WARN [21697,64bd5779d50c4773] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:01.8684+08:00 INFO [21697,64bd5779d50c4773] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=64bd5779d50c4773, body=] 2024-11-29T09:56:02.10842+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:56:02.70608+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000894, result count: 52 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000894, max_request_id: 275000945 2024-11-29T09:56:02.74188+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:245:updateMetaInfo: successfully init sqlAudit tenant config 2024-11-29T09:56:02.78246+08:00 INFO [21697,3b84d42cce12765c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:56:02.78252+08:00 INFO [21697,3b84d42cce12765c] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:56:02.765817568 +0800 CST m=+10835327.882427101, end time:2024-11-29 09:56:02 +0800 CST, Metrics number 0 2024-11-29T09:56:02.78261+08:00 WARN [21697,3b84d42cce12765c] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:56:02.85746+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:56:02.85894+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:56:02.85901+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:56:02.85903+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:56:02.85905+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:56:02.85908+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:56:02.8591+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:56:02.85912+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:56:02.85914+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:56:02.85916+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:56:02.85918+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:56:02.85919+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:56:02.87319+08:00 INFO [21697,9f9f018292a5d480] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:56:02 +0800 CST, start_request_id:0 2024-11-29T09:56:02.87325+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:02.87333+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:796:refreshTenants: configured tenants: map[1:sys 1002:ocp 1004:dbamonitor 1006:dbmanager 1008:datatransfer] 2024-11-29T09:56:02.87338+08:00 INFO [21697,9f9f018292a5d480] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:56:02 +0800 CST, start_request_id:0 2024-11-29T09:56:02.87341+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:02.87346+08:00 INFO [21697,9f9f018292a5d480] caller=oceanbase/sql_plan_monitor.go:385:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1, collect time:2024-11-29 09:56:02 +0800 CST, start_request_id:544444, min_request_id:544445, max_request_id:544455 2024-11-29T09:56:02.87352+08:00 INFO [21697,9f9f018292a5d480] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:56:02 +0800 CST, start_request_id:0 2024-11-29T09:56:02.87354+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:02.87567+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544444, min_request_id: 544445, max_request_id: 544455, collected metrics number: 11 2024-11-29T09:56:02.88217+08:00 INFO [21697,9f9f018292a5d480] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:56:02 +0800 CST, start_request_id:0 2024-11-29T09:56:02.88223+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:02.88229+08:00 INFO [21697,9f9f018292a5d480] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 2 complete, collect time:2024-11-29 09:56:02 +0800 CST, start_request_id:544455 2024-11-29T09:56:02.88231+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 2, start_request_id: 544455, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:02.8824+08:00 INFO [21697,9f9f018292a5d480] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:56:02 +0800 CST, start_request_id:387 2024-11-29T09:56:02.88242+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:02.88592+08:00 INFO [21697,9f9f018292a5d480] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:56:02 +0800 CST, start_request_id:7154 2024-11-29T09:56:02.88595+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:02.88608+08:00 INFO [21697,9f9f018292a5d480] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:56:02 +0800 CST, start_request_id:0 2024-11-29T09:56:02.8861+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:02.89982+08:00 INFO [21697,9f9f018292a5d480] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:56:02 +0800 CST, start_request_id:0 2024-11-29T09:56:02.89987+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:02.91039+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:56:02.91045+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:02.91047+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:56:02.91057+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807619, got 0 result 2024-11-29T09:56:02.91059+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807619, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:02.91063+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:56:02.91073+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:56:02.91075+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:02.91077+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:56:02.91089+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907753, got 2 result 2024-11-29T09:56:02.91099+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:56:02.91101+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:02.91108+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:56:02.91109+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:02.91113+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 2, planExplainMetrics number 0, planMetaMetrics number 2 2024-11-29T09:56:02.91115+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 2, planExplainMetrics number 0, planMetaMetrics number 2 2024-11-29T09:56:02.91207+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907753, got 2 result, planStatMetrics number 2, planMetaMetrics number 2, planExplainMetrics number 0 2024-11-29T09:56:02.9121+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 2, planExplainMetrics number 0, planMetaMetrics number 2 2024-11-29T09:56:02.91754+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:56:02.91759+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:02.91761+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 2, planExplainMetrics number 0, planMetaMetrics number 2 2024-11-29T09:56:02.91824+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:56:02.91826+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:02.91828+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 2, planExplainMetrics number 0, planMetaMetrics number 2 2024-11-29T09:56:02.97327+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:56:02.97334+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:02.97336+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 2, planExplainMetrics number 0, planMetaMetrics number 2 2024-11-29T09:56:02.97341+08:00 INFO [21697,5d80727463d30b81] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:56:02 +0800 CST, end time:2024-11-29 09:56:02 +0800 CST, planMetaMetrics number 2, planExplainMetrics number 0, planStatMetrics number 2 2024-11-29T09:56:03.70663+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 53 2024-11-29T09:56:03.7067+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:56:03.70674+08:00 INFO [21697,54b1fa81e5f69d82] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1398, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 53, sql_audit_progress Metrics number 1 2024-11-29T09:56:03.70946+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275000946, result count: 106 auditMsg number: 0, diagMsg number: 0, min_request_id: 275000946, max_request_id: 275001051 2024-11-29T09:56:04.48141+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:56:04.48148+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:56:04.56233+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields: error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:56:04.71131+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001052, result count: 49 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001052, max_request_id: 275001100 2024-11-29T09:56:04.86461+08:00 INFO [21697,4cae26b300ea4999] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=4cae26b300ea4999, body=] 2024-11-29T09:56:04.86483+08:00 WARN [21697,4cae26b300ea4999] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:04.86486+08:00 WARN [21697,4cae26b300ea4999] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:04.86488+08:00 WARN [21697,4cae26b300ea4999] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:04.86489+08:00 WARN [21697,4cae26b300ea4999] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:04.8649+08:00 WARN [21697,4cae26b300ea4999] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:04.86853+08:00 INFO [21697,4cae26b300ea4999] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=4cae26b300ea4999, body=] 2024-11-29T09:56:05.04188+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:56:05.29447+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:56:05.2946+08:00 INFO [21697,259ef80362b6fde4] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:56:05.31462+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1008 from request_id 117581950, result count: 5 auditMsg number: 0, diagMsg number: 0, min_request_id: 117585588, max_request_id: 117585592 2024-11-29T09:56:05.31468+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1008 minRequestId 117585588 is larger than startRequestId 117581950, maybe lost 3638 data 2024-11-29T09:56:05.71349+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001101, result count: 25 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001101, max_request_id: 275001125 2024-11-29T09:56:06.19496+08:00 INFO [21697,a5c995dd35fb798e] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=a5c995dd35fb798e, body=] 2024-11-29T09:56:06.23154+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 12 2024-11-29T09:56:06.23162+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:56:06.23166+08:00 INFO [21697,bba108ab980f01ef] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 310, lost data number: 585, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 12, sql_audit_progress Metrics number 1 2024-11-29T09:56:06.23434+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163143672, result count: 8 auditMsg number: 0, diagMsg number: 0, min_request_id: 163144037, max_request_id: 163144053 2024-11-29T09:56:06.23439+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163144037 is larger than startRequestId 163143672, maybe lost 365 data 2024-11-29T09:56:06.72211+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001126, result count: 74 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001126, max_request_id: 275001199 2024-11-29T09:56:06.96907+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:56:06.96917+08:00 INFO [21697,d662f441ac577100] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:56:06.99028+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1006 from request_id 136527397, result count: 5 auditMsg number: 0, diagMsg number: 0, min_request_id: 136531581, max_request_id: 136531586 2024-11-29T09:56:06.99033+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1006 minRequestId 136531581 is larger than startRequestId 136527397, maybe lost 4184 data 2024-11-29T09:56:07.05835+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:56:07.05846+08:00 INFO [21697,f18be8a978b646fc] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:56:07.095+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:56:07.3011+08:00 INFO [21697,cfe4ed5d1e49dcc4] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=cfe4ed5d1e49dcc4, body=] 2024-11-29T09:56:07.3015+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:56:07.30189+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:56:07.72544+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001200, result count: 58 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001200, max_request_id: 275001257 2024-11-29T09:56:07.79294+08:00 INFO [21697,272f51400685e302] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:56:07.79302+08:00 INFO [21697,272f51400685e302] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:56:07.765853595 +0800 CST m=+10835332.882463131, end time:2024-11-29 09:56:07 +0800 CST, Metrics number 0 2024-11-29T09:56:07.7931+08:00 WARN [21697,272f51400685e302] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:56:07.86389+08:00 INFO [21697,a78e412e7315474c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=a78e412e7315474c, body=] 2024-11-29T09:56:07.8641+08:00 WARN [21697,a78e412e7315474c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:07.86413+08:00 WARN [21697,a78e412e7315474c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:07.86415+08:00 WARN [21697,a78e412e7315474c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:07.86417+08:00 WARN [21697,a78e412e7315474c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:07.86418+08:00 WARN [21697,a78e412e7315474c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:07.87018+08:00 INFO [21697,a78e412e7315474c] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=a78e412e7315474c, body=] 2024-11-29T09:56:08.72768+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001258, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001258, max_request_id: 275001288 2024-11-29T09:56:09.72904+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001289, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001289, max_request_id: 275001321 2024-11-29T09:56:10.02001+08:00 INFO [21697,46df227be67b433d] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=46df227be67b433d, body=] 2024-11-29T09:56:10.02025+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:56:10.04115+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:56:10.13638+08:00 INFO [21697,47a822bf3580ce6c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=47a822bf3580ce6c, body=] 2024-11-29T09:56:10.13676+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:10.1368+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:10.13681+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:10.13683+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:10.13684+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:10.13718+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1506, err: 2024-11-29T09:56:10.73068+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001322, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001322, max_request_id: 275001358 2024-11-29T09:56:10.77753+08:00 INFO [21697,35e08c234e95a233] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=35e08c234e95a233, body=] 2024-11-29T09:56:10.86556+08:00 INFO [21697,ab1fed9522a14dd0] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=ab1fed9522a14dd0, body=] 2024-11-29T09:56:10.86582+08:00 WARN [21697,ab1fed9522a14dd0] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:10.86585+08:00 WARN [21697,ab1fed9522a14dd0] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:10.86587+08:00 WARN [21697,ab1fed9522a14dd0] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:10.86588+08:00 WARN [21697,ab1fed9522a14dd0] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:10.8659+08:00 WARN [21697,ab1fed9522a14dd0] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:10.86984+08:00 INFO [21697,ab1fed9522a14dd0] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=ab1fed9522a14dd0, body=] 2024-11-29T09:56:11.73308+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001359, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001359, max_request_id: 275001399 2024-11-29T09:56:11.74366+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:56:12.10877+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:56:12.73525+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001400, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001400, max_request_id: 275001439 2024-11-29T09:56:12.78215+08:00 INFO [21697,b1969b1af069533b] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:56:12.78221+08:00 INFO [21697,b1969b1af069533b] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:56:12.765262035 +0800 CST m=+10835337.881871571, end time:2024-11-29 09:56:12 +0800 CST, Metrics number 0 2024-11-29T09:56:12.78229+08:00 WARN [21697,b1969b1af069533b] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:56:13.73802+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001440, result count: 59 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001440, max_request_id: 275001498 2024-11-29T09:56:13.86387+08:00 INFO [21697,961c589dbe4f4e80] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=961c589dbe4f4e80, body=] 2024-11-29T09:56:13.86406+08:00 WARN [21697,961c589dbe4f4e80] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:13.86409+08:00 WARN [21697,961c589dbe4f4e80] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:13.86411+08:00 WARN [21697,961c589dbe4f4e80] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:13.86412+08:00 WARN [21697,961c589dbe4f4e80] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:13.86413+08:00 WARN [21697,961c589dbe4f4e80] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:13.86774+08:00 INFO [21697,961c589dbe4f4e80] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=961c589dbe4f4e80, body=] 2024-11-29T09:56:14.73951+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001499, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001499, max_request_id: 275001529 2024-11-29T09:56:15.04113+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:56:15.08711+08:00 INFO [21697,5974755914984f95] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=5974755914984f95, body=] 2024-11-29T09:56:15.08729+08:00 INFO [21697,5974755914984f95] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=5974755914984f95, body=] 2024-11-29T09:56:15.08777+08:00 WARN [21697,5974755914984f95] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:15.08781+08:00 WARN [21697,5974755914984f95] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:15.08782+08:00 WARN [21697,5974755914984f95] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:15.08784+08:00 WARN [21697,5974755914984f95] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:15.08785+08:00 WARN [21697,5974755914984f95] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:15.09661+08:00 INFO [21697,5974755914984f95] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:56:15.10236+08:00 INFO [21697,5974755914984f95] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=5974755914984f95, body=] 2024-11-29T09:56:15.10541+08:00 INFO [21697,5974755914984f95] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=5974755914984f95, body=] 2024-11-29T09:56:15.10552+08:00 INFO [21697,5974755914984f95] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=5974755914984f95, body=] 2024-11-29T09:56:15.10564+08:00 WARN [21697,5974755914984f95] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:56:15.10705+08:00 WARN [21697,5974755914984f95] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:56:15.10902+08:00 INFO [21697,5974755914984f95] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:56:15.74149+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001530, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001530, max_request_id: 275001566 2024-11-29T09:56:16.74443+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001567, result count: 54 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001567, max_request_id: 275001620 2024-11-29T09:56:16.86528+08:00 INFO [21697,7d9775665bec4c38] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=7d9775665bec4c38, body=] 2024-11-29T09:56:16.86557+08:00 WARN [21697,7d9775665bec4c38] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:16.86562+08:00 WARN [21697,7d9775665bec4c38] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:16.86564+08:00 WARN [21697,7d9775665bec4c38] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:16.86567+08:00 WARN [21697,7d9775665bec4c38] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:16.8657+08:00 WARN [21697,7d9775665bec4c38] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:16.8726+08:00 INFO [21697,7d9775665bec4c38] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=7d9775665bec4c38, body=] 2024-11-29T09:56:17.09574+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:56:17.76337+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001621, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001621, max_request_id: 275001658 2024-11-29T09:56:17.78114+08:00 INFO [21697,9db195dbe9f8a3bc] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:56:17.78119+08:00 INFO [21697,9db195dbe9f8a3bc] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:56:17.764987169 +0800 CST m=+10835342.881596706, end time:2024-11-29 09:56:17 +0800 CST, Metrics number 0 2024-11-29T09:56:17.78127+08:00 WARN [21697,9db195dbe9f8a3bc] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:56:18.76624+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001659, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001659, max_request_id: 275001689 2024-11-29T09:56:19.48092+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields: coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:56:19.48101+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:56:19.559+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:56:19.77478+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001690, result count: 45 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001690, max_request_id: 275001734 2024-11-29T09:56:19.86494+08:00 INFO [21697,4e2c2875ec974d91] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=4e2c2875ec974d91, body=] 2024-11-29T09:56:19.86519+08:00 WARN [21697,4e2c2875ec974d91] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:19.86523+08:00 WARN [21697,4e2c2875ec974d91] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:19.86525+08:00 WARN [21697,4e2c2875ec974d91] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:19.86526+08:00 WARN [21697,4e2c2875ec974d91] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:19.86528+08:00 WARN [21697,4e2c2875ec974d91] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:19.87183+08:00 INFO [21697,4e2c2875ec974d91] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=4e2c2875ec974d91, body=] 2024-11-29T09:56:20.04159+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:56:20.77689+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001735, result count: 27 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001735, max_request_id: 275001761 2024-11-29T09:56:21.18901+08:00 INFO [21697,6917e4d3e758cf05] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=6917e4d3e758cf05, body=] 2024-11-29T09:56:21.7499+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:56:21.77982+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001762, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001762, max_request_id: 275001801 2024-11-29T09:56:22.10854+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:56:22.24346+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163144054, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163144382, max_request_id: 163144384 2024-11-29T09:56:22.24351+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163144382 is larger than startRequestId 163144054, maybe lost 328 data 2024-11-29T09:56:22.30095+08:00 INFO [21697,3500bae68f8b8243] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=3500bae68f8b8243, body=] 2024-11-29T09:56:22.3013+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:56:22.30222+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:56:22.78144+08:00 INFO [21697,5cf6b756bcc16afe] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:56:22.7815+08:00 INFO [21697,5cf6b756bcc16afe] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:56:22.765036131 +0800 CST m=+10835347.881645669, end time:2024-11-29 09:56:22 +0800 CST, Metrics number 0 2024-11-29T09:56:22.78157+08:00 WARN [21697,5cf6b756bcc16afe] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:56:22.78295+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001802, result count: 47 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001802, max_request_id: 275001848 2024-11-29T09:56:22.86554+08:00 INFO [21697,0417128f8e3f46ba] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=0417128f8e3f46ba, body=] 2024-11-29T09:56:22.86574+08:00 WARN [21697,0417128f8e3f46ba] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:22.86577+08:00 WARN [21697,0417128f8e3f46ba] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:22.86579+08:00 WARN [21697,0417128f8e3f46ba] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:22.8658+08:00 WARN [21697,0417128f8e3f46ba] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:22.86581+08:00 WARN [21697,0417128f8e3f46ba] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:22.87091+08:00 INFO [21697,0417128f8e3f46ba] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=0417128f8e3f46ba, body=] 2024-11-29T09:56:23.78547+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001849, result count: 50 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001849, max_request_id: 275001898 2024-11-29T09:56:24.78718+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001899, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001899, max_request_id: 275001929 2024-11-29T09:56:25.02284+08:00 INFO [21697,0011f176025330b6] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=0011f176025330b6, body=] 2024-11-29T09:56:25.02306+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:56:25.04198+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:56:25.13645+08:00 INFO [21697,dcd70c0ac276186b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=dcd70c0ac276186b, body=] 2024-11-29T09:56:25.13687+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:25.13692+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:25.13693+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:25.13695+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:25.13696+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:25.13739+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:56:25.77722+08:00 INFO [21697,2f663e5f9e3368f3] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=2f663e5f9e3368f3, body=] 2024-11-29T09:56:25.79873+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001930, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001930, max_request_id: 275001968 2024-11-29T09:56:25.86365+08:00 INFO [21697,7840563bf3124d6d] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=7840563bf3124d6d, body=] 2024-11-29T09:56:25.86388+08:00 WARN [21697,7840563bf3124d6d] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:25.86393+08:00 WARN [21697,7840563bf3124d6d] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:25.86394+08:00 WARN [21697,7840563bf3124d6d] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:25.86396+08:00 WARN [21697,7840563bf3124d6d] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:25.86398+08:00 WARN [21697,7840563bf3124d6d] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:25.86763+08:00 INFO [21697,7840563bf3124d6d] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=7840563bf3124d6d, body=] 2024-11-29T09:56:26.80071+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275001969, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 275001969, max_request_id: 275002009 2024-11-29T09:56:27.09484+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:56:27.78254+08:00 INFO [21697,90c37c81ed9eaf24] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:56:27.78262+08:00 INFO [21697,90c37c81ed9eaf24] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:56:27.765838503 +0800 CST m=+10835352.882448043, end time:2024-11-29 09:56:27 +0800 CST, Metrics number 0 2024-11-29T09:56:27.78269+08:00 WARN [21697,90c37c81ed9eaf24] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:56:27.8032+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002010, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002010, max_request_id: 275002048 2024-11-29T09:56:28.80537+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002049, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002049, max_request_id: 275002088 2024-11-29T09:56:28.86528+08:00 INFO [21697,7920fd1e248e48d9] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=7920fd1e248e48d9, body=] 2024-11-29T09:56:28.86549+08:00 WARN [21697,7920fd1e248e48d9] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:28.86552+08:00 WARN [21697,7920fd1e248e48d9] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:28.86554+08:00 WARN [21697,7920fd1e248e48d9] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:28.86555+08:00 WARN [21697,7920fd1e248e48d9] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:28.86557+08:00 WARN [21697,7920fd1e248e48d9] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:28.86981+08:00 INFO [21697,7920fd1e248e48d9] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=7920fd1e248e48d9, body=] 2024-11-29T09:56:29.80755+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002089, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002089, max_request_id: 275002123 2024-11-29T09:56:30.04103+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:56:30.08671+08:00 INFO [21697,b6eaddbc562d433c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=b6eaddbc562d433c, body=] 2024-11-29T09:56:30.08701+08:00 INFO [21697,b6eaddbc562d433c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=b6eaddbc562d433c, body=] 2024-11-29T09:56:30.08742+08:00 WARN [21697,b6eaddbc562d433c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:30.08749+08:00 WARN [21697,b6eaddbc562d433c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:30.08751+08:00 WARN [21697,b6eaddbc562d433c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:30.08753+08:00 WARN [21697,b6eaddbc562d433c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:30.08755+08:00 WARN [21697,b6eaddbc562d433c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:30.09616+08:00 INFO [21697,b6eaddbc562d433c] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1505, err: 2024-11-29T09:56:30.10173+08:00 INFO [21697,b6eaddbc562d433c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=b6eaddbc562d433c, body=] 2024-11-29T09:56:30.11629+08:00 INFO [21697,b6eaddbc562d433c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=b6eaddbc562d433c, body=] 2024-11-29T09:56:30.11812+08:00 WARN [21697,b6eaddbc562d433c] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:56:30.11862+08:00 INFO [21697,b6eaddbc562d433c] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:56:30.11939+08:00 INFO [21697,b6eaddbc562d433c] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=b6eaddbc562d433c, body=] 2024-11-29T09:56:30.11957+08:00 WARN [21697,b6eaddbc562d433c] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:56:30.80913+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002124, result count: 44 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002124, max_request_id: 275002167 2024-11-29T09:56:31.32396+08:00 INFO [21697,dbc48005116bc647] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.98.86, ocpServerIp=, traceId=dbc48005116bc647, body=] 2024-11-29T09:56:31.32417+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:31.3242+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:31.32422+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:31.32423+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:31.32424+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:31.74551+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:56:31.81072+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002168, result count: 51 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002168, max_request_id: 275002218 2024-11-29T09:56:31.86383+08:00 INFO [21697,5b1dcbbcc60b415d] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=5b1dcbbcc60b415d, body=] 2024-11-29T09:56:31.86405+08:00 WARN [21697,5b1dcbbcc60b415d] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:31.86409+08:00 WARN [21697,5b1dcbbcc60b415d] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:31.8641+08:00 WARN [21697,5b1dcbbcc60b415d] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:31.86411+08:00 WARN [21697,5b1dcbbcc60b415d] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:31.86413+08:00 WARN [21697,5b1dcbbcc60b415d] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:31.86798+08:00 INFO [21697,5b1dcbbcc60b415d] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=5b1dcbbcc60b415d, body=] 2024-11-29T09:56:32.09497+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:56:32.7824+08:00 INFO [21697,3f4997a0cf0bf119] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:56:32.78248+08:00 INFO [21697,3f4997a0cf0bf119] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:56:32.765871853 +0800 CST m=+10835357.882481386, end time:2024-11-29 09:56:32 +0800 CST, Metrics number 0 2024-11-29T09:56:32.78258+08:00 WARN [21697,3f4997a0cf0bf119] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:56:32.81298+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002219, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002219, max_request_id: 275002254 2024-11-29T09:56:32.857+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:56:32.85838+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:56:32.85845+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:56:32.85847+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:56:32.85849+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:56:32.85852+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:56:32.85854+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:56:32.85857+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:56:32.85859+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:56:32.85862+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:56:32.85864+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:56:32.85865+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:56:32.87521+08:00 INFO [21697,61360e7debb2004b] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:56:32 +0800 CST, start_request_id:0 2024-11-29T09:56:32.87532+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:32.87543+08:00 INFO [21697,61360e7debb2004b] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:56:32 +0800 CST, start_request_id:0 2024-11-29T09:56:32.87547+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:32.87556+08:00 INFO [21697,61360e7debb2004b] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:56:32 +0800 CST, start_request_id:0 2024-11-29T09:56:32.87559+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:32.88271+08:00 INFO [21697,61360e7debb2004b] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:56:32 +0800 CST, start_request_id:544455 2024-11-29T09:56:32.88277+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544455, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:32.88283+08:00 INFO [21697,61360e7debb2004b] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:56:32 +0800 CST, start_request_id:0 2024-11-29T09:56:32.88284+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:32.88295+08:00 INFO [21697,61360e7debb2004b] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:56:32 +0800 CST, start_request_id:7154 2024-11-29T09:56:32.88296+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:32.88668+08:00 INFO [21697,61360e7debb2004b] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:56:32 +0800 CST, start_request_id:387 2024-11-29T09:56:32.88673+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:32.88683+08:00 INFO [21697,61360e7debb2004b] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:56:32 +0800 CST, start_request_id:0 2024-11-29T09:56:32.88685+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:32.89133+08:00 INFO [21697,61360e7debb2004b] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:56:32 +0800 CST, start_request_id:0 2024-11-29T09:56:32.89138+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:56:32.9088+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:56:32.90885+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:32.90888+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:56:32.90897+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:56:32.909+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:32.90908+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:56:32.9091+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:32.90917+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:56:32.90918+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:32.90924+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:56:32.90926+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:32.9093+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:56:32.90932+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:56:32.90934+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:56:32.90935+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:56:32.90949+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907757, got 4 result 2024-11-29T09:56:32.91016+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807661, got 39 result 2024-11-29T09:56:32.91077+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:56:32.9108+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:32.91082+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 43, planExplainMetrics number 0, planMetaMetrics number 43 2024-11-29T09:56:32.91192+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907757, got 4 result, planStatMetrics number 4, planMetaMetrics number 4, planExplainMetrics number 0 2024-11-29T09:56:32.91196+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 43, planExplainMetrics number 0, planMetaMetrics number 43 2024-11-29T09:56:32.92429+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807661, got 39 result, planStatMetrics number 39, planMetaMetrics number 39, planExplainMetrics number 0 2024-11-29T09:56:32.92434+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 43, planExplainMetrics number 0, planMetaMetrics number 43 2024-11-29T09:56:32.96381+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:56:32.9639+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:56:32.96392+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 43, planExplainMetrics number 0, planMetaMetrics number 43 2024-11-29T09:56:32.96397+08:00 INFO [21697,4091695304da2d1d] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:56:32 +0800 CST, end time:2024-11-29 09:56:32 +0800 CST, planMetaMetrics number 43, planExplainMetrics number 0, planStatMetrics number 43 2024-11-29T09:56:33.00382+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:56:33.00393+08:00 INFO [21697,36dc31b0f5e2a732] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 5, lost data number: 4184, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:56:33.24671+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 3 2024-11-29T09:56:33.24682+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:56:33.24693+08:00 INFO [21697,4c08d6dfa02fc4f8] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 11, lost data number: 693, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 3, sql_audit_progress Metrics number 1 2024-11-29T09:56:33.25432+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163144385, result count: 315 auditMsg number: 0, diagMsg number: 0, min_request_id: 163144574, max_request_id: 163144960 2024-11-29T09:56:33.25438+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163144574 is larger than startRequestId 163144385, maybe lost 189 data 2024-11-29T09:56:33.81377+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 51 2024-11-29T09:56:33.81384+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:56:33.81388+08:00 INFO [21697,0e10ed5fcdeb0eaa] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1309, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 51, sql_audit_progress Metrics number 1 2024-11-29T09:56:33.81701+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002255, result count: 166 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002255, max_request_id: 275002420 2024-11-29T09:56:34.4809+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:56:34.48099+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:56:34.55569+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:56:34.82105+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002421, result count: 47 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002421, max_request_id: 275002467 2024-11-29T09:56:34.86443+08:00 INFO [21697,10733612913548e2] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=10733612913548e2, body=] 2024-11-29T09:56:34.86465+08:00 WARN [21697,10733612913548e2] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:34.86468+08:00 WARN [21697,10733612913548e2] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:34.8647+08:00 WARN [21697,10733612913548e2] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:34.86471+08:00 WARN [21697,10733612913548e2] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:34.86472+08:00 WARN [21697,10733612913548e2] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:34.87203+08:00 INFO [21697,10733612913548e2] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=10733612913548e2, body=] 2024-11-29T09:56:35.04106+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:56:35.82289+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002468, result count: 23 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002468, max_request_id: 275002490 2024-11-29T09:56:36.18916+08:00 INFO [21697,1033d909a2cef1f8] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=1033d909a2cef1f8, body=] 2024-11-29T09:56:36.32901+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:56:36.32913+08:00 INFO [21697,991c43d7a358d5c4] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 5, lost data number: 3638, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:56:36.82543+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002491, result count: 68 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002491, max_request_id: 275002558 2024-11-29T09:56:37.09578+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:56:37.15511+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:56:37.15528+08:00 INFO [21697,956e6b8b1e6913d9] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:56:37.30064+08:00 INFO [21697,215e94c073e669b8] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=215e94c073e669b8, body=] 2024-11-29T09:56:37.30104+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:56:37.30168+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:56:37.79061+08:00 INFO [21697,6d4cb107ccbc7105] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:56:37.79072+08:00 INFO [21697,6d4cb107ccbc7105] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:56:37.765339249 +0800 CST m=+10835362.881948781, end time:2024-11-29 09:56:37 +0800 CST, Metrics number 0 2024-11-29T09:56:37.79086+08:00 WARN [21697,6d4cb107ccbc7105] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:56:37.82762+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002559, result count: 56 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002559, max_request_id: 275002614 2024-11-29T09:56:37.86295+08:00 INFO [21697,f9a39232bb6a4cec] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=f9a39232bb6a4cec, body=] 2024-11-29T09:56:37.86317+08:00 WARN [21697,f9a39232bb6a4cec] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:37.86321+08:00 WARN [21697,f9a39232bb6a4cec] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:37.86323+08:00 WARN [21697,f9a39232bb6a4cec] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:37.86324+08:00 WARN [21697,f9a39232bb6a4cec] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:37.86325+08:00 WARN [21697,f9a39232bb6a4cec] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:37.86713+08:00 INFO [21697,f9a39232bb6a4cec] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=f9a39232bb6a4cec, body=] 2024-11-29T09:56:38.82966+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002615, result count: 26 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002615, max_request_id: 275002640 2024-11-29T09:56:39.25855+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163144961, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163145040, max_request_id: 163145042 2024-11-29T09:56:39.25863+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163145040 is larger than startRequestId 163144961, maybe lost 79 data 2024-11-29T09:56:39.83257+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002641, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002641, max_request_id: 275002679 2024-11-29T09:56:40.01866+08:00 INFO [21697,ca76d57f3574b838] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=ca76d57f3574b838, body=] 2024-11-29T09:56:40.01889+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:56:40.04179+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:56:40.13543+08:00 INFO [21697,aa6d15d33eb5579d] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=aa6d15d33eb5579d, body=] 2024-11-29T09:56:40.13589+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:40.13594+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:40.13595+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:40.13597+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:40.13598+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:40.13642+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:56:40.77633+08:00 INFO [21697,5922a4ab06d06305] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=5922a4ab06d06305, body=] 2024-11-29T09:56:40.83443+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002680, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002680, max_request_id: 275002719 2024-11-29T09:56:40.86444+08:00 INFO [21697,06a43ca617ee4389] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=06a43ca617ee4389, body=] 2024-11-29T09:56:40.86515+08:00 INFO [21697,06a43ca617ee4389] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=06a43ca617ee4389, body=] 2024-11-29T09:56:40.8688+08:00 WARN [21697,06a43ca617ee4389] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:40.86885+08:00 WARN [21697,06a43ca617ee4389] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:40.86887+08:00 WARN [21697,06a43ca617ee4389] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:40.86889+08:00 WARN [21697,06a43ca617ee4389] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:40.8689+08:00 WARN [21697,06a43ca617ee4389] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:41.74349+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:56:41.83637+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002720, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002720, max_request_id: 275002758 2024-11-29T09:56:42.108+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:56:42.78578+08:00 INFO [21697,2bcd704e5c96aa5d] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:56:42.78583+08:00 INFO [21697,2bcd704e5c96aa5d] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:56:42.76518328 +0800 CST m=+10835367.881792813, end time:2024-11-29 09:56:42 +0800 CST, Metrics number 0 2024-11-29T09:56:42.78589+08:00 WARN [21697,2bcd704e5c96aa5d] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:56:42.83857+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002759, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002759, max_request_id: 275002795 2024-11-29T09:56:43.84103+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002796, result count: 58 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002796, max_request_id: 275002853 2024-11-29T09:56:43.86281+08:00 INFO [21697,50498088c25e443f] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=50498088c25e443f, body=] 2024-11-29T09:56:43.86304+08:00 WARN [21697,50498088c25e443f] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:43.86307+08:00 WARN [21697,50498088c25e443f] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:43.86309+08:00 WARN [21697,50498088c25e443f] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:43.8631+08:00 WARN [21697,50498088c25e443f] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:43.86311+08:00 WARN [21697,50498088c25e443f] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:43.86848+08:00 INFO [21697,50498088c25e443f] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=50498088c25e443f, body=] 2024-11-29T09:56:44.84267+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002854, result count: 29 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002854, max_request_id: 275002882 2024-11-29T09:56:45.04126+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:56:45.08625+08:00 INFO [21697,8fdc1ed9eab24d64] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=8fdc1ed9eab24d64, body=] 2024-11-29T09:56:45.08666+08:00 WARN [21697,8fdc1ed9eab24d64] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:45.08669+08:00 WARN [21697,8fdc1ed9eab24d64] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:45.08671+08:00 WARN [21697,8fdc1ed9eab24d64] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:45.08672+08:00 WARN [21697,8fdc1ed9eab24d64] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:45.08673+08:00 WARN [21697,8fdc1ed9eab24d64] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:45.08718+08:00 INFO [21697,8fdc1ed9eab24d64] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:56:45.08764+08:00 INFO [21697,8fdc1ed9eab24d64] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=8fdc1ed9eab24d64, body=] 2024-11-29T09:56:45.09133+08:00 INFO [21697,8fdc1ed9eab24d64] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=8fdc1ed9eab24d64, body=] 2024-11-29T09:56:45.09149+08:00 WARN [21697,8fdc1ed9eab24d64] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:56:45.09161+08:00 INFO [21697,8fdc1ed9eab24d64] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=8fdc1ed9eab24d64, body=] 2024-11-29T09:56:45.09175+08:00 INFO [21697,8fdc1ed9eab24d64] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=8fdc1ed9eab24d64, body=] 2024-11-29T09:56:45.09461+08:00 WARN [21697,8fdc1ed9eab24d64] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:56:45.0999+08:00 INFO [21697,8fdc1ed9eab24d64] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:56:45.84421+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002883, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002883, max_request_id: 275002917 2024-11-29T09:56:46.84706+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002918, result count: 52 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002918, max_request_id: 275002969 2024-11-29T09:56:46.86827+08:00 INFO [21697,97ce011d89d1415b] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=97ce011d89d1415b, body=] 2024-11-29T09:56:46.87847+08:00 INFO [21697,97ce011d89d1415b] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=97ce011d89d1415b, body=] 2024-11-29T09:56:46.87881+08:00 WARN [21697,97ce011d89d1415b] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:46.87885+08:00 WARN [21697,97ce011d89d1415b] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:46.87887+08:00 WARN [21697,97ce011d89d1415b] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:46.87889+08:00 WARN [21697,97ce011d89d1415b] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:46.8789+08:00 WARN [21697,97ce011d89d1415b] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:47.09469+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:56:47.78177+08:00 INFO [21697,eef2c88378fa3ffe] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:56:47.78183+08:00 INFO [21697,eef2c88378fa3ffe] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:56:47.765531492 +0800 CST m=+10835372.882141025, end time:2024-11-29 09:56:47 +0800 CST, Metrics number 0 2024-11-29T09:56:47.78191+08:00 WARN [21697,eef2c88378fa3ffe] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:56:47.84893+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275002970, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275002970, max_request_id: 275003009 2024-11-29T09:56:48.30083+08:00 INFO [21697,6877e84903d16916] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.98.86, ocpServerIp=, traceId=6877e84903d16916, body=] 2024-11-29T09:56:48.85054+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003010, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003010, max_request_id: 275003041 2024-11-29T09:56:49.4818+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:56:49.48191+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:56:49.55249+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:56:49.853+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003042, result count: 46 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003042, max_request_id: 275003087 2024-11-29T09:56:49.86271+08:00 INFO [21697,0bd5538e5e9f4dcd] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=0bd5538e5e9f4dcd, body=] 2024-11-29T09:56:49.86294+08:00 WARN [21697,0bd5538e5e9f4dcd] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:49.86298+08:00 WARN [21697,0bd5538e5e9f4dcd] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:49.863+08:00 WARN [21697,0bd5538e5e9f4dcd] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:49.86301+08:00 WARN [21697,0bd5538e5e9f4dcd] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:49.86302+08:00 WARN [21697,0bd5538e5e9f4dcd] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:49.86787+08:00 INFO [21697,0bd5538e5e9f4dcd] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=0bd5538e5e9f4dcd, body=] 2024-11-29T09:56:50.04166+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:56:50.85452+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003088, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003088, max_request_id: 275003119 2024-11-29T09:56:51.18831+08:00 INFO [21697,9bd9f7eba6b153d0] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=9bd9f7eba6b153d0, body=] 2024-11-29T09:56:51.74868+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:56:51.85749+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003120, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003120, max_request_id: 275003160 2024-11-29T09:56:52.10862+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:56:52.29977+08:00 INFO [21697,0035f995c3c98de9] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=0035f995c3c98de9, body=] 2024-11-29T09:56:52.30016+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:56:52.30056+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:56:52.37837+08:00 INFO [21697,ed85e7e5ed336283] caller=host/custom.go:178:func10: collect observer dir exists info 2024-11-29T09:56:52.78184+08:00 INFO [21697,dcb5cbe8478b85f6] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 1 result 2024-11-29T09:56:52.78191+08:00 INFO [21697,dcb5cbe8478b85f6] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:56:52.765401245 +0800 CST m=+10835377.882010778, end time:2024-11-29 09:56:52 +0800 CST, Metrics number 1 2024-11-29T09:56:52.78199+08:00 INFO [21697,dcb5cbe8478b85f6] caller=engine/pipeline.go:145:pipelinePush: push pipeline ob_sql_transaction metrics count 1 2024-11-29T09:56:52.86024+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003161, result count: 61 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003161, max_request_id: 275003221 2024-11-29T09:56:52.86405+08:00 INFO [21697,9b786d743fdf48f9] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=9b786d743fdf48f9, body=] 2024-11-29T09:56:52.86422+08:00 WARN [21697,9b786d743fdf48f9] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:52.86425+08:00 WARN [21697,9b786d743fdf48f9] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:52.86426+08:00 WARN [21697,9b786d743fdf48f9] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:52.86427+08:00 WARN [21697,9b786d743fdf48f9] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:52.86429+08:00 WARN [21697,9b786d743fdf48f9] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:52.86809+08:00 INFO [21697,9b786d743fdf48f9] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=9b786d743fdf48f9, body=] 2024-11-29T09:56:53.86308+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003222, result count: 46 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003222, max_request_id: 275003267 2024-11-29T09:56:54.86474+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003268, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003268, max_request_id: 275003302 2024-11-29T09:56:55.01834+08:00 INFO [21697,c4502136a4cd8e17] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=c4502136a4cd8e17, body=] 2024-11-29T09:56:55.01858+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:56:55.04206+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:56:55.05035+08:00 WARN [21697,] caller=common/process.go:106:refreshProcesses: failed to get process name of pid: 168679 fields: error="open /proc/168679/status: no such file or directory" 2024-11-29T09:56:55.05042+08:00 WARN [21697,] caller=common/process.go:106:refreshProcesses: failed to get process name of pid: 168680 fields: error="open /proc/168680/status: no such file or directory" 2024-11-29T09:56:55.05045+08:00 WARN [21697,] caller=common/process.go:106:refreshProcesses: failed to get process name of pid: 168681 fields:, error="open /proc/168681/status: no such file or directory" 2024-11-29T09:56:55.05047+08:00 WARN [21697,] caller=common/process.go:106:refreshProcesses: failed to get process name of pid: 168682 fields: error="open /proc/168682/status: no such file or directory" 2024-11-29T09:56:55.13565+08:00 INFO [21697,8ffc4c6c118feffe] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=8ffc4c6c118feffe, body=] 2024-11-29T09:56:55.13608+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:55.13612+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:55.13614+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:55.13615+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:55.13616+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:55.13651+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:56:55.26763+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163145043, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163145369, max_request_id: 163145373 2024-11-29T09:56:55.26769+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163145369 is larger than startRequestId 163145043, maybe lost 326 data 2024-11-29T09:56:55.77589+08:00 INFO [21697,5ce4425c5cf6583d] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=5ce4425c5cf6583d, body=] 2024-11-29T09:56:55.86257+08:00 INFO [21697,bdbc96f0edeb4241] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=bdbc96f0edeb4241, body=] 2024-11-29T09:56:55.8628+08:00 WARN [21697,bdbc96f0edeb4241] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:55.86283+08:00 WARN [21697,bdbc96f0edeb4241] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:55.86285+08:00 WARN [21697,bdbc96f0edeb4241] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:55.86287+08:00 WARN [21697,bdbc96f0edeb4241] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:55.86288+08:00 WARN [21697,bdbc96f0edeb4241] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:55.866+08:00 INFO [21697,bdbc96f0edeb4241] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=bdbc96f0edeb4241, body=] 2024-11-29T09:56:55.8879+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003303, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003303, max_request_id: 275003339 2024-11-29T09:56:56.8894+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003340, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003340, max_request_id: 275003379 2024-11-29T09:56:57.09561+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:56:57.7815+08:00 INFO [21697,361941d74c2d6fd8] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:56:57.78158+08:00 INFO [21697,361941d74c2d6fd8] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:56:57.765220605 +0800 CST m=+10835382.881830145, end time:2024-11-29 09:56:57 +0800 CST, Metrics number 0 2024-11-29T09:56:57.78168+08:00 WARN [21697,361941d74c2d6fd8] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:56:57.89161+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003380, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003380, max_request_id: 275003419 2024-11-29T09:56:58.86428+08:00 INFO [21697,5a6c013e41b542cb] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=5a6c013e41b542cb, body=] 2024-11-29T09:56:58.86451+08:00 WARN [21697,5a6c013e41b542cb] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:56:58.86455+08:00 WARN [21697,5a6c013e41b542cb] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:56:58.86457+08:00 WARN [21697,5a6c013e41b542cb] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:56:58.86458+08:00 WARN [21697,5a6c013e41b542cb] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:58.8646+08:00 WARN [21697,5a6c013e41b542cb] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:56:58.86956+08:00 INFO [21697,5a6c013e41b542cb] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=5a6c013e41b542cb, body=] 2024-11-29T09:56:58.90704+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003420, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003420, max_request_id: 275003457 2024-11-29T09:56:59.90891+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003458, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003458, max_request_id: 275003488 2024-11-29T09:57:00.0413+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:57:00.08585+08:00 INFO [21697,b286696f72b1405b] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=b286696f72b1405b, body=] 2024-11-29T09:57:00.08606+08:00 INFO [21697,b286696f72b1405b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=b286696f72b1405b, body=] 2024-11-29T09:57:00.09336+08:00 WARN [21697,b286696f72b1405b] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:00.09342+08:00 WARN [21697,b286696f72b1405b] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:00.09344+08:00 WARN [21697,b286696f72b1405b] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:00.09345+08:00 WARN [21697,b286696f72b1405b] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:00.09346+08:00 WARN [21697,b286696f72b1405b] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:00.09609+08:00 INFO [21697,b286696f72b1405b] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:57:00.10143+08:00 INFO [21697,b286696f72b1405b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=b286696f72b1405b, body=] 2024-11-29T09:57:00.10499+08:00 INFO [21697,b286696f72b1405b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=b286696f72b1405b, body=] 2024-11-29T09:57:00.10512+08:00 INFO [21697,b286696f72b1405b] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=b286696f72b1405b, body=] 2024-11-29T09:57:00.10525+08:00 WARN [21697,b286696f72b1405b] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:57:00.10667+08:00 WARN [21697,b286696f72b1405b] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:57:00.10847+08:00 INFO [21697,b286696f72b1405b] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:57:00.45908+08:00 INFO [21697,44ffea2e14ff9bd6] caller=common/middleware.go:59:func1: API request: [GET /api/v1/log/alarms, client=1.1.1.69.58, ocpServerIp=1.1.1.69.58, traceId=44ffea2e14ff9bd6, body=] 2024-11-29T09:57:00.45924+08:00 INFO [21697,] caller=alarmscache/alarm_handler.go:73:getPipelieGroupReader: get alarms, result size:0, cache list size:4 fields: uri=/api/v1/log/alarms 2024-11-29T09:57:00.9138+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003489, result count: 79 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003489, max_request_id: 275003567 2024-11-29T09:57:01.27237+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163145374, result count: 1 auditMsg number: 0, diagMsg number: 0, min_request_id: 163145586, max_request_id: 163145586 2024-11-29T09:57:01.27242+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163145586 is larger than startRequestId 163145374, maybe lost 212 data 2024-11-29T09:57:01.74786+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:57:01.86311+08:00 INFO [21697,28cf4ed6c7bd444c] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=28cf4ed6c7bd444c, body=] 2024-11-29T09:57:01.86338+08:00 WARN [21697,28cf4ed6c7bd444c] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:01.86342+08:00 WARN [21697,28cf4ed6c7bd444c] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:01.86344+08:00 WARN [21697,28cf4ed6c7bd444c] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:01.86345+08:00 WARN [21697,28cf4ed6c7bd444c] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:01.86346+08:00 WARN [21697,28cf4ed6c7bd444c] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:01.86765+08:00 INFO [21697,28cf4ed6c7bd444c] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=28cf4ed6c7bd444c, body=] 2024-11-29T09:57:01.91601+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003568, result count: 73 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003568, max_request_id: 275003640 2024-11-29T09:57:02.10809+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:57:02.74202+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:245:updateMetaInfo: successfully init sqlAudit tenant config 2024-11-29T09:57:02.78228+08:00 INFO [21697,892fe441ef0e0a12] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:57:02.78235+08:00 INFO [21697,892fe441ef0e0a12] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:57:02.765573985 +0800 CST m=+10835387.882183534, end time:2024-11-29 09:57:02 +0800 CST, Metrics number 0 2024-11-29T09:57:02.78244+08:00 WARN [21697,892fe441ef0e0a12] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:57:02.85654+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:57:02.858+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:57:02.85809+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:57:02.85812+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:57:02.85816+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:57:02.8582+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:57:02.85823+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:57:02.85826+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:57:02.85829+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:57:02.85833+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:57:02.85836+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:57:02.85838+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:57:02.86542+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:796:refreshTenants: configured tenants: map[1:sys 1002:ocp 1004:dbamonitor 1006:dbmanager 1008:datatransfer] 2024-11-29T09:57:02.8658+08:00 INFO [21697,85a1ea466717774c] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:57:02 +0800 CST, start_request_id:0 2024-11-29T09:57:02.86584+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:02.86591+08:00 INFO [21697,85a1ea466717774c] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:57:02 +0800 CST, start_request_id:0 2024-11-29T09:57:02.86594+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:02.87276+08:00 INFO [21697,85a1ea466717774c] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:57:02 +0800 CST, start_request_id:544455 2024-11-29T09:57:02.87282+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544455, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:02.87293+08:00 INFO [21697,85a1ea466717774c] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:57:02 +0800 CST, start_request_id:0 2024-11-29T09:57:02.87297+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:02.87303+08:00 INFO [21697,85a1ea466717774c] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:57:02 +0800 CST, start_request_id:7154 2024-11-29T09:57:02.87304+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:02.8766+08:00 INFO [21697,85a1ea466717774c] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:57:02 +0800 CST, start_request_id:0 2024-11-29T09:57:02.87664+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:02.87673+08:00 INFO [21697,85a1ea466717774c] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:57:02 +0800 CST, start_request_id:0 2024-11-29T09:57:02.87676+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:02.87991+08:00 INFO [21697,85a1ea466717774c] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:57:02 +0800 CST, start_request_id:387 2024-11-29T09:57:02.87996+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:02.8822+08:00 INFO [21697,85a1ea466717774c] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:57:02 +0800 CST, start_request_id:0 2024-11-29T09:57:02.88224+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:02.91424+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807709, got 48 result 2024-11-29T09:57:02.91488+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:57:02.91493+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:02.91495+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 48, planExplainMetrics number 0, planMetaMetrics number 48 2024-11-29T09:57:02.91505+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:57:02.91507+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:02.91561+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907760, got 3 result 2024-11-29T09:57:02.91586+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:57:02.91597+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:02.91851+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 51 2024-11-29T09:57:02.91856+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:57:02.91961+08:00 INFO [21697,779f49a9985818c0] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1386, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 51, sql_audit_progress Metrics number 1 2024-11-29T09:57:02.91972+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:57:02.91975+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:02.92046+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 51, planExplainMetrics number 0, planMetaMetrics number 51 2024-11-29T09:57:02.92061+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 51, planExplainMetrics number 0, planMetaMetrics number 51 2024-11-29T09:57:02.9217+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 51, planExplainMetrics number 0, planMetaMetrics number 51 2024-11-29T09:57:02.92191+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:57:02.92194+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:02.92196+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 51, planExplainMetrics number 0, planMetaMetrics number 51 2024-11-29T09:57:02.92204+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:57:02.92234+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:02.92275+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 51, planExplainMetrics number 0, planMetaMetrics number 51 2024-11-29T09:57:02.92699+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003641, result count: 84 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003641, max_request_id: 275003724 2024-11-29T09:57:02.92854+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907760, got 3 result, planStatMetrics number 3, planMetaMetrics number 3, planExplainMetrics number 0 2024-11-29T09:57:02.92859+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 51, planExplainMetrics number 0, planMetaMetrics number 51 2024-11-29T09:57:02.95822+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807709, got 48 result, planStatMetrics number 48, planMetaMetrics number 48, planExplainMetrics number 0 2024-11-29T09:57:02.95829+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 51, planExplainMetrics number 0, planMetaMetrics number 51 2024-11-29T09:57:02.97034+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:57:02.9704+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:02.97043+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 51, planExplainMetrics number 0, planMetaMetrics number 51 2024-11-29T09:57:02.97048+08:00 INFO [21697,cae9a6d1e5e8058c] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:57:02 +0800 CST, end time:2024-11-29 09:57:02 +0800 CST, planMetaMetrics number 51, planExplainMetrics number 0, planStatMetrics number 51 2024-11-29T09:57:03.05637+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:57:03.05648+08:00 INFO [21697,8f14c8c987b30b19] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:57:03.93291+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003725, result count: 153 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003725, max_request_id: 275003877 2024-11-29T09:57:04.48099+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields: coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:57:04.48108+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:57:04.55342+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:57:04.88901+08:00 INFO [21697,e4cacbb4d2a94d0d] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=e4cacbb4d2a94d0d, body=] 2024-11-29T09:57:04.88922+08:00 WARN [21697,e4cacbb4d2a94d0d] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:04.88925+08:00 WARN [21697,e4cacbb4d2a94d0d] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:04.88927+08:00 WARN [21697,e4cacbb4d2a94d0d] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:04.88929+08:00 WARN [21697,e4cacbb4d2a94d0d] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:04.8893+08:00 WARN [21697,e4cacbb4d2a94d0d] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:04.89294+08:00 INFO [21697,e4cacbb4d2a94d0d] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=e4cacbb4d2a94d0d, body=] 2024-11-29T09:57:04.93505+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003878, result count: 51 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003878, max_request_id: 275003928 2024-11-29T09:57:05.04158+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:57:05.93719+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003929, result count: 21 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003929, max_request_id: 275003949 2024-11-29T09:57:06.18739+08:00 INFO [21697,5bbb8039ecce959d] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=5bbb8039ecce959d, body=] 2024-11-29T09:57:06.36232+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:57:06.36243+08:00 INFO [21697,21cc32f9725659cc] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:57:06.94301+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275003950, result count: 73 auditMsg number: 0, diagMsg number: 0, min_request_id: 275003950, max_request_id: 275004022 2024-11-29T09:57:07.09616+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:57:07.26967+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:57:07.26978+08:00 INFO [21697,7745dace604606fc] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:57:07.27413+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 15 2024-11-29T09:57:07.27417+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:57:07.27421+08:00 INFO [21697,d8af46719b7427e7] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 322, lost data number: 806, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 15, sql_audit_progress Metrics number 1 2024-11-29T09:57:07.27586+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163145587, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163145755, max_request_id: 163145758 2024-11-29T09:57:07.27589+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163145755 is larger than startRequestId 163145587, maybe lost 168 data 2024-11-29T09:57:07.30005+08:00 INFO [21697,8404f5bea54e7b59] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=8404f5bea54e7b59, body=] 2024-11-29T09:57:07.30036+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:57:07.30068+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:57:07.79635+08:00 INFO [21697,a47c0a9f2559c2fa] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:57:07.79641+08:00 INFO [21697,a47c0a9f2559c2fa] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:57:07.765527594 +0800 CST m=+10835392.882137129, end time:2024-11-29 09:57:07 +0800 CST, Metrics number 0 2024-11-29T09:57:07.79648+08:00 WARN [21697,a47c0a9f2559c2fa] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:57:07.86252+08:00 INFO [21697,0c2cdc15a8344a87] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=0c2cdc15a8344a87, body=] 2024-11-29T09:57:07.86275+08:00 WARN [21697,0c2cdc15a8344a87] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:07.86278+08:00 WARN [21697,0c2cdc15a8344a87] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:07.8628+08:00 WARN [21697,0c2cdc15a8344a87] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:07.86281+08:00 WARN [21697,0c2cdc15a8344a87] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:07.86282+08:00 WARN [21697,0c2cdc15a8344a87] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:07.86681+08:00 INFO [21697,0c2cdc15a8344a87] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=0c2cdc15a8344a87, body=] 2024-11-29T09:57:07.94677+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004023, result count: 59 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004023, max_request_id: 275004081 2024-11-29T09:57:08.94869+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004082, result count: 21 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004082, max_request_id: 275004102 2024-11-29T09:57:09.95103+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004103, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004103, max_request_id: 275004135 2024-11-29T09:57:10.01831+08:00 INFO [21697,12bc38f5b0c6e890] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=12bc38f5b0c6e890, body=] 2024-11-29T09:57:10.01858+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:57:10.04105+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:57:10.13513+08:00 INFO [21697,eca86defe1e676f1] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=eca86defe1e676f1, body=] 2024-11-29T09:57:10.13551+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:10.13556+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:10.13559+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:10.1356+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:10.13561+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:10.13598+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:57:10.77538+08:00 INFO [21697,9d2556d808043607] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=9d2556d808043607, body=] 2024-11-29T09:57:10.8622+08:00 INFO [21697,663ef1e249aa4422] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=663ef1e249aa4422, body=] 2024-11-29T09:57:10.86242+08:00 WARN [21697,663ef1e249aa4422] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:10.86245+08:00 WARN [21697,663ef1e249aa4422] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:10.86246+08:00 WARN [21697,663ef1e249aa4422] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:10.86248+08:00 WARN [21697,663ef1e249aa4422] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:10.86249+08:00 WARN [21697,663ef1e249aa4422] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:10.86617+08:00 INFO [21697,663ef1e249aa4422] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=663ef1e249aa4422, body=] 2024-11-29T09:57:10.95285+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004136, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004136, max_request_id: 275004177 2024-11-29T09:57:11.74843+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:57:11.95526+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004178, result count: 47 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004178, max_request_id: 275004224 2024-11-29T09:57:12.10853+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:57:12.78439+08:00 INFO [21697,5e054c59c9925dbb] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:57:12.78448+08:00 INFO [21697,5e054c59c9925dbb] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:57:12.765432484 +0800 CST m=+10835397.882042024, end time:2024-11-29 09:57:12 +0800 CST, Metrics number 0 2024-11-29T09:57:12.78458+08:00 WARN [21697,5e054c59c9925dbb] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:57:12.95749+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004225, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004225, max_request_id: 275004263 2024-11-29T09:57:13.86259+08:00 INFO [21697,8b709f1492334321] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=8b709f1492334321, body=] 2024-11-29T09:57:13.86282+08:00 WARN [21697,8b709f1492334321] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:13.86286+08:00 WARN [21697,8b709f1492334321] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:13.86287+08:00 WARN [21697,8b709f1492334321] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:13.86289+08:00 WARN [21697,8b709f1492334321] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:13.8629+08:00 WARN [21697,8b709f1492334321] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:13.86692+08:00 INFO [21697,8b709f1492334321] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=8b709f1492334321, body=] 2024-11-29T09:57:13.9599+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004264, result count: 52 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004264, max_request_id: 275004315 2024-11-29T09:57:14.96226+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004316, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004316, max_request_id: 275004347 2024-11-29T09:57:15.0419+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:57:15.09022+08:00 INFO [21697,f4332dcf53014704] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=f4332dcf53014704, body=] 2024-11-29T09:57:15.09044+08:00 INFO [21697,f4332dcf53014704] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=f4332dcf53014704, body=] 2024-11-29T09:57:15.09102+08:00 INFO [21697,f4332dcf53014704] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=f4332dcf53014704, body=] 2024-11-29T09:57:15.10048+08:00 WARN [21697,f4332dcf53014704] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:15.10053+08:00 WARN [21697,f4332dcf53014704] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:15.10054+08:00 WARN [21697,f4332dcf53014704] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:15.10056+08:00 WARN [21697,f4332dcf53014704] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:15.10057+08:00 WARN [21697,f4332dcf53014704] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:15.10127+08:00 INFO [21697,f4332dcf53014704] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:57:15.1185+08:00 INFO [21697,f4332dcf53014704] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=f4332dcf53014704, body=] 2024-11-29T09:57:15.11888+08:00 WARN [21697,f4332dcf53014704] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:57:15.11923+08:00 INFO [21697,f4332dcf53014704] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 610, err: 2024-11-29T09:57:15.11974+08:00 INFO [21697,f4332dcf53014704] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=f4332dcf53014704, body=] 2024-11-29T09:57:15.11985+08:00 WARN [21697,f4332dcf53014704] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:57:15.96562+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004348, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004348, max_request_id: 275004387 2024-11-29T09:57:16.86225+08:00 INFO [21697,f5953a4101bb47b8] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=f5953a4101bb47b8, body=] 2024-11-29T09:57:16.86248+08:00 WARN [21697,f5953a4101bb47b8] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:16.86253+08:00 WARN [21697,f5953a4101bb47b8] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:16.86254+08:00 WARN [21697,f5953a4101bb47b8] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:16.86256+08:00 WARN [21697,f5953a4101bb47b8] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:16.86258+08:00 WARN [21697,f5953a4101bb47b8] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:16.86666+08:00 INFO [21697,f5953a4101bb47b8] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=f5953a4101bb47b8, body=] 2024-11-29T09:57:16.9688+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004388, result count: 60 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004388, max_request_id: 275004447 2024-11-29T09:57:17.09464+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:57:17.78421+08:00 INFO [21697,0d83345af2f0a876] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:57:17.78428+08:00 INFO [21697,0d83345af2f0a876] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:57:17.765460113 +0800 CST m=+10835402.882069645, end time:2024-11-29 09:57:17 +0800 CST, Metrics number 0 2024-11-29T09:57:17.78436+08:00 WARN [21697,0d83345af2f0a876] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:57:17.97283+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004448, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004448, max_request_id: 275004484 2024-11-29T09:57:18.97466+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004485, result count: 21 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004485, max_request_id: 275004505 2024-11-29T09:57:19.4811+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields: coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:57:19.48119+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:57:19.55201+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:57:19.86253+08:00 INFO [21697,32155226165d4a5f] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=32155226165d4a5f, body=] 2024-11-29T09:57:19.86279+08:00 WARN [21697,32155226165d4a5f] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:19.86283+08:00 WARN [21697,32155226165d4a5f] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:19.86285+08:00 WARN [21697,32155226165d4a5f] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:19.86287+08:00 WARN [21697,32155226165d4a5f] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:19.86288+08:00 WARN [21697,32155226165d4a5f] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:19.86753+08:00 INFO [21697,32155226165d4a5f] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=32155226165d4a5f, body=] 2024-11-29T09:57:19.97738+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004506, result count: 45 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004506, max_request_id: 275004550 2024-11-29T09:57:20.04101+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:57:20.97972+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004551, result count: 27 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004551, max_request_id: 275004577 2024-11-29T09:57:21.18762+08:00 INFO [21697,f246e2670e7d14b5] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=f246e2670e7d14b5, body=] 2024-11-29T09:57:21.74356+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:57:21.98245+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004578, result count: 47 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004578, max_request_id: 275004624 2024-11-29T09:57:22.11446+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:57:22.29931+08:00 INFO [21697,2f09a48dbdb6dfe0] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=2f09a48dbdb6dfe0, body=] 2024-11-29T09:57:22.29967+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:57:22.29998+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:57:22.7824+08:00 INFO [21697,16aef288eda9bc7d] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:57:22.78246+08:00 INFO [21697,16aef288eda9bc7d] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:57:22.765624522 +0800 CST m=+10835407.882234055, end time:2024-11-29 09:57:22 +0800 CST, Metrics number 0 2024-11-29T09:57:22.78253+08:00 WARN [21697,16aef288eda9bc7d] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:57:22.86344+08:00 INFO [21697,cb6e4277bc2d443f] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=cb6e4277bc2d443f, body=] 2024-11-29T09:57:22.86369+08:00 WARN [21697,cb6e4277bc2d443f] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:22.86372+08:00 WARN [21697,cb6e4277bc2d443f] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:22.86374+08:00 WARN [21697,cb6e4277bc2d443f] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:22.86376+08:00 WARN [21697,cb6e4277bc2d443f] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:22.86377+08:00 WARN [21697,cb6e4277bc2d443f] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:22.86901+08:00 INFO [21697,cb6e4277bc2d443f] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=cb6e4277bc2d443f, body=] 2024-11-29T09:57:22.98445+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004625, result count: 50 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004625, max_request_id: 275004674 2024-11-29T09:57:23.28772+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163145759, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163146087, max_request_id: 163146090 2024-11-29T09:57:23.28778+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163146087 is larger than startRequestId 163145759, maybe lost 328 data 2024-11-29T09:57:23.98705+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004675, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004675, max_request_id: 275004713 2024-11-29T09:57:24.99026+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004714, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004714, max_request_id: 275004753 2024-11-29T09:57:25.01752+08:00 INFO [21697,f5d2449c86a8b684] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=f5d2449c86a8b684, body=] 2024-11-29T09:57:25.01782+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:57:25.04147+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:57:25.13471+08:00 INFO [21697,fa3c79a36c71a8ec] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=fa3c79a36c71a8ec, body=] 2024-11-29T09:57:25.13507+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:25.13511+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:25.13512+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:25.13513+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:25.13514+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:25.13551+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:57:25.77589+08:00 INFO [21697,a140ef624ab2c649] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=a140ef624ab2c649, body=] 2024-11-29T09:57:25.86168+08:00 INFO [21697,a95e4badd4ef49be] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=a95e4badd4ef49be, body=] 2024-11-29T09:57:25.86189+08:00 WARN [21697,a95e4badd4ef49be] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:25.86193+08:00 WARN [21697,a95e4badd4ef49be] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:25.86194+08:00 WARN [21697,a95e4badd4ef49be] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:25.86195+08:00 WARN [21697,a95e4badd4ef49be] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:25.86197+08:00 WARN [21697,a95e4badd4ef49be] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:25.86603+08:00 INFO [21697,a95e4badd4ef49be] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=a95e4badd4ef49be, body=] 2024-11-29T09:57:25.99272+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004754, result count: 64 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004754, max_request_id: 275004817 2024-11-29T09:57:26.40633+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1008 from request_id 117585593, result count: 21 auditMsg number: 0, diagMsg number: 0, min_request_id: 117587157, max_request_id: 117587177 2024-11-29T09:57:26.4064+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1008 minRequestId 117587157 is larger than startRequestId 117585593, maybe lost 1564 data 2024-11-29T09:57:26.99601+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004818, result count: 46 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004818, max_request_id: 275004863 2024-11-29T09:57:27.09601+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:57:27.80242+08:00 INFO [21697,ea08f6c59328c848] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:57:27.8025+08:00 INFO [21697,ea08f6c59328c848] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:57:27.765549798 +0800 CST m=+10835412.882159334, end time:2024-11-29 09:57:27 +0800 CST, Metrics number 0 2024-11-29T09:57:27.8026+08:00 WARN [21697,ea08f6c59328c848] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:57:27.99929+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004864, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004864, max_request_id: 275004904 2024-11-29T09:57:28.12812+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1006 from request_id 136531587, result count: 21 auditMsg number: 0, diagMsg number: 0, min_request_id: 136533388, max_request_id: 136533419 2024-11-29T09:57:28.12818+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1006 minRequestId 136533388 is larger than startRequestId 136531587, maybe lost 1801 data 2024-11-29T09:57:28.87304+08:00 INFO [21697,3a2d140a7b16448a] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=3a2d140a7b16448a, body=] 2024-11-29T09:57:28.8785+08:00 INFO [21697,3a2d140a7b16448a] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=3a2d140a7b16448a, body=] 2024-11-29T09:57:28.87871+08:00 WARN [21697,3a2d140a7b16448a] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:28.87874+08:00 WARN [21697,3a2d140a7b16448a] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:28.87876+08:00 WARN [21697,3a2d140a7b16448a] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:28.87878+08:00 WARN [21697,3a2d140a7b16448a] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:28.87879+08:00 WARN [21697,3a2d140a7b16448a] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:29.00221+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004905, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004905, max_request_id: 275004936 2024-11-29T09:57:29.29412+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163146091, result count: 48 auditMsg number: 0, diagMsg number: 0, min_request_id: 163146180, max_request_id: 163146328 2024-11-29T09:57:29.29418+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163146180 is larger than startRequestId 163146091, maybe lost 89 data 2024-11-29T09:57:30.00437+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004937, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004937, max_request_id: 275004971 2024-11-29T09:57:30.04132+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:57:30.08503+08:00 INFO [21697,c57498082cae4727] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=c57498082cae4727, body=] 2024-11-29T09:57:30.08524+08:00 INFO [21697,c57498082cae4727] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=c57498082cae4727, body=] 2024-11-29T09:57:30.09393+08:00 WARN [21697,c57498082cae4727] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:30.09398+08:00 WARN [21697,c57498082cae4727] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:30.094+08:00 WARN [21697,c57498082cae4727] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:30.09401+08:00 WARN [21697,c57498082cae4727] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:30.09402+08:00 WARN [21697,c57498082cae4727] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:30.09632+08:00 INFO [21697,c57498082cae4727] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:57:30.09964+08:00 INFO [21697,c57498082cae4727] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=c57498082cae4727, body=] 2024-11-29T09:57:30.10648+08:00 INFO [21697,c57498082cae4727] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=c57498082cae4727, body=] 2024-11-29T09:57:30.10667+08:00 INFO [21697,c57498082cae4727] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=c57498082cae4727, body=] 2024-11-29T09:57:30.10683+08:00 WARN [21697,c57498082cae4727] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:57:30.11261+08:00 WARN [21697,c57498082cae4727] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:57:30.11763+08:00 INFO [21697,c57498082cae4727] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:57:30.29679+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163146329, result count: 16 auditMsg number: 0, diagMsg number: 0, min_request_id: 163146329, max_request_id: 163146349 2024-11-29T09:57:31.00677+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275004972, result count: 88 auditMsg number: 0, diagMsg number: 0, min_request_id: 275004972, max_request_id: 275005059 2024-11-29T09:57:31.32255+08:00 INFO [21697,d28c75893cff98f1] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.98.86, ocpServerIp=, traceId=d28c75893cff98f1, body=] 2024-11-29T09:57:31.32278+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:31.32282+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:31.32284+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:31.32285+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:31.32286+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:31.74407+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:57:31.86151+08:00 INFO [21697,b663792b76a84084] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=b663792b76a84084, body=] 2024-11-29T09:57:31.86173+08:00 WARN [21697,b663792b76a84084] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:31.86177+08:00 WARN [21697,b663792b76a84084] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:31.86178+08:00 WARN [21697,b663792b76a84084] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:31.8618+08:00 WARN [21697,b663792b76a84084] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:31.86181+08:00 WARN [21697,b663792b76a84084] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:31.86573+08:00 INFO [21697,b663792b76a84084] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=b663792b76a84084, body=] 2024-11-29T09:57:32.00985+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005060, result count: 139 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005060, max_request_id: 275005198 2024-11-29T09:57:32.10832+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:57:32.78264+08:00 INFO [21697,c4eee7d14acd8ce8] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 2 result 2024-11-29T09:57:32.78272+08:00 INFO [21697,c4eee7d14acd8ce8] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:57:32.765317664 +0800 CST m=+10835417.881927196, end time:2024-11-29 09:57:32 +0800 CST, Metrics number 2 2024-11-29T09:57:32.78282+08:00 INFO [21697,c4eee7d14acd8ce8] caller=engine/pipeline.go:145:pipelinePush: push pipeline ob_sql_transaction metrics count 2 2024-11-29T09:57:32.85684+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:57:32.85836+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:57:32.85844+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:57:32.85846+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:57:32.85849+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:57:32.85851+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:57:32.85854+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:57:32.85859+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:57:32.85862+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:57:32.85865+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:57:32.85867+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:57:32.85869+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:57:32.86957+08:00 INFO [21697,37264ea9bd0f1cca] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:57:32 +0800 CST, start_request_id:0 2024-11-29T09:57:32.86968+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:32.8698+08:00 INFO [21697,37264ea9bd0f1cca] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:57:32 +0800 CST, start_request_id:0 2024-11-29T09:57:32.86982+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:32.86991+08:00 INFO [21697,37264ea9bd0f1cca] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:57:32 +0800 CST, start_request_id:0 2024-11-29T09:57:32.86993+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:32.87641+08:00 INFO [21697,37264ea9bd0f1cca] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:57:32 +0800 CST, start_request_id:544455 2024-11-29T09:57:32.87647+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544455, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:32.87654+08:00 INFO [21697,37264ea9bd0f1cca] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:57:32 +0800 CST, start_request_id:0 2024-11-29T09:57:32.87657+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:32.87662+08:00 INFO [21697,37264ea9bd0f1cca] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:57:32 +0800 CST, start_request_id:7154 2024-11-29T09:57:32.87664+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:32.88137+08:00 INFO [21697,37264ea9bd0f1cca] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:57:32 +0800 CST, start_request_id:0 2024-11-29T09:57:32.88142+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:32.88153+08:00 INFO [21697,37264ea9bd0f1cca] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:57:32 +0800 CST, start_request_id:387 2024-11-29T09:57:32.88157+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:32.88845+08:00 INFO [21697,37264ea9bd0f1cca] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:57:32 +0800 CST, start_request_id:0 2024-11-29T09:57:32.88852+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:57:32.90496+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:57:32.90503+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:32.90506+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:57:32.90517+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907760, got 0 result 2024-11-29T09:57:32.90519+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907760, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:32.90523+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:57:32.9056+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807714, got 5 result 2024-11-29T09:57:32.90571+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:57:32.90574+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:32.90575+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:57:32.90583+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:57:32.90586+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:32.90594+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:57:32.90596+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:32.90604+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:57:32.90606+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:32.9061+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:57:32.90613+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:57:32.90614+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:57:32.90813+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807714, got 5 result, planStatMetrics number 5, planMetaMetrics number 5, planExplainMetrics number 0 2024-11-29T09:57:32.90817+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:57:32.91488+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:57:32.91492+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:32.91495+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:57:32.95983+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:57:32.9599+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:57:32.95993+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 5, planExplainMetrics number 0, planMetaMetrics number 5 2024-11-29T09:57:32.95999+08:00 INFO [21697,2dea5020bc2f2a3a] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:57:32 +0800 CST, end time:2024-11-29 09:57:32 +0800 CST, planMetaMetrics number 5, planExplainMetrics number 0, planStatMetrics number 5 2024-11-29T09:57:33.0106+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 50 2024-11-29T09:57:33.01067+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:57:33.01071+08:00 INFO [21697,7af36c9738c32ced] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1558, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 50, sql_audit_progress Metrics number 1 2024-11-29T09:57:33.01288+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005199, result count: 75 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005199, max_request_id: 275005273 2024-11-29T09:57:34.02035+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005274, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005274, max_request_id: 275005312 2024-11-29T09:57:34.13583+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1006 topSqlMetrics length: 7 2024-11-29T09:57:34.13594+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:57:34.13599+08:00 INFO [21697,346ec715b7406617] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 21, lost data number: 1801, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 7, sql_audit_progress Metrics number 1 2024-11-29T09:57:34.48163+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields: coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:57:34.48173+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:57:34.5522+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields: error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:57:34.86304+08:00 INFO [21697,d78c9fbe0db94a26] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=d78c9fbe0db94a26, body=] 2024-11-29T09:57:34.86329+08:00 WARN [21697,d78c9fbe0db94a26] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:34.86333+08:00 WARN [21697,d78c9fbe0db94a26] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:34.86334+08:00 WARN [21697,d78c9fbe0db94a26] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:34.86336+08:00 WARN [21697,d78c9fbe0db94a26] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:34.86337+08:00 WARN [21697,d78c9fbe0db94a26] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:34.86816+08:00 INFO [21697,d78c9fbe0db94a26] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=d78c9fbe0db94a26, body=] 2024-11-29T09:57:35.02294+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005313, result count: 50 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005313, max_request_id: 275005362 2024-11-29T09:57:35.04101+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:57:35.04753+08:00 WARN [21697,] caller=common/process.go:106:refreshProcesses: failed to get process name of pid: 169424 fields: error="open /proc/169424/status: no such file or directory" 2024-11-29T09:57:35.04759+08:00 WARN [21697,] caller=common/process.go:106:refreshProcesses: failed to get process name of pid: 169425 fields: error="open /proc/169425/status: no such file or directory" 2024-11-29T09:57:35.04762+08:00 WARN [21697,] caller=common/process.go:106:refreshProcesses: failed to get process name of pid: 169426 fields:, error="open /proc/169426/status: no such file or directory" 2024-11-29T09:57:35.04765+08:00 WARN [21697,] caller=common/process.go:106:refreshProcesses: failed to get process name of pid: 169427 fields:, error="open /proc/169427/status: no such file or directory" 2024-11-29T09:57:36.0246+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005363, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005363, max_request_id: 275005393 2024-11-29T09:57:36.18662+08:00 INFO [21697,b610f05f54d7d143] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=b610f05f54d7d143, body=] 2024-11-29T09:57:36.29894+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 18 2024-11-29T09:57:36.29901+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:57:36.29905+08:00 INFO [21697,a34e2cb756a2d1d8] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 70, lost data number: 585, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 18, sql_audit_progress Metrics number 1 2024-11-29T09:57:36.31007+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163146350, result count: 265 auditMsg number: 0, diagMsg number: 0, min_request_id: 163146395, max_request_id: 163146752 2024-11-29T09:57:36.31047+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163146395 is larger than startRequestId 163146350, maybe lost 45 data 2024-11-29T09:57:37.02745+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005394, result count: 72 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005394, max_request_id: 275005465 2024-11-29T09:57:37.095+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:57:37.2988+08:00 INFO [21697,3d3e6cf33010af8f] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=3d3e6cf33010af8f, body=] 2024-11-29T09:57:37.29915+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:57:37.29949+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:57:37.41231+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1008 topSqlMetrics length: 7 2024-11-29T09:57:37.41238+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:57:37.41242+08:00 INFO [21697,4fcb66b35835932b] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 21, lost data number: 1564, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 7, sql_audit_progress Metrics number 1 2024-11-29T09:57:37.43078+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:57:37.43089+08:00 INFO [21697,5ba743e54cf0c31f] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:57:37.78159+08:00 INFO [21697,1713b00c8bb7498f] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:57:37.78165+08:00 INFO [21697,1713b00c8bb7498f] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:57:37.765269017 +0800 CST m=+10835422.881878600, end time:2024-11-29 09:57:37 +0800 CST, Metrics number 0 2024-11-29T09:57:37.78173+08:00 WARN [21697,1713b00c8bb7498f] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:57:37.86119+08:00 INFO [21697,0fd53dddbe5641d8] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=0fd53dddbe5641d8, body=] 2024-11-29T09:57:37.86143+08:00 WARN [21697,0fd53dddbe5641d8] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:37.86147+08:00 WARN [21697,0fd53dddbe5641d8] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:37.86149+08:00 WARN [21697,0fd53dddbe5641d8] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:37.8615+08:00 WARN [21697,0fd53dddbe5641d8] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:37.86151+08:00 WARN [21697,0fd53dddbe5641d8] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:37.86658+08:00 INFO [21697,0fd53dddbe5641d8] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=0fd53dddbe5641d8, body=] 2024-11-29T09:57:38.02965+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005466, result count: 56 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005466, max_request_id: 275005521 2024-11-29T09:57:39.031+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005522, result count: 17 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005522, max_request_id: 275005538 2024-11-29T09:57:40.01762+08:00 INFO [21697,5145a4cd467bcb42] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=5145a4cd467bcb42, body=] 2024-11-29T09:57:40.01786+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:57:40.0327+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005539, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005539, max_request_id: 275005574 2024-11-29T09:57:40.04202+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:57:40.13493+08:00 INFO [21697,a189d5c3a1441f63] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=a189d5c3a1441f63, body=] 2024-11-29T09:57:40.13541+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:40.13546+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:40.13549+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:40.13551+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:40.13553+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:40.13611+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:57:40.77542+08:00 INFO [21697,a8ba92aab2da06d9] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=a8ba92aab2da06d9, body=] 2024-11-29T09:57:40.86141+08:00 INFO [21697,4f6d8e8226df40ce] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=4f6d8e8226df40ce, body=] 2024-11-29T09:57:40.86165+08:00 WARN [21697,4f6d8e8226df40ce] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:40.86169+08:00 WARN [21697,4f6d8e8226df40ce] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:40.8617+08:00 WARN [21697,4f6d8e8226df40ce] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:40.86171+08:00 WARN [21697,4f6d8e8226df40ce] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:40.86173+08:00 WARN [21697,4f6d8e8226df40ce] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:40.86593+08:00 INFO [21697,4f6d8e8226df40ce] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=4f6d8e8226df40ce, body=] 2024-11-29T09:57:41.03479+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005575, result count: 46 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005575, max_request_id: 275005620 2024-11-29T09:57:41.74935+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields: coredump-path= 2024-11-29T09:57:42.03731+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005621, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005621, max_request_id: 275005659 2024-11-29T09:57:42.10868+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:57:42.78275+08:00 INFO [21697,2be2394cc4c5c888] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 6 result 2024-11-29T09:57:42.78285+08:00 INFO [21697,2be2394cc4c5c888] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:57:42.765964382 +0800 CST m=+10835427.882573941, end time:2024-11-29 09:57:42 +0800 CST, Metrics number 6 2024-11-29T09:57:42.78297+08:00 INFO [21697,2be2394cc4c5c888] caller=engine/pipeline.go:145:pipelinePush: push pipeline ob_sql_transaction metrics count 6 2024-11-29T09:57:43.04036+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005660, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005660, max_request_id: 275005697 2024-11-29T09:57:43.86124+08:00 INFO [21697,fc99c8a12fd44d4a] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=fc99c8a12fd44d4a, body=] 2024-11-29T09:57:43.86176+08:00 WARN [21697,fc99c8a12fd44d4a] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:43.86183+08:00 WARN [21697,fc99c8a12fd44d4a] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:43.86187+08:00 WARN [21697,fc99c8a12fd44d4a] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:43.8619+08:00 WARN [21697,fc99c8a12fd44d4a] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:43.86192+08:00 WARN [21697,fc99c8a12fd44d4a] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:43.86664+08:00 INFO [21697,fc99c8a12fd44d4a] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=fc99c8a12fd44d4a, body=] 2024-11-29T09:57:44.04304+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005698, result count: 54 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005698, max_request_id: 275005751 2024-11-29T09:57:45.04169+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:57:45.0497+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005752, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005752, max_request_id: 275005783 2024-11-29T09:57:45.08466+08:00 INFO [21697,b810f6f06e494eac] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=b810f6f06e494eac, body=] 2024-11-29T09:57:45.08509+08:00 WARN [21697,b810f6f06e494eac] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:45.08513+08:00 WARN [21697,b810f6f06e494eac] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:45.08515+08:00 WARN [21697,b810f6f06e494eac] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:45.08516+08:00 WARN [21697,b810f6f06e494eac] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:45.08518+08:00 WARN [21697,b810f6f06e494eac] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:45.08558+08:00 INFO [21697,b810f6f06e494eac] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1506, err: 2024-11-29T09:57:45.08601+08:00 INFO [21697,b810f6f06e494eac] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=b810f6f06e494eac, body=] 2024-11-29T09:57:45.10246+08:00 INFO [21697,b810f6f06e494eac] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=b810f6f06e494eac, body=] 2024-11-29T09:57:45.11439+08:00 INFO [21697,b810f6f06e494eac] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=b810f6f06e494eac, body=] 2024-11-29T09:57:45.11458+08:00 INFO [21697,b810f6f06e494eac] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=b810f6f06e494eac, body=] 2024-11-29T09:57:45.11475+08:00 WARN [21697,b810f6f06e494eac] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:57:45.11558+08:00 WARN [21697,b810f6f06e494eac] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:57:45.12222+08:00 INFO [21697,b810f6f06e494eac] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 610, err: 2024-11-29T09:57:46.0522+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005784, result count: 50 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005784, max_request_id: 275005833 2024-11-29T09:57:46.8623+08:00 INFO [21697,920faa35784440a3] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=920faa35784440a3, body=] 2024-11-29T09:57:46.86255+08:00 WARN [21697,920faa35784440a3] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:46.8626+08:00 WARN [21697,920faa35784440a3] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:46.86261+08:00 WARN [21697,920faa35784440a3] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:46.86263+08:00 WARN [21697,920faa35784440a3] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:46.86264+08:00 WARN [21697,920faa35784440a3] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:46.86691+08:00 INFO [21697,920faa35784440a3] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=920faa35784440a3, body=] 2024-11-29T09:57:47.05509+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005834, result count: 51 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005834, max_request_id: 275005884 2024-11-29T09:57:47.09587+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:57:47.7872+08:00 INFO [21697,1e8f40437ad76330] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:57:47.78728+08:00 INFO [21697,1e8f40437ad76330] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:57:47.765301163 +0800 CST m=+10835432.881910732, end time:2024-11-29 09:57:47 +0800 CST, Metrics number 0 2024-11-29T09:57:47.78739+08:00 WARN [21697,1e8f40437ad76330] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:57:48.05744+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005885, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005885, max_request_id: 275005924 2024-11-29T09:57:48.29936+08:00 INFO [21697,756a2285ece71ed0] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.98.86, ocpServerIp=, traceId=756a2285ece71ed0, body=] 2024-11-29T09:57:49.05929+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005925, result count: 24 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005925, max_request_id: 275005948 2024-11-29T09:57:49.48308+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:57:49.48321+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:57:49.55403+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:57:49.86115+08:00 INFO [21697,cb6e530232714484] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=cb6e530232714484, body=] 2024-11-29T09:57:49.86138+08:00 WARN [21697,cb6e530232714484] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:49.86143+08:00 WARN [21697,cb6e530232714484] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:49.86145+08:00 WARN [21697,cb6e530232714484] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:49.86146+08:00 WARN [21697,cb6e530232714484] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:49.86147+08:00 WARN [21697,cb6e530232714484] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:49.86557+08:00 INFO [21697,cb6e530232714484] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=cb6e530232714484, body=] 2024-11-29T09:57:50.04153+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:57:50.06259+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005949, result count: 47 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005949, max_request_id: 275005995 2024-11-29T09:57:51.06474+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275005996, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 275005996, max_request_id: 275006031 2024-11-29T09:57:51.18683+08:00 INFO [21697,da3cbd105d7cbfc7] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=da3cbd105d7cbfc7, body=] 2024-11-29T09:57:51.74407+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:57:52.07257+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006032, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006032, max_request_id: 275006073 2024-11-29T09:57:52.10875+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:57:52.29889+08:00 INFO [21697,f582b3b18b0317e3] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=f582b3b18b0317e3, body=] 2024-11-29T09:57:52.2992+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:57:52.29952+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:57:52.32306+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163146753, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163147084, max_request_id: 163147086 2024-11-29T09:57:52.32311+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163147084 is larger than startRequestId 163146753, maybe lost 331 data 2024-11-29T09:57:52.37818+08:00 INFO [21697,ed85e7e5ed336283] caller=host/custom.go:178:func10: collect observer dir exists info 2024-11-29T09:57:52.78421+08:00 INFO [21697,ba25faebb8df0766] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:57:52.78429+08:00 INFO [21697,ba25faebb8df0766] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:57:52.765590374 +0800 CST m=+10835437.882199907, end time:2024-11-29 09:57:52 +0800 CST, Metrics number 0 2024-11-29T09:57:52.78437+08:00 WARN [21697,ba25faebb8df0766] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:57:52.86259+08:00 INFO [21697,3afa6ea49ea849de] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=3afa6ea49ea849de, body=] 2024-11-29T09:57:52.86284+08:00 WARN [21697,3afa6ea49ea849de] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:52.86288+08:00 WARN [21697,3afa6ea49ea849de] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:52.8629+08:00 WARN [21697,3afa6ea49ea849de] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:52.86291+08:00 WARN [21697,3afa6ea49ea849de] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:52.86293+08:00 WARN [21697,3afa6ea49ea849de] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:52.86813+08:00 INFO [21697,3afa6ea49ea849de] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=3afa6ea49ea849de, body=] 2024-11-29T09:57:53.07481+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006074, result count: 54 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006074, max_request_id: 275006127 2024-11-29T09:57:54.07683+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006128, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006128, max_request_id: 275006167 2024-11-29T09:57:55.01714+08:00 INFO [21697,6e3668909a1a9b3c] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=6e3668909a1a9b3c, body=] 2024-11-29T09:57:55.01737+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:57:55.04187+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:57:55.07863+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006168, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006168, max_request_id: 275006199 2024-11-29T09:57:55.13311+08:00 INFO [21697,436673016fbeb8f8] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=436673016fbeb8f8, body=] 2024-11-29T09:57:55.13346+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:55.13349+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:55.13351+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:55.13352+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:55.13353+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:55.13389+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:57:55.77507+08:00 INFO [21697,61ad4d09dfc0830a] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=61ad4d09dfc0830a, body=] 2024-11-29T09:57:55.86081+08:00 INFO [21697,961bff9e97744fdf] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=961bff9e97744fdf, body=] 2024-11-29T09:57:55.86103+08:00 WARN [21697,961bff9e97744fdf] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:55.86107+08:00 WARN [21697,961bff9e97744fdf] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:55.86109+08:00 WARN [21697,961bff9e97744fdf] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:55.8611+08:00 WARN [21697,961bff9e97744fdf] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:55.86111+08:00 WARN [21697,961bff9e97744fdf] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:55.86541+08:00 INFO [21697,961bff9e97744fdf] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=961bff9e97744fdf, body=] 2024-11-29T09:57:56.08104+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006200, result count: 47 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006200, max_request_id: 275006246 2024-11-29T09:57:57.08342+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006247, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006247, max_request_id: 275006281 2024-11-29T09:57:57.09539+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:57:57.78333+08:00 INFO [21697,da0dfd26d4886cf6] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:57:57.78342+08:00 INFO [21697,da0dfd26d4886cf6] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:57:57.765793596 +0800 CST m=+10835442.882403154, end time:2024-11-29 09:57:57 +0800 CST, Metrics number 0 2024-11-29T09:57:57.78357+08:00 WARN [21697,da0dfd26d4886cf6] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:57:58.08524+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006282, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006282, max_request_id: 275006323 2024-11-29T09:57:58.86238+08:00 INFO [21697,89c0b5b2849d4098] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=89c0b5b2849d4098, body=] 2024-11-29T09:57:58.86264+08:00 WARN [21697,89c0b5b2849d4098] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:57:58.86268+08:00 WARN [21697,89c0b5b2849d4098] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:57:58.86269+08:00 WARN [21697,89c0b5b2849d4098] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:57:58.86271+08:00 WARN [21697,89c0b5b2849d4098] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:58.86272+08:00 WARN [21697,89c0b5b2849d4098] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:57:58.86781+08:00 INFO [21697,89c0b5b2849d4098] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=89c0b5b2849d4098, body=] 2024-11-29T09:57:59.08773+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006324, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006324, max_request_id: 275006355 2024-11-29T09:58:00.04101+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:58:00.08418+08:00 INFO [21697,e04f387c3b214677] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=e04f387c3b214677, body=] 2024-11-29T09:58:00.08456+08:00 WARN [21697,e04f387c3b214677] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:00.08459+08:00 WARN [21697,e04f387c3b214677] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:00.08461+08:00 WARN [21697,e04f387c3b214677] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:00.08462+08:00 WARN [21697,e04f387c3b214677] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:00.08463+08:00 WARN [21697,e04f387c3b214677] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:00.08502+08:00 INFO [21697,e04f387c3b214677] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:58:00.08539+08:00 INFO [21697,e04f387c3b214677] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=e04f387c3b214677, body=] 2024-11-29T09:58:00.08875+08:00 INFO [21697,e04f387c3b214677] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=e04f387c3b214677, body=] 2024-11-29T09:58:00.08887+08:00 INFO [21697,e04f387c3b214677] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=e04f387c3b214677, body=] 2024-11-29T09:58:00.08901+08:00 WARN [21697,e04f387c3b214677] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:58:00.08911+08:00 INFO [21697,e04f387c3b214677] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=e04f387c3b214677, body=] 2024-11-29T09:58:00.08942+08:00 WARN [21697,e04f387c3b214677] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:58:00.09976+08:00 INFO [21697,e04f387c3b214677] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:58:00.10702+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006356, result count: 34 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006356, max_request_id: 275006389 2024-11-29T09:58:00.83252+08:00 INFO [21697,ead9b1ea66821a86] caller=common/middleware.go:59:func1: API request: [GET /api/v1/log/alarms, client=1.1.1.69.58, ocpServerIp=1.1.1.69.58, traceId=ead9b1ea66821a86, body=] 2024-11-29T09:58:00.83287+08:00 INFO [21697,] caller=alarmscache/alarm_handler.go:73:getPipelieGroupReader: get alarms, result size:0, cache list size:4 fields: uri=/api/v1/log/alarms 2024-11-29T09:58:01.10917+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006390, result count: 102 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006390, max_request_id: 275006491 2024-11-29T09:58:01.74822+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:58:01.86388+08:00 INFO [21697,0ec259614e0b44f7] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=0ec259614e0b44f7, body=] 2024-11-29T09:58:01.86412+08:00 WARN [21697,0ec259614e0b44f7] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:01.86415+08:00 WARN [21697,0ec259614e0b44f7] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:01.86417+08:00 WARN [21697,0ec259614e0b44f7] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:01.86418+08:00 WARN [21697,0ec259614e0b44f7] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:01.86419+08:00 WARN [21697,0ec259614e0b44f7] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:01.86818+08:00 INFO [21697,0ec259614e0b44f7] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=0ec259614e0b44f7, body=] 2024-11-29T09:58:02.10874+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:58:02.11226+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006492, result count: 89 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006492, max_request_id: 275006580 2024-11-29T09:58:02.45178+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1008 from request_id 117587178, result count: 22 auditMsg number: 0, diagMsg number: 0, min_request_id: 117587890, max_request_id: 117587915 2024-11-29T09:58:02.45184+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1008 minRequestId 117587890 is larger than startRequestId 117587178, maybe lost 712 data 2024-11-29T09:58:02.74196+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:245:updateMetaInfo: successfully init sqlAudit tenant config 2024-11-29T09:58:02.78207+08:00 INFO [21697,065c11a973b42fdd] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:58:02.78217+08:00 INFO [21697,065c11a973b42fdd] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:58:02.765401864 +0800 CST m=+10835447.882011403, end time:2024-11-29 09:58:02 +0800 CST, Metrics number 0 2024-11-29T09:58:02.78226+08:00 WARN [21697,065c11a973b42fdd] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:58:02.86192+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:58:02.86341+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:58:02.86347+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:58:02.86349+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:58:02.86351+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:58:02.86353+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:58:02.86356+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:58:02.8636+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:58:02.86363+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:58:02.86366+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:58:02.8637+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:58:02.86372+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:58:02.86646+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:796:refreshTenants: configured tenants: map[1:sys 1002:ocp 1004:dbamonitor 1006:dbmanager 1008:datatransfer] 2024-11-29T09:58:02.87663+08:00 INFO [21697,c799df773bbbb169] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:58:02 +0800 CST, start_request_id:0 2024-11-29T09:58:02.87668+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:02.87684+08:00 INFO [21697,c799df773bbbb169] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:58:02 +0800 CST, start_request_id:0 2024-11-29T09:58:02.87687+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:02.87696+08:00 INFO [21697,c799df773bbbb169] caller=oceanbase/sql_plan_monitor.go:385:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1, collect time:2024-11-29 09:58:02 +0800 CST, start_request_id:544455, min_request_id:544456, max_request_id:544466 2024-11-29T09:58:02.87721+08:00 INFO [21697,c799df773bbbb169] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:58:02 +0800 CST, start_request_id:0 2024-11-29T09:58:02.87723+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:02.87974+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544455, min_request_id: 544456, max_request_id: 544466, collected metrics number: 11 2024-11-29T09:58:02.88632+08:00 INFO [21697,c799df773bbbb169] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:58:02 +0800 CST, start_request_id:0 2024-11-29T09:58:02.88636+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:02.88645+08:00 INFO [21697,c799df773bbbb169] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:58:02 +0800 CST, start_request_id:387 2024-11-29T09:58:02.88646+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:02.88652+08:00 INFO [21697,c799df773bbbb169] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:58:02 +0800 CST, start_request_id:7154 2024-11-29T09:58:02.88654+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:02.88663+08:00 INFO [21697,c799df773bbbb169] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 2 complete, collect time:2024-11-29 09:58:02 +0800 CST, start_request_id:544466 2024-11-29T09:58:02.88664+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 2, start_request_id: 544466, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:02.89182+08:00 INFO [21697,c799df773bbbb169] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:58:02 +0800 CST, start_request_id:0 2024-11-29T09:58:02.89186+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:02.89387+08:00 INFO [21697,c799df773bbbb169] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:58:02 +0800 CST, start_request_id:0 2024-11-29T09:58:02.89391+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:02.92737+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:58:02.92743+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:02.92746+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:58:02.92755+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907760, got 0 result 2024-11-29T09:58:02.92758+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907760, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:02.92764+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:58:02.92766+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:02.92771+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:58:02.92773+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:58:02.9278+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:58:02.92782+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:02.92784+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:58:02.92789+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:58:02.92791+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:02.92802+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807715, got 1 result 2024-11-29T09:58:02.9281+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:58:02.92811+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:02.92817+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:58:02.9282+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:02.92823+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:58:02.92825+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:58:02.92827+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:58:02.92878+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807715, got 1 result, planStatMetrics number 1, planMetaMetrics number 1, planExplainMetrics number 0 2024-11-29T09:58:02.9288+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:58:02.98376+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:58:02.98383+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:02.98386+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 1, planExplainMetrics number 0, planMetaMetrics number 1 2024-11-29T09:58:02.98391+08:00 INFO [21697,fdecb50c475f6509] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:58:02 +0800 CST, end time:2024-11-29 09:58:02 +0800 CST, planMetaMetrics number 1, planExplainMetrics number 0, planStatMetrics number 1 2024-11-29T09:58:03.11347+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 51 2024-11-29T09:58:03.11354+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:58:03.1136+08:00 INFO [21697,62e7b18b2d82147f] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1382, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 51, sql_audit_progress Metrics number 1 2024-11-29T09:58:03.1159+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006581, result count: 83 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006581, max_request_id: 275006663 2024-11-29T09:58:03.32721+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 5 2024-11-29T09:58:03.32727+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:58:03.3273+08:00 INFO [21697,72c44f904b2e1d7a] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 268, lost data number: 376, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 5, sql_audit_progress Metrics number 1 2024-11-29T09:58:03.33024+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163147087, result count: 21 auditMsg number: 0, diagMsg number: 0, min_request_id: 163147342, max_request_id: 163147372 2024-11-29T09:58:03.33028+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163147342 is larger than startRequestId 163147087, maybe lost 255 data 2024-11-29T09:58:03.45248+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1008 topSqlMetrics length: 7 2024-11-29T09:58:03.45255+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:58:03.45261+08:00 INFO [21697,983aa45f9e0b8b8b] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 22, lost data number: 712, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 7, sql_audit_progress Metrics number 1 2024-11-29T09:58:04.11891+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006664, result count: 60 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006664, max_request_id: 275006723 2024-11-29T09:58:04.15827+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:58:04.15838+08:00 INFO [21697,736bfcae69742233] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:58:04.16325+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1006 from request_id 136533420, result count: 21 auditMsg number: 0, diagMsg number: 0, min_request_id: 136534240, max_request_id: 136534270 2024-11-29T09:58:04.16329+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1006 minRequestId 136534240 is larger than startRequestId 136533420, maybe lost 820 data 2024-11-29T09:58:04.48134+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields: coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:58:04.48143+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:58:04.553+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:58:04.86897+08:00 INFO [21697,c7d2648ab4cc43fd] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=c7d2648ab4cc43fd, body=] 2024-11-29T09:58:04.8707+08:00 INFO [21697,c7d2648ab4cc43fd] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=c7d2648ab4cc43fd, body=] 2024-11-29T09:58:04.87087+08:00 WARN [21697,c7d2648ab4cc43fd] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:04.87092+08:00 WARN [21697,c7d2648ab4cc43fd] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:04.87093+08:00 WARN [21697,c7d2648ab4cc43fd] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:04.87096+08:00 WARN [21697,c7d2648ab4cc43fd] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:04.87098+08:00 WARN [21697,c7d2648ab4cc43fd] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:05.0413+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:58:05.12171+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006724, result count: 50 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006724, max_request_id: 275006773 2024-11-29T09:58:06.12442+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006774, result count: 34 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006774, max_request_id: 275006807 2024-11-29T09:58:06.1865+08:00 INFO [21697,cf4c5a1a2c085520] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=cf4c5a1a2c085520, body=] 2024-11-29T09:58:07.09526+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:58:07.12661+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006808, result count: 63 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006808, max_request_id: 275006870 2024-11-29T09:58:07.29863+08:00 INFO [21697,9d2c660b5a0f356f] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=9d2c660b5a0f356f, body=] 2024-11-29T09:58:07.29941+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:58:07.3001+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:58:07.56839+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:58:07.56851+08:00 INFO [21697,46e5f1c93f58b340] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:58:07.78403+08:00 INFO [21697,7fdc0b605160d24d] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:58:07.78409+08:00 INFO [21697,7fdc0b605160d24d] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:58:07.765105532 +0800 CST m=+10835452.881715064, end time:2024-11-29 09:58:07 +0800 CST, Metrics number 0 2024-11-29T09:58:07.78417+08:00 WARN [21697,7fdc0b605160d24d] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:58:07.86115+08:00 INFO [21697,b3c16c3f709a4630] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=b3c16c3f709a4630, body=] 2024-11-29T09:58:07.86142+08:00 WARN [21697,b3c16c3f709a4630] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:07.86146+08:00 WARN [21697,b3c16c3f709a4630] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:07.86147+08:00 WARN [21697,b3c16c3f709a4630] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:07.86148+08:00 WARN [21697,b3c16c3f709a4630] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:07.86151+08:00 WARN [21697,b3c16c3f709a4630] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:07.86537+08:00 INFO [21697,b3c16c3f709a4630] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=b3c16c3f709a4630, body=] 2024-11-29T09:58:08.12842+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006871, result count: 53 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006871, max_request_id: 275006923 2024-11-29T09:58:09.13064+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006924, result count: 20 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006924, max_request_id: 275006943 2024-11-29T09:58:09.33562+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163147373, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163147491, max_request_id: 163147493 2024-11-29T09:58:09.33568+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163147491 is larger than startRequestId 163147373, maybe lost 118 data 2024-11-29T09:58:10.01659+08:00 INFO [21697,f430ceba20b9a3e7] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=f430ceba20b9a3e7, body=] 2024-11-29T09:58:10.01683+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:58:10.04138+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:58:10.13232+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006944, result count: 41 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006944, max_request_id: 275006984 2024-11-29T09:58:10.13267+08:00 INFO [21697,49c703114821d0a7] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=49c703114821d0a7, body=] 2024-11-29T09:58:10.13302+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:10.13306+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:10.13308+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:10.13309+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:10.1331+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:10.13344+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:58:10.77482+08:00 INFO [21697,3362affd3b2201b2] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=3362affd3b2201b2, body=] 2024-11-29T09:58:10.87223+08:00 INFO [21697,b8ebf3462a764ea1] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=b8ebf3462a764ea1, body=] 2024-11-29T09:58:10.87252+08:00 WARN [21697,b8ebf3462a764ea1] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:10.87257+08:00 WARN [21697,b8ebf3462a764ea1] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:10.87259+08:00 WARN [21697,b8ebf3462a764ea1] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:10.8726+08:00 WARN [21697,b8ebf3462a764ea1] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:10.87261+08:00 WARN [21697,b8ebf3462a764ea1] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:10.8764+08:00 INFO [21697,b8ebf3462a764ea1] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=b8ebf3462a764ea1, body=] 2024-11-29T09:58:11.13477+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275006985, result count: 50 auditMsg number: 0, diagMsg number: 0, min_request_id: 275006985, max_request_id: 275007034 2024-11-29T09:58:11.74391+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:58:12.1115+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:58:12.13658+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007035, result count: 29 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007035, max_request_id: 275007063 2024-11-29T09:58:12.79958+08:00 INFO [21697,b678783b25fe08d2] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:58:12.79964+08:00 INFO [21697,b678783b25fe08d2] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:58:12.765685857 +0800 CST m=+10835457.882295390, end time:2024-11-29 09:58:12 +0800 CST, Metrics number 0 2024-11-29T09:58:12.79972+08:00 WARN [21697,b678783b25fe08d2] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:58:13.13921+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007064, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007064, max_request_id: 275007102 2024-11-29T09:58:13.86782+08:00 INFO [21697,d18f4edd054745ef] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=d18f4edd054745ef, body=] 2024-11-29T09:58:13.87358+08:00 INFO [21697,d18f4edd054745ef] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=d18f4edd054745ef, body=] 2024-11-29T09:58:13.87389+08:00 WARN [21697,d18f4edd054745ef] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:13.87392+08:00 WARN [21697,d18f4edd054745ef] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:13.87394+08:00 WARN [21697,d18f4edd054745ef] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:13.87396+08:00 WARN [21697,d18f4edd054745ef] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:13.87397+08:00 WARN [21697,d18f4edd054745ef] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:14.14252+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007103, result count: 52 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007103, max_request_id: 275007154 2024-11-29T09:58:15.04161+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:58:15.08381+08:00 INFO [21697,09e242147a80407d] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=09e242147a80407d, body=] 2024-11-29T09:58:15.08398+08:00 INFO [21697,09e242147a80407d] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=09e242147a80407d, body=] 2024-11-29T09:58:15.08447+08:00 WARN [21697,09e242147a80407d] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:15.08451+08:00 WARN [21697,09e242147a80407d] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:15.08454+08:00 WARN [21697,09e242147a80407d] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:15.08457+08:00 WARN [21697,09e242147a80407d] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:15.08458+08:00 WARN [21697,09e242147a80407d] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:15.09424+08:00 INFO [21697,09e242147a80407d] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:58:15.09946+08:00 INFO [21697,09e242147a80407d] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=09e242147a80407d, body=] 2024-11-29T09:58:15.11798+08:00 INFO [21697,09e242147a80407d] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=09e242147a80407d, body=] 2024-11-29T09:58:15.11814+08:00 INFO [21697,09e242147a80407d] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=09e242147a80407d, body=] 2024-11-29T09:58:15.11829+08:00 WARN [21697,09e242147a80407d] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:58:15.11882+08:00 WARN [21697,09e242147a80407d] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:58:15.11969+08:00 INFO [21697,09e242147a80407d] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:58:15.14436+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007155, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007155, max_request_id: 275007189 2024-11-29T09:58:16.1468+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007190, result count: 53 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007190, max_request_id: 275007242 2024-11-29T09:58:16.86177+08:00 INFO [21697,38cb8eb982e9457a] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=38cb8eb982e9457a, body=] 2024-11-29T09:58:16.86204+08:00 WARN [21697,38cb8eb982e9457a] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:16.86209+08:00 WARN [21697,38cb8eb982e9457a] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:16.86211+08:00 WARN [21697,38cb8eb982e9457a] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:16.86213+08:00 WARN [21697,38cb8eb982e9457a] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:16.86214+08:00 WARN [21697,38cb8eb982e9457a] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:16.86717+08:00 INFO [21697,38cb8eb982e9457a] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=38cb8eb982e9457a, body=] 2024-11-29T09:58:17.09531+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:58:17.1488+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007243, result count: 44 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007243, max_request_id: 275007286 2024-11-29T09:58:17.78379+08:00 INFO [21697,1fa4beaf9f6bb5ff] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:58:17.78386+08:00 INFO [21697,1fa4beaf9f6bb5ff] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:58:17.765549299 +0800 CST m=+10835462.882158832, end time:2024-11-29 09:58:17 +0800 CST, Metrics number 0 2024-11-29T09:58:17.78394+08:00 WARN [21697,1fa4beaf9f6bb5ff] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:58:18.15108+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007287, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007287, max_request_id: 275007325 2024-11-29T09:58:19.15336+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007326, result count: 20 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007326, max_request_id: 275007345 2024-11-29T09:58:19.4815+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:58:19.48162+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:58:19.55194+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:58:19.86077+08:00 INFO [21697,3639c303c3ae4fa5] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=3639c303c3ae4fa5, body=] 2024-11-29T09:58:19.861+08:00 WARN [21697,3639c303c3ae4fa5] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:19.86103+08:00 WARN [21697,3639c303c3ae4fa5] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:19.86105+08:00 WARN [21697,3639c303c3ae4fa5] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:19.86107+08:00 WARN [21697,3639c303c3ae4fa5] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:19.86108+08:00 WARN [21697,3639c303c3ae4fa5] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:19.86512+08:00 INFO [21697,3639c303c3ae4fa5] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=3639c303c3ae4fa5, body=] 2024-11-29T09:58:20.04708+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:58:20.15619+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007346, result count: 54 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007346, max_request_id: 275007399 2024-11-29T09:58:21.15976+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007400, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007400, max_request_id: 275007439 2024-11-29T09:58:21.18593+08:00 INFO [21697,88448c1d0711a862] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=88448c1d0711a862, body=] 2024-11-29T09:58:21.74379+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:58:22.10874+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:58:22.16151+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007440, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007440, max_request_id: 275007471 2024-11-29T09:58:22.29742+08:00 INFO [21697,dae72cea66c3ad4b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=dae72cea66c3ad4b, body=] 2024-11-29T09:58:22.29778+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:58:22.29812+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 610, err: 2024-11-29T09:58:22.78176+08:00 INFO [21697,d4f59bdca45445e0] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:58:22.78183+08:00 INFO [21697,d4f59bdca45445e0] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:58:22.76505892 +0800 CST m=+10835467.881668453, end time:2024-11-29 09:58:22 +0800 CST, Metrics number 0 2024-11-29T09:58:22.78191+08:00 WARN [21697,d4f59bdca45445e0] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:58:22.8616+08:00 INFO [21697,6554ecab596a4ef8] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=6554ecab596a4ef8, body=] 2024-11-29T09:58:22.86195+08:00 WARN [21697,6554ecab596a4ef8] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:22.86199+08:00 WARN [21697,6554ecab596a4ef8] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:22.862+08:00 WARN [21697,6554ecab596a4ef8] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:22.86201+08:00 WARN [21697,6554ecab596a4ef8] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:22.86203+08:00 WARN [21697,6554ecab596a4ef8] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:22.86642+08:00 INFO [21697,6554ecab596a4ef8] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=6554ecab596a4ef8, body=] 2024-11-29T09:58:23.16328+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007472, result count: 51 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007472, max_request_id: 275007522 2024-11-29T09:58:24.16567+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007523, result count: 38 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007523, max_request_id: 275007560 2024-11-29T09:58:25.01613+08:00 INFO [21697,fd3909660e11281b] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=fd3909660e11281b, body=] 2024-11-29T09:58:25.01637+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:58:25.04186+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:58:25.13318+08:00 INFO [21697,2919d418b6ca6c6b] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=2919d418b6ca6c6b, body=] 2024-11-29T09:58:25.13354+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:25.13359+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:25.13361+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:25.13362+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:25.13363+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:25.13402+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:58:25.16761+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007561, result count: 36 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007561, max_request_id: 275007596 2024-11-29T09:58:25.34714+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163147494, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163147822, max_request_id: 163147825 2024-11-29T09:58:25.34719+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163147822 is larger than startRequestId 163147494, maybe lost 328 data 2024-11-29T09:58:25.77366+08:00 INFO [21697,22ac8c0e7bec5af7] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=22ac8c0e7bec5af7, body=] 2024-11-29T09:58:25.86022+08:00 INFO [21697,8f8af1b535de4580] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=8f8af1b535de4580, body=] 2024-11-29T09:58:25.86054+08:00 WARN [21697,8f8af1b535de4580] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:25.86058+08:00 WARN [21697,8f8af1b535de4580] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:25.8606+08:00 WARN [21697,8f8af1b535de4580] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:25.86061+08:00 WARN [21697,8f8af1b535de4580] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:25.86062+08:00 WARN [21697,8f8af1b535de4580] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:25.86808+08:00 INFO [21697,8f8af1b535de4580] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=8f8af1b535de4580, body=] 2024-11-29T09:58:26.17052+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007597, result count: 53 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007597, max_request_id: 275007649 2024-11-29T09:58:27.09587+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:58:27.18095+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007650, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007650, max_request_id: 275007680 2024-11-29T09:58:27.78226+08:00 INFO [21697,59a1f8cddddbd8a0] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:58:27.78234+08:00 INFO [21697,59a1f8cddddbd8a0] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:58:27.765929879 +0800 CST m=+10835472.882539412, end time:2024-11-29 09:58:27 +0800 CST, Metrics number 0 2024-11-29T09:58:27.78241+08:00 WARN [21697,59a1f8cddddbd8a0] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:58:28.18328+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007681, result count: 40 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007681, max_request_id: 275007720 2024-11-29T09:58:28.85993+08:00 INFO [21697,18b7fe8e456349ef] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=18b7fe8e456349ef, body=] 2024-11-29T09:58:28.86015+08:00 WARN [21697,18b7fe8e456349ef] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:28.86019+08:00 WARN [21697,18b7fe8e456349ef] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:28.86021+08:00 WARN [21697,18b7fe8e456349ef] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:28.86022+08:00 WARN [21697,18b7fe8e456349ef] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:28.86023+08:00 WARN [21697,18b7fe8e456349ef] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:28.86417+08:00 INFO [21697,18b7fe8e456349ef] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=18b7fe8e456349ef, body=] 2024-11-29T09:58:29.18531+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007721, result count: 32 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007721, max_request_id: 275007752 2024-11-29T09:58:30.04155+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:58:30.08342+08:00 INFO [21697,59824532378e4f1f] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=59824532378e4f1f, body=] 2024-11-29T09:58:30.08383+08:00 WARN [21697,59824532378e4f1f] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:30.08387+08:00 WARN [21697,59824532378e4f1f] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:30.08388+08:00 WARN [21697,59824532378e4f1f] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:30.0839+08:00 WARN [21697,59824532378e4f1f] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:30.08391+08:00 WARN [21697,59824532378e4f1f] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:30.08428+08:00 INFO [21697,59824532378e4f1f] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:58:30.08476+08:00 INFO [21697,59824532378e4f1f] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=59824532378e4f1f, body=] 2024-11-29T09:58:30.09804+08:00 INFO [21697,59824532378e4f1f] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=59824532378e4f1f, body=] 2024-11-29T09:58:30.10326+08:00 INFO [21697,59824532378e4f1f] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=59824532378e4f1f, body=] 2024-11-29T09:58:30.10342+08:00 INFO [21697,59824532378e4f1f] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=59824532378e4f1f, body=] 2024-11-29T09:58:30.10383+08:00 WARN [21697,59824532378e4f1f] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:58:30.10872+08:00 WARN [21697,59824532378e4f1f] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:58:30.11446+08:00 INFO [21697,59824532378e4f1f] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 610, err: 2024-11-29T09:58:30.18698+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007753, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007753, max_request_id: 275007794 2024-11-29T09:58:31.19006+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007795, result count: 126 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007795, max_request_id: 275007920 2024-11-29T09:58:31.32098+08:00 INFO [21697,31b6c9a6fca67e2e] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.98.86, ocpServerIp=, traceId=31b6c9a6fca67e2e, body=] 2024-11-29T09:58:31.3212+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:31.32124+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:31.32125+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:31.32127+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:31.32128+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:31.35226+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163147826, result count: 47 auditMsg number: 0, diagMsg number: 0, min_request_id: 163148004, max_request_id: 163148063 2024-11-29T09:58:31.35231+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163148004 is larger than startRequestId 163147826, maybe lost 178 data 2024-11-29T09:58:31.74401+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:58:31.86023+08:00 INFO [21697,40f087e5ce674045] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=40f087e5ce674045, body=] 2024-11-29T09:58:31.86045+08:00 WARN [21697,40f087e5ce674045] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:31.86048+08:00 WARN [21697,40f087e5ce674045] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:31.86049+08:00 WARN [21697,40f087e5ce674045] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:31.8605+08:00 WARN [21697,40f087e5ce674045] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:31.86051+08:00 WARN [21697,40f087e5ce674045] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:31.8643+08:00 INFO [21697,40f087e5ce674045] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=40f087e5ce674045, body=] 2024-11-29T09:58:32.10837+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:58:32.19401+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275007921, result count: 87 auditMsg number: 0, diagMsg number: 0, min_request_id: 275007921, max_request_id: 275008007 2024-11-29T09:58:32.35739+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163148064, result count: 262 auditMsg number: 0, diagMsg number: 0, min_request_id: 163148114, max_request_id: 163148385 2024-11-29T09:58:32.35744+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163148114 is larger than startRequestId 163148064, maybe lost 50 data 2024-11-29T09:58:32.78162+08:00 INFO [21697,b53be1e739eb211f] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:58:32.7817+08:00 INFO [21697,b53be1e739eb211f] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:58:32.765219413 +0800 CST m=+10835477.881828946, end time:2024-11-29 09:58:32 +0800 CST, Metrics number 0 2024-11-29T09:58:32.78178+08:00 WARN [21697,b53be1e739eb211f] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:58:32.86199+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:58:32.86344+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:58:32.86351+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:58:32.86352+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:58:32.86355+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:58:32.86358+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:58:32.8636+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:58:32.86362+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:58:32.86364+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:58:32.86366+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:58:32.86368+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:58:32.86369+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:58:32.87244+08:00 INFO [21697,fe0be5edcfac8822] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:58:32 +0800 CST, start_request_id:0 2024-11-29T09:58:32.8725+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:32.87259+08:00 INFO [21697,fe0be5edcfac8822] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:58:32 +0800 CST, start_request_id:0 2024-11-29T09:58:32.87261+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:32.8727+08:00 INFO [21697,fe0be5edcfac8822] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:58:32 +0800 CST, start_request_id:544466 2024-11-29T09:58:32.87272+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544466, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:32.88002+08:00 INFO [21697,fe0be5edcfac8822] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:58:32 +0800 CST, start_request_id:0 2024-11-29T09:58:32.88009+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:32.88016+08:00 INFO [21697,fe0be5edcfac8822] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:58:32 +0800 CST, start_request_id:7154 2024-11-29T09:58:32.88018+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:32.88383+08:00 INFO [21697,fe0be5edcfac8822] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:58:32 +0800 CST, start_request_id:0 2024-11-29T09:58:32.88387+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:32.88396+08:00 INFO [21697,fe0be5edcfac8822] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:58:32 +0800 CST, start_request_id:0 2024-11-29T09:58:32.88399+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:32.88983+08:00 INFO [21697,fe0be5edcfac8822] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:58:32 +0800 CST, start_request_id:387 2024-11-29T09:58:32.88991+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:32.90868+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:58:32.90877+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:32.90879+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:58:32.90894+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:58:32.90896+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:32.90897+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:58:32.90905+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907760, got 0 result 2024-11-29T09:58:32.90908+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907760, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:32.90915+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:58:32.90917+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:32.90926+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:58:32.90929+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:32.90938+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:58:32.9094+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:32.90947+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:58:32.9095+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:58:32.90952+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:58:32.90954+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:58:32.91178+08:00 INFO [21697,fe0be5edcfac8822] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:58:32 +0800 CST, start_request_id:0 2024-11-29T09:58:32.91181+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:58:32.91244+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807715, got 0 result 2024-11-29T09:58:32.91248+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807715, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:32.9125+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:58:32.91968+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:58:32.91972+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:32.91975+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:58:32.97058+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:58:32.97066+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:58:32.97069+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:58:32.97074+08:00 INFO [21697,4b4ff80014e1ec22] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:58:32 +0800 CST, end time:2024-11-29 09:58:32 +0800 CST, planMetaMetrics number 0, planExplainMetrics number 0, planStatMetrics number 0 2024-11-29T09:58:33.1948+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 53 2024-11-29T09:58:33.19487+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:58:33.19491+08:00 INFO [21697,6f7d8956bb30160d] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1427, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 53, sql_audit_progress Metrics number 1 2024-11-29T09:58:33.19837+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008008, result count: 67 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008008, max_request_id: 275008074 2024-11-29T09:58:33.35863+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1004 topSqlMetrics length: 20 2024-11-29T09:58:33.3587+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1004 sampleSqlMetrics length: 1 2024-11-29T09:58:33.35874+08:00 INFO [21697,f734e181a1d04c6c] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1004 total process data number 336, lost data number: 929, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 20, sql_audit_progress Metrics number 1 2024-11-29T09:58:33.468+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:58:33.46813+08:00 INFO [21697,3cd77836bd4d49a8] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:58:34.2047+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008075, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008075, max_request_id: 275008117 2024-11-29T09:58:34.48119+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:58:34.48128+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:58:34.5523+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:58:34.85978+08:00 INFO [21697,46984b68b10e42f8] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=46984b68b10e42f8, body=] 2024-11-29T09:58:34.86001+08:00 WARN [21697,46984b68b10e42f8] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:34.86006+08:00 WARN [21697,46984b68b10e42f8] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:34.86008+08:00 WARN [21697,46984b68b10e42f8] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:34.8601+08:00 WARN [21697,46984b68b10e42f8] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:34.86012+08:00 WARN [21697,46984b68b10e42f8] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:34.86437+08:00 INFO [21697,46984b68b10e42f8] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=46984b68b10e42f8, body=] 2024-11-29T09:58:35.04145+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:58:35.18529+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1006 topSqlMetrics length: 7 2024-11-29T09:58:35.18535+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1006 sampleSqlMetrics length: 1 2024-11-29T09:58:35.1854+08:00 INFO [21697,3ab8317b4b37979e] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1006 total process data number 21, lost data number: 820, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 7, sql_audit_progress Metrics number 1 2024-11-29T09:58:35.2065+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008118, result count: 55 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008118, max_request_id: 275008172 2024-11-29T09:58:36.18544+08:00 INFO [21697,f2f4b423ca639815] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=f2f4b423ca639815, body=] 2024-11-29T09:58:36.20919+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008173, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008173, max_request_id: 275008209 2024-11-29T09:58:37.09575+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:58:37.21092+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008210, result count: 57 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008210, max_request_id: 275008266 2024-11-29T09:58:37.29736+08:00 INFO [21697,2733723a8c3e756c] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=2733723a8c3e756c, body=] 2024-11-29T09:58:37.29785+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:58:37.29837+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:58:37.75764+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:58:37.75775+08:00 INFO [21697,d6ef286dd3bc030e] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:58:37.78143+08:00 INFO [21697,796260cb2c72ad48] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:58:37.7815+08:00 INFO [21697,796260cb2c72ad48] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:58:37.765047984 +0800 CST m=+10835482.881657538, end time:2024-11-29 09:58:37 +0800 CST, Metrics number 0 2024-11-29T09:58:37.78159+08:00 WARN [21697,796260cb2c72ad48] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:58:37.85969+08:00 INFO [21697,746ba3a4c8c44281] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=746ba3a4c8c44281, body=] 2024-11-29T09:58:37.85992+08:00 WARN [21697,746ba3a4c8c44281] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:37.85995+08:00 WARN [21697,746ba3a4c8c44281] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:37.85996+08:00 WARN [21697,746ba3a4c8c44281] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:37.85998+08:00 WARN [21697,746ba3a4c8c44281] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:37.85999+08:00 WARN [21697,746ba3a4c8c44281] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:37.86378+08:00 INFO [21697,746ba3a4c8c44281] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=746ba3a4c8c44281, body=] 2024-11-29T09:58:38.21438+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008267, result count: 52 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008267, max_request_id: 275008318 2024-11-29T09:58:38.37358+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163148386, result count: 9 auditMsg number: 0, diagMsg number: 0, min_request_id: 163148462, max_request_id: 163148473 2024-11-29T09:58:38.37364+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163148462 is larger than startRequestId 163148386, maybe lost 76 data 2024-11-29T09:58:39.21598+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008319, result count: 19 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008319, max_request_id: 275008337 2024-11-29T09:58:40.01593+08:00 INFO [21697,c88ba15f5a72dad6] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=c88ba15f5a72dad6, body=] 2024-11-29T09:58:40.01617+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:58:40.04182+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:58:40.13275+08:00 INFO [21697,eb08bf86709aecd9] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=eb08bf86709aecd9, body=] 2024-11-29T09:58:40.13312+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:40.13316+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:40.13319+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:40.1332+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:40.13321+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:40.1336+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1506, err: 2024-11-29T09:58:40.21763+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008338, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008338, max_request_id: 275008380 2024-11-29T09:58:40.77361+08:00 INFO [21697,35be8b5211992d14] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=35be8b5211992d14, body=] 2024-11-29T09:58:40.85968+08:00 INFO [21697,392e9095023a4186] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=392e9095023a4186, body=] 2024-11-29T09:58:40.8599+08:00 WARN [21697,392e9095023a4186] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:40.85994+08:00 WARN [21697,392e9095023a4186] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:40.85995+08:00 WARN [21697,392e9095023a4186] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:40.85997+08:00 WARN [21697,392e9095023a4186] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:40.85998+08:00 WARN [21697,392e9095023a4186] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:40.86519+08:00 INFO [21697,392e9095023a4186] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=392e9095023a4186, body=] 2024-11-29T09:58:41.21957+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008381, result count: 53 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008381, max_request_id: 275008433 2024-11-29T09:58:41.74409+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:58:42.09462+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:58:42.22153+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008434, result count: 24 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008434, max_request_id: 275008457 2024-11-29T09:58:42.78274+08:00 INFO [21697,c19ef12dfbcb0ac3] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:58:42.78282+08:00 INFO [21697,c19ef12dfbcb0ac3] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:58:42.765930475 +0800 CST m=+10835487.882540008, end time:2024-11-29 09:58:42 +0800 CST, Metrics number 0 2024-11-29T09:58:42.78291+08:00 WARN [21697,c19ef12dfbcb0ac3] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:58:43.22427+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008458, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008458, max_request_id: 275008494 2024-11-29T09:58:43.86021+08:00 INFO [21697,d002364d5a8e4039] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=d002364d5a8e4039, body=] 2024-11-29T09:58:43.86055+08:00 WARN [21697,d002364d5a8e4039] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:43.86062+08:00 WARN [21697,d002364d5a8e4039] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:43.86065+08:00 WARN [21697,d002364d5a8e4039] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:43.86068+08:00 WARN [21697,d002364d5a8e4039] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:43.8607+08:00 WARN [21697,d002364d5a8e4039] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:43.86762+08:00 INFO [21697,d002364d5a8e4039] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=d002364d5a8e4039, body=] 2024-11-29T09:58:44.22707+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008495, result count: 56 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008495, max_request_id: 275008550 2024-11-29T09:58:45.04177+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:58:45.083+08:00 INFO [21697,62e3df0eaabc4af8] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=62e3df0eaabc4af8, body=] 2024-11-29T09:58:45.08322+08:00 INFO [21697,62e3df0eaabc4af8] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=62e3df0eaabc4af8, body=] 2024-11-29T09:58:45.08349+08:00 WARN [21697,62e3df0eaabc4af8] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:45.08352+08:00 WARN [21697,62e3df0eaabc4af8] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:45.08353+08:00 WARN [21697,62e3df0eaabc4af8] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:45.08354+08:00 WARN [21697,62e3df0eaabc4af8] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:45.08356+08:00 WARN [21697,62e3df0eaabc4af8] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:45.09213+08:00 INFO [21697,62e3df0eaabc4af8] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:58:45.09723+08:00 INFO [21697,62e3df0eaabc4af8] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=62e3df0eaabc4af8, body=] 2024-11-29T09:58:45.10048+08:00 INFO [21697,62e3df0eaabc4af8] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=62e3df0eaabc4af8, body=] 2024-11-29T09:58:45.1006+08:00 INFO [21697,62e3df0eaabc4af8] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=62e3df0eaabc4af8, body=] 2024-11-29T09:58:45.10072+08:00 WARN [21697,62e3df0eaabc4af8] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:58:45.10225+08:00 WARN [21697,62e3df0eaabc4af8] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:58:45.10415+08:00 INFO [21697,62e3df0eaabc4af8] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:58:45.22891+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008551, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008551, max_request_id: 275008587 2024-11-29T09:58:46.2316+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008588, result count: 56 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008588, max_request_id: 275008643 2024-11-29T09:58:46.86119+08:00 INFO [21697,8a70baf428854afa] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=8a70baf428854afa, body=] 2024-11-29T09:58:46.86142+08:00 WARN [21697,8a70baf428854afa] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:46.86146+08:00 WARN [21697,8a70baf428854afa] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:46.86148+08:00 WARN [21697,8a70baf428854afa] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:46.8615+08:00 WARN [21697,8a70baf428854afa] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:46.86151+08:00 WARN [21697,8a70baf428854afa] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:46.86628+08:00 INFO [21697,8a70baf428854afa] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=8a70baf428854afa, body=] 2024-11-29T09:58:47.09502+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:58:47.23336+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008644, result count: 44 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008644, max_request_id: 275008687 2024-11-29T09:58:47.79086+08:00 INFO [21697,37c6913b30bb59a0] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:58:47.79093+08:00 INFO [21697,37c6913b30bb59a0] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:58:47.765285637 +0800 CST m=+10835492.881895169, end time:2024-11-29 09:58:47 +0800 CST, Metrics number 0 2024-11-29T09:58:47.791+08:00 WARN [21697,37c6913b30bb59a0] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:58:48.23592+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008688, result count: 35 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008688, max_request_id: 275008722 2024-11-29T09:58:48.29779+08:00 INFO [21697,895e1b5e5b79643c] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.98.86, ocpServerIp=, traceId=895e1b5e5b79643c, body=] 2024-11-29T09:58:49.24629+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008723, result count: 25 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008723, max_request_id: 275008747 2024-11-29T09:58:49.48104+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:58:49.48114+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:58:49.55183+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields:, error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:58:49.85946+08:00 INFO [21697,08ce9c6138e84e52] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=08ce9c6138e84e52, body=] 2024-11-29T09:58:49.85973+08:00 WARN [21697,08ce9c6138e84e52] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:49.85977+08:00 WARN [21697,08ce9c6138e84e52] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:49.85979+08:00 WARN [21697,08ce9c6138e84e52] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:49.85981+08:00 WARN [21697,08ce9c6138e84e52] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:49.85982+08:00 WARN [21697,08ce9c6138e84e52] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:49.86429+08:00 INFO [21697,08ce9c6138e84e52] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=08ce9c6138e84e52, body=] 2024-11-29T09:58:50.04133+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:58:50.24929+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008748, result count: 54 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008748, max_request_id: 275008801 2024-11-29T09:58:51.18519+08:00 INFO [21697,4d9eb44d52af6e1f] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.98.86, ocpServerIp=, traceId=4d9eb44d52af6e1f, body=] 2024-11-29T09:58:51.25125+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008802, result count: 43 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008802, max_request_id: 275008844 2024-11-29T09:58:51.74778+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:58:52.10834+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:58:52.25299+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008845, result count: 31 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008845, max_request_id: 275008875 2024-11-29T09:58:52.2968+08:00 INFO [21697,990d434819319456] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=990d434819319456, body=] 2024-11-29T09:58:52.29716+08:00 WARN [21697,] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:58:52.2975+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 610, err: 2024-11-29T09:58:52.37898+08:00 INFO [21697,ed85e7e5ed336283] caller=host/custom.go:178:func10: collect observer dir exists info 2024-11-29T09:58:52.79011+08:00 INFO [21697,5dcccaea17b173f0] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:58:52.79018+08:00 INFO [21697,5dcccaea17b173f0] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:58:52.766309216 +0800 CST m=+10835497.882918754, end time:2024-11-29 09:58:52 +0800 CST, Metrics number 0 2024-11-29T09:58:52.79026+08:00 WARN [21697,5dcccaea17b173f0] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:58:52.86395+08:00 INFO [21697,4bf7001f7c0d4a07] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=4bf7001f7c0d4a07, body=] 2024-11-29T09:58:52.86418+08:00 WARN [21697,4bf7001f7c0d4a07] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:52.86421+08:00 WARN [21697,4bf7001f7c0d4a07] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:52.86424+08:00 WARN [21697,4bf7001f7c0d4a07] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:52.86426+08:00 WARN [21697,4bf7001f7c0d4a07] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:52.86428+08:00 WARN [21697,4bf7001f7c0d4a07] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:52.87211+08:00 INFO [21697,4bf7001f7c0d4a07] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=4bf7001f7c0d4a07, body=] 2024-11-29T09:58:53.25624+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008876, result count: 81 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008876, max_request_id: 275008956 2024-11-29T09:58:54.25794+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008957, result count: 42 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008957, max_request_id: 275008998 2024-11-29T09:58:54.38403+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1004 from request_id 163148474, result count: 3 auditMsg number: 0, diagMsg number: 0, min_request_id: 163148807, max_request_id: 163148809 2024-11-29T09:58:54.3841+08:00 WARN [21697,] caller=oceanbase/sql_audit_merge.go:444:collectRawMsgsByTenant: collect by tenantId 1004 minRequestId 163148807 is larger than startRequestId 163148474, maybe lost 333 data 2024-11-29T09:58:55.01551+08:00 INFO [21697,31c0f6fa7cabbd74] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.98.86, ocpServerIp=, traceId=31c0f6fa7cabbd74, body=] 2024-11-29T09:58:55.01577+08:00 WARN [21697,] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:58:55.04154+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:58:55.13226+08:00 INFO [21697,fb70c257b6a2ff28] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.98.86, ocpServerIp=, traceId=fb70c257b6a2ff28, body=] 2024-11-29T09:58:55.1327+08:00 WARN [21697,] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:55.13274+08:00 WARN [21697,] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:55.13276+08:00 WARN [21697,] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:55.13277+08:00 WARN [21697,] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:55.13279+08:00 WARN [21697,] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:55.13325+08:00 INFO [21697,] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1508, err: 2024-11-29T09:58:55.26035+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275008999, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 275008999, max_request_id: 275009035 2024-11-29T09:58:55.77253+08:00 INFO [21697,a4b0037a70c0e8d3] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.98.86, ocpServerIp=, traceId=a4b0037a70c0e8d3, body=] 2024-11-29T09:58:55.85981+08:00 INFO [21697,a29cbd37c0f04ee0] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=a29cbd37c0f04ee0, body=] 2024-11-29T09:58:55.86003+08:00 WARN [21697,a29cbd37c0f04ee0] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:55.86006+08:00 WARN [21697,a29cbd37c0f04ee0] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:55.86007+08:00 WARN [21697,a29cbd37c0f04ee0] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:55.86008+08:00 WARN [21697,a29cbd37c0f04ee0] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:55.86009+08:00 WARN [21697,a29cbd37c0f04ee0] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:55.86473+08:00 INFO [21697,a29cbd37c0f04ee0] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=a29cbd37c0f04ee0, body=] 2024-11-29T09:58:56.26215+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275009036, result count: 53 auditMsg number: 0, diagMsg number: 0, min_request_id: 275009036, max_request_id: 275009088 2024-11-29T09:58:57.09536+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:58:57.26368+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275009089, result count: 26 auditMsg number: 0, diagMsg number: 0, min_request_id: 275009089, max_request_id: 275009114 2024-11-29T09:58:57.78215+08:00 INFO [21697,ad616322660bb529] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 1 result 2024-11-29T09:58:57.78223+08:00 INFO [21697,ad616322660bb529] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:58:57.765483945 +0800 CST m=+10835502.882093484, end time:2024-11-29 09:58:57 +0800 CST, Metrics number 1 2024-11-29T09:58:57.7823+08:00 INFO [21697,ad616322660bb529] caller=engine/pipeline.go:145:pipelinePush: push pipeline ob_sql_transaction metrics count 1 2024-11-29T09:58:58.26652+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275009115, result count: 37 auditMsg number: 0, diagMsg number: 0, min_request_id: 275009115, max_request_id: 275009151 2024-11-29T09:58:58.85956+08:00 INFO [21697,426a038cd5374c54] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=426a038cd5374c54, body=] 2024-11-29T09:58:58.85982+08:00 WARN [21697,426a038cd5374c54] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:58:58.85988+08:00 WARN [21697,426a038cd5374c54] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:58:58.8599+08:00 WARN [21697,426a038cd5374c54] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:58:58.85993+08:00 WARN [21697,426a038cd5374c54] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:58.85995+08:00 WARN [21697,426a038cd5374c54] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:58:58.86647+08:00 INFO [21697,426a038cd5374c54] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=426a038cd5374c54, body=] 2024-11-29T09:58:59.26795+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275009152, result count: 33 auditMsg number: 0, diagMsg number: 0, min_request_id: 275009152, max_request_id: 275009184 2024-11-29T09:59:00.04205+08:00 INFO [21697,] caller=common/process.go:83:refreshProcesses: start refresh processes 2024-11-29T09:59:00.08255+08:00 INFO [21697,526a4cd683d5439e] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/ob, client=1.1.1.180, ocpServerIp=, traceId=526a4cd683d5439e, body=] 2024-11-29T09:59:00.08293+08:00 WARN [21697,526a4cd683d5439e] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:59:00.08296+08:00 WARN [21697,526a4cd683d5439e] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:59:00.08298+08:00 WARN [21697,526a4cd683d5439e] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:59:00.08299+08:00 WARN [21697,526a4cd683d5439e] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:59:00.083+08:00 WARN [21697,526a4cd683d5439e] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:59:00.08335+08:00 INFO [21697,526a4cd683d5439e] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 1507, err: 2024-11-29T09:59:00.08374+08:00 INFO [21697,526a4cd683d5439e] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/host, client=1.1.1.180, ocpServerIp=, traceId=526a4cd683d5439e, body=] 2024-11-29T09:59:00.08692+08:00 INFO [21697,526a4cd683d5439e] caller=common/middleware.go:59:func1: API request: [GET /metrics/node/obproxy, client=1.1.1.180, ocpServerIp=, traceId=526a4cd683d5439e, body=] 2024-11-29T09:59:00.08704+08:00 INFO [21697,526a4cd683d5439e] caller=common/middleware.go:59:func1: API request: [GET /metrics/obproxy, client=1.1.1.180, ocpServerIp=, traceId=526a4cd683d5439e, body=] 2024-11-29T09:59:00.08716+08:00 WARN [21697,526a4cd683d5439e] caller=engine/route_manager.go:248:pipelinePull: pull pipeline parallel compute result metrics is nil 2024-11-29T09:59:00.08726+08:00 INFO [21697,526a4cd683d5439e] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/basic, client=1.1.1.180, ocpServerIp=, traceId=526a4cd683d5439e, body=] 2024-11-29T09:59:00.09198+08:00 WARN [21697,526a4cd683d5439e] caller=coredump/coredump.go:125:Collect: coredump value not exists in cache 2024-11-29T09:59:00.12297+08:00 INFO [21697,526a4cd683d5439e] caller=engine/route_manager.go:100:GetPipelieGroupReader: transformMetric to buffer size 611, err: 2024-11-29T09:59:00.2702+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275009185, result count: 39 auditMsg number: 0, diagMsg number: 0, min_request_id: 275009185, max_request_id: 275009223 2024-11-29T09:59:00.3271+08:00 INFO [21697,ba7bba8848150493] caller=common/middleware.go:59:func1: API request: [GET /api/v1/log/alarms, client=1.1.1.69.58, ocpServerIp=1.1.1.69.58, traceId=ba7bba8848150493, body=] 2024-11-29T09:59:00.32726+08:00 INFO [21697,] caller=alarmscache/alarm_handler.go:73:getPipelieGroupReader: get alarms, result size:0, cache list size:4 fields: uri=/api/v1/log/alarms 2024-11-29T09:59:01.2729+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275009224, result count: 96 auditMsg number: 0, diagMsg number: 0, min_request_id: 275009224, max_request_id: 275009319 2024-11-29T09:59:01.74408+08:00 WARN [21697,0466798dcf23faf3] caller=host/custom.go:676:doCollectCoredumpTime: get observer coredump time failed, err: open : no such file or directory fields:, coredump-path= 2024-11-29T09:59:01.85953+08:00 INFO [21697,ce594e55fe374cce] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=ce594e55fe374cce, body=] 2024-11-29T09:59:01.87197+08:00 INFO [21697,ce594e55fe374cce] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=ce594e55fe374cce, body=] 2024-11-29T09:59:01.87232+08:00 WARN [21697,ce594e55fe374cce] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:59:01.87236+08:00 WARN [21697,ce594e55fe374cce] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:59:01.87239+08:00 WARN [21697,ce594e55fe374cce] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:59:01.8724+08:00 WARN [21697,ce594e55fe374cce] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:59:01.87242+08:00 WARN [21697,ce594e55fe374cce] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:59:02.11563+08:00 INFO [21697,0466798dcf23faf3] caller=process/process.go:232:doCollect: process uptime down, proc ntpd count 0, uptime 0 2024-11-29T09:59:02.27514+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275009320, result count: 54 auditMsg number: 0, diagMsg number: 0, min_request_id: 275009320, max_request_id: 275009373 2024-11-29T09:59:02.74261+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:245:updateMetaInfo: successfully init sqlAudit tenant config 2024-11-29T09:59:02.78134+08:00 INFO [21697,110d90b443f7afda] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:59:02.78142+08:00 INFO [21697,110d90b443f7afda] caller=oceanbase/sql_trans.go:183:Collect: sql transaction pipeline collect data complete, collect start time:2024-11-29 09:59:02.765030648 +0800 CST m=+10835507.881640180, end time:2024-11-29 09:59:02 +0800 CST, Metrics number 0 2024-11-29T09:59:02.7815+08:00 WARN [21697,110d90b443f7afda] caller=engine/pipeline.go:142:pipelinePush: push pipeline ob_sql_transaction parallel compute result metrics is nil 2024-11-29T09:59:02.85649+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_trans.go:237:CollectTransaction: query sql transaction with 0 result 2024-11-29T09:59:02.85797+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:59:02.85804+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 114759697 bigger than end request id 0 2024-11-29T09:59:02.85806+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 15083 bigger than end request id 0 2024-11-29T09:59:02.85809+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:59:02.85811+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:59:02.85814+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:59:02.85817+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 163070714 bigger than end request id 0 2024-11-29T09:59:02.85819+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 9223372036854775807 bigger than end request id 0 2024-11-29T09:59:02.85821+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:237:checkCallBackQuerySqlAuditSample: sql audit no need to call back query, start request id 134134346 bigger than end request id 0 2024-11-29T09:59:02.85823+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_transaction.go:58:doCallBack: sql audit pipeline call back end, get 0 realtime transaction metrics, get 0 sql sample metrics 2024-11-29T09:59:02.85825+08:00 INFO [21697,db6d9b906a6bd98c] caller=oceanbase/sql_audit_merge.go:575:callbackWithTimeout: sql_audit_merge pipeline collect sql_transaction Metrics number 0 2024-11-29T09:59:02.86459+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:796:refreshTenants: configured tenants: map[1:sys 1002:ocp 1004:dbamonitor 1006:dbmanager 1008:datatransfer] 2024-11-29T09:59:02.8721+08:00 INFO [21697,32f7ff4cefee2857] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1003 data round 1 complete, collect time:2024-11-29 09:59:02 +0800 CST, start_request_id:0 2024-11-29T09:59:02.87221+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1003 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:59:02.87234+08:00 INFO [21697,32f7ff4cefee2857] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1 data round 1 complete, collect time:2024-11-29 09:59:02 +0800 CST, start_request_id:544466 2024-11-29T09:59:02.87236+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1 data round 1, start_request_id: 544466, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:59:02.87243+08:00 INFO [21697,32f7ff4cefee2857] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1001 data round 1 complete, collect time:2024-11-29 09:59:02 +0800 CST, start_request_id:0 2024-11-29T09:59:02.87246+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1001 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:59:02.87941+08:00 INFO [21697,32f7ff4cefee2857] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1002 data round 1 complete, collect time:2024-11-29 09:59:02 +0800 CST, start_request_id:0 2024-11-29T09:59:02.87948+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1002 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:59:02.87958+08:00 INFO [21697,32f7ff4cefee2857] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1005 data round 1 complete, collect time:2024-11-29 09:59:02 +0800 CST, start_request_id:0 2024-11-29T09:59:02.87961+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1005 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:59:02.88034+08:00 INFO [21697,32f7ff4cefee2857] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1004 data round 1 complete, collect time:2024-11-29 09:59:02 +0800 CST, start_request_id:7154 2024-11-29T09:59:02.8804+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1004 data round 1, start_request_id: 7154, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:59:02.88336+08:00 INFO [21697,32f7ff4cefee2857] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1006 data round 1 complete, collect time:2024-11-29 09:59:02 +0800 CST, start_request_id:387 2024-11-29T09:59:02.8834+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1006 data round 1, start_request_id: 387, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:59:02.88352+08:00 INFO [21697,32f7ff4cefee2857] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1007 data round 1 complete, collect time:2024-11-29 09:59:02 +0800 CST, start_request_id:0 2024-11-29T09:59:02.88353+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1007 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:59:02.88875+08:00 INFO [21697,32f7ff4cefee2857] caller=oceanbase/sql_plan_monitor.go:382:collectByTenant: sql plan monitor pipeline collect tenant 1008 data round 1 complete, collect time:2024-11-29 09:59:02 +0800 CST, start_request_id:0 2024-11-29T09:59:02.8888+08:00 INFO [21697,] caller=oceanbase/sql_plan_monitor.go:399:collectByTenant: sql plan monitor collect tenant 1008 data round 1, start_request_id: 0, min_request_id: 0, max_request_id: 0, collected metrics number: 0 2024-11-29T09:59:02.91398+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result 2024-11-29T09:59:02.91407+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1, from plan id: 40566, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:59:02.9141+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:59:02.91422+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result 2024-11-29T09:59:02.91424+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1003, from plan id: 45774, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:59:02.91432+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result 2024-11-29T09:59:02.91434+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1004, from plan id: 269049, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:59:02.91439+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1003, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:59:02.91441+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1004, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:59:02.91452+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result 2024-11-29T09:59:02.91454+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1007, from plan id: 56999, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:59:02.91457+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1007, got planStatMetrics number 0, planExplainMetrics number 0, planMetaMetrics number 0 2024-11-29T09:59:02.91465+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result 2024-11-29T09:59:02.91467+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1006, from plan id: 45968, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:59:02.91489+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1005, from plan id: 1807718, got 3 result 2024-11-29T09:59:02.91505+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1001, from plan id: 1907761, got 1 result 2024-11-29T09:59:02.91517+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result 2024-11-29T09:59:02.91519+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1002, from plan id: 24388, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:59:02.91524+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1006, got planStatMetrics number 4, planExplainMetrics number 0, planMetaMetrics number 4 2024-11-29T09:59:02.91527+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1002, got planStatMetrics number 4, planExplainMetrics number 0, planMetaMetrics number 4 2024-11-29T09:59:02.91566+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1001, from plan id: 1907761, got 1 result, planStatMetrics number 1, planMetaMetrics number 1, planExplainMetrics number 0 2024-11-29T09:59:02.91568+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1001, got planStatMetrics number 4, planExplainMetrics number 0, planMetaMetrics number 4 2024-11-29T09:59:02.91646+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1005, from plan id: 1807718, got 3 result, planStatMetrics number 3, planMetaMetrics number 3, planExplainMetrics number 0 2024-11-29T09:59:02.91649+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1005, got planStatMetrics number 4, planExplainMetrics number 0, planMetaMetrics number 4 2024-11-29T09:59:02.93792+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1002 sampleSqlMetrics length: 1 2024-11-29T09:59:02.93804+08:00 INFO [21697,5070c2ca327ef2ca] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1002 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:59:02.97581+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:403:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result 2024-11-29T09:59:02.97587+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:412:collectByTenant: query sql plan for tenant 1008, from plan id: 6462, got 0 result, planStatMetrics number 0, planMetaMetrics number 0, planExplainMetrics number 0 2024-11-29T09:59:02.9759+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:430:collectByTenant: finish collect sql plan for tenant 1008, got planStatMetrics number 4, planExplainMetrics number 0, planMetaMetrics number 4 2024-11-29T09:59:02.97595+08:00 INFO [21697,32e18ed500408bcf] caller=oceanbase/sql_plan.go:217:doCollect: sql plan pipeline collect data complete, collect start time:2024-11-29 09:59:02 +0800 CST, end time:2024-11-29 09:59:02 +0800 CST, planMetaMetrics number 4, planExplainMetrics number 0, planStatMetrics number 4 2024-11-29T09:59:03.27679+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:592:aggregateMapToMessages: tenant_id 1 topSqlMetrics length: 51 2024-11-29T09:59:03.27686+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1 sampleSqlMetrics length: 1 2024-11-29T09:59:03.2769+08:00 INFO [21697,016cbece43a3f5f7] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1 total process data number 1366, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 51, sql_audit_progress Metrics number 1 2024-11-29T09:59:03.27963+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275009374, result count: 95 auditMsg number: 0, diagMsg number: 0, min_request_id: 275009374, max_request_id: 275009468 2024-11-29T09:59:03.50974+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:611:aggregateMapToMessages: tenant_id 1008 sampleSqlMetrics length: 1 2024-11-29T09:59:03.50984+08:00 INFO [21697,06231f4e706bd9c9] caller=oceanbase/sql_audit_merge.go:618:aggregateMapToMessages: sql_audit_merge input collect data number statistics: tenant_id 1008 total process data number 0, lost data number: 0, sql_text Metrics number 0, transaction sql Metrics number 0, slow sql Metrics number 0, parallel sql Metrics number 0, top sql Metrics number 0, sql_audit_progress Metrics number 1 2024-11-29T09:59:04.2822+08:00 INFO [21697,] caller=oceanbase/sql_audit_merge.go:439:collectRawMsgsByTenant: collect by tenantId 1 from request_id 275009469, result count: 53 auditMsg number: 0, diagMsg number: 0, min_request_id: 275009469, max_request_id: 275009521 2024-11-29T09:59:04.48158+08:00 WARN [21697,68bb25a45f0494c5] caller=coredump/coredump.go:140:doCollect: get coredump time failed, err: open /opt/oceanbase/obproxy/minidump: no such file or directory fields:, coredump-path=/opt/oceanbase/obproxy/minidump 2024-11-29T09:59:04.48168+08:00 WARN [21697,68bb25a45f0494c5] caller=common/input_cache.go:47:Update: update cache for key obproxy_coredump_time_seconds, err: open /opt/oceanbase/obproxy/minidump: no such file or directory 2024-11-29T09:59:04.55191+08:00 ERROR [21697,90eb1320f4b01a71] caller=runtime/asm_amd64.s:1594:goexit: read text format failed fields: error="text format parsing error in line 47: invalid escape sequence '\\\n'" 2024-11-29T09:59:04.85937+08:00 INFO [21697,110d8bbb7a784c02] caller=common/middleware.go:59:func1: API request: [GET /metrics/ob/extra, client=1.1.1.180, ocpServerIp=, traceId=110d8bbb7a784c02, body=] 2024-11-29T09:59:04.85964+08:00 WARN [21697,110d8bbb7a784c02] caller=host/custom.go:730:Collect: tcp_retrans is not exists in cache 2024-11-29T09:59:04.85968+08:00 WARN [21697,110d8bbb7a784c02] caller=host/custom.go:731:Collect: io_infos is not exists in cache 2024-11-29T09:59:04.8597+08:00 WARN [21697,110d8bbb7a784c02] caller=host/custom.go:734:Collect: bandwidth_infos is not exists in cache 2024-11-29T09:59:04.85971+08:00 WARN [21697,110d8bbb7a784c02] caller=host/custom.go:735:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:59:04.85973+08:00 WARN [21697,110d8bbb7a784c02] caller=host/custom.go:736:Collect: node_ntp_offset_seconds is not exists in cache 2024-11-29T09:59:04.8644+08:00 INFO [21697,110d8bbb7a784c02] caller=common/middleware.go:59:func1: API request: [GET /metrics/stat, client=1.1.1.180, ocpServerIp=, traceId=110d8bbb7a784c02, body=]