应用服务报这个(也是启用了Quartz框架），我在博客上看了下，大概是存在死锁造成的断连；从proxy的日志来看，执行
2023-11-21 15:43:18.563907,obproxyssd1,obssd1:ict:ict_test,OB_MYSQL,QRTZ_LOCKS,qrtz_locks,OB_MYSQL_COM_QUERY,SELECT,failed,1317,SELECT * FROM QRTZ_LOCKS WHERE SCHED_NAME = ‘ictpmsReportQuartzScheduler’ AND LOCK_NAME = ‘TRIGGER_ACCESS’ FOR UPDATE,1101722244us,1804us,0us,407us,Y0-7FE603918140,0,10.125.11.234:2881,Query execution was interrupted被拒绝了
根据租户id查了下存在琐事务
大概300多个锁等待
有社区的老师帮看下吗
TRACE [TRACE]ob_trans_ctx.cpp:266 [14877][1828][YB420A7D0BEA-0006016FBE1FDC16] [lt=7] [dc=0] [force print](TRACE=begin_ts=1700555452476169 2023-11-21 08:30:52.476169|[init] u=0 arg1:140052522507344, ctx_type:“participant”, trans_type:0, trans_id:{hash:8560887474004427275, inc:1475933133, addr:{ip:“10.125.11.234”, port:2882}, t:1700555452476057}, pkey:{tid:1128098930148193, partition_id:0, part_cnt:0}, arg1:false, uref:1073741824|[start_trans] u=20 ret:0, left_time:2999999650, uref:1073741825|[snapshot] u=3 snapshot:1700555452475855, uref:1073741825|[update_gts] u=4 ret:0, sql_no:4294967297, uref:1073741825, snapshot:1700555452475855, tenant_id:1026|[start_task] u=3 ret:0, sql_no:4294967297, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39ECBD0, start_time:1700555452475832|[end_task] u=476 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=100 snapshot:1700555452475855, uref:1073741825|[update_gts] u=0 ret:0, sql_no:8589934594, uref:1073741825, snapshot:1700555452475855, tenant_id:1026|[start_task] u=1 ret:0, sql_no:8589934594, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39ECBD0, start_time:1700555452475832|[end_task] u=177 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=10496802 snapshot:1700555462929930, uref:1073741825|[update_gts] u=1 ret:0, sql_no:12884901891, uref:1073741825, snapshot:1700555462929930, tenant_id:1026|[start_task] u=1 ret:0, sql_no:12884901891, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39ECBD0, start_time:1700555452475832|[end_task] u=200 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=76 snapshot:1700555462929930, uref:1073741825|[update_gts] u=1 ret:0, sql_no:17179869188, uref:1073741825, snapshot:1700555462929930, tenant_id:1026|[start_task] u=0 ret:0, sql_no:17179869188, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39ECBD0, start_time:1700555452475832|[end_task] u=178 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=10003439 snapshot:1700555472946283, uref:1073741825|[update_gts] u=1 ret:0, sql_no:21474836485, uref:1073741825, snapshot:1700555472946283, tenant_id:1026|[start_task] u=1 ret:0, sql_no:21474836485, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39ECBD0, start_time:1700555452475832|[end_task] u=183 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=86 snapshot:1700555472946283, uref:1073741825|[update_gts] u=0 ret:0, sql_no:25769803782, uref:1073741825, snapshot:1700555472946283, tenant_id:1026|[start_task] u=1 ret:0, sql_no:25769803782, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39ECBD0, start_time:1700555452475832|[end_task] u=162 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=10014193 snapshot:1700555482978058, uref:1073741825|[update_gts] u=2 ret:0, sql_no:30064771079, uref:1073741825, snapshot:1700555482978058, tenant_id:1026|[start_task] u=1 ret:0, sql_no:30064771079, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39ECBD0, start_time:1700555452475832|[end_task] u=193 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=92 snapshot:1700555482978058, uref:1073741825|[update_gts] u=1 ret:0, sql_no:34359738376, uref:1073741825, snapshot:1700555482978058, tenant_id:1026|[start_task] u=1 ret:0, sql_no:34359738376, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39ECBD0, start_time:1700555452475832|[end_task] u=157 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=10006166 snapshot:1700555492916308, uref:1073741825|[update_gts] u=2 ret:0, sql_no:38654705673, uref:1073741825, snapshot:1700555492916308, tenant_id:1026|[start_task] u=1 ret:0, sql_no:38654705673, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39ECBD0, start_time:1700555452475832|[end_task] u=202 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=79 snapshot:1700555492916308, uref:1073741825|[update_gts] u=1 ret:0, sql_no:42949672970, uref:1073741825, snapshot:1700555492916308, tenant_id:1026|[start_task] u=0 ret:0, sql_no:42949672970, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39ECBD0, start_time:1700555452475832|[end_task] u=194 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=9998291 |[update_gts] u=1 |[start_task] u=1 |[end_task] u=197 |[snapshot] u=70 |[update_gts] u=1 |[start_task] u=0 |[end_task] u=158 |[snapshot] u=10003622 |[update_gts] u=1 |[start_task] u=1 |[end_task] u=188 |[snapshot] u=68 |[update_gts] u=1 |[start_task] u=1 |[end_task] u=146 |[snapshot] u=10010630 |[update_gts] u=2 |[start_task] u=1 |[end_task] u=193 |[snapshot] u=81 |[update_gts] u=1 |[start_task] u=1 |[end_task] u=184 |[snapshot] u=10005796 |[update_gts] u=2 |[start_task] u=1 |[end_task] u=194 |[snapshot] u=81 |[update_gts] u=1 |[start_task] u=1 |[end_task] u=196 |[snapshot] u=10004011 |[update_gts] u=1 |[start_task] u=1 |[end_task] u=236 |[snapshot] u=79 |[update_gts] u=1 |[start_task] u=0 |[end_task] u=169 |[snapshot] u=10004421 |[update_gts] u=1 |[start_task] u=1 |[end_task] u=159 |[snapshot] u=72 |[update_gts] u=0 |[start_task] u=1 |[end_task] u=153 |[snapshot] u=10008582 |[update_gts] u=1 |[start_task] u=1 |[end_task] u=182 |[snapshot] u=75 |[update_gts] u=0 |[start_task] u=1 |[end_task] u=146 |[snapshot] u=10000604|[end_task] u=550222167 |total_timeu=670784378 DROPPED_EVENTS=446)[2023-11-21 16:42:07.106436] TRACE [TRACE]ob_trans_ctx.cpp:266 [14877][1828][YB420A7D0BEA-0006016FBE1FDC16] [lt=10] [dc=0] [force print](TRACE=begin_ts=1700554875554729 2023-11-21 08:21:15.554729|[init] u=0 arg1:140062495513936, ctx_type:“participant”,trans_type:0, trans_id:{hash:7187006584976986621, inc:1475845169, addr:{ip:“10.125.11.234”, port:2882}, t:1700554875554673}, pkey:{tid:1128098930148193, partition_id:0, part_cnt:0}, arg1:false, uref:1073741824|[start_trans] u=8 ret:0, left_time:2999999766, uref:1073741825|[snapshot] u=1 snapshot:1700554875523382, uref:1073741825|[update_gts] u=2 ret:0, sql_no:4294967297, uref:1073741825, snapshot:1700554875523382, tenant_id:1026|[start_task] u=2 ret:0, sql_no:4294967297, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39EC501, start_time:1700554875554499|[end_task] u=339 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=88 snapshot:1700554875523382, uref:1073741825|[update_gts] u=1 ret:0, sql_no:8589934594, uref:1073741825, snapshot:1700554875523382, tenant_id:1026|[start_task] u=1 ret:0, sql_no:8589934594, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39EC501, start_time:1700554875554499|[end_task] u=162 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=7122344 snapshot:1700554882614310, uref:1073741825|[update_gts] u=1 ret:0, sql_no:12884901891, uref:1073741825, snapshot:1700554882614310, tenant_id:1026|[start_task] u=2 ret:0, sql_no:12884901891, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39EC501, start_time:1700554875554499|[end_task] u=170 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=72 snapshot:1700554882614310, uref:1073741825|[update_gts] u=0 ret:0, sql_no:17179869188, uref:1073741825, snapshot:1700554882614310, tenant_id:1026|[start_task] u=1 ret:0, sql_no:17179869188, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39EC501, start_time:1700554875554499|[end_task] u=149 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=10010060 snapshot:1700554892686645, uref:1073741825|[update_gts] u=2 ret:0, sql_no:21474836485, uref:1073741825, snapshot:1700554892686645, tenant_id:1026|[start_task] u=1 ret:0, sql_no:21474836485, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39EC501, start_time:1700554875554499|[end_task] u=175 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=95 snapshot:1700554892686645, uref:1073741825|[update_gts] u=1 ret:0, sql_no:25769803782, uref:1073741825, snapshot:1700554892686645, tenant_id:1026|[start_task] u=1 ret:0, sql_no:25769803782, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39EC501, start_time:1700554875554499|[end_task] u=148 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=10002144 snapshot:1700554902680204, uref:1073741825|[update_gts] u=1 ret:0, sql_no:30064771079, uref:1073741825, snapshot:1700554902680204, tenant_id:1026|[start_task] u=2 ret:0, sql_no:30064771079, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39EC501, start_time:1700554875554499|[end_task] u=371 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=161 snapshot:1700554902680204, uref:1073741825|[update_gts] u=0 ret:0, sql_no:34359738376, uref:1073741825, snapshot:1700554902680204, tenant_id:1026|[start_task] u=1 ret:0, sql_no:34359738376, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39EC501, start_time:1700554875554499|[end_task] u=190 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=9998350 snapshot:1700554912678924, uref:1073741825|[update_gts] u=1 ret:0, sql_no:38654705673, uref:1073741825, snapshot:1700554912678924, tenant_id:1026|[start_task] u=1 ret:0, sql_no:38654705673, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39EC501, start_time:1700554875554499|[end_task] u=173 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=68 snapshot:1700554912678924, uref:1073741825|[update_gts] u=0 ret:0, sql_no:42949672970, uref:1073741825, snapshot:1700554912678924, tenant_id:1026|[start_task] u=1 ret:0, sql_no:42949672970, uref:1073741825, need_update_gts:true, trace_id:YB420A7D0BEA-0006016FA39EC501, start_time:1700554875554499|[end_task] u=140 ret:0, is_rollback:true, uref:1073741825|[snapshot] u=10008358 |[update_gts] u=2 |[start_task] u=2 |[end_task] u=216 |[snapshot] u=83 |[update_gts] u=0 |[start_task] u=1 |[end_task] u=199 |[snapshot] u=9996792 |[update_gts] u=2 |[start_task] u=1 |[end_task] u=199 |[snapshot] u=72 |[update_gts] u=0 |[start_task] u=1 |[end_task] u=157 |[snapshot] u=10007215 |[update_gts] u=1 |[start_task] u=2 |[end_task] u=165 |[snapshot] u=79 |[update_gts] u=1 |[start_task] u=0 |[end_task] u=157 |[snapshot] u=10005659 |[update_gts] u=2 |[start_task] u=2 |[end_task] u=187 |[snapshot] u=95 |[update_gts] u=1:
2023-11-21 15:43:18.562644,obproxyssd1,obssd1:ict:ict_test,OB_MYSQL,QRTZ_LOCKS,qrtz_locks,OB_MYSQL_COM_QUERY,SELECT,failed,1317,SELECT * FROM QRTZ_LOCKS WHERE SCHED_NAME = ‘ictpmsReportQuartzScheduler’ AND LOCK_NAME = ‘TRIGGER_ACCESS’ FOR UPDATE,634672075us,489us,0us,347us,Y0-7FE62FCC5BE0,0,10.125.11.234:2881,Query execution was interrupted
2023-11-21 15:43:18.563907,obproxyssd1,obssd1:ict:ict_test,OB_MYSQL,QRTZ_LOCKS,qrtz_locks,OB_MYSQL_COM_QUERY,SELECT,failed,1317,SELECT * FROM QRTZ_LOCKS WHERE SCHED_NAME = ‘ictpmsReportQuartzScheduler’ AND LOCK_NAME = ‘TRIGGER_ACCESS’ FOR UPDATE,1101722244us,1804us,0us,407us,Y0-7FE603918140,0,10.125.11.234:2881,Query execution was interrupted
2023-11-21 15:43:18.564503,obproxyssd1,obssd1:ict:ict_test,OB_MYSQL,QRTZ_LOCKS,qrtz_locks,OB_MYSQL_COM_QUERY,SELECT,failed,1317,SELECT * FROM QRTZ_LOCKS WHERE SCHED_NAME = ‘ictpmsReportQuartzScheduler’ AND LOCK_NAME = ‘STATE_ACCESS’ FOR UPDATE,577251673us,413us,0us,321us,Y0-7FE605CF0240,0,10.125.11.234:2881,Query execution was interrupted
2023-11-21 15:43:18.564666,obproxyssd1,obssd1:ict:ict_test,OB_MYSQL,QRTZ_LOCKS,qrtz_locks,OB_MYSQL_COM_QUERY,SELECT,failed,1317,SELECT * FROM QRTZ_LOCKS WHERE SCHED_NAME = ‘ictpmsReportQuartzScheduler’ AND LOCK_NAME = ‘TRIGGER_ACCESS’ FOR UPDATE,607215557us,1841us,0us,501us,Y0-7FE605D22650,0,10.125.11.234:2881,Query execution was interrupted
2023-11-21 15:43:18.564697,obproxyssd1,obssd1:ict:ict_test,OB_MYSQL,QRTZ_LOCKS,qrtz_locks,OB_MYSQL_COM_QUERY,SELECT,failed,1317,SELECT * FROM QRTZ_LOCKS WHERE SCHED_NAME = ‘ictpmsReportQuartzScheduler’ AND LOCK_NAME = ‘TRIGGER_ACCESS’ FOR UPDATE,772051330us,1967us,0us,458us,Y0-7FE6038278D0,0,10.125.11.234:2881,Query execution was interrupted
2023-11-21 15:43:18.564757,obproxyssd1,obssd1:ict:ict_test,OB_MYSQL,QRTZ_LOCKS,qrtz_locks,OB_MYSQL_COM_QUERY,SELECT,failed,1317,SELECT * FROM QRTZ_LOCKS WHERE SCHED_NAME = ‘ictpmsReportQuartzScheduler’ AND LOCK_NAME = ‘STATE_ACCESS’ FOR UPDATE,632162108us,2284us,0us,471us,Y0-7FE603826CD0,0
proxy日志看 前端一直向数据库发送 select for update 行数查询，最终出现死锁。建议业务侧先确认下业务逻辑。
