log sync timeout on leader经常日志刷屏,导致数据库无法正常使用
请问一下这是什么原因导致的
看下clog盘是不是满了
预留空间是够的,不是一直卡死,有时半个小时,有时好几个小时。
确认下你这个redo的空间还会上涨么?另外提供下你对应的租户配置,看下租户内存会不会爆
select * from __all_unit_config; 看下
你可以去clog下面看下哪个租户对应的日志比较大
1010租户是dev_tenant吗 ?
select now(),t.tenant_name, m.svr_ip, m.active_span / 1073741824 active_G, m.memstore_used / 1073741824 total_G, m.freeze_trigger / 1073741824 trigger_G, m.memstore_limit / 1073741824 limit_G, (m.memstore_used / m.memstore_limit) * 100 used_rate from __all_virtual_tenant_memstore_info m, __all_tenant t where t.tenant_id = m.tenant_id order by used_rate desc;
这个租户你在做什么操作
21.11 这个机器上的memstore 一直这么高吗
嗯,有大数据插入与处理。
原始日志太大,截取了两个 “log sync timeout on leader” 之间的内容(附件 observer_timeout.log):
内容似乎多次出现 “end_lsn is larger than max reuse pos”
observer_timeout.log (535.0 KB)
租户下执行下 show parameters like ‘%writing_throttling_trigger_percentage%’; 看下写入限速是否打开了
mysql> show parameters like '%writing_throttling_trigger_percentage%';
+-------+----------+---------------+----------+---------------------------------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------+---------+--------+---------+-------------------+
| zone | svr_type | svr_ip | svr_port | name | data_type | value | info | section | scope | source | edit_level |
+-------+----------+---------------+----------+---------------------------------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------+---------+--------+---------+-------------------+
| zone1 | observer | 192.168.21.11 | 2882 | writing_throttling_trigger_percentage | NULL | 80 | the threshold of the size of the mem store when writing_limit will be triggered. Rang:(0,100]. setting 100 means turn off writing limit | TRANS | TENANT | DEFAULT | DYNAMIC_EFFECTIVE |
| zone2 | observer | 192.168.21.12 | 2882 | writing_throttling_trigger_percentage | NULL | 80 | the threshold of the size of the mem store when writing_limit will be triggered. Rang:(0,100]. setting 100 means turn off writing limit | TRANS | TENANT | DEFAULT | DYNAMIC_EFFECTIVE |
| zone3 | observer | 192.168.21.13 | 2882 | writing_throttling_trigger_percentage | NULL | 80 | the threshold of the size of the mem store when writing_limit will be triggered. Rang:(0,100]. setting 100 means turn off writing limit | TRANS | TENANT | DEFAULT | DYNAMIC_EFFECTIVE |
+-------+----------+---------------+----------+---------------------------------------+-----------+-------+-------------------------------------------------------------------------------------------------------------------------------------------+---------+--------+---------+-------------------+
调大下这个参数 compaction_mid_thread_score 试下
默认是 0 ,调多少合适?
改成10 试下
收到。谢谢~