obd cluster start 一直卡在Initialize oceanbase-ce阶段

yufei · 2024 年11 月 24 日 13:09

【使用环境】银河麒麟v10 ARM架构
【使用版本】ARM 社区版 4.2.1 或社区版4.3.4
【问题描述】
我使用 obd cluster start 本地单机部署一直卡在Initialize oceanbase-ce阶段
obd cluster deploy mydb -c mini-local-example.yaml
obd cluster start mydb

配置文件使用mini-local-example.yaml 未修改
按官网文档配置了 limits.conf、sysctl.conf

【日志】
OBD-5000: after system bootstrap REGION “sys_region” ZONE zone1 SERVER “127.0.0.1:2882” execute failed
[DEBUG] – execute sql: alter user “root” IDENTIFIED BY %s. args: [‘xxxxxxx’]

banjin · 2024 年11 月 24 日 20:40

看下资源还多吗，资源少了很容易卡在这些地方

旭辉 · 2024 年11 月 25 日 10:36

麻烦发下obd.log，observer.log，另外不建议使用mini模式

yufei · 2024 年11 月 26 日 10:40

logs.tar.gz (9.0 MB)
这是从安装到启动的日志

旭辉 · 2024 年11 月 26 日 15:13

[2024-11-25 15:29:11.017873] WDIAG [RPC] send (ob_poc_rpc_proxy.h:170) [33277][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=4][errcode=-4224] execute rpc fail(addr="127.0.0.1:2882", pcode=264, ret=-4224, timeout=984705651)

[2024-11-25 15:29:11.017892] WDIAG log_user_error_and_warn (ob_poc_rpc_proxy.cpp:244) [33277][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=16][errcode=-4224] Out of range value for column 'sval1' at row 1

[2024-11-25 15:29:11.017904] ERROR issue_dba_error (ob_log.cpp:1875) [33277][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=5][errcode=-4388] Unexpected internal error happen, please checkout the internal errcode(errcode=-4224, file="ob_service.cpp", line_no=1467, info="execute bootstrap fail")

[2024-11-25 15:29:11.017914] EDIAG [BOOTSTRAP] bootstrap (ob_service.cpp:1467) [33277][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=10][errcode=-4224] execute bootstrap fail(ret=-4224, ret="OB_DATA_OUT_OF_RANGE", rpc_svr="0.0.0.0:0", master_rs="127.0.0.1:2882", rpc_timeout=984705651) BACKTRACE:0x107e424c 0xac68124 0xabf7804 0xabf73ec 0xabf731c 0xabf71b8 0x8e7a3d0 0x8e793a0 0x862e09c 0xadb2b94 0xa94be6c 0xa93631c 0x823a5d0 0x10ddfba0 0x4000211a878c 0x4000212b508c

[2024-11-25 15:29:11.018078] WDIAG [RPC] send (ob_poc_rpc_proxy.h:170) [33283][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=19][errcode=-4224] execute rpc fail(addr="127.0.0.1:2882", pcode=257, ret=-4224, timeout=999999800)

[2024-11-25 15:29:11.018098] WDIAG log_user_error_and_warn (ob_poc_rpc_proxy.cpp:244) [33283][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=18][errcode=-4224] Out of range value for column 'sval1' at row 1

[2024-11-25 15:29:11.018112] WDIAG [SQL.ENG] execute (ob_alter_system_executor.cpp:1670) [33283][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=6][errcode=-4224] rpc proxy bootstrap failed(ret=-4224, rpc_timeout=999999800)

[2024-11-25 15:29:11.018119] WDIAG [BOOTSTRAP] execute (ob_alter_system_executor.cpp:1671) [33283][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=6][errcode=-4224] STEP_0.1:alter_system execute fail

[2024-11-25 15:29:11.018184] INFO [SHARE] add_event (ob_event_history_table_operator.h:295) [33283][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=5] event table add task(ret=0, event_table_name="__all_server_event_history", sql=INSERT INTO __all_server_event_history (gmt_create, module, event, name1, value1, name2, value2, name3, value3, name4, value4, svr_ip, svr_port) VALUES (usec_to_time(1732519751018126), 'sql', 'execute_cmd', 'cmd_type', 77, 'sql_text', X'616C7465722073797374656D20626F6F74737472617020524547494F4E20227379735F726567696F6E22205A4F4E4520227A6F6E6531222053455256455220223132372E302E302E313A3238383222', 'return_code', -4224, 'tenant_id', 1, '127.0.0.1', 2882))

[2024-11-25 15:29:11.018206] WDIAG [SQL] open_cmd (ob_result_set.cpp:99) [33283][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=19][errcode=-4224] execute cmd failed(ret=-4224)

[2024-11-25 15:29:11.018213] WDIAG [SQL] open (ob_result_set.cpp:158) [33283][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=5][errcode=-4224] execute plan failed(ret=-4224)

[2024-11-25 15:29:11.018223] WDIAG [SERVER] response_result (ob_sync_cmd_driver.cpp:145) [33283][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=8][errcode=-4224] close result set fail(cret=-4224)

[2024-11-25 15:29:11.018265] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:947) [33283][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=39][errcode=-4224] [RETRY] check if need retry(v={force_local_retry:false, stmt_retry_times:0, local_retry_times:0, err_:-4224, err_:"OB_DATA_OUT_OF_RANGE", retry_type:0, client_ret:-4224}, need_retry=false, THIS_WORKER.can_retry()=true, v.ctx_.multi_stmt_item_={is_part_of_multi_stmt:false, seq_num:0, sql:"alter system bootstrap REGION "sys_region" ZONE "zone1" SERVER "127.0.0.1:2882"", batched_queries:NULL, is_ps_mode:false, ab_cnt:0})

[2024-11-25 15:29:11.018279] WDIAG [SERVER] response_result (ob_sync_cmd_driver.cpp:151) [33283][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=13][errcode=-4224] result set open failed, check if need retry(ret=-4224, cli_ret=-4224, retry_ctrl_.need_retry()=0)

[2024-11-25 15:29:11.018308] INFO [SERVER] send_error_packet (obmp_packet_sender.cpp:319) [33283][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=5] sending error packet(ob_error=-4224, client error=1264, extra_err_info=NULL, lbt()="0x107e424c 0x83c80b8 0x83791f8 0xab5d8e0 0xa95e9c8 0xa952f70 0xa9479b0 0xa93631c 0x823a5d0 0x10ddfba0 0x4000211a878c 0x4000212b508c")

[2024-11-25 15:29:11.018327] WDIAG [SERVER] do_process (obmp_query.cpp:808) [33283][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=5][errcode=-4224] execute query fail(ret=-4224, timeout_timestamp=1732520698252089)

[2024-11-25 15:29:11.018383] TRACE [TRACE] after_process (obmp_base.cpp:145) [33283][T1_L0_G0][T1][YB427F000001-000627B7AE249FBC-0-0] [lt=5] [slow query](TRACE=begin_ts=1732519698252111 2024-11-25 07:28:18.252111|[process_begin] u=0 in_queue_time:21, receive_ts:1732519698252089, enqueue_ts:1732519698252091|[start_sql] u=1 addr:{ip:"127.0.0.1", port:37706}|[query_begin] u=1 trace_id:YB427F000001-000627B7AE249FBC-0-0|[before_processor_run] u=8 |[session] u=2 sid:3221225473, tenant_id:1|[parse_begin] u=32 stmt:"alter system bootstrap REGION "sys_region" ZONE "zone1" SERVER "127.0.0.1:2882"", stmt_len:79|[transform_with_outline_begin] u=56 |[transform_with_outline_end] u=2 |[resolve_begin] u=4 |[resolve_end] u=26 |[session] u=52766074 sid:3221225473, tenant_id:1|[get_location_cache_begin] u=44 |[query_end] u=9 |[session] u=1 sid:3221225473, tenant_id:1|[process_end] u=10 run_ts:1732519698252121|total_timeu=52766270)

[2024-11-25 15:29:12.061152] INFO [MDS] try_gc_mds_table (mds_table_handler.cpp:128) [33097][T1_Occam][T1][YB427F000001-000627B7ADA49FE9-0-0] [lt=13] [GC]success to gc mds_table(ret=0, ret="OB_SUCCESS", valid_node_cnt=0, *this={mds_table_handle:{p_mds_table_base:{ctrl_ptr:{ref:1, p_data_block:{this:0x400089216270, ls_id:{id:1}, tablet_id:{id:115}, flushing_scn:{val:18446744073709551615, v:3}, rec_scn:{val:4611686018427387903, v:0}, last_inner_recycled_scn:{val:1732519747114370132, v:0}, total_node_cnt:0, construct_sequence:64, debug_info:{do_init_tablet_pointer:0x40008f1b0688, do_remove_tablet_pointer:null, init_ts:"2024-11-25 15:29:03.934286", last_reset_ts:"1970-01-01 08:00:00.0", remove_ts:"1970-01-01 08:00:00.0", last_flush_ts:"2024-11-25 15:29:07.261962", switch_to_empty_shell_ts:"1970-01-01 08:00:00.0", init_trace_id:YB427F000001-000627B7AE249FBC-0-0, remove_trace_id:Y0-0000000000000000-0-0}}}}, mds_table_id:1}})

淇铭 · 2024 年11 月 26 日 15:33

把配置 mini-local-example.yaml发一下

yufei · 2024 年11 月 26 日 16:13

默认的配置

旭辉 · 2024 年11 月 26 日 16:17

/root/observer/bin/observer -r 127.0.0.1:2882:2881 -p 2881 -P 2882 -z zone1 -n mydb -c 1 -d /root/observer/store -I 127.0.0.1 -o __min_full_resource_pool_memory=2147483648,memory_limit=6G,system_memory=1G,datafile_size=2G,datafile_next=2G,datafile_maxsize=20G,log_disk_size=14G,cpu_count=16,enable_syslog_wf=False,max_syslog_file_count=4
observer (OceanBase_CE 4.2.1.9)

分析中