obd单机部署服务启动失败

【 使用环境 】测试环境
【 OB or 其他组件 】ob
【 使用版本 】3.1.4
【问题描述】清晰明确描述问题
observer 服务单机版部署的,重启后一直卡在 observer init 环节,obd 日志也显示 init 字样,observer.log中有 ERROR partition table update task cost too much time to execute ;分区表更新任务执行过长错误。

[2022-09-15 18:34:22.418152] INFO [SERVER] ob_partition_table_updater.cpp:898 [1296][140][YB42AC119776-0005E8B19A5A9376] [lt=4]
[dc=0] batch update partition table fail, reput to queue(tasks.count()=122)
[2022-09-15 18:34:22.418215] ERROR [SERVER] check_task_status (ob_partition_table_updater.cpp:114) [1296][140][YB42AC119776-0005E
8B19A5A9377] [lt=7] [dc=0] partition table update task cost too much time to execute(*this={part_key:{tid:1100611139403779, parti
tion_id:0, part_cnt:0}, data_version:0, first_submit_time:1663224410560890, is_remove:false, with_role:true}, safe_interval=12000
0000, cost_time=13651857315, interval=1800000000) BACKTRACE:0x9a98e9e 0x986d141 0x22e9b7e 0x22e97bb 0x22e94e7 0x93f842b 0x936119c
0x9368127 0x9451863 0x944dd63 0x2ca95d4 0x2cabf02 0x9820da5 0x981f792 0x981c24f
[2022-09-15 18:34:22.418250] ERROR [SERVER] check_task_status (ob_partition_table_updater.cpp:114) [1296][140][YB42AC119776-0005E
8B19A5A9377] [lt=25] [dc=0] partition table update task cost too much time to execute(*this={part_key:{tid:1100611139403780, part
ition_id:0, part_cnt:0}, data_version:0, first_submit_time:1663224407328695, is_remove:false, with_role:true}, safe_interval=1200
00000, cost_time=13655089553, interval=1800000000) BACKTRACE:0x9a98e9e 0x986d141 0x22e9b7e 0x22e97bb 0x22e94e7 0x93f842b 0x936119
c 0x9368127 0x9451863 0x944dd63 0x2ca95d4 0x2cabf02 0x9820da5 0x981f792 0x981c24f

【复现路径】问题出现前后相关操作
【问题现象及影响】


正在处理:observer-log.png…

【附件】

列表条目

请问 yaml配置可以贴一下

Only need to configure when remote login is required

user:
username: root
password: 123456
key_file: /root/.ssh/id_rsa
port: 22

oceanbase-ce:
servers:
# Please don’t use hostname, only IP can be supported

  • 172.17.151.118
    global:

    The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.

    home_path: /home/admin/oceanbase/single
    data_dir: /oceanbase/single/data
    redo_dir: /oceanbase/single/redo
    devname: ens192
    mysql_port: 2881
    rpc_port: 2882
    zone: zone1

    The maximum running memory for an observer. When ignored, autodeploy calculates this value based on the current server available resource.

    memory_limit: 6G
    memory_limit_percentage: 80

    The reserved system memory. system_memory is reserved for general tenants. The default value is 30G. Autodeploy calculates this value based on the current server available resource.

    system_memory: 10G
    datafile_size: 100G

    The percentage of the data_dir space to the total disk space. This value takes effect only when datafile_size is 0. The default value is 90.

    datafile_disk_percentage: 90

    syslog_level: INFO

    Print system logs whose levels are higher than WARNING to a separate log file. The default value is true. The default value for autodeploy mode is false.

    enable_syslog_wf: false

    Enable auto system log recycling or not. The default value is false. The default value for autodeploy mode is on.

    enable_syslog_recycle: true

    The maximum number of reserved log files before enabling auto recycling. When set to 0, no logs are deleted. The default value for autodeploy mode is 4.

    max_syslog_file_count: 4

    Cluster name for OceanBase Database. The default value is obcluster. When you deploy OceanBase Database and obproxy, this value must be the same as the cluster_name for obproxy.

    appname: obcluster

    Password for root. The default value is empty.

    root_password:

    Password for proxyro. proxyro_password must be the same as observer_sys_password. The default value is empty.

    proxyro_password:

172.17.151.118:
syslog_level: INFO
enable_syslog_recycle: true
enable_syslog_wf: true
max_syslog_file_count: 4
system_memory: 4G
cpu_count: 16

同学,你可以试下用官方.yaml配置重新运行一下。

# Only need to configure when remote login is required
# user:
#   username: your username
#   password: your password if need
#   key_file: your ssh-key file path if need
#   port: your ssh port, default 22
#   timeout: ssh connection timeout (second), default 30
oceanbase-ce:
  servers:
    # Please don't use hostname, only IP can be supported
    - 172.17.151.118
  global:
    #  The working directory for OceanBase Database. OceanBase Database is started under this directory. This is a required field.
    home_path: /home/admin/oceanbase-ce/single
    # The directory for data storage. The default value is $home_path/store.
    data_dir: /oceanbase/single/data
    # The directory for clog, ilog, and slog. The default value is the same as the data_dir value.
    redo_dir: /oceanbase/single/redo
    # Please set devname as the network adaptor's name whose ip is  in the setting of severs.
    # if set severs as "127.0.0.1", please set devname as "lo"
    # if current ip is 192.168.1.10, and the ip's network adaptor's name is "eth0", please use "eth0"
    devname: ens192
    mysql_port: 2881 # External port for OceanBase Database. The default value is 2881.
    rpc_port: 2882 # Internal port for OceanBase Database. The default value is 2882.
    zone: zone1
    cluster_id: 1
    # please set memory limit to a suitable value which is matching resource.
    memory_limit: 8G # The maximum running memory for an observer
    system_memory: 3G # The reserved system memory. system_memory is reserved for general tenants. The default value is 30G.
    stack_size: 512K
    cpu_count: 16
    cache_wash_threshold: 1G
    __min_full_resource_pool_memory: 268435456
    workers_per_cpu_quota: 10
    schema_history_expire_time: 1d
    # The value of net_thread_count had better be same as cpu's core number.
    net_thread_count: 4
    major_freeze_duty_time: Disable
    minor_freeze_times: 10
    enable_separate_sys_clog: 0
    enable_merge_by_turn: FALSE
    datafile_disk_percentage: 90 # The percentage of the data_dir space to the total disk space. This value takes effect only when datafile_size is 0. The default value is 90.
    #datafile_size: 50G
    syslog_level: WARN # System log level. The default value is INFO.
    enable_syslog_wf: false # Print system logs whose levels are higher than WARNING to a separate log file. The default value is true.
    enable_syslog_recycle: true # Enable auto system log recycling or not. The default value is false.
    max_syslog_file_count: 10 # The maximum number of reserved log files before enabling auto recycling. The default value is 0.
    root_password: 123456  # root user password, can be empty
  • memory_limit: 6G
  • system_memory: 10G
    memory_limit要大于system_memory
1 个赞

所以这个 system_memory 不是保留给操作系统的内存,而是保留给后台系统线程的内存是吧

是的 内存模型可以参考 https://mp.weixin.qq.com/s/DYTOri_Cq6XWKOmZc5EeBg

1 个赞