oms 社区版3.2.2-bp1 安装改进

longda · 2022 年2 月 7 日 16:07

安装oms

改进意见:

oms 安装问题

1.1 /data/config.yaml: --》需要解释这个配置文件为在上文配置的文件, 需要重新设置准确路径, 另外备注/home/admin/conf/config.yaml 路径不用修改

1.2 需要备注: 提前把目录/data/oms/oms_logs /data/oms/oms_store /data/oms/oms_run 提前创建好

1.3 出现 “Unable to find image ‘reg.docker.alibaba-inc.com/oceanbase/oms:feature_3.2.2-ce’ locally”, 需要列出步骤, 怎么找到仓库中准确oms docker 的名字

1.4 把下面启动做成一个脚本, 方便用户进行修改和运行

OMS_HOST_IP=xxx CONTAINER_NAME=oms_xxx docker run -dit --net host \ -v /data/config.yaml:/home/admin/conf/config.yaml \ -v /data/oms/oms_logs:/home/admin/logs \ -v /data/oms/oms_store:/home/ds/store \ -v /data/oms/oms_run:/home/ds/run \ -e OMS_HOST_IP=${OMS_HOST_IP} \ --privileged=true \ --pids-limit -1 \ --ulimit nproc=65535:65535 \ --name ${CONTAINER_NAME} \ reg.docker.alibaba-inc.com/oceanbase/oms:feature_3.2.2-ce

longda · 2022 年2 月 7 日 16:09

执行

bash /root/docker_init.sh

出现

----------------------------------------------------------------------------------------------------------------------------------------------------------------
python -m omsflow.scripts.units.oms_cluster_config --check /home/admin/conf/config.yaml


/root/omsflow
Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/root/omsflow/scripts/units/oms_cluster_config.py", line 5, in <module>
    from .units import get_sqlalchemy_uri_from_dict
  File "omsflow/scripts/units/units.py", line 45, in <module>
    "Access-Token": OmsConfig.hnbc_access_token()
  File "/root/omsflow/next/config.py", line 15, in hnbc_access_token
    _ = yaml.load(fr, Loader=yaml.FullLoader)
  File "/usr/lib64/python2.7/site-packages/yaml/__init__.py", line 114, in load
    return loader.get_single_data()
  File "/usr/lib64/python2.7/site-packages/yaml/constructor.py", line 80, in get_single_data
    node = self.get_single_node()
  File "/usr/lib64/python2.7/site-packages/yaml/composer.py", line 36, in get_single_node
    document = self.compose_document()
  File "/usr/lib64/python2.7/site-packages/yaml/composer.py", line 55, in compose_document
    node = self.compose_node(None, None)
  File "/usr/lib64/python2.7/site-packages/yaml/composer.py", line 84, in compose_node
    node = self.compose_mapping_node(anchor)
  File "/usr/lib64/python2.7/site-packages/yaml/composer.py", line 127, in compose_mapping_node
    while not self.check_event(MappingEndEvent):
  File "/usr/lib64/python2.7/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File "/usr/lib64/python2.7/site-packages/yaml/parser.py", line 428, in parse_block_mapping_key
    if self.check_token(KeyToken):
  File "/usr/lib64/python2.7/site-packages/yaml/scanner.py", line 115, in check_token
    while self.need_more_tokens():
  File "/usr/lib64/python2.7/site-packages/yaml/scanner.py", line 152, in need_more_tokens
    self.stale_possible_simple_keys()
  File "/usr/lib64/python2.7/site-packages/yaml/scanner.py", line 292, in stale_possible_simple_keys
    "could not find expected ':'", self.get_mark())
yaml.scanner.ScannerError: while scanning a simple key
  in "/home/admin/conf/config.yaml", line 36, column 1
could not find expected ':'
  in "/home/admin/conf/config.yaml", line 37, column 1


----------------------------------------------------------------------------------------------------------------------------------------------------------------
# 【结束】初始化失败，当前命令: python -m omsflow.scripts.units.oms_cluster_config --check /home/admin/conf/config.yaml
----------------------------------------------------------------------------------------------------------------------------------------------------------------

longda · 2022 年2 月 7 日 16:13

建议 check yaml 的这个脚本放到启动docker 之前, 这样提前能检查yaml 文件是否合法

python -m omsflow.scripts.units.oms_cluster_config --check /home/admin/conf/config.yaml

另外, 这个/root/docker_init.sh 脚本增加日志, 让用户可以去什么地方查看日志

, 建议增加日志, 把检查和

longda · 2022 年2 月 7 日 16:25

这段配置文件第36行有问题, 需要增加注释

# OMS 社区版部署必填配置
# OMS 社区版元数据库信息，即创建的租户所在 OceanBase 集群对应的信息
# 您需要提前准备一个 OceanBase 数据库，用于存放 OMS 社区版的元信息
oms_meta_host: ${oms_meta_host}
oms_meta_port: ${oms_meta_port}
oms_meta_user: ${oms_meta_user}
oms_meta_password: ${oms_meta_password}


# 用户可以自定义以下三个数据库的名称，OMS 社区版部署时会在元信息库中创建出这三个数据库
drc_rm_db: ${drc_rm_db}
drc_cm_db: ${drc_cm_db}
drc_cm_heartbeat_db: ${drc_cm_heartbeat_db}


# OceanBase 社区版增量用户
# 当需要从 OceanBase 社区版消费增量日志时，请在 sys 租户下创建用户
# drc user 需要在待迁移的 OceanBase 集群 sys 租户下创建，然后在 OMS 社区版 yaml 文件中指定
drc_user: ${drc_user}
drc_password: ${drc_passwrord}


# OMS 社区版集群配置
# 通常配置为当前 OMS 社区版机器 IP
cm_url: ${cm_url}
cm_location: ${cm_location}
cm_region: ${cm_region}
cm_region_cn: ${cm_region_cn}
cm_is_default: true
cm_nodes:
 - ${cm_nodes}


# 下述参数均为可选参数
# 时序数据库配置
tsdb_service: 'INFLUXDB'
tsdb_enabled: false


# 部署 InfluxDB 的机器地址
当 tsdb_enabled 为 true 时，请根据实际环境修改下述参数
tsdb_url: ${tsdb_url}
tsdb_username: ${tsdb_user}
tsdb_password: ${tsdb_password}

longda · 2022 年2 月 7 日 16:35

2022-02-07 16:30:58,505-urllib3.connectionpool-DEBUG connectionpool._new_conn.231 :Starting new HTTP connection (1): 8.134.89.241:8088
^@^@Traceback (most recent call last):
  File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/root/omsflow/scripts/units/oms_cluster_manager.py", line 195, in <module>
    main()
  File "/root/omsflow/scripts/units/oms_cluster_manager.py", line 191, in main
    o.add_resource()
  File "/root/omsflow/scripts/units/oms_cluster_manager.py", line 174, in add_resource
    self.add_resource_nodes(self.role)
  File "/root/omsflow/scripts/units/oms_cluster_manager.py", line 166, in add_resource_nodes
    self.add_resource_node(cm_endpoint, node)
  File "/root/omsflow/scripts/units/oms_cluster_manager.py", line 43, in add_resource_node
    c = requests.post(url, data=data)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 117, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='8.134.89.241', port=8088): Max retries exceeded with url: /resource/host/add (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f63cd5fc790>: Failed to establish a new connection: [Errno 110] Connection timed out',))


----------------------------------------------------------------------------------------------------------------------------------------------------------------
# 【结束】初始化失败，当前命令: python -m omsflow.scripts.units.oms_cluster_manager add_resource

longda · 2022 年2 月 7 日 16:40

执行 bash /root/docker_init.sh需要增加一个说明, 当前执行路径需要在/root 目录下,

另外,这个脚本比较简单, 也可以用python 重写这个脚本, 方便打日志, 或者进行重试, shell 脚本调试性, 也不好打日志

longda · 2022 年2 月 7 日 17:12

网络问题，把圈住的地方改成内网的ip就好了，原来是公网ip访问不通。

这里修改配置注释, 建议用户使用内网ip 非公网ip

longda · 2022 年2 月 7 日 18:37

需要在文档里面补充说明

添加ob 数据源需要增加说明, ocp 只可以连接obproxy, 不允许直连observer

https://open.oceanbase.com/docs/oceanbase-migration-service/oms-community-edition/V3.2.2/add-oceanbase-community-edition-data-source

longda · 2022 年2 月 7 日 18:50

从ob 导出到mysql, 需要sys 租户创建在启动oms 时的配置文件中

# OceanBase 社区版增量用户
# 当需要从 OceanBase 社区版消费增量日志时，请在 sys 租户下创建用户
# drc user 需要在待迁移的 OceanBase 集群 sys 租户下创建，然后在 OMS 社区版 yaml 文件中指定
drc_user: ${drc_user}
drc_password: ${drc_passwrord}

需要在文档中 https://open.oceanbase.com/docs/oceanbase-migration-service/oms-community-edition/V3.2.2/user-permission-settings 增加一个说明

longda · 2022 年2 月 7 日 18:51

第10楼解决当前这个问题(oms 增量同步到mysql)