创建租户失败:ERROR 4009 (58030): IO error

我从obd白屏上面创建租户的日志打出来:
############{EXECUTE}{2024-02-02T11:40:23.624+08:00}############2024-02-02 11:40:23.628 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.t.engine.runner.JavaSubtaskRunner : Run subtask, id=12846, context=Context{parallelIdx=-1, stringMap={tenant_name=monitor, ob_tenant_parameter_map=, prohibit_rollback=false, task_instance_id=12780, task_operation=execute, whitelist=%, target_tenant_status=NORMAL, old_password=, new_password=, tenant_mode=MYSQL, system_variable_map=, create_tenant_param_json={“charset”:“utf8mb4”,“collation”:“utf8mb4_general_ci”,“mode”:“MYSQL”,“name”:“monitor”,“parameters”:[],“primaryZone”:"",“rootPassword”:"",“whitelist”:"%",“zones”:[{“name”:“zone1”,“replicaType”:“FULL”,“resourcePool”:{“unitCount”:1,“unitSpec”:{“cpuCore”:3.00,“memoryBytes”:2147483648,“memorySize”:2}}},{“name”:“zone2”,“replicaType”:“FULL”,“resourcePool”:{“unitCount”:1,“unitSpec”:{“cpuCore”:3.00,“memoryBytes”:2147483648,“memorySize”:2}}},{“name”:“zone3”,“replicaType”:“FULL”,“resourcePool”:{“unitCount”:1,“unitSpec”:{“cpuCore”:3.00,“memoryBytes”:2147483648,“memorySize”:2}}}]}, latest_execution_start_time=2024-02-02T11:40:23.617+08:00, sub_task_instance_id=12846}, listMap={}}, executor=11.162.218.226
2024-02-02 11:40:23.631 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.o.i.tenant.task.CreateTenantTask : begin create tenant, param=CreateTenantParam(name=monitor, mode=MYSQL, primaryZone=, charset=utf8mb4, collation=utf8mb4_general_ci, description=null, whitelist=%, timeZone=null, rootPassword=
, zones=[CreateTenantParam.ZoneParam(name=zone1, replicaType=FULL, resourcePool=CreateTenantParam.PoolParam(unitSpec=UnitSpecParam(cpuCore=3.0, memorySize=2), unitCount=1)), CreateTenantParam.ZoneParam(name=zone2, replicaType=FULL, resourcePool=CreateTenantParam.PoolParam(unitSpec=UnitSpecParam(cpuCore=3.0, memorySize=2), unitCount=1)), CreateTenantParam.ZoneParam(name=zone3, replicaType=FULL, resourcePool=CreateTenantParam.PoolParam(unitSpec=UnitSpecParam(cpuCore=3.0, memorySize=2), unitCount=1))], parameters=[])
2024-02-02 11:40:23.635 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.c.obsdk.ConnectPropertiesBuilder : build connect address for sys tenant, obServerAddrList=[ObServerAddr(address=11.162.218.35, port=2881), ObServerAddr(address=11.162.218.226, port=2881), ObServerAddr(address=11.162.218.58, port=2881)]
2024-02-02 11:40:23.639 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=null, obClusterId=null, tenantName=sys, username=root, address=11.162.218.35, port=2881, database=oceanbase)
2024-02-02 11:40:23.642 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:11.162.218.226, sql port:2881
2024-02-02 11:40:23.644 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=null, obClusterId=null, tenantName=sys, username=root, address=11.162.218.35, port=2881, database=oceanbase)
2024-02-02 11:40:23.648 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:11.162.218.226, sql port:2881
2024-02-02 11:40:23.651 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000]
2024-02-02 11:40:23.654 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = ‘min_observer_version’
2024-02-02 11:40:23.660 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=null, obClusterId=null, tenantName=sys, username=root, address=11.162.218.35, port=2881, database=oceanbase)
2024-02-02 11:40:23.663 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:11.162.218.226, sql port:2881
2024-02-02 11:40:23.666 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000]
2024-02-02 11:40:23.670 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: CREATE RESOURCE UNIT config_monitor_zone2_3C2G_ksv MAX_CPU = ?, MIN_CPU = ?, MEMORY_SIZE = ?, args: [3.0, 3.0, 2147483648]
2024-02-02 11:40:23.680 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=null, obClusterId=null, tenantName=sys, username=root, address=11.162.218.35, port=2881, database=oceanbase)
2024-02-02 11:40:23.684 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:11.162.218.226, sql port:2881
2024-02-02 11:40:23.687 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000]
2024-02-02 11:40:23.691 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT UNIT_CONFIG_ID, NAME, MAX_CPU, MIN_CPU, MEMORY_SIZE AS MAX_MEMORY, MEMORY_SIZE AS MIN_MEMORY, LOG_DISK_SIZE, MAX_IOPS, MIN_IOPS, IOPS_WEIGHT FROM oceanbase.DBA_OB_UNIT_CONFIGS WHERE NAME = ?, args: [config_monitor_zone2_3C2G_ksv]
2024-02-02 11:40:23.701 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=null, obClusterId=null, tenantName=sys, username=root, address=11.162.218.35, port=2881, database=oceanbase)
2024-02-02 11:40:23.705 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:11.162.218.226, sql port:2881
2024-02-02 11:40:23.708 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000]
2024-02-02 11:40:23.711 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: CREATE RESOURCE POOL pool_monitor_zone2_ksv UNIT = ?, UNIT_NUM = ?, ZONE_LIST=(‘zone2’), args: [config_monitor_zone2_3C2G_ksv, 1]
2024-02-02 11:40:23.721 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=null, obClusterId=null, tenantName=sys, username=root, address=11.162.218.35, port=2881, database=oceanbase)
2024-02-02 11:40:23.725 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:11.162.218.226, sql port:2881
2024-02-02 11:40:23.728 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000]
2024-02-02 11:40:23.732 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: SELECT /*+ QUERY_TIMEOUT(60000000) */ time_to_usec(t1.MODIFY_TIME) AS UPDATE_TIME, t1.RESOURCE_POOL_ID, t1.NAME, t1.UNIT_COUNT, t1.UNIT_CONFIG_ID, t1.ZONE_LIST, t1.TENANT_ID, t1.REPLICA_TYPE, t2.NAME AS UNIT_CONFIG_NAME, t2.MAX_CPU, t2.MIN_CPU, t2.MEMORY_SIZE AS MAX_MEMORY, t2.MEMORY_SIZE AS MIN_MEMORY, t2.MAX_IOPS, t2.MIN_IOPS FROM oceanbase.DBA_OB_RESOURCE_POOLS AS t1 JOIN oceanbase.DBA_OB_UNIT_CONFIGS AS t2 ON t1.UNIT_CONFIG_ID = t2.UNIT_CONFIG_ID WHERE t1.name = ?, args: [pool_monitor_zone2_ksv]
2024-02-02 11:40:23.747 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.o.i.tenant.task.CreateTenantTask : create resource pool success, resourcePoolList=[ResourcePool(id=1035, name=pool_monitor_zone1_sai, unitCount=1, unitConfig=UnitConfig(maxCpuCoreCount=3.0, minCpuCoreCount=3.0, maxMemoryByte=2147483648, minMemoryByte=2147483648, maxDiskSizeByte=null, maxDiskSize=null, maxIops=null, minIops=null, maxSessionNum=null, name=config_monitor_zone1_3C2G_sai), zoneList=[zone1]), ResourcePool(id=1034, name=pool_monitor_zone2_ksv, unitCount=1, unitConfig=UnitConfig(maxCpuCoreCount=3.0, minCpuCoreCount=3.0, maxMemoryByte=2147483648, minMemoryByte=2147483648, maxDiskSizeByte=null, maxDiskSize=null, maxIops=null, minIops=null, maxSessionNum=null, name=config_monitor_zone2_3C2G_ksv), zoneList=[zone2]), ResourcePool(id=1036, name=pool_monitor_zone3_rip, unitCount=1, unitConfig=UnitConfig(maxCpuCoreCount=3.0, minCpuCoreCount=3.0, maxMemoryByte=2147483648, minMemoryByte=2147483648, maxDiskSizeByte=null, maxDiskSize=null, maxIops=null, minIops=null, maxSessionNum=null, name=config_monitor_zone3_3C2G_rip), zoneList=[zone3])]
2024-02-02 11:40:23.751 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.c.obsdk.ConnectPropertiesBuilder : get credential from obsdk context, clusterName=test2024_lzh_ob, tenantName=sys, dbUser=root
2024-02-02 11:40:23.754 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.c.obsdk.ConnectPropertiesBuilder : build connect address for sys tenant, obServerAddrList=[ObServerAddr(address=11.162.218.35, port=2881), ObServerAddr(address=11.162.218.226, port=2881), ObServerAddr(address=11.162.218.58, port=2881)]
2024-02-02 11:40:23.757 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=null, obClusterId=null, tenantName=sys, username=root, address=11.162.218.35, port=2881, database=oceanbase)
2024-02-02 11:40:23.760 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:11.162.218.226, sql port:2881
2024-02-02 11:40:23.763 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=null, obClusterId=null, tenantName=sys, username=root, address=11.162.218.35, port=2881, database=oceanbase)
2024-02-02 11:40:23.765 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:11.162.218.226, sql port:2881
2024-02-02 11:40:23.768 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000]
2024-02-02 11:40:23.771 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = ‘min_observer_version’
2024-02-02 11:40:23.777 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=null, obClusterId=null, tenantName=sys, username=root, address=11.162.218.35, port=2881, database=oceanbase)
2024-02-02 11:40:23.781 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:11.162.218.226, sql port:2881
2024-02-02 11:40:23.783 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000]
2024-02-02 11:40:23.786 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: CREATE TENANT monitor resource_pool_list=(‘pool_monitor_zone1_sai’,‘pool_monitor_zone2_ksv’,‘pool_monitor_zone3_rip’), LOCALITY = ?, CHARSET = ?, COLLATE = ? SET ob_tcp_invited_nodes=’%’, ob_compatibility_mode = ?, args: [FULL@zone1,FULL@zone2,FULL@zone3, utf8mb4, utf8mb4_general_ci, mysql]
2024-02-02 11:40:24.760 ERROR 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] update failed, sql:[CREATE TENANT monitor resource_pool_list=(‘pool_monitor_zone1_sai’,‘pool_monitor_zone2_ksv’,‘pool_monitor_zone3_rip’), LOCALITY = ?, CHARSET = ?, COLLATE = ? SET ob_tcp_invited_nodes=’%’, ob_compatibility_mode = ?], error message:[PreparedStatementCallback; SQL [CREATE TENANT monitor resource_pool_list=(‘pool_monitor_zone1_sai’,‘pool_monitor_zone2_ksv’,‘pool_monitor_zone3_rip’), LOCALITY = ?, CHARSET = ?, COLLATE = ? SET ob_tcp_invited_nodes=’%’, ob_compatibility_mode = ?]; (conn=3221699732) IO error; nested exception is java.sql.SQLTransientConnectionException: (conn=3221699732) IO error]
2024-02-02 11:40:24.765 WARN 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.o.task.engine.runner.RunnerFactory : Execute task failed, subtask=SubtaskInstanceOverview{id=12846, name=Create ob tenant, state=FAILED, operation=EXECUTE, className=com.oceanbase.ocp.obops.internal.tenant.task.CreateTenantTask, seriesId=1, startTime=2024-02-02T11:40:23.618+08:00, endTime=null}

com.oceanbase.ocp.obsdk.exception.OceanBaseException: (conn=3221699732) IO error
at com.oceanbase.ocp.obsdk.connector.ConnectTemplate.customizeException(ConnectTemplate.java:342) ~[ocp-express-obsdk-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.obsdk.connector.ConnectTemplate.customizeException(ConnectTemplate.java:334) ~[ocp-express-obsdk-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.obsdk.connector.ConnectTemplate.customizeException(ConnectTemplate.java:329) ~[ocp-express-obsdk-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.obsdk.connector.ConnectTemplate.updateInner(ConnectTemplate.java:207) ~[ocp-express-obsdk-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.obsdk.connector.ConnectTemplate.update(ConnectTemplate.java:185) ~[ocp-express-obsdk-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.obsdk.operator.tenant.MysqlTenantOperator.createTenant(MysqlTenantOperator.java:119) ~[ocp-express-obsdk-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.obops.internal.tenant.TenantOperationServiceImpl.createTenantOnResourcePool(TenantOperationServiceImpl.java:169) ~[ocp-express-obops-service-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.obops.internal.tenant.TenantOperationServiceImpl.createTenantOnResourcePool(TenantOperationServiceImpl.java:148) ~[ocp-express-obops-service-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.obops.internal.tenant.task.CreateTenantTask.run(CreateTenantTask.java:72) ~[ocp-express-obops-service-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.task.engine.runner.JavaSubtaskRunner.execute(JavaSubtaskRunner.java:63) ~[ocp-express-task-service-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.task.engine.runner.JavaSubtaskRunner.doRun(JavaSubtaskRunner.java:35) ~[ocp-express-task-service-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.task.engine.runner.JavaSubtaskRunner.run(JavaSubtaskRunner.java:29) ~[ocp-express-task-service-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.task.engine.runner.RunnerFactory.doRun(RunnerFactory.java:114) ~[ocp-express-task-service-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.task.engine.runner.RunnerFactory.redirectOutputIfNotSysSchedule(RunnerFactory.java:185) ~[ocp-express-task-service-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.task.engine.runner.RunnerFactory.run(RunnerFactory.java:104) ~[ocp-express-task-service-4.2.1-20231116.jar!/:4.2.1-20231116]
at com.oceanbase.ocp.task.engine.coordinator.worker.subtask.ReadySubtaskWorker.lambda$null$1(ReadySubtaskWorker.java:112) ~[ocp-express-task-service-4.2.1-20231116.jar!/:4.2.1-20231116]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_252]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_252]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_252]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_252]
Caused by: java.sql.SQLTransientConnectionException: (conn=3221699732) IO error
at com.oceanbase.jdbc.internal.util.exceptions.ExceptionFactory.createException(ExceptionFactory.java:128) ~[oceanbase-client-2.4.3.jar!/:na]
at com.oceanbase.jdbc.internal.util.exceptions.ExceptionFactory.create(ExceptionFactory.java:202) ~[oceanbase-client-2.4.3.jar!/:na]
at com.oceanbase.jdbc.OceanBaseStatement.executeExceptionEpilogue(OceanBaseStatement.java:315) ~[oceanbase-client-2.4.3.jar!/:na]
at com.oceanbase.jdbc.JDBC4PreparedStatement.executeInternal(JDBC4PreparedStatement.java:243) ~[oceanbase-client-2.4.3.jar!/:na]
at com.oceanbase.jdbc.JDBC4PreparedStatement.execute(JDBC4PreparedStatement.java:161) ~[oceanbase-client-2.4.3.jar!/:na]
at com.oceanbase.jdbc.JDBC4PreparedStatement.executeUpdate(JDBC4PreparedStatement.java:195) ~[oceanbase-client-2.4.3.jar!/:na]
at com.alibaba.druid.pool.DruidPooledPreparedStatement.executeUpdate(DruidPooledPreparedStatement.java:255) ~[druid-1.2.8.jar!/:1.2.8]
at org.springframework.jdbc.core.JdbcTemplate.lambda$update$2(JdbcTemplate.java:965) ~[spring-jdbc-5.3.27.jar!/:5.3.27]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:651) ~[spring-jdbc-5.3.27.jar!/:5.3.27]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:960) ~[spring-jdbc-5.3.27.jar!/:5.3.27]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:1015) ~[spring-jdbc-5.3.27.jar!/:5.3.27]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:1025) ~[spring-jdbc-5.3.27.jar!/:5.3.27]
at com.oceanbase.ocp.obsdk.connector.ConnectTemplate.updateInner(ConnectTemplate.java:204) ~[ocp-express-obsdk-4.2.1-20231116.jar!/:4.2.1-20231116]
… 16 common frames omitted
Caused by: com.oceanbase.jdbc.internal.util.exceptions.OceanBaseSqlException: IO error
at com.oceanbase.jdbc.internal.util.exceptions.OceanBaseSqlException.of(OceanBaseSqlException.java:79) ~[oceanbase-client-2.4.3.jar!/:na]
at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.exceptionWithQuery(AbstractQueryProtocol.java:199) ~[oceanbase-client-2.4.3.jar!/:na]
at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.exceptionWithQuery(AbstractQueryProtocol.java:180) ~[oceanbase-client-2.4.3.jar!/:na]
at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:372) ~[oceanbase-client-2.4.3.jar!/:na]
at com.oceanbase.jdbc.JDBC4PreparedStatement.executeInternal(JDBC4PreparedStatement.java:234) ~[oceanbase-client-2.4.3.jar!/:na]
… 25 common frames omitted
Caused by: java.sql.SQLException: IO error
at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.readErrorPacket(AbstractQueryProtocol.java:2192) ~[oceanbase-client-2.4.3.jar!/:na]
at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.readPacket(AbstractQueryProtocol.java:2057) ~[oceanbase-client-2.4.3.jar!/:na]
at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:1951) ~[oceanbase-client-2.4.3.jar!/:na]
at com.oceanbase.jdbc.internal.protocol.AbstractQueryProtocol.executeQuery(AbstractQueryProtocol.java:370) ~[oceanbase-client-2.4.3.jar!/:na]
… 26 common frames omitted

Set state for subtask: 12846, operation:EXECUTE, state: FAILED############{ROLLBACK}{2024-02-02T11:44:53.167+08:00}############2024-02-02 11:44:53.170 WARN 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.o.t.engine.runner.JavaSubtaskRunner : Rollback subtask, id=12846, context=Context{parallelIdx=-1, stringMap={tenant_name=monitor, ob_tenant_parameter_map=, prohibit_rollback=false, task_instance_id=12780, task_operation=rollback, whitelist=%, target_tenant_status=NORMAL, resource_pool_list_json=[{“id”:1035,“name”:“pool_monitor_zone1_sai”,“unitConfig”:{“maxCpuCoreCount”:3.00,“maxMemoryByte”:2147483648,“maxMemorySize”:2,“minCpuCoreCount”:3.00,“minMemoryByte”:2147483648,“minMemorySize”:2},“unitCount”:1},{“id”:1034,“name”:“pool_monitor_zone2_ksv”,“unitConfig”:{“maxCpuCoreCount”:3.00,“maxMemoryByte”:2147483648,“maxMemorySize”:2,“minCpuCoreCount”:3.00,“minMemoryByte”:2147483648,“minMemorySize”:2},“unitCount”:1},{“id”:1036,“name”:“pool_monitor_zone3_rip”,“unitConfig”:{“maxCpuCoreCount”:3.00,“maxMemoryByte”:2147483648,“maxMemorySize”:2,“minCpuCoreCount”:3.00,“minMemoryByte”:2147483648,“minMemorySize”:2},“unitCount”:1}], old_password=, new_password=, tenant_mode=MYSQL, system_variable_map=, create_tenant_param_json={“charset”:“utf8mb4”,“collation”:“utf8mb4_general_ci”,“mode”:“MYSQL”,“name”:“monitor”,“parameters”:[],“primaryZone”:"",“rootPassword”:"******",“whitelist”:"%",“zones”:[{“name”:“zone1”,“replicaType”:“FULL”,“resourcePool”:{“unitCount”:1,“unitSpec”:{“cpuCore”:3.00,“memoryBytes”:2147483648,“memorySize”:2}}},{“name”:“zone2”,“replicaType”:“FULL”,“resourcePool”:{“unitCount”:1,“unitSpec”:{“cpuCore”:3.00,“memoryBytes”:2147483648,“memorySize”:2}}},{“name”:“zone3”,“replicaType”:“FULL”,“resourcePool”:{“unitCount”:1,“unitSpec”:{“cpuCore”:3.00,“memoryBytes”:2147483648,“memorySize”:2}}}]}, latest_execution_start_time=2024-02-02T11:44:53.161+08:00, sub_task_instance_id=12846}, listMap={}}, executor=11.162.218.226
2024-02-02 11:44:53.175 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.o.c.obsdk.ConnectPropertiesBuilder : build connect address for sys tenant, obServerAddrList=[ObServerAddr(address=11.162.218.35, port=2881), ObServerAddr(address=11.162.218.226, port=2881), ObServerAddr(address=11.162.218.58, port=2881)]
2024-02-02 11:44:53.178 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=null, obClusterId=null, tenantName=sys, username=root, address=11.162.218.35, port=2881, database=oceanbase)
2024-02-02 11:44:53.181 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:11.162.218.226, sql port:2881
2024-02-02 11:44:53.184 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=null, obClusterId=null, tenantName=sys, username=root, address=11.162.218.35, port=2881, database=oceanbase)
2024-02-02 11:44:53.187 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:11.162.218.226, sql port:2881
2024-02-02 11:44:53.191 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000]
2024-02-02 11:44:53.195 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: select max(value) value from oceanbase.__all_virtual_sys_parameter_stat where name = ‘min_observer_version’
2024-02-02 11:44:53.201 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.o.o.o.r.MysqlResourceOperator : start to delete resource pool, pool_name = pool_monitor_zone2_ksv
2024-02-02 11:44:53.204 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.o.obsdk.connector.ObConnectorHolder : [obsdk] no ob connector found in holder, key=ObConnectorKey(connectionMode=direct, clusterName=null, obClusterId=null, tenantName=sys, username=root, address=11.162.218.35, port=2881, database=oceanbase)
2024-02-02 11:44:53.207 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.ocp.obsdk.connector.ObConnectors : [obsdk]:connected server ip:11.162.218.226, sql port:2881
2024-02-02 11:44:53.210 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: set ob_query_timeout = ?, args: [10000000]
2024-02-02 11:44:53.213 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.ocp.obsdk.connector.ConnectTemplate : [obsdk] sql: DROP RESOURCE POOL IF EXISTS pool_monitor_zone2_ksv
2024-02-02 11:44:53.221 WARN 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.o.o.i.t.ResourcePoolServiceImpl : delete resource unit config failed. unitConfigName=null, error message:input unit config name is empty
2024-02-02 11:44:53.224 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.o.o.i.t.ResourcePoolServiceImpl : resource pool deleted, poolName=pool_monitor_zone2_ksv, poolId=1034
2024-02-02 11:44:53.228 INFO 490524 — [pool-manual-subtask-executor16,da5e8e6eaea349b8,fee736e227d6] c.o.o.o.i.tenant.task.CreateTenantTask : rollback, delete resource pool, resourcePoolList=[ResourcePool(id=1035, name=pool_monitor_zone1_sai, unitCount=1, unitConfig=UnitConfig(maxCpuCoreCount=3.0, minCpuCoreCount=3.0, maxMemoryByte=2147483648, minMemoryByte=2147483648, maxDiskSizeByte=null, maxDiskSize=null, maxIops=null, minIops=null, maxSessionNum=null, name=null), zoneList=null), ResourcePool(id=1034, name=pool_monitor_zone2_ksv, unitCount=1, unitConfig=UnitConfig(maxCpuCoreCount=3.0, minCpuCoreCount=3.0, maxMemoryByte=2147483648, minMemoryByte=2147483648, maxDiskSizeByte=null, maxDiskSize=null, maxIops=null, minIops=null, maxSessionNum=null, name=null), zoneList=null), ResourcePool(id=1036, name=pool_monitor_zone3_rip, unitCount=1, unitConfig=UnitConfig(maxCpuCoreCount=3.0, minCpuCoreCount=3.0, maxMemoryByte=2147483648, minMemoryByte=2147483648, maxDiskSizeByte=null, maxDiskSize=null, maxIops=null, minIops=null, maxSessionNum=null, name=null), zoneList=null)]
Set state for subtask: 12846, operation:ROLLBACK, state: PENDING

报错行

2024-02-02 11:40:24.760 ERROR 490524 --- [pool-manual-subtask-executor16,da5e8e6eaea349b8,33879543baa0] c.o.ocp.obsdk.connector.ConnectTemplate  : [obsdk] update failed, sql:[CREATE TENANT `monitor` resource_pool_list=('pool_monitor_zone1_sai','pool_monitor_zone2_ksv','pool_monitor_zone3_rip'), LOCALITY = ?, CHARSET = ?, COLLATE = ? SET ob_tcp_invited_nodes='%', ob_compatibility_mode = ?], error message:[PreparedStatementCallback; SQL [CREATE TENANT `monitor` resource_pool_list=('pool_monitor_zone1_sai','pool_monitor_zone2_ksv','pool_monitor_zone3_rip'), LOCALITY = ?, CHARSET = ?, COLLATE = ? SET ob_tcp_invited_nodes='%', ob_compatibility_mode = ?]; (conn=3221699732) IO error; nested exception is java.sql.SQLTransientConnectionException: (conn=3221699732) IO error]

select last_trace_id();
定位具体的实现方式:
grep ‘trace id’ observer.log
试试

这样啥东西也没有啊

已解决,存在节点安装目录磁盘满导致。 清理部分日志后,创建正常。