calc_result_typeN（）请问这种类型推导是在哪个阶段完成的？

OBFan · 2023 年11 月 3 日 10:33

我发现每个表达式都会有这样的类型推导过程，我好奇的是他是在哪个阶段完成的？逻辑计划应该不可能完成吧，应该是在执行阶段调用的吗？确定一下猜想，求助内核大佬。

OBFan · 2023 年11 月 3 日 10:41

第二个问题是：
还有其他形式的，例如：calc_result_type0()，calc_result_type1()，calc_result_type2()，所以我推断如果没有参数就使用calc_result_type0()，一个参数使用calc_result_type1()，两个参数使用calc_result_type2()，三个及三个以上使用…typeN()
求证这种猜想是否准确？

兹拉坦 · 2023 年11 月 3 日 11:13

猜想是正确的。

兹拉坦 · 2023 年11 月 3 日 11:17

类型推导 calc_result_type 在 Resolver 阶段中完成。

SQL是一个静态类型系统，数据是什么类型，需要哪些操作，在编译期（cg 完成之前，也就是物理执行计划生成之前）就能全部确定，不需要在算子执行阶段根据数据动态解释。MySQL不同类型数据间的转换非常灵活，但具体怎么转也是在编译期确定，因此静态类型并不与MySQL兼容性冲突(MySQL这种允许类型互相隐式转换的行为，称为弱类型，与之对应的是强类型)。由于sql是静态类型，很多事情可以在编译期确定，我们需要计算尽量提到编译期做，而不是在执行期重复的做。OB 4.x 版本的数据类型信息只在编译期推导，执行期时数据按之前推导的类型进行解释。

在 calc_result_type 里用 gdb 打个断点，看下调用栈，就知道是在啥阶段被调用的了~
附一个学生刚开始学习 OB 3.1 版本开源代码过程时的笔记（部分），供您参考：

回顾一条 SQL 语句的执行流程
总入口是 ObSql::stmt_query 函数。文档在这
● Parser: 入口是 ObParser 的 parse 函数
● Plan Cache
● Resolver：接口类是 ObResolver
○ 表达式解析类是位于 sql/resolver/expr 的 ObRawExprResolver，它输入 ParseNode 树，输出表达式树 ObRawExpr。这个ObRawExpr 仅用于优化器阶段的语义分析优化，在生成物理执行计划后会被转换为为执行效率优化的 ObExpr。
○ 通过查找 ObExprOperatorFactory 类中的 get_type_by_name 决定一个函数是否存在.
● Transfomer: 改写规则的入口类是 ObTransformerImpl ，输入输出都是 ObDMLStmt
● Optimizer: 优化器（ sql/optimizer ）的接口类是 ObOptimizer ，以上一步生成的 ObDMLStmt （含select）为输入，执行基于代价的优化，生成一个逻辑执行计划（ ObLogPlan ）。入口函数是 int ObOptimizer::optimize(ObDMLStmt& stmt, ObLogPlan*& logical_plan)
○ 逻辑执行计划 ObLogPlan 是由OB的关系运算算子 ObLogicalOperator 组成的树状结构
● Code Generator: 模块负责把逻辑执行计划转换为能够高效执行的物理执行计划。接口类是 ObCodeGenerator。
● Executor: SQL执行引擎，也叫物理执行计划（ObPhysicalPlan），是由物理算子（ObPhyOperator）组成的树状结构
不考虑 PC 的情况下，各个模块执行的先后顺序是从上往下

SELECT crc32("");

可用 ALTER SYSTEM FLUSH PLAN CACHE 来清除缓存以查看不命中 cache 的过程
SELECT crc32 的日志
通过 trace id 定位到 observer.log 中 debug 级别以上到日志
cat observer.log | grep YB437F000001-0005C60BDA16C410 > select_crc32.log
执行计划如下：


MySQL [test]> EXPLAIN SELECT CRC32("")\G;
Query Plan: ===================================
|ID|OPERATOR  |NAME|EST. ROWS|COST|
-----------------------------------
|0 |EXPRESSION|    |1        |1   |
===================================

Outputs & filters:
-------------------------------------
  0 - output([?]), filter(nil)
      values({?})

用 gdb attach 追踪程序执行过程
常用命令

ps -aux | grep observer | grep <user_name> 获取 observer 的 pid
gdb attach 进入程序
bt 看调用栈
info threads 查看所有线程
stop 暂停线程
在 obclient 中用 HINT 来 select /*+ query_timeout(100000000) */ crc32(“abc”); 设置超时时间，防止影响查询结果
b ObSql::stmt_query 设置一下断点；后面加 if 可以变为条件断点。比如可以 b ObSql::stmt_query if stmt.data_length_==19, b ObParser::parse if query.data_length_==19
c 继续运行代码
frame 查看当前栈帧，frame N 切换栈帧

ObExprCrc32 中的函数是什么时候执行的
● 如果没有命中 cache，执行 calc_result_type1 → cg_expr → calc_crc32_expr
● 如果命中了 cache，则只会执行 calc_crc32_expr
未命中 cache
首先执行到 ObExprCrc32::calc_result_type1
bt 信息如下。可以看出，类型检查在 Resolver 的一个阶段中完成

(gdb) bt
#0  oceanbase::sql::ObExprCrc32::calc_result_type1 (this=0x7f3e4b9b8640, type=..., type1=..., type_ctx=...) at ./src/sql/engine/expr/ob_expr_crc32.cpp:33
#1  0x00000000048ca2d8 in oceanbase::sql::ObRawExprDeduceType::calc_result_type (this=0x7f3f611203b8, expr=..., types=..., cast_mode=@0x7f3f6111fde8: 289,
    row_dimension=-1) at ./src/sql/resolver/expr/ob_raw_expr_deduce_type.cpp:325
#2  0x00000000048e6de4 in oceanbase::sql::ObRawExprDeduceType::visit (this=0x7f3f611203b8, expr=...)
    at ./src/sql/resolver/expr/ob_raw_expr_deduce_type.cpp:1706
#3  0x00000000048b38a1 in oceanbase::sql::ObSysFunRawExpr::do_visit (this=0x7f3e4b9b7d70, visitor=...) at ./src/sql/resolver/expr/ob_raw_expr.cpp:2759
#4  0x000000000489d5da in oceanbase::sql::ObRawExpr::postorder_accept (this=0x7f3e4b9b7d70, visitor=...) at ./src/sql/resolver/expr/ob_raw_expr.cpp:426
#5  0x000000000489c2d0 in oceanbase::sql::ObRawExprDeduceType::deduce (this=0x7f3f611203b8, expr=...)
    at ./src/sql/resolver/expr/ob_raw_expr_deduce_type.cpp:36
#6  0x000000000489c15e in oceanbase::sql::ObRawExpr::deduce_type (this=0x7f3e4b9b7d70, session_info=0x7f404063fb40)
    at ./src/sql/resolver/expr/ob_raw_expr.cpp:325
#7  0x0000000004894d74 in oceanbase::sql::ObRawExpr::formalize (this=0x7f3e4b9b7d70, session_info=0x7f404063fb40)
    at ./src/sql/resolver/expr/ob_raw_expr.cpp:337
#8  0x00000000064ef847 in oceanbase::sql::ObDMLStmt::formalize_stmt (this=0x7f3efee01250, session_info=0x7f404063fb40)
    at ./src/sql/resolver/dml/ob_dml_stmt.cpp:1784
#9  0x00000000065f91f2 in oceanbase::sql::ObSelectResolver::resolve_normal_query (this=0x7f3e60fe9d30, parse_tree=...)
    at ./src/sql/resolver/dml/ob_select_resolver.cpp:814
#10 0x0000000006603d90 in oceanbase::sql::ObSelectResolver::resolve (this=0x7f3e60fe9d30, parse_tree=...)
    at ./src/sql/resolver/dml/ob_select_resolver.cpp:900
#11 0x0000000004a604fb in oceanbase::sql::ObResolver::select_stmt_resolver_func<oceanbase::sql::ObSelectResolver> (this=0x7f3f61121b88, params=...,
    parse_tree=..., stmt=@0x7f3f61122b80: 0x0) at ./src/sql/resolver/ob_resolver.cpp:141
#12 0x00000000049b90b2 in oceanbase::sql::ObResolver::resolve (this=0x7f3f61121b88, if_prepared=oceanbase::sql::ObResolver::IS_NOT_PREPARED_STMT,
    parse_tree=..., stmt=@0x7f3f61122b80: 0x0) at ./src/sql/resolver/ob_resolver.cpp:268
#13 0x0000000005e56f1b in oceanbase::sql::ObSql::generate_stmt (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    parse_result=..., pc_ctx=0x7f3e60fd9960, context=..., allocator=..., result=..., stmt=@0x7f3f61122b80: 0x0) at ./src/sql/ob_sql.cpp:1389
#14 0x0000000005e5b79e in oceanbase::sql::ObSql::generate_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    parse_result=..., pc_ctx=0x7f3e60fd9960, sql_ctx=..., result=..., is_ps_mode=false) at ./src/sql/ob_sql.cpp:1493
#15 0x0000000005e5a018 in oceanbase::sql::ObSql::handle_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    trimed_stmt=..., context=..., result=..., pc_ctx=..., get_plan_err=-5138) at ./src/sql/ob_sql.cpp:3155
#16 0x0000000005edad7d in oceanbase::sql::ObSql::handle_text_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    stmt=..., context=..., result=...) at ./src/sql/ob_sql.cpp:1184
#17 0x0000000005e4c611 in oceanbase::sql::ObSql::stmt_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>, stmt=...,
    context=..., result=...) at ./src/sql/ob_sql.cpp:169

其次执行到 ObExprCrc32::cg_expr
bt 信息如下：

(gdb) bt
#0  oceanbase::sql::ObExprCrc32::cg_expr (this=0x7f3ef8630de0, expr_cg_ctx=..., raw_expr=..., rt_expr=...) at ./src/sql/engine/expr/ob_expr_crc32.cpp:106
#1  0x000000000446d632 in oceanbase::sql::ObStaticEngineExprCG::cg_expr_by_operator (this=0x7f3f61121c50, raw_exprs=..., total_ctx_cnt=@0x7f3ee73d06f8: 0)
    at ./src/sql/code_generator/ob_static_engine_expr_cg.cpp:297
#2  0x000000000446bd68 in oceanbase::sql::ObStaticEngineExprCG::cg_exprs (this=0x7f3f61121c50, raw_exprs=..., expr_info=...)
    at ./src/sql/code_generator/ob_static_engine_expr_cg.cpp:120
#3  0x0000000004378c09 in oceanbase::sql::ObStaticEngineExprCG::generate (this=0x7f3f61121c50, all_raw_exprs=..., expr_info=...)
    at ./src/sql/code_generator/ob_static_engine_expr_cg.cpp:55
#4  0x00000000044743ca in oceanbase::sql::ObStaticEngineExprCG::inner_generate_calculable_exprs (this=0x7f3f61121c50, calculable_exprs=..., expr_info=...)
    at ./src/sql/code_generator/ob_static_engine_expr_cg.cpp:821
#5  0x00000000044737fc in oceanbase::sql::ObStaticEngineExprCG::generate_calculable_exprs (this=0x7f3f61121c50, calculable_exprs=..., pre_calc_frame=...)
    at ./src/sql/code_generator/ob_static_engine_expr_cg.cpp:736
#6  0x0000000005e64fa7 in oceanbase::sql::ObSql::calc_pre_calculable_exprs (stmt=..., calculable_exprs=..., is_ignore_stmt=false, exec_ctx=...,
    phy_plan=...) at ./src/sql/ob_sql.cpp:3415
#7  0x0000000005e64501 in oceanbase::sql::ObSql::calc_pre_calculable_exprs (calculable_exprs=..., exec_ctx=..., stmt=..., phy_plan=...)
    at ./src/sql/ob_sql.cpp:1715
#8  0x0000000005e6087b in oceanbase::sql::ObSql::calc_pre_calculable_exprs (exec_ctx=..., stmt=..., phy_plan=...) at ./src/sql/ob_sql.cpp:1669
#9  0x0000000005e5cbba in oceanbase::sql::ObSql::generate_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    parse_result=..., pc_ctx=0x7f3e60fd9960, sql_ctx=..., result=..., is_ps_mode=false) at ./src/sql/ob_sql.cpp:1574
#10 0x0000000005e5a018 in oceanbase::sql::ObSql::handle_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    trimed_stmt=..., context=..., result=..., pc_ctx=..., get_plan_err=-5138) at ./src/sql/ob_sql.cpp:3155
#11 0x0000000005edad7d in oceanbase::sql::ObSql::handle_text_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    stmt=..., context=..., result=...) at ./src/sql/ob_sql.cpp:1184
#12 0x0000000005e4c611 in oceanbase::sql::ObSql::stmt_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>, stmt=...,
    context=..., result=...) at ./src/sql/ob_sql.cpp:169

在 ObStaticEngineExprCG::cg_expr_by_operator 中由 const ObExprOperator* op 所指向的对象类型确定具体的 cg_expr

最后执行到 ObExprCrc32::calc_crc32_expr
bt 信息如下：

(gdb) bt
#0  oceanbase::sql::ObExprCrc32::calc_crc32_expr (expr=..., ctx=..., res_datum=...) at ./src/sql/engine/expr/ob_expr_crc32.cpp:85
#1  0x000000000622b177 in oceanbase::sql::ObExpr::eval (this=0x7f3e945bf490, ctx=..., datum=@0x7f3f60cab7a0: 0x7f3e4b9c0f00)
    at ./src/sql/engine/expr/ob_expr.h:605
#2  oceanbase::sql::ObCacheObject::pre_calculation (is_ignore_stmt=false, pre_calc_frame=..., exec_ctx=...) at ./src/sql/plan_cache/ob_cache_object.cpp:195
#3  0x0000000005e65287 in oceanbase::sql::ObSql::calc_pre_calculable_exprs (stmt=..., calculable_exprs=..., is_ignore_stmt=false, exec_ctx=...,
    phy_plan=...) at ./src/sql/ob_sql.cpp:3429
#4  0x0000000005e64501 in oceanbase::sql::ObSql::calc_pre_calculable_exprs (calculable_exprs=..., exec_ctx=..., stmt=..., phy_plan=...)
    at ./src/sql/ob_sql.cpp:1715
#5  0x0000000005e6087b in oceanbase::sql::ObSql::calc_pre_calculable_exprs (exec_ctx=..., stmt=..., phy_plan=...) at ./src/sql/ob_sql.cpp:1669
#6  0x0000000005e5cbba in oceanbase::sql::ObSql::generate_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    parse_result=..., pc_ctx=0x7f3ee4799960, sql_ctx=..., result=..., is_ps_mode=false) at ./src/sql/ob_sql.cpp:1574
#7  0x0000000005e5a018 in oceanbase::sql::ObSql::handle_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    trimed_stmt=..., context=..., result=..., pc_ctx=..., get_plan_err=-5138) at ./src/sql/ob_sql.cpp:3155
#8  0x0000000005edad7d in oceanbase::sql::ObSql::handle_text_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    stmt=..., context=..., result=...) at ./src/sql/ob_sql.cpp:1184
#9  0x0000000005e4c611 in oceanbase::sql::ObSql::stmt_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>, stmt=...,
    context=..., result=...) at ./src/sql/ob_sql.cpp:169

以上这 3 个函数在 ObSql::generate_physical_plan 中先后进行，其中
● calc_result_type1 在 generate_stmt 中
● cg_expr 和 calc_crc32_expr 在 ObSql::calc_pre_calculable_exprs 中

命中 cache
bt 信息如下。可以看到从 pre_calculation 那里开始和未命中 cache 时是一样的，而 #2 之前都是不一样的

(gdb) bt
#0  oceanbase::sql::ObExprCrc32::calc_crc32_expr (expr=..., ctx=..., res_datum=...) at ./src/sql/engine/expr/ob_expr_crc32.cpp:85
#1  0x000000000622b177 in oceanbase::sql::ObExpr::eval (this=0x7f3e96206640, ctx=..., datum=@0x7f3f5d156370: 0x7f3e4b9b7c10)
    at ./src/sql/engine/expr/ob_expr.h:605
#2  oceanbase::sql::ObCacheObject::pre_calculation (is_ignore_stmt=false, pre_calc_frame=..., exec_ctx=...) at ./src/sql/plan_cache/ob_cache_object.cpp:195
#3  0x000000000626612a in oceanbase::sql::ObPlanSet::pre_calc_exprs (this=0x7f3e962002b0, exec_ctx=...) at ./src/sql/plan_cache/ob_plan_set.cpp:821
#4  0x000000000626019d in oceanbase::sql::ObPlanSet::match_params_info (this=0x7f3e962002b0, params=0x7f3e4b9b7130, pc_ctx=..., outline_param_idx=-1,
    is_same=@0x7f3f5d156c27: true) at ./src/sql/plan_cache/ob_plan_set.cpp:135
#5  0x000000000623e82b in oceanbase::sql::ObPlanCacheValue::choose_plan (this=0x7f3e961ec840, pc_ctx=..., schema_array=..., plan_out=@0x7f3f5d1574b8: 0x0)
    at ./src/sql/plan_cache/ob_plan_cache_value.cpp:356
#6  0x000000000623b3c9 in oceanbase::sql::ObPCVSet::get_plan (this=0x7f3e96259660, pc_ctx=..., plan=@0x7f3f5d1574b8: 0x0)
    at ./src/sql/plan_cache/ob_pcv_set.cpp:176
#7  0x000000000624760e in oceanbase::sql::ObPlanCache::get_cache_obj (this=0x7f3f8b748870, pc_ctx=..., cache_obj=@0x7f3f5d1574b8: 0x0)
    at ./src/sql/plan_cache/ob_plan_cache.cpp:325
#8  0x000000000624a71b in oceanbase::sql::ObPlanCache::get_plan (this=0x7f3f8b748870, ref_handle=oceanbase::sql::CLI_QUERY_HANDLE, allocator=...,
    pc_ctx=..., plan=@0x7f3f5d1578b8: 0x0) at ./src/sql/plan_cache/ob_plan_cache.cpp:388
#9  0x0000000005e6cbb4 in oceanbase::sql::ObSql::execute_get_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    plan_cache=..., pc_ctx=..., plan=@0x7f3f5d1578b8: 0x0) at ./src/sql/ob_sql.cpp:2528
#10 0x0000000005e5f2e7 in oceanbase::sql::ObSql::pc_get_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>, pc_ctx=...,
    plan=@0x7f3f5d1578b8: 0x0, get_plan_err=@0x7f3f5d157a24: 0, need_disconnect=@0x7f405334a930: true) at ./src/sql/ob_sql.cpp:2648
#11 0x0000000005e5915d in oceanbase::sql::ObSql::pc_get_plan_and_fill_result (
    this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>, pc_ctx=..., result=..., get_plan_err=@0x7f3f5d157a24: 0,
    need_disconnect=@0x7f405334a930: true) at ./src/sql/ob_sql.cpp:2619
#12 0x0000000005edabdc in oceanbase::sql::ObSql::handle_text_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    stmt=..., context=..., result=...) at ./src/sql/ob_sql.cpp:1169
#13 0x0000000005e4c611 in oceanbase::sql::ObSql::stmt_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>, stmt=...,
    context=..., result=...) at ./src/sql/ob_sql.cpp:169

在 ObSql::handle_text_query 中，如果命中了 PC，那么在 pc_get_plan_and_fill_result 就可以完成查询；如果未命中，则得在之后的 handle_physical_plan 那里进行 SQL 语句除了 parse 之外的一整个流程
在 ObCacheObject::pre_calculation 中获取到 ObExpr 之后，用 eval 方法计算。 eval 方法中的 eval_func_ 在 cg_expr 中由 calc_crc32_expr 赋值
ObSql::stmt_query 总入口
整个过程执行如下：

进入 ObSql::stmt_query
a. 调用 ObSql::handle_text_query
ObSql::handle_text_query
a. handle_text_query 首先 init_result_set，在 init_result_set 中初始化 ObVirtualTableCtx 之后再调用 init_exec_context。在 init_exec_context 中给 ObTaskExecutorCtx 和 ObExecContext 赋值
b. handle_text_query 随后获取 ObIAllocator, ObSQLSessionInfo, ObExecContext, ObPlanCacheCtx 等类的对象
c. 随后调用 pc_get_plan_and_fill_result → pc_get_plan → execute_get_plan
d. 调用 handle_large_query,执行 ret = (tmp_ret == OB_PC_LOCK_CONFLICT) ? OB_SUCCESS : tmp_ret; 后退出
e. 调用 handle_physical_plan
f. 调用 after_get_plan
ObSql::handle_physical_plan
ObSql::after_get_plan

ObParser::parse
ObParser::parse 的调用栈如下：

(gdb) bt
#0  oceanbase::sql::ObParser::parse (this=0x7f9d73195288, query=..., parse_result=..., parse_mode=FP_MODE, is_batched_multi_stmt_split_on=false)
    at ./src/sql/parser/ob_parser.cpp:205
#1  0x000000000624ba73 in oceanbase::sql::ObSqlParameterization::fast_parser (allocator=..., sql_mode=4194304,
    connection_collation=oceanbase::common::CS_TYPE_UTF8MB4_GENERAL_CI, sql=..., enable_batched_multi_stmt=false, fp_result=...)
    at ./src/sql/plan_cache/ob_sql_parameterization.cpp:873
#2  0x000000000624b688 in oceanbase::sql::ObPlanCache::construct_fast_parser_result (allocator=..., pc_ctx=..., raw_sql=..., fp_result=...)
    at ./src/sql/plan_cache/ob_plan_cache.cpp:465
#3  0x000000000624a621 in oceanbase::sql::ObPlanCache::get_plan (this=0x7f9d9c3a3900, ref_handle=oceanbase::sql::CLI_QUERY_HANDLE, allocator=...,
    pc_ctx=..., plan=@0x7f9d731958b8: 0x0) at ./src/sql/plan_cache/ob_plan_cache.cpp:382
#4  0x0000000005e6cbb4 in oceanbase::sql::ObSql::execute_get_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    plan_cache=..., pc_ctx=..., plan=@0x7f9d731958b8: 0x0) at ./src/sql/ob_sql.cpp:2528
#5  0x0000000005e5f2e7 in oceanbase::sql::ObSql::pc_get_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>, pc_ctx=...,
    plan=@0x7f9d731958b8: 0x0, get_plan_err=@0x7f9d73195a24: 0, need_disconnect=@0x7f9e4d201630: true) at ./src/sql/ob_sql.cpp:2648
#6  0x0000000005e5915d in oceanbase::sql::ObSql::pc_get_plan_and_fill_result (
    this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>, pc_ctx=..., result=..., get_plan_err=@0x7f9d73195a24: 0,
    need_disconnect=@0x7f9e4d201630: true) at ./src/sql/ob_sql.cpp:2619
#7  0x0000000005edabdc in oceanbase::sql::ObSql::handle_text_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    stmt=..., context=..., result=...) at ./src/sql/ob_sql.cpp:1169
#8  0x0000000005e4c611 in oceanbase::sql::ObSql::stmt_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>, stmt=...,
    context=..., result=...) at ./src/sql/ob_sql.cpp:169

…
● ObParser:parse: → ObParser:parse_sql → ObSQLParser::parse → sql_parser_base,c 中的 parse_sql，在其中先后调用 obsql_mysql_yy_scan_bytes , obsql_mysql_yy_switch_to_buffer , obsql_mysql_fast_parse , obsql_mysql_yy_delete_buffer
○ ParseResult 对象的 is_fp_ 表示是否启用 fast parse
○ obsql_mysql_xxxx 等函数位于 src/sql/parser 中，用 bison 和 flex

ObResolver::resolve
ObResolver::resolve 的调用栈如下：

(gdb) bt
#0  oceanbase::sql::ObResolver::resolve (this=0x7fcf72e98b68, if_prepared=oceanbase::sql::ObResolver::IS_NOT_PREPARED_STMT, parse_tree=...,
    stmt=@0x7fcf72e99b60: 0x0) at ./src/sql/resolver/ob_resolver.cpp:161
#1  0x0000000005e56f1b in oceanbase::sql::ObSql::generate_stmt (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    parse_result=..., pc_ctx=0x7fce2e74ff20, context=..., allocator=..., result=..., stmt=@0x7fcf72e99b60: 0x0) at ./src/sql/ob_sql.cpp:1389
#2  0x0000000005e5b79e in oceanbase::sql::ObSql::generate_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    parse_result=..., pc_ctx=0x7fce2e74ff20, sql_ctx=..., result=..., is_ps_mode=false) at ./src/sql/ob_sql.cpp:1493
#3  0x0000000005e5a018 in oceanbase::sql::ObSql::handle_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    trimed_stmt=..., context=..., result=..., pc_ctx=..., get_plan_err=-5138) at ./src/sql/ob_sql.cpp:3155
#4  0x0000000005edad7d in oceanbase::sql::ObSql::handle_text_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    stmt=..., context=..., result=...) at ./src/sql/ob_sql.cpp:1184
#5  0x0000000005e4c611 in oceanbase::sql::ObSql::stmt_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>, stmt=...,
    context=..., result=...) at ./src/sql/ob_sql.cpp:169

…
● 在 ObResolver::resolve 中， parse_tree.type_ 的值为 T_SELECT，进入 select_stmt_resolver_func 函数中，随后调用 ObSelectResolver::resolve
● ObSelectResolver::resolve 创建一个 ObSelectResolver ，随后调用 ObSelectResolver::resolve_normal_query
● ObSelectResolver::resolve_normal_query：按照 ObSelectResolver::resolve 中的顺序，解析一条 Select 语句可能出现的各种语义，大多数函数定义在 ObDMLResolver 中的 resolveXXXXX
● 由于解析的 SQL 语句中含有 Expression （crc32），所以进入 ObRawExprResolverImpl::resolve 中，调用 ObRawExprResolverImpl::recursive_resolve 对 const ParseNode* node 进行解析，结果在 ObRawExpr*& expr 中
疑问：是否存在打印一棵 ParseNode 树的函数？
附上 ObResolverUtils::resolve_const 的 bt 信息：

(gdb) bt
#0  oceanbase::sql::ObResolverUtils::resolve_const (node=0x7fce24f14068, stmt_type=oceanbase::sql::stmt::T_NONE, allocator=...,
    connection_collation=oceanbase::common::CS_TYPE_UTF8MB4_GENERAL_CI, nchar_collation=oceanbase::common::CS_TYPE_UTF8MB4_BIN, tz_info=0x7fcef58423d8,
    val=..., is_paramlize=false, literal_prefix=..., default_length_semantics=2, server_collation=oceanbase::common::CS_TYPE_INVALID,
    parents_expr_info=0x7fcfb9694888) at ./src/sql/resolver/ob_resolver_utils.cpp:459
#1  0x00000000049436fd in oceanbase::sql::ObRawExprResolverImpl::process_datatype_or_questionmark (this=0x7fcfb9694848, node=..., expr=@0x7fcfb96936f0: 0x0)
    at ./src/sql/resolver/expr/ob_raw_expr_resolver_impl.cpp:1447
#2  0x000000000493d104 in oceanbase::sql::ObRawExprResolverImpl::do_recursive_resolve (this=0x7fcfb9694848, node=0x7fce24f14068, expr=@0x7fcfb96936f0: 0x0)
    at ./src/sql/resolver/expr/ob_raw_expr_resolver_impl.cpp:223
#3  0x000000000493bf39 in oceanbase::sql::ObRawExprResolverImpl::recursive_resolve (this=0x7fcfb9694848, node=0x7fce24f14068, expr=@0x7fcfb96936f0: 0x0)
    at ./src/sql/resolver/expr/ob_raw_expr_resolver_impl.cpp:204
#4  0x000000000494a49d in oceanbase::sql::ObRawExprResolverImpl::process_operator_node (this=0x7fcfb9694848, node=0x7fce24f140f8, expr=@0x7fcfb9695068: 0x0)
    at ./src/sql/resolver/expr/ob_raw_expr_resolver_impl.cpp:1907
#5  0x000000000493fa2b in oceanbase::sql::ObRawExprResolverImpl::do_recursive_resolve (this=0x7fcfb9694848, node=0x7fce24f140f8, expr=@0x7fcfb9695068: 0x0)
    at ./src/sql/resolver/expr/ob_raw_expr_resolver_impl.cpp:428
#6  0x000000000493bf39 in oceanbase::sql::ObRawExprResolverImpl::recursive_resolve (this=0x7fcfb9694848, node=0x7fce24f140f8, expr=@0x7fcfb9695068: 0x0)
    at ./src/sql/resolver/expr/ob_raw_expr_resolver_impl.cpp:204
#7  0x000000000493bd90 in oceanbase::sql::ObRawExprResolverImpl::resolve (this=0x7fcfb9694848, node=0x7fce24f140f8, expr=@0x7fcfb9695068: 0x0, columns=...,
    sys_vars=..., sub_query_info=..., aggr_exprs=..., win_exprs=..., op_exprs=..., user_var_exprs=...)
    at ./src/sql/resolver/expr/ob_raw_expr_resolver_impl.cpp:47
#8  0x00000000064fd920 in oceanbase::sql::ObDMLResolver::resolve_sql_expr (this=0x7fce24f1c8b0, node=..., expr=@0x7fcfb9695068: 0x0,
    output_columns=0x7fcfb9694c80) at ./src/sql/resolver/dml/ob_dml_resolver.cpp:207
#9  0x0000000006533848 in oceanbase::sql::ObDMLResolver::resolve_and_split_sql_expr (this=0x7fce24f1c8b0, node=..., and_exprs=...)
    at ./src/sql/resolver/dml/ob_dml_resolver.cpp:2992
#10 0x0000000006520b57 in oceanbase::sql::ObDMLResolver::resolve_and_split_sql_expr_with_bool_expr (this=0x7fce24f1c8b0, node=..., and_exprs=...)
    at ./src/sql/resolver/dml/ob_dml_resolver.cpp:3023
#11 0x00000000064e5f91 in oceanbase::sql::ObDMLResolver::resolve_where_clause (this=0x7fce24f1c8b0, node=0x7fce24f14188)
    at ./src/sql/resolver/dml/ob_dml_resolver.cpp:3299
#12 0x00000000065f7daa in oceanbase::sql::ObSelectResolver::resolve_normal_query (this=0x7fce24f1c8b0, parse_tree=...)
    at ./src/sql/resolver/dml/ob_select_resolver.cpp:758
#13 0x0000000006603d90 in oceanbase::sql::ObSelectResolver::resolve (this=0x7fce24f1c8b0, parse_tree=...)
    at ./src/sql/resolver/dml/ob_select_resolver.cpp:900
#14 0x0000000004a604fb in oceanbase::sql::ObResolver::select_stmt_resolver_func<oceanbase::sql::ObSelectResolver> (this=0x7fcfb9696648, params=...,
    parse_tree=..., stmt=@0x7fcfb9697640: 0x0) at ./src/sql/resolver/ob_resolver.cpp:141
#15 0x00000000049b90b2 in oceanbase::sql::ObResolver::resolve (this=0x7fcfb9696648, if_prepared=oceanbase::sql::ObResolver::IS_NOT_PREPARED_STMT,
    parse_tree=..., stmt=@0x7fcfb9697640: 0x0) at ./src/sql/resolver/ob_resolver.cpp:268
#16 0x0000000005e56f1b in oceanbase::sql::ObSql::generate_stmt (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    parse_result=..., pc_ctx=0x7fce24f0cea0, context=..., allocator=..., result=..., stmt=@0x7fcfb9697640: 0x0) at ./src/sql/ob_sql.cpp:1389
#17 0x0000000005e5b79e in oceanbase::sql::ObSql::generate_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    parse_result=..., pc_ctx=0x7fce24f0cea0, sql_ctx=..., result=..., is_ps_mode=false) at ./src/sql/ob_sql.cpp:1493
#18 0x0000000005e5a018 in oceanbase::sql::ObSql::handle_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    trimed_stmt=..., context=..., result=..., pc_ctx=..., get_plan_err=-4654) at ./src/sql/ob_sql.cpp:3155
#19 0x0000000005edad7d in oceanbase::sql::ObSql::handle_text_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    stmt=..., context=..., result=...) at ./src/sql/ob_sql.cpp:1184
#20 0x0000000005e4c611 in oceanbase::sql::ObSql::stmt_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>, stmt=...,
    context=..., result=...) at ./src/sql/ob_sql.cpp:169

…
附上 ObExprOperatorFactory::get_type_by_name 的 bt 信息：

(gdb) bt
#0  oceanbase::sql::ObExprOperatorFactory::get_type_by_name (name=...) at ./src/sql/engine/expr/ob_expr_operator_factory.cpp:356
#1  0x0000000004960578 in oceanbase::sql::ObRawExprResolverImpl::process_fun_sys_node (this=0x7fcecb3e74a8, node=0x7fcdd89dfbe8, expr=@0x7fcecb3e8178: 0x0)
    at ./src/sql/resolver/expr/ob_raw_expr_resolver_impl.cpp:3532
#2  0x0000000004941d5d in oceanbase::sql::ObRawExprResolverImpl::do_recursive_resolve (this=0x7fcecb3e74a8, node=0x7fcdd89dfbe8, expr=@0x7fcecb3e8178: 0x0)
    at ./src/sql/resolver/expr/ob_raw_expr_resolver_impl.cpp:732
#3  0x000000000493bf39 in oceanbase::sql::ObRawExprResolverImpl::recursive_resolve (this=0x7fcecb3e74a8, node=0x7fcdd89dfbe8, expr=@0x7fcecb3e8178: 0x0)
    at ./src/sql/resolver/expr/ob_raw_expr_resolver_impl.cpp:204
#4  0x000000000493bd90 in oceanbase::sql::ObRawExprResolverImpl::resolve (this=0x7fcecb3e74a8, node=0x7fcdd89dfbe8, expr=@0x7fcecb3e8178: 0x0, columns=...,
    sys_vars=..., sub_query_info=..., aggr_exprs=..., win_exprs=..., op_exprs=..., user_var_exprs=...)
    at ./src/sql/resolver/expr/ob_raw_expr_resolver_impl.cpp:47
#5  0x00000000064fd920 in oceanbase::sql::ObDMLResolver::resolve_sql_expr (this=0x7fcdd89eed30, node=..., expr=@0x7fcecb3e8178: 0x0,
    output_columns=0x7fcecb3e78e0) at ./src/sql/resolver/dml/ob_dml_resolver.cpp:207
#6  0x00000000065fd899 in oceanbase::sql::ObSelectResolver::resolve_field_list (this=0x7fcdd89eed30, node=...)
    at ./src/sql/resolver/dml/ob_select_resolver.cpp:1371
#7  0x00000000065f7ffb in oceanbase::sql::ObSelectResolver::resolve_normal_query (this=0x7fcdd89eed30, parse_tree=...)
    at ./src/sql/resolver/dml/ob_select_resolver.cpp:768
#8  0x0000000006603d90 in oceanbase::sql::ObSelectResolver::resolve (this=0x7fcdd89eed30, parse_tree=...)
    at ./src/sql/resolver/dml/ob_select_resolver.cpp:900

…

ObTransformerImpl::transform
ObTransformerImpl::transform 的 bt 信息如下：

(gdb) bt
#0  oceanbase::sql::ObTransformerImpl::transform (this=0x7fcecb3e9a18, stmt=@0x7fcecb3e9fe8: 0x7fcde07cc4d0) at ./src/sql/rewrite/ob_transformer_impl.cpp:51
#1  0x0000000005e6125b in oceanbase::sql::ObSql::transform_stmt (sql_schema_guard=0x7fcdd91cc5b0, partition_location_cache=0x7fcfa699db88,
    partition_service=0xe32a9c0 <oceanbase::storage::ObPartitionService::get_instance()::instance>,
    stat_mgr=0xec5e9a0 <oceanbase::common::ObStatManager::get_instance()::instance_>,
    opt_stat_mgr=0xec5d100 <oceanbase::common::ObOptStatManager::get_instance()::instance_>,
    self_addr=0xac09938 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111992>, merged_version=13, phy_plan=0x7fce06d31cd0, exec_ctx=...,
    stmt=@0x7fcecb3eaa58: 0x7fcde07cc4d0) at ./src/sql/ob_sql.cpp:1808
#2  0x0000000005e5cd9d in oceanbase::sql::ObSql::generate_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    parse_result=..., pc_ctx=0x7fcdd91de960, sql_ctx=..., result=..., is_ps_mode=false) at ./src/sql/ob_sql.cpp:1576
#3  0x0000000005e5a018 in oceanbase::sql::ObSql::handle_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    trimed_stmt=..., context=..., result=..., pc_ctx=..., get_plan_err=-5138) at ./src/sql/ob_sql.cpp:3155
#4  0x0000000005edad7d in oceanbase::sql::ObSql::handle_text_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    stmt=..., context=..., result=...) at ./src/sql/ob_sql.cpp:1184
#5  0x0000000005e4c611 in oceanbase::sql::ObSql::stmt_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>, stmt=...,
    context=..., result=...) at ./src/sql/ob_sql.cpp:169

…
● ObTransformerImpl::transform 先后调用 do_transform_pre_precessing, formalize_stmt_expr_reference, do_transform, do_transform_post_precessing, formalize_stmt_expr_reference, print_trans_stat。

ObOptimizer::optimize
ObOptimizer::optimize 的 bt 如下：

(gdb) bt
#0  oceanbase::sql::ObOptimizer::optimize (this=0x7fcecb3ea598, stmt=..., logical_plan=@0x7fcecb3ea590: 0x0) at ./src/sql/optimizer/ob_optimizer.cpp:26
#1  0x0000000005e624ae in oceanbase::sql::ObSql::optimize_stmt (optimizer=..., session_info=..., stmt=..., logical_plan=@0x7fcecb3ea590: 0x0)
    at ./src/sql/ob_sql.cpp:1886
#2  0x0000000005e5ceb4 in oceanbase::sql::ObSql::generate_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    parse_result=..., pc_ctx=0x7fcdf41de960, sql_ctx=..., result=..., is_ps_mode=false) at ./src/sql/ob_sql.cpp:1591
#3  0x0000000005e5a018 in oceanbase::sql::ObSql::handle_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    trimed_stmt=..., context=..., result=..., pc_ctx=..., get_plan_err=-5138) at ./src/sql/ob_sql.cpp:3155
#4  0x0000000005edad7d in oceanbase::sql::ObSql::handle_text_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    stmt=..., context=..., result=...) at ./src/sql/ob_sql.cpp:1184
#5  0x0000000005e4c611 in oceanbase::sql::ObSql::stmt_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>, stmt=...,
    context=..., result=...) at ./src/sql/ob_sql.cpp:169

● ObOptimizer::optimize 先后调用 init_env_info, generate_plan_for_temp_table, ObLogPlanFactory::create, init_plan_info, generate_plan 等函数

ObSql::code_generate
ObSql::code_generate 的 bt 如下：

(gdb) bt
#0  oceanbase::sql::ObSql::code_generate (sql_ctx=..., result=..., stmt=0x7f3efee01250, stmt_need_privs=..., stmt_ora_need_privs=...,
    logical_plan=0x7f3e4b9b9110, phy_plan=@0x7f3f5ce46588: 0x7f3ee73a7cd0) at ./src/sql/ob_sql.cpp:1960
#1  0x0000000005e5d0df in oceanbase::sql::ObSql::generate_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    parse_result=..., pc_ctx=0x7f3e861d9960, sql_ctx=..., result=..., is_ps_mode=false) at ./src/sql/ob_sql.cpp:1597
#2  0x0000000005e5a018 in oceanbase::sql::ObSql::handle_physical_plan (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    trimed_stmt=..., context=..., result=..., pc_ctx=..., get_plan_err=-5138) at ./src/sql/ob_sql.cpp:3155
#3  0x0000000005edad7d in oceanbase::sql::ObSql::handle_text_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>,
    stmt=..., context=..., result=...) at ./src/sql/ob_sql.cpp:1184
#4  0x0000000005e4c611 in oceanbase::sql::ObSql::stmt_query (this=0xac09700 <oceanbase::observer::ObServer::get_instance()::THE_ONE+1111424>, stmt=...,
    context=..., result=...) at ./src/sql/ob_sql.cpp:169

use_static_typing_engine_ 为 true，也就是使用新引擎，目前 4.x 版本只有新引擎了，3.1 版本还有新引擎和老引擎。

● ObSql::code_generate
○ 调用 ObCodeGenerator::generate 函数，由 logical_plan 生成 phy_plan
○ 设置 query_hint 的 query_timeout_
● ObCodeGenerator::generate 先后调用 generate_exprs 和 generate_operators
● ObCodeGenerator::generate_exprs 调用 ObStaticEngineExprCG::generate
● ObCodeGenerator::generate_operators 调用 ObStaticEngineCG::generate

以上仅为笔记的部分内容。

AntTech_AC0E2Y · 2023 年11 月 3 日 11:43

OBFan · 2023 年11 月 3 日 13:30

好的，非常感谢解答我的问题。还有一个问题是如果使用源码编译oceanbase，例如使用：

sh build.sh debug

之后进入相应目录使用：

make -j{N} observer

编译后的debug版本是默认开启了编译优化吗？如果开启了，优化级别是o2吗？如果想要不开启编译优化，即 -o0 那么应该怎样操作？

兹拉坦 · 2023 年11 月 7 日 13:52

麻烦 @官明看下这个编译相关的问题？