目录
摘要:
查询SQL:
处理该问题的思路分析:
削减查询SQL的操作符
tianmu引擎的查询SQL
innodb引擎的查询SQL
mysql/sql和innodb执行分析:
执行过程的trace日志:
摘出一些涉及查询执行的部分
思路分析:
mysql/sql层的核心处理:
Item_func_case::fix_fields
调用堆栈:
函数实现:
参数ref的数据:
函数目的分析:
Item_func_case::val_int
调用堆栈:
函数实现:
函数实现的运算逻辑的数据:
函数目的分析:
tianmu执行分析:
构建查询序列:
tianmu的构建查询序列的分析思路:
tianmu执行的核心处理:
MysqlExpression::EvalType
调用堆栈:
函数实现:
函数目的分析:
MysqlExpression::GetTianmufieldItem
调用堆栈:
函数实现:
函数目的分析:
Tianmu::core::Query::Preexecute
执行结果的临时表的case里属性的数据:
此函数目的:
Query::AddColumnForMysqlExpression
调用堆栈:
函数实现:
此函数目的:
Tianmu::core::Query::AddColumnForMysqlExpression
调用堆栈:
函数实现:
此函数目的:
TempTable::Materialize
调用堆栈:
函数实现:
此函数执行完后的case列属性中的数据
此函数目的:
AggregationAlgorithm::MultiDimensionalGroupByScan
调用堆栈:
对比当派生中有值且聚合能非NULL的结果时的数据:
此函数目的:
分析该查询出错的问题总结:
tianmu引擎对于case列属性以及聚合处理的设计意图:
修复该查询错误的问题:
摘要:
stonedb-在派生表的场景查询为空无法传递默认值-问题分析.
本文对该问题的成因, 相关功能的代码设计, 在下一步设计时如何应对这种问题, 做相关的分析。
https://stoneatom.yuque.com/staff-ft8n1u/lsztbl/rxlhws22n0f1otxn/edit#AqyB
相关ISSUE: https://github.com/stoneatom/stonedb/issues/1784
查询SQL:
SELECT 'aaaaa' inner_code,
a.exchange_rate,
sum(a.balance) AS balance,
'b' std_balance
FROM (SELECT b.inner_code,
CASE
WHEN r.row_id IS NOT NULL THEN
r.exchange_rate
ELSE
1.0000
END AS exchange_rate,
c.fiscal_date,
c.balance
FROM c1md_bank_acct a
JOIN c1md_company b
ON a.company_id = b.row_id
AND b.deleted_flag = '0'
LEFT JOIN c1cd_exchange_rate r
ON a.currency_id = r.currency_id
AND r.deleted_flag = '0'
AND r.using_flag = '1'
AND r.inure_date <= now()
AND (r.abate_date >= now() OR r.abate_date IS NULL)
LEFT JOIN (SELECT b.account_id, b.fiscal_date, b.balance
FROM (SELECT account_id, max(fiscal_date) fiscal_date
FROM c1am_acct_day
WHERE deleted_flag = '0'
GROUP BY account_id) a
JOIN c1am_acct_day b
ON a.account_id = b.account_id
AND a.fiscal_date = b.fiscal_date
WHERE b.deleted_flag = '0') c
ON a.row_id = c.account_id
WHERE a.deleted_flag = '0'
AND a.acct_flag IN ('2', '4')
AND a.company_id IN (3000000000027265)) a;
处理该问题的思路分析:
- 对于复杂的查询SQL, 涉及的操作符过多, 首先要进行问题定位
- 缩减到具体哪些操作符的执行出现了问题
- 这个过程涉及条件削减, 排查, 甚至摸索试探
- 找到出问题的操作符的对应的代码实现, 分析该处代码的设计意图，以及为什么会导致当前查询出错
- 该过程涉及架构能力，代码能力，以及理解他人代码设计意图的能力
- 对于复杂的代码, 牵扯到的因素过多，存在试探来猜测设计意图的情况, 但是一定需要事实的验证来辅助对设计意图的定位
- 重新对该操作符做设计
- 需要注意要兼容此前的设计意图
- 为了避免出现类似的问题, 需要评估代码修改对其他模块的影响
削减查询SQL的操作符
tianmu引擎的查询SQL
create table t1 (age int) engine=tianmu;select case when age IS NOT NULL THEN age else 33 end, sum(age) from ( select * from t1) ta;
innodb引擎的查询SQL
create table t1_innodb (age int) engine=innodb;select case when age IS NOT NULL THEN age else 33 end, sum(age) from ( select * from t1_innodb) ta;mysql/sql和innodb执行分析:
执行过程的trace日志:
2022-10-18 mysql-5.7-开启debug日志_mysql 5.7 debug_财阀悟世的博客-CSDN博客
https://download.csdn.net/download/adofsauron/87864107
摘出一些涉及查询执行的部分
T@3: 2008: sql_executor.cc: 887: | | | | | | >do_select
T@3: 2009: sql_executor.cc: 1226: | | | | | | | >sub_select
T@3: 2010: records.cc: 194: | | | | | | | | >init_read_record
T@3: 2011: records.cc: 333: | | | | | | | | | info: using rr_sequential
T@3: 2012: handler.cc: 2924: | | | | | | | | | >ha_rnd_init
T@3: 2013: ha_innodb.cc: 8960: | | | | | | | | | | >change_active_index
T@3: 2014: ha_innodb.cc: 8905: | | | | | | | | | | | >innobase_get_index
T@3: 2015: ha_innodb.cc: 8946: | | | | | | | | | | | <innobase_get_index 8946
T@3: 2016: ha_innodb.cc: 9064: | | | | | | | | | | <change_active_index 9064
T@3: 2017: handler.cc: 2931: | | | | | | | | | <ha_rnd_init 2931
T@3: 2018: ha_innodb.cc: 2887: | | | | | | | | | >ha_innobase::update_thd
T@3: 2019: ha_innodb.cc: 2889: | | | | | | | | | | ha_innobase::update_thd: user_thd: 0x7fd254000e10 -> 0x7fd254000e10
T@3: 2020: ha_innodb.cc: 2618: | | | | | | | | | | >innobase_trx_init
T@3: 2021: ha_innodb.cc: 2628: | | | | | | | | | | <innobase_trx_init 2628
T@3: 2022: ha_innodb.cc: 2913: | | | | | | | | | <ha_innobase::update_thd 2913
T@3: 2023: records.cc: 362: | | | | | | | | <init_read_record 362
T@3: 2024: handler.cc: 2971: | | | | | | | | >handler::ha_rnd_next
T@3: 2025: ha_innodb.cc: 9315: | | | | | | | | | >rnd_next
T@3: 2026: ha_innodb.cc: 9218: | | | | | | | | | | >index_first
T@3: 2027: ha_innodb.cc: 8711: | | | | | | | | | | | >index_read
T@3: 2028: row0sel.cc: 4681: | | | | | | | | | | | | >row_search_mvcc
T@3: 2029: row0sel.cc: 6434: | | | | | | | | | | | | <row_search_mvcc 6434
T@3: 2030: ha_innodb.cc: 8871: | | | | | | | | | | | <index_read 8871
T@3: 2031: ha_innodb.cc: 9230: | | | | | | | | | | <index_first 9230
T@3: 2032: ha_innodb.cc: 9331: | | | | | | | | | <rnd_next 9331
T@3: 2033: handler.cc: 2986: | | | | | | | | <handler::ha_rnd_next 2986
T@3: 2034: sql_executor.cc: 1316: | | | | | | | <sub_select 1316
T@3: 2035: sql_executor.cc: 1226: | | | | | | | >sub_select
T@3: 2036: sql_executor.cc: 3019: | | | | | | | | >end_send_group
T@3: 2037: item_sum.cc: 1438: | | | | | | | | | >Item_sum_sum::clear
T@3: 2038: item_sum.cc: 1447: | | | | | | | | | <Item_sum_sum::clear 1447
T@3: 2039: sql_class.cc: 2726: | | | | | | | | | >Query_result_send::send_data
T@3: 2040: sql_class.cc: 4753: | | | | | | | | | | >send_result_set_row
T@3: 2041: sql_class.cc: 4765: | | | | | | | | | | <send_result_set_row 4765
T@3: 2042: sql_class.cc: 2749: | | | | | | | | | <Query_result_send::send_data 2749
T@3: 2043: protocol_classic.cc: 1202: | | | | | | | | | >Protocol_classic::end_row
T@3: 2044: protocol_classic.cc: 1205: | | | | | | | | | <Protocol_classic::end_row 1205
T@3: 2045: sql_executor.cc: 355: | | | | | | | | | packet_header: Memory: 0x7fd519f6e944 Bytes: (4)
04 00 00 04
T@3: 2046: sql_executor.cc: 3084: | | | | | | | | <end_send_group 3084
T@3: 2047: sql_executor.cc: 1234: | | | | | | | <sub_select 1234
T@3: 2048: sql_select.cc: 2546: | | | | | | | >JOIN::join_free
T@3: 2049: sql_select.cc: 2608: | | | | | | | | >JOIN::cleanup
T@3: 2050: handler.cc: 2945: | | | | | | | | | >ha_rnd_end
T@3: 2051: handler.cc: 2953: | | | | | | | | | <ha_rnd_end 2953
T@3: 2052: ha_innodb.cc: 8580: | | | | | | | | | >index_end
T@3: 2053: handler.cc: 6761: | | | | | | | | | | >DsMrr_impl::dsmrr_close
T@3: 2054: handler.cc: 6770: | | | | | | | | | | <DsMrr_impl::dsmrr_close 6770
T@3: 2055: ha_innodb.cc: 8592: | | | | | | | | | <index_end 8592
T@3: 2056: sql_base.cc: 1147: | | | | | | | | | >free_io_cache
T@3: 2057: sql_base.cc: 1154: | | | | | | | | | <free_io_cache 1154
T@3: 2058: filesort.cc: 591: | | | | | | | | | >filesort_free_buffers
T@3: 2059: filesort.cc: 604: | | | | | | | | | <filesort_free_buffers 604
T@3: 2060: sql_select.cc: 2656: | | | | | | | | <JOIN::cleanup 2656
T@3: 2061: lock.cc: 449: | | | | | | | | >mysql_unlock_read_tables
T@3: 2062: lock.cc: 666: | | | | | | | | | >unlock_external
T@3: 2063: handler.cc: 8073: | | | | | | | | | | >handler::ha_external_lock
T@3: 2064: ha_innodb.cc: 15636: | | | | | | | | | | | >ha_innobase::external_lock
T@3: 2065: ha_innodb.cc: 15637: | | | | | | | | | | | | enter: lock_type: 2
T@3: 2066: ha_innodb.cc: 2887: | | | | | | | | | | | | >ha_innobase::update_thd
T@3: 2067: ha_innodb.cc: 2889: | | | | | | | | | | | | | ha_innobase::update_thd: user_thd: 0x7fd254000e10 -> 0x7fd254000e10
T@3: 2068: ha_innodb.cc: 2618: | | | | | | | | | | | | | >innobase_trx_init
T@3: 2069: ha_innodb.cc: 2628: | | | | | | | | | | | | | <innobase_trx_init 2628
T@3: 2070: ha_innodb.cc: 2913: | | | | | | | | | | | | <ha_innobase::update_thd 2913
T@3: 2071: ha_innodb.cc: 4379: | | | | | | | | | | | | >innobase_commit
T@3: 2072: ha_innodb.cc: 4381: | | | | | | | | | | | | | trans: ending transaction
T@3: 2073: ha_innodb.cc: 2618: | | | | | | | | | | | | | >innobase_trx_init
T@3: 2074: ha_innodb.cc: 2628: | | | | | | | | | | | | | <innobase_trx_init 2628
T@3: 2075: ha_innodb.cc: 4515: | | | | | | | | | | | | <innobase_commit 4515
T@3: 2076: ha_innodb.cc: 15895: | | | | | | | | | | | <ha_innobase::external_lock 15895
T@3: 2077: handler.cc: 8135: | | | | | | | | | | <handler::ha_external_lock 8135
T@3: 2078: lock.cc: 682: | | | | | | | | | <unlock_external 682
T@3: 2079: lock.cc: 499: | | | | | | | | <mysql_unlock_read_tables 499
T@3: 2080: sql_select.cc: 2590: | | | | | | | <JOIN::join_free 2590
T@3: 2081: sql_error.cc: 409: | | | | | | | >set_eof_status
T@3: 2082: sql_error.cc: 431: | | | | | | | <set_eof_status 431
T@3: 2083: sql_executor.cc: 1008: | | | | | | | info: 1 records output
T@3: 2084: sql_executor.cc: 1019: | | | | | | <do_select 1019
T@3: 2085: sql_executor.cc: 210: | | | | | | counts: thd->examined_row_count: 0
T@3: 2086: sql_executor.cc: 212: | | | | | <JOIN::exec 212
T@3: 2087: sql_select.cc: 286: | | | | | opt: steps: ending struct
T@3: 2088: sql_select.cc: 286: | | | | | opt: join_execution: ending struct
T@3: 2089: sql_select.cc: 286: | | | | | opt: (null): ending struct
T@3: 2090: sql_select.cc: 741: | | | | | THD::enter_stage: 'end' /root/work/stonedb-dev-20230605/sql/sql_select.cc:210
T@3: 2091: sql_profile.cc: 365: | | | | | >PROFILING::status_change
T@3: 2092: sql_profile.cc: 371: | | | | | <PROFILING::status_change 371
T@3: 2093: sql_union.cc: 920: | | | | | >st_select_lex_unit::cleanup
T@3: 2094: sql_union.cc: 1078: | | | | | | >st_select_lex::cleanup()
T@3: 2095: sql_select.cc: 2608: | | | | | | | >JOIN::cleanup
T@3: 2096: sql_base.cc: 1147: | | | | | | | | >free_io_cache
T@3: 2097: sql_base.cc: 1154: | | | | | | | | <free_io_cache 1154
T@3: 2098: filesort.cc: 591: | | | | | | | | >filesort_free_buffers
T@3: 2099: filesort.cc: 604: | | | | | | | | <filesort_free_buffers 604
T@3: 2100: sql_select.cc: 2656: | | | | | | | <JOIN::cleanup 2656
T@3: 2101: sql_union.cc: 1100: | | | | | | <st_select_lex::cleanup() 1100
T@3: 2102: sql_union.cc: 952: | | | | | <st_select_lex_unit::cleanup 952
T@3: 2103: sql_select.cc: 215: | | | | <handle_query 215
T@3: 2104: sql_parse.cc: 741: | | | | THD::enter_stage: 'query end' /root/work/stonedb-dev-20230605/sql/sql_parse.cc:4993
T@3: 2105: sql_profile.cc: 365: | | | | >PROFILING::status_change
T@3: 2106: sql_profile.cc: 371: | | | | <PROFILING::status_change 371
T@3: 2107: sql_audit.cc: 1111: | | | | >mysql_audit_acquire_plugins
T@3: 2108: sql_audit.cc: 1149: | | | | <mysql_audit_acquire_plugins 1149
T@3: 2109: transaction.cc: 445: | | | | >trans_commit_stmt
T@3: 2110: transaction.cc: 293: | | | | | debug: add_unsafe_rollback_flags: 0
T@3: 2111: handler.cc: 1597: | | | | | >commit_owned_gtids(...)
T@3: 2112: handler.cc: 1626: | | | | | <commit_owned_gtids(...) 1626
T@3: 2113: handler.cc: 1718: | | | | | >ha_commit_trans
T@3: 2114: handler.cc: 1721: | | | | | | info: all=0 thd->in_sub_stmt=0 ha_info=0x7fd254002338 is_real_trans=1
T@3: 2115: binlog.cc: 8639: | | | | | | >MYSQL_BIN_LOG::commit
T@3: 2116: binlog.cc: 8641: | | | | | | | info: query='select case when age IS NOT NULL THEN age else 33 end, sum(age) from ( select * from t1_innodb) ta'
T@3: 2117: binlog.cc: 8651: | | | | | | | enter: thd: 0x7fd254000e10, all: no, xid: 6, cache_mngr: 0x0
T@3: 2118: handler.cc: 1905: | | | | | | | >ha_commit_low
T@3: 2119: ha_innodb.cc: 4379: | | | | | | | | >innobase_commit
T@3: 2120: ha_innodb.cc: 4381: | | | | | | | | | trans: ending transaction
T@3: 2121: ha_innodb.cc: 2618: | | | | | | | | | >innobase_trx_init
T@3: 2122: ha_innodb.cc: 2628: | | | | | | | | | <innobase_trx_init 2628
T@3: 2123: ha_innodb.cc: 4515: | | | | | | | | <innobase_commit 4515
T@3: 2124: transaction_info.h: 97: | | | | | | | | >Ha_trx_info::reset
T@3: 2125: transaction_info.h: 101: | | | | | | | | <Ha_trx_info::reset 101
T@3: 2126: handler.cc: 1966: | | | | | | | <ha_commit_low 1966
T@3: 2127: binlog.cc: 8661: | | | | | | <MYSQL_BIN_LOG::commit 8661
T@3: 2128: transaction_info.h: 409: | | | | | | >Transaction_ctx::cleanup
T@3: 2129: rpl_transaction_ctx.cc: 40: | | | | | | | >Rpl_transaction_ctx::cleanup
T@3: 2130: rpl_transaction_ctx.cc: 47: | | | | | | | <Rpl_transaction_ctx::cleanup 47
T@3: 2131: my_alloc.c: 445: | | | | | | | >free_root
T@3: 2132: my_alloc.c: 446: | | | | | | | | enter: root: 0x7fd2540048b0 flags: 1
T@3: 2133: my_alloc.c: 489: | | | | | | | <free_root 489
T@3: 2134: transaction_info.h: 416: | | | | | | <Transaction_ctx::cleanup 416
T@3: 2135: handler.cc: 1875: | | | | | <ha_commit_trans 1875
T@3: 2136: rpl_context.cc: 65: | | | | | >Rpl_consistency_ctx::notify_after_transaction_commit
T@3: 2137: rpl_context.cc: 70: | | | | | <Rpl_consistency_ctx::notify_after_transaction_commit 70
T@3: 2138: transaction.cc: 298: | | | | | debug: reset_unsafe_rollback_flags
T@3: 2139: transaction.cc: 481: | | | | <trans_commit_stmt 481
T@3: 2140: sql_union.cc: 920: | | | | >st_select_lex_unit::cleanup
T@3: 2141: sql_union.cc: 1078: | | | | | >st_select_lex::cleanup()
T@3: 2142: sql_select.cc: 939: | | | | | | >JOIN::destroy
T@3: 2143: filesort.cc: 591: | | | | | | | >filesort_free_buffers
T@3: 2144: filesort.cc: 604: | | | | | | | <filesort_free_buffers 604
T@3: 2145: ha_innodb.cc: 2887: | | | | | | | >ha_innobase::update_thd
T@3: 2146: ha_innodb.cc: 2889: | | | | | | | | ha_innobase::update_thd: user_thd: 0x7fd254000e10 -> 0x7fd254000e10
T@3: 2147: ha_innodb.cc: 2618: | | | | | | | | >innobase_trx_init
T@3: 2148: ha_innodb.cc: 2628: | | | | | | | | <innobase_trx_init 2628
T@3: 2149: ha_innodb.cc: 2913: | | | | | | | <ha_innobase::update_thd 2913
T@3: 2150: sql_base.cc: 1147: | | | | | | | >free_io_cache
T@3: 2151: sql_base.cc: 1154: | | | | | | | <free_io_cache 1154
T@3: 2152: filesort.cc: 591: | | | | | | | >filesort_free_buffers
T@3: 2153: filesort.cc: 604: | | | | | | | <filesort_free_buffers 604
T@3: 2154: item.cc: 798: | | | | | | | >Item::cleanup
T@3: 2155: item.cc: 804: | | | | | | | <Item::cleanup 804
T@3: 2156: item.cc: 10834: | | | | | | | >Item_result_field::cleanup()
T@3: 2157: item.cc: 798: | | | | | | | | >Item::cleanup
T@3: 2158: item.cc: 804: | | | | | | | | <Item::cleanup 804
T@3: 2159: item.cc: 10837: | | | | | | | <Item_result_field::cleanup() 10837
T@3: 2160: sql_select.cc: 986: | | | | | | <JOIN::destroy 986
T@3: 2161: sql_union.cc: 1100: | | | | | <st_select_lex::cleanup() 1100
T@3: 2162: sql_union.cc: 952: | | | | <st_select_lex_unit::cleanup 952
T@3: 2163: sql_parse.cc: 741: | | | | THD::enter_stage: 'closing tables' /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5052
T@3: 2164: sql_profile.cc: 365: | | | | >PROFILING::status_change
T@3: 2165: sql_profile.cc: 371: | | | | <PROFILING::status_change 371
T@3: 2166: sql_base.cc: 1583: | | | | >close_thread_tables
T@3: 2167: sql_base.cc: 1607: | | | | | tcache: table: 't1_innodb' query_id: 0
T@3: 2168: ha_innodb.cc: 2887: | | | | | >ha_innobase::update_thd
T@3: 2169: ha_innodb.cc: 2889: | | | | | | ha_innobase::update_thd: user_thd: 0x7fd254000e10 -> 0x7fd254000e10
T@3: 2170: ha_innodb.cc: 2618: | | | | | | >innobase_trx_init
T@3: 2171: ha_innodb.cc: 2628: | | | | | | <innobase_trx_init 2628
T@3: 2172: ha_innodb.cc: 2913: | | | | | <ha_innobase::update_thd 2913
T@3: 2173: binlog.cc: 11975: | | | | | >THD::binlog_flush_pending_rows_event
T@3: 2174: binlog.cc: 12001: | | | | | <THD::binlog_flush_pending_rows_event 12001
T@3: 2175: binlog.cc: 11975: | | | | | >THD::binlog_flush_pending_rows_event
T@3: 2176: binlog.cc: 12001: | | | | | <THD::binlog_flush_pending_rows_event 12001
T@3: 2177: lock.cc: 419: | | | | | >mysql_unlock_tables
T@3: 2178: my_malloc.c: 297: | | | | | | >my_free
T@3: 2179: my_malloc.c: 298: | | | | | | | my: ptr: 0x7fd254012370
T@3: 2180: my_malloc.c: 304: | | | | | | <my_free 304
T@3: 2181: lock.cc: 425: | | | | | <mysql_unlock_tables 425
T@3: 2182: sql_base.cc: 1468: | | | | | info: thd->open_tables: 0x7fd2540150c0
T@3: 2183: sql_base.cc: 1758: | | | | | >close_thread_table
T@3: 2184: ha_innodb.cc: 2887: | | | | | | >ha_innobase::update_thd
T@3: 2185: ha_innodb.cc: 2889: | | | | | | | ha_innobase::update_thd: user_thd: 0x7fd254000e10 -> 0x7fd254000e10
T@3: 2186: ha_innodb.cc: 2618: | | | | | | | >innobase_trx_init
T@3: 2187: ha_innodb.cc: 2628: | | | | | | | <innobase_trx_init 2628
T@3: 2188: ha_innodb.cc: 2913: | | | | | | <ha_innobase::update_thd 2913
T@3: 2189: handler.cc: 8146: | | | | | | >handler::ha_reset
T@3: 2190: sql_base.cc: 1147: | | | | | | | >free_io_cache
T@3: 2191: sql_base.cc: 1154: | | | | | | | <free_io_cache 1154
T@3: 2192: handler.cc: 6776: | | | | | | | >DsMrr_impl::reset
T@3: 2193: handler.cc: 6788: | | | | | | | <DsMrr_impl::reset 6788
T@3: 2194: handler.cc: 8165: | | | | | | <handler::ha_reset 8165
T@3: 2195: sql_base.cc: 1805: | | | | | <close_thread_table 1805
T@3: 2196: sql_base.cc: 1749: | | | | <close_thread_tables 1749
T@3: 2197: sql_parse.cc: 278: | | | | >stmt_causes_implicit_commit
T@3: 2198: sql_parse.cc: 281: | | | | <stmt_causes_implicit_commit 281
T@3: 2199: mdl.cc: 4720: | | | | >MDL_context::release_transactional_locks
T@3: 2200: mdl.cc: 4441: | | | | | >MDL_context::release_locks_stored_before
T@3: 2201: mdl.cc: 4444: | | | | | <MDL_context::release_locks_stored_before 4444
T@3: 2202: mdl.cc: 4441: | | | | | >MDL_context::release_locks_stored_before
T@3: 2203: mdl.cc: 4448: | | | | | | info: found lock to release ticket=0x7fd254012730
T@3: 2204: mdl.cc: 4288: | | | | | | >MDL_context::release_lock
T@3: 2205: mdl.cc: 4290: | | | | | | | enter: db=test name=t1_innodb
T@3: 2206: mdl.cc: 4404: | | | | | | <MDL_context::release_lock 4404
T@3: 2207: mdl.cc: 4452: | | | | | <MDL_context::release_locks_stored_before 4452
T@3: 2208: mdl.cc: 4723: | | | | <MDL_context::release_transactional_locks 4723
T@3: 2209: sql_parse.cc: 2393: | | | | >binlog_gtid_end_transaction
T@3: 2210: sql_parse.cc: 278: | | | | | >stmt_causes_implicit_commit
T@3: 2211: sql_parse.cc: 281: | | | | | <stmt_causes_implicit_commit 281
T@3: 2212: sql_parse.cc: 2435: | | | | <binlog_gtid_end_transaction 2435
T@3: 2213: sql_parse.cc: 5150: | | | <mysql_execute_command 5150
T@3: 2214: sql_parse.cc: 286: | | | opt: steps: ending struct
T@3: 2215: sql_parse.cc: 286: | | | opt: (null): ending struct
T@3: 2216: opt_trace2server.cc: 236: | | | >~opt_trace_start
T@3: 2217: my_malloc.c: 187: | | | | >my_raw_malloc
T@3: 2218: my_malloc.c: 188: | | | | | my: size: 40 my_flags: 16
T@3: 2219: my_malloc.c: 230: | | | | | exit: ptr: 0x7fd254027bd0
T@3: 2220: my_malloc.c: 231: | | | | <my_raw_malloc 231
Complete optimizer trace:
T@3: 2221: opt_trace.cc: 1155: | | | | >Opt_trace_context::purge_stmts
T@3: 2222: my_malloc.c: 297: | | | | | >my_free
T@3: 2223: my_malloc.c: 298: | | | | | | my: ptr: 0x7fd254027bd0
T@3: 2224: my_malloc.c: 304: | | | | | <my_free 304
T@3: 2225: opt_trace.cc: 1246: | | | | <Opt_trace_context::purge_stmts 1246
T@3: 2226: opt_trace2server.cc: 239: | | | <~opt_trace_start 239
T@3: 2227: sql_parse.cc: 741: | | | THD::enter_stage: 'freeing items' /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5680
T@3: 2228: sql_profile.cc: 365: | | | >PROFILING::status_change
T@3: 2229: sql_profile.cc: 371: | | | <PROFILING::status_change 371
T@3: 2230: sql_lex.cc: 528: | | | >lex_end
T@3: 2231: sql_lex.cc: 529: | | | | enter: lex: 0x7fd254003138
T@3: 2232: sql_plugin.cc: 1203: | | | | >plugin_unlock_list
T@3: 2233: sql_plugin.cc: 1132: | | | | | >intern_plugin_unlock
T@3: 2234: my_malloc.c: 297: | | | | | | >my_free
T@3: 2235: my_malloc.c: 298: | | | | | | | my: ptr: 0x7fd254007b20
T@3: 2236: my_malloc.c: 304: | | | | | | <my_free 304
T@3: 2237: sql_plugin.cc: 1149: | | | | | | info: unlocking plugin, name= InnoDB, ref_count= 18
T@3: 2238: sql_plugin.cc: 1177: | | | | | <intern_plugin_unlock 1177
T@3: 2239: sql_plugin.cc: 1132: | | | | | >intern_plugin_unlock
T@3: 2240: my_malloc.c: 297: | | | | | | >my_free
T@3: 2241: my_malloc.c: 298: | | | | | | | my: ptr: 0x7fd2540115e0
T@3: 2242: my_malloc.c: 304: | | | | | | <my_free 304
T@3: 2243: sql_plugin.cc: 1149: | | | | | | info: unlocking plugin, name= InnoDB, ref_count= 17
T@3: 2244: sql_plugin.cc: 1177: | | | | | <intern_plugin_unlock 1177
T@3: 2245: sql_plugin.cc: 1218: | | | | <plugin_unlock_list 1218
T@3: 2246: sql_lex.cc: 537: | | | <lex_end 537
T@3: 2247: sql_class.cc: 3412: | | | >Query_arena::free_items
T@3: 2248: item.cc: 798: | | | | >Item::cleanup
T@3: 2249: item.cc: 804: | | | | <Item::cleanup 804
T@3: 2250: item.cc: 8287: | | | | >Item_ref::cleanup
T@3: 2251: item.cc: 944: | | | | | >Item_ident::cleanup
T@3: 2252: item.cc: 798: | | | | | | >Item::cleanup
T@3: 2253: item.cc: 804: | | | | | | <Item::cleanup 804
T@3: 2254: item.cc: 949: | | | | | <Item_ident::cleanup 949
T@3: 2255: item.cc: 8292: | | | | <Item_ref::cleanup 8292
T@3: 2256: item.cc: 8287: | | | | >Item_ref::cleanup
T@3: 2257: item.cc: 944: | | | | | >Item_ident::cleanup
T@3: 2258: item.cc: 798: | | | | | | >Item::cleanup
T@3: 2259: item.cc: 804: | | | | | | <Item::cleanup 804
T@3: 2260: item.cc: 949: | | | | | <Item_ident::cleanup 949
T@3: 2261: item.cc: 8292: | | | | <Item_ref::cleanup 8292
T@3: 2262: item.cc: 8287: | | | | >Item_ref::cleanup
T@3: 2263: item.cc: 944: | | | | | >Item_ident::cleanup
T@3: 2264: item.cc: 798: | | | | | | >Item::cleanup
T@3: 2265: item.cc: 804: | | | | | | <Item::cleanup 804
T@3: 2266: item.cc: 949: | | | | | <Item_ident::cleanup 949
T@3: 2267: item.cc: 8292: | | | | <Item_ref::cleanup 8292
T@3: 2268: item.cc: 6071: | | | | >Item_field::cleanup
T@3: 2269: item.cc: 944: | | | | | >Item_ident::cleanup
T@3: 2270: item.cc: 798: | | | | | | >Item::cleanup
T@3: 2271: item.cc: 804: | | | | | | <Item::cleanup 804
T@3: 2272: item.cc: 949: | | | | | <Item_ident::cleanup 949
T@3: 2273: item.cc: 6082: | | | | <Item_field::cleanup 6082
T@3: 2274: item.cc: 6071: | | | | >Item_field::cleanup
T@3: 2275: item.cc: 944: | | | | | >Item_ident::cleanup
T@3: 2276: item.cc: 798: | | | | | | >Item::cleanup
T@3: 2277: item.cc: 804: | | | | | | <Item::cleanup 804
T@3: 2278: item.cc: 949: | | | | | <Item_ident::cleanup 949
T@3: 2279: item.cc: 6082: | | | | <Item_field::cleanup 6082
T@3: 2280: item.cc: 6071: | | | | >Item_field::cleanup
T@3: 2281: item.cc: 944: | | | | | >Item_ident::cleanup
T@3: 2282: item.cc: 798: | | | | | | >Item::cleanup
T@3: 2283: item.cc: 804: | | | | | | <Item::cleanup 804
T@3: 2284: item.cc: 949: | | | | | <Item_ident::cleanup 949
T@3: 2285: item.cc: 6082: | | | | <Item_field::cleanup 6082
T@3: 2286: item.cc: 798: | | | | >Item::cleanup
T@3: 2287: item.cc: 804: | | | | <Item::cleanup 804
T@3: 2288: item.cc: 798: | | | | >Item::cleanup
T@3: 2289: item.cc: 804: | | | | <Item::cleanup 804
T@3: 2290: item.cc: 10834: | | | | >Item_result_field::cleanup()
T@3: 2291: item.cc: 798: | | | | | >Item::cleanup
T@3: 2292: item.cc: 804: | | | | | <Item::cleanup 804
T@3: 2293: item.cc: 10837: | | | | <Item_result_field::cleanup() 10837
T@3: 2294: item.cc: 798: | | | | >Item::cleanup
T@3: 2295: item.cc: 804: | | | | <Item::cleanup 804
T@3: 2296: item.cc: 6071: | | | | >Item_field::cleanup
T@3: 2297: item.cc: 944: | | | | | >Item_ident::cleanup
T@3: 2298: item.cc: 798: | | | | | | >Item::cleanup
T@3: 2299: item.cc: 804: | | | | | | <Item::cleanup 804
T@3: 2300: item.cc: 949: | | | | | <Item_ident::cleanup 949
T@3: 2301: item.cc: 6082: | | | | <Item_field::cleanup 6082
T@3: 2302: item.cc: 798: | | | | >Item::cleanup
T@3: 2303: item.cc: 804: | | | | <Item::cleanup 804
T@3: 2304: item.cc: 6071: | | | | >Item_field::cleanup
T@3: 2305: item.cc: 944: | | | | | >Item_ident::cleanup
T@3: 2306: item.cc: 798: | | | | | | >Item::cleanup
T@3: 2307: item.cc: 804: | | | | | | <Item::cleanup 804
T@3: 2308: item.cc: 949: | | | | | <Item_ident::cleanup 949
T@3: 2309: item.cc: 6082: | | | | <Item_field::cleanup 6082
T@3: 2310: item.cc: 798: | | | | >Item::cleanup
T@3: 2311: item.cc: 804: | | | | <Item::cleanup 804
T@3: 2312: item.cc: 10834: | | | | >Item_result_field::cleanup()
T@3: 2313: item.cc: 798: | | | | | >Item::cleanup
T@3: 2314: item.cc: 804: | | | | | <Item::cleanup 804
T@3: 2315: item.cc: 10837: | | | | <Item_result_field::cleanup() 10837
T@3: 2316: item_cmpfunc.cc: 4243: | | | | >Item_func_case::cleanup
T@3: 2317: item.cc: 10834: | | | | | >Item_result_field::cleanup()
T@3: 2318: item.cc: 798: | | | | | | >Item::cleanup
T@3: 2319: item.cc: 804: | | | | | | <Item::cleanup 804
T@3: 2320: item.cc: 10837: | | | | | <Item_result_field::cleanup() 10837
T@3: 2321: item_cmpfunc.cc: 4250: | | | | <Item_func_case::cleanup 4250思路分析:
- 该debug的trace日志包含这个查询SQL的所有关键函数的执行过程
- 从这个日志中可以分析出mysql/sql层所有的查询优化和查询执行的细节
- 但是将问题聚焦下, 当前的目标是解决tianmu查询出错, 那么对应的就需要关注mysql/sql在处理case属性时是如何处理的
- 也就是说，要将重点，放在mysql/sql层是如何处理case的item的
- 这里需要注意, 分析case的item并不是为了要记住这块代码的处理逻辑, 而是作为一个正确处理case的解决方案的对标, 分析tianmu引擎在处理case的属性时的逻辑，从而有个逻辑处理的对比
- tianmu引擎对于case的属性有自己的一套处理逻辑, 在此处分析的一个歧途就是容易将mysql/sql当作唯一正确的接, 从而照抄mysql/sql的方案, 而忽略了tianmu引擎对于列属性处理的特殊性, 此特殊的地方在以下分析tianmu引擎对查询执行时具体分析
mysql/sql层的核心处理:
Item_func_case::fix_fields
调用堆栈:
#0 Item_func_case::fix_fields (this=0x7fd254005850, thd=0x7fd254000e10, ref=0x7fd254005ac0) at /root/work/stonedb-dev-20230605/sql/item_cmpfunc.cc:4001
#1 0x00000000023a8a7e in setup_fields (thd=0x7fd254000e10, ref_pointer_array=..., fields=..., want_privilege=1, sum_func_list=0x7fd254004d30, allow_sum_func=true, column_update=false)
at /root/work/stonedb-dev-20230605/sql/sql_base.cc:9138
#2 0x00000000024650da in st_select_lex::prepare (this=0x7fd254004bd0, thd=0x7fd254000e10) at /root/work/stonedb-dev-20230605/sql/sql_resolver.cc:200
#3 0x000000000247165f in handle_query (thd=0x7fd254000e10, lex=0x7fd254003138, result=0x7fd254006a88, added_options=0, removed_options=0, is_optimize_after_tianmu=0, free_join_from_tianmu=0)
at /root/work/stonedb-dev-20230605/sql/sql_select.cc:139
#4 0x0000000002427ae2 in execute_sqlcom_select (thd=0x7fd254000e10, all_tables=0x7fd254014a88) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5205
#5 0x0000000002420e1e in mysql_execute_command (thd=0x7fd254000e10, first_level=true) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:2847
#6 0x0000000002428b0d in mysql_parse (thd=0x7fd254000e10, parser_state=0x7fd519f6ff90) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5642
#7 0x000000000241db04 in dispatch_command (thd=0x7fd254000e10, com_data=0x7fd519f70730, command=COM_QUERY) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1495
#8 0x000000000241c945 in do_command (thd=0x7fd254000e10) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1034
#9 0x000000000254eeb5 in handle_connection (arg=0x965db20) at /root/work/stonedb-dev-20230605/sql/conn_handler/connection_handler_per_thread.cc:313
#10 0x0000000002c1e6f4 in pfs_spawn_thread (arg=0x9490860) at /root/work/stonedb-dev-20230605/storage/perfschema/pfs.cc:2197
#11 0x00007fd569c2b1ca in start_thread () from /lib64/libpthread.so.0
#12 0x00007fd566aa1e73 in clone () from /lib64/libc.so.6
函数实现:
bool Item_func_case::fix_fields(THD *thd, Item **ref)
{
/*
buff should match stack usage from
Item_func_case::val_int() -> Item_func_case::find_item()
*/
uchar buff[MAX_FIELD_WIDTH*2+sizeof(String)*2+sizeof(String*)*2+sizeof(double)*2+sizeof(longlong)*2];
bool res= Item_func::fix_fields(thd, ref);
/*
Call check_stack_overrun after fix_fields to be sure that stack variable
is not optimized away
*/
if (check_stack_overrun(thd, STACK_MIN_SIZE, buff))
return TRUE; // Fatal error flag is set!
return res;
}参数ref的数据:
(gdb) p ref[0][0]
$4 = (Item_func_case) {
<Item_func> = {
<Item_result_field> = {
<Item> = {
<Parse_tree_node> = {
_vptr.Parse_tree_node = 0x4355418 <vtable for Item_func_case+16>,
contextualized = true,
transitional = false
},
members of Item:
is_expensive_cache = -1 '\377',
rsize = 0,
str_value = {
m_ptr = 0x0,
m_length = 0,
m_charset = 0x44664e0 <my_charset_bin>,
m_alloced_length = 0,
m_is_alloced = false
},
item_name = {
<Name_string> = {
<Simple_cstring> = {
m_str = 0x7fd2540065e0 "case when age IS NOT NULL THEN age else 33 end",
m_length = 47
}, <No data fields>},
members of Item_name_string:
m_is_autogenerated = true
},
orig_name = {
<Name_string> = {
<Simple_cstring> = {
m_str = 0x0,
m_length = 0
}, <No data fields>},
members of Item_name_string:
m_is_autogenerated = true
},
next = 0x7fd2540059d0,
max_length = 0,
marker = 0,
decimals = 0 '\000',
maybe_null = 0 '\000',
null_value = 0 '\000',
unsigned_flag = 0 '\000',
with_sum_func = 0 '\000',
fixed = 0 '\000',
collation = {
collation = 0x44664e0 <my_charset_bin>,
--Type <RET> for more, q to quit, c to continue without paging--
derivation = DERIVATION_COERCIBLE,
repertoire = 3
},
cmp_context = 4294967295,
runtime_item = false,
derived_used = false,
with_subselect = 0 '\000',
with_stored_program = 0 '\000',
tables_locked_cache = false,
is_parser_item = true
},
members of Item_result_field:
result_field = 0x0
},
members of Item_func:
args = 0x7fd2540059b8,
tmp_arg = {0x8f8f8f8f8f8f8f8f, 0x8f8f8f8f8f8f8f8f},
const_item_cache = 143,
allowed_arg_cols = 1,
used_tables_cache = 10344644715844964239,
not_null_tables_cache = 10344644715844964239,
arg_count = 3
},
members of Item_func_case:
first_expr_num = -1,
else_expr_num = 2,
cached_result_type = INT_RESULT,
left_result_type = INT_RESULT,
tmp_value = {
m_ptr = 0x0,
m_length = 0,
m_charset = 0x44664e0 <my_charset_bin>,
m_alloced_length = 0,
m_is_alloced = false
},
ncases = 2,
cmp_type = 2408550287,
cmp_collation = {
collation = 0x44664e0 <my_charset_bin>,
derivation = DERIVATION_NONE,
repertoire = 3
},
cached_field_type = 2408550287,
cmp_items = {0x0, 0x0, 0x0, 0x0, 0x0},
case_item = 0x0
}
(gdb) p ( (Item_func*) ref[0]).args[0][0]
$8 = (Item_func_isnotnull) {
<Item_bool_func> = {
<Item_int_func> = {
<Item_func> = {
<Item_result_field> = {
<Item> = {
<Parse_tree_node> = {
_vptr.Parse_tree_node = 0x4351ba0 <vtable for Item_func_isnotnull+16>,
contextualized = true,
transitional = false
},
members of Item:
is_expensive_cache = -1 '\377',
rsize = 0,
str_value = {
m_ptr = 0x0,
m_length = 0,
m_charset = 0x44664e0 <my_charset_bin>,
m_alloced_length = 0,
m_is_alloced = false
},
item_name = {
<Name_string> = {
<Simple_cstring> = {
m_str = 0x0,
m_length = 0
}, <No data fields>},
members of Item_name_string:
m_is_autogenerated = true
},
orig_name = {
<Name_string> = {
<Simple_cstring> = {
m_str = 0x0,
m_length = 0
}, <No data fields>},
members of Item_name_string:
m_is_autogenerated = true
},
next = 0x7fd254005850,
max_length = 21,
marker = 0,
decimals = 0 '\000',
maybe_null = 0 '\000',
null_value = 0 '\000',
unsigned_flag = 0 '\000',
with_sum_func = 0 '\000',
fixed = 0 '\000',
--Type <RET> for more, q to quit, c to continue without paging--
collation = {
collation = 0x446eae0 <my_charset_latin1>,
derivation = DERIVATION_NUMERIC,
repertoire = 1
},
cmp_context = 4294967295,
runtime_item = false,
derived_used = false,
with_subselect = 0 '\000',
with_stored_program = 0 '\000',
tables_locked_cache = false,
is_parser_item = true
},
members of Item_result_field:
result_field = 0x0
},
members of Item_func:
args = 0x7fd254005678,
tmp_arg = {0x7fd2540063a0, 0x8f8f8f8f8f8f8f8f},
const_item_cache = 143,
allowed_arg_cols = 1,
used_tables_cache = 10344644715844964239,
not_null_tables_cache = 10344644715844964239,
arg_count = 1
}, <No data fields>},
members of Item_bool_func:
m_created_by_in2exists = false
},
members of Item_func_isnotnull:
abort_on_null = false
}
(gdb) p ( (Item_func*) ref[0]).args[1][0]
$9 = (Item_field) {
<Item_ident> = {
<Item> = {
<Parse_tree_node> = {
_vptr.Parse_tree_node = 0x43485f8 <vtable for Item_field+16>,
contextualized = true,
transitional = false
},
members of Item:
is_expensive_cache = -1 '\377',
rsize = 0,
str_value = {
m_ptr = 0x0,
m_length = 0,
m_charset = 0x44664e0 <my_charset_bin>,
m_alloced_length = 0,
m_is_alloced = false
},
item_name = {
<Name_string> = {
<Simple_cstring> = {
m_str = 0x7fd2540056a8 "age",
m_length = 3
}, <No data fields>},
members of Item_name_string:
m_is_autogenerated = true
},
orig_name = {
<Name_string> = {
<Simple_cstring> = {
m_str = 0x0,
m_length = 0
}, <No data fields>},
members of Item_name_string:
m_is_autogenerated = true
},
next = 0x7fd2540056b0,
max_length = 0,
marker = 0,
decimals = 0 '\000',
maybe_null = 0 '\000',
null_value = 0 '\000',
unsigned_flag = 0 '\000',
with_sum_func = 0 '\000',
fixed = 0 '\000',
collation = {
collation = 0x44664e0 <my_charset_bin>,
derivation = DERIVATION_IMPLICIT,
--Type <RET> for more, q to quit, c to continue without paging--
repertoire = 3
},
cmp_context = 4294967295,
runtime_item = false,
derived_used = false,
with_subselect = 0 '\000',
with_stored_program = 0 '\000',
tables_locked_cache = false,
is_parser_item = true
},
members of Item_ident:
orig_db_name = 0x0,
orig_table_name = 0x0,
orig_field_name = 0x7fd2540056a8 "age",
m_alias_of_expr = false,
context = 0x7fd254004c30,
db_name = 0x0,
table_name = 0x0,
field_name = 0x7fd2540056a8 "age",
cached_field_index = 4294967295,
cached_table = 0x0,
depended_from = 0x0
},
members of Item_field:
table_ref = 0x0,
field = 0x0,
result_field = 0x0,
item_equal = 0x0,
no_const_subst = false,
have_privileges = 0,
any_privileges = false
}
函数目的分析:
- 此处是构建case属性的相关列属性的过程, ref的参数包含两种列属性
- 参与case运算的 Item_func_isnotnull
- 表的列属性Item_field
- 此时尚未读取列属性Item_field中的值进行计算取值, 目的在于构建case列属性相关信息的完整结构
Item_func_case::val_int
调用堆栈:
#0 Item_func_case::val_int (this=0x7fd254005850) at /root/work/stonedb-dev-20230605/sql/item_cmpfunc.cc:3897
#1 0x0000000001dde09f in Item_copy_int::copy (this=0x7fd2540217b0, thd=0x7fd254000e10) at /root/work/stonedb-dev-20230605/sql/item.cc:4868
#2 0x00000000023e3292 in copy_fields (param=0x7fd254020ff8, thd=0x7fd254000e10) at /root/work/stonedb-dev-20230605/sql/sql_executor.cc:4381
#3 0x000000000247902f in JOIN::clear (this=0x7fd254020ec0) at /root/work/stonedb-dev-20230605/sql/sql_select.cc:3365
#4 0x00000000023dfb7b in end_send_group (join=0x7fd254020ec0, qep_tab=0x7fd254021730, end_of_records=true) at /root/work/stonedb-dev-20230605/sql/sql_executor.cc:3060
#5 0x00000000023db32a in sub_select (join=0x7fd254020ec0, qep_tab=0x7fd2540215b8, end_of_records=true) at /root/work/stonedb-dev-20230605/sql/sql_executor.cc:1233
#6 0x00000000023dae9e in do_select (join=0x7fd254020ec0) at /root/work/stonedb-dev-20230605/sql/sql_executor.cc:959
#7 0x00000000023d8ded in JOIN::exec (this=0x7fd254020ec0) at /root/work/stonedb-dev-20230605/sql/sql_executor.cc:206
#8 0x00000000024717e3 in handle_query (thd=0x7fd254000e10, lex=0x7fd254003138, result=0x7fd254006a88, added_options=0, removed_options=0, is_optimize_after_tianmu=0, free_join_from_tianmu=0)
at /root/work/stonedb-dev-20230605/sql/sql_select.cc:195
#9 0x0000000002427ae2 in execute_sqlcom_select (thd=0x7fd254000e10, all_tables=0x7fd254014a88) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5205
#10 0x0000000002420e1e in mysql_execute_command (thd=0x7fd254000e10, first_level=true) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:2847
#11 0x0000000002428b0d in mysql_parse (thd=0x7fd254000e10, parser_state=0x7fd519f6ff90) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5642
#12 0x000000000241db04 in dispatch_command (thd=0x7fd254000e10, com_data=0x7fd519f70730, command=COM_QUERY) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1495
#13 0x000000000241c945 in do_command (thd=0x7fd254000e10) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1034
#14 0x000000000254eeb5 in handle_connection (arg=0x965db20) at /root/work/stonedb-dev-20230605/sql/conn_handler/connection_handler_per_thread.cc:313
#15 0x0000000002c1e6f4 in pfs_spawn_thread (arg=0x9490860) at /root/work/stonedb-dev-20230605/storage/perfschema/pfs.cc:2197
#16 0x00007fd569c2b1ca in start_thread () from /lib64/libpthread.so.0
#17 0x00007fd566aa1e73 in clone () from /lib64/libc.so.6
函数实现:
longlong Item_func_case::val_int()
{
assert(fixed == 1);
char buff[MAX_FIELD_WIDTH];
String dummy_str(buff,sizeof(buff),default_charset());
Item *item=find_item(&dummy_str);
longlong res;
if (!item)
{
null_value=1;
return 0;
}
res=item->val_int();
null_value=item->null_value;
return res;
}函数实现的运算逻辑的数据:
函数目的分析:
- 该函数实现了case属性的取值和计算, 代码逻辑非常简单直接，这里不做过多分析
- 值得注意的是该函数被调用的位置, 是在end_send_group中, 由copy_fields完成
- 需要注意myql/sql中的item的概念, 是作为一个基本元素来理解, 即使是一个包含case运算的列, 也是一个item. 而这个包含case运算的item又包含了其他属性的item。在当前场景下具体为 Item_func_case
- item对上层提供的统一抽象接口是val_int和val_str, 即使是逻辑分支运算例如case, 也是被包括在了val_int中进行
tianmu执行分析:
构建查询序列:
T:-1 = TABLE_ALIAS(T:0,"t1")
T:-2 = TMP_TABLE(T:4294967295)
VC:-2.0 = CREATE_VC(T:-2,EXPR("case"))
A:-1 = T:-2.ADD_COLUMN(VC:-2.0,LIST,"case when age IS NOT NULL THEN age else 33 end","ALL"
VC:-2.1 = CREATE_VC(T:-2,PHYS_COL(T:-1,A:0))
A:-2 = T:-2.ADD_COLUMN(VC:-2.1,SUM,"sum(age)","ALL")
T:-2.APPLY_CONDS()
RESULT(T:-2)tianmu的构建查询序列的分析思路:
- 需要注意这个日志仅仅是将mysql/sql的查询树, 转换成tianmu的查询序列的过程, 是一个构建的过程，而非执行查询序列的过程
- 查询序列的执行的入口是APPLY_CONDS()
- 查询序列的每一个操作符的具体的执行过程, 需要从APPLY_CONDS()入口开始逐步定位
- 具体到本问题, 就是要分析EXPR的列属性，是被如何具体处理
tianmu执行的核心处理:
MysqlExpression::EvalType
调用堆栈:
#0 Tianmu::core::MysqlExpression::EvalType (this=0x7fd254024e70, tv=0x7fd254eb5bd8) at /root/work/stonedb-dev-20230605/storage/tianmu/core/mysql_expression.cpp:373
#1 0x0000000002e406d2 in Tianmu::vcolumn::ExpressionColumn::ExpressionColumn (this=0x7fd254eb5a30, expr=0x7fd254024e70, temp_table=0x7fd254eb56f0, temp_table_alias=-2,
multi_index=0x7fd254026e40) at /root/work/stonedb-dev-20230605/storage/tianmu/vc/expr_column.cpp:72
#2 0x0000000002cfd078 in Tianmu::core::Query::CreateColumnFromExpression (this=0x7fd519f6e750, exprs=std::vector of length 1, capacity 1 = {...}, temp_table=0x7fd254eb56f0,
temp_table_alias=-2, mind=0x7fd254026e40) at /root/work/stonedb-dev-20230605/storage/tianmu/core/query.cpp:498
#3 0x0000000002cff9d5 in Tianmu::core::Query::Preexecute (this=0x7fd519f6e750, qu=..., sender=0x71c5c40, display_now=true) at /root/work/stonedb-dev-20230605/storage/tianmu/core/query.cpp:820
#4 0x0000000002cebb38 in Tianmu::core::Engine::Execute (this=0x7218d50, thd=0x7fd254000e10, lex=0x7fd254003138, result_output=0x7fd254006a68, unit_for_union=0x0)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/engine_execute.cpp:513
#5 0x0000000002cea825 in Tianmu::core::Engine::HandleSelect (this=0x7218d50, thd=0x7fd254000e10, lex=0x7fd254003138, result=@0x7fd519f6edc8: 0x7fd254006a68, setup_tables_done_option=0,
res=@0x7fd519f6edc4: 0, is_optimize_after_tianmu=@0x7fd519f6edbc: 1, tianmu_free_join=@0x7fd519f6edc0: 1, with_insert=0)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/engine_execute.cpp:243
#6 0x0000000003084b13 in Tianmu::DBHandler::ha_my_tianmu_query (thd=0x7fd254000e10, lex=0x7fd254003138, result_output=@0x7fd519f6edc8: 0x7fd254006a68, setup_tables_done_option=0,
res=@0x7fd519f6edc4: 0, is_optimize_after_tianmu=@0x7fd519f6edbc: 1, tianmu_free_join=@0x7fd519f6edc0: 1, with_insert=0)
at /root/work/stonedb-dev-20230605/storage/tianmu/sql/ha_my_tianmu.cpp:95
#7 0x0000000002427aa8 in execute_sqlcom_select (thd=0x7fd254000e10, all_tables=0x7fd254014a88) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5204
#8 0x0000000002420e1e in mysql_execute_command (thd=0x7fd254000e10, first_level=true) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:2847
#9 0x0000000002428b0d in mysql_parse (thd=0x7fd254000e10, parser_state=0x7fd519f6ff90) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5642
#10 0x000000000241db04 in dispatch_command (thd=0x7fd254000e10, com_data=0x7fd519f70730, command=COM_QUERY) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1495
#11 0x000000000241c945 in do_command (thd=0x7fd254000e10) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1034
#12 0x000000000254eeb5 in handle_connection (arg=0x965db20) at /root/work/stonedb-dev-20230605/sql/conn_handler/connection_handler_per_thread.cc:313
#13 0x0000000002c1e6f4 in pfs_spawn_thread (arg=0x9490860) at /root/work/stonedb-dev-20230605/storage/perfschema/pfs.cc:2197
#14 0x00007fd569c2b1ca in start_thread () from /lib64/libpthread.so.0
#15 0x00007fd566aa1e73 in clone () from /lib64/libc.so.6
函数实现:
DataType MysqlExpression::EvalType(TypOfVars *tv) {
// set types of variables (_tianmufieldsCache)
if (tv) {
DataType fieldtype;
auto tianmufield_set = tianmu_fields_cache.begin();
while (tianmufield_set != tianmu_fields_cache.end()) {
auto it = tv->find(tianmufield_set->first);
if (it != tv->end()) {
for (auto &tianmufield : tianmufield_set->second) {
fieldtype = it->second;
tianmufield->SetType(fieldtype);
}
}
tianmufield_set++;
}
}函数目的分析:
- 最大的一个问题便是为什么要在这个函数打断点做追踪?
- 对expr的处理的代码并不熟悉, 不知道tianmu具体是怎么处理case的, 无法直接在逻辑处理的地方打断点, 或者说对代码的熟悉程度不足以直接定位问题。这也是现在大部分的解决问题的时候面临的难题, 对这块代码的熟悉程度不足, 是在解决这块代码的问题时才第一次理解该块逻辑的代码
- 但是EvalType是必定要走到的, 必须要拿到expr的类型
- 从这个函数的调用堆栈出发, 理解expr的属性的处理流程
MysqlExpression::GetTianmufieldItem
调用堆栈:
#0 Tianmu::core::MysqlExpression::GetTianmufieldItem (this=0x7fd254024e70, ifield=0x7fd254eb2a08) at /root/work/stonedb-dev-20230605/storage/tianmu/core/mysql_expression.cpp:165
#1 0x00000000030c5a05 in Tianmu::core::MysqlExpression::TransformTree (this=0x7fd254024e70, root=0x7fd254eb2a08, dir=Tianmu::core::MysqlExpression::TransformDirection::FORWARD)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/mysql_expression.cpp:240
#2 0x00000000030c5f09 in Tianmu::core::MysqlExpression::TransformTree (this=0x7fd254024e70, root=0x7fd254eb2f50, dir=Tianmu::core::MysqlExpression::TransformDirection::FORWARD)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/mysql_expression.cpp:333
#3 0x00000000030c5e05 in Tianmu::core::MysqlExpression::TransformTree (this=0x7fd254024e70, root=0x7fd2540055c0, dir=Tianmu::core::MysqlExpression::TransformDirection::FORWARD)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/mysql_expression.cpp:317
#4 0x00000000030c5e05 in Tianmu::core::MysqlExpression::TransformTree (this=0x7fd254024e70, root=0x7fd254005840, dir=Tianmu::core::MysqlExpression::TransformDirection::FORWARD)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/mysql_expression.cpp:317
#5 0x00000000030c4ce1 in Tianmu::core::MysqlExpression::MysqlExpression (this=0x7fd254024e70, item=0x7fd254005840, item2varid=std::map with 1 element = {...})
at /root/work/stonedb-dev-20230605/storage/tianmu/core/mysql_expression.cpp:50
#6 0x0000000002d24303 in Tianmu::core::Query::WrapMysqlExpression (this=0x7fd519f6e750, item=0x7fd254005840, tmp_table=..., expr=@0x7fd519f6e1c8: 0x0, in_where=false, aggr_used=false)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/query_compile.cpp:834
#7 0x0000000002d22279 in Tianmu::core::Query::AddFields (this=0x7fd519f6e750, fields=..., tmp_table=..., base_table=..., group_by_clause=false, num_of_added_fields=@0x7fd519f6e4c4: 0,
ignore_minmax=false, aggregation_used=@0x7fd519f6e40f: false) at /root/work/stonedb-dev-20230605/storage/tianmu/core/query_compile.cpp:495
#8 0x0000000002d25efd in Tianmu::core::Query::Compile (this=0x7fd519f6e750, compiled_query=0x7fd519f6e680, selects_list=0x7fd254004bc0, last_distinct=0x0, res_tab=0x0, ignore_limit=false,
left_expr_for_subselect=0x0, oper_for_subselect=0x0, ignore_minmax=false, for_subq_in_where=false) at /root/work/stonedb-dev-20230605/storage/tianmu/core/query_compile.cpp:1210
#9 0x0000000002ceb69c in Tianmu::core::Engine::Execute (this=0x7218d50, thd=0x7fd254000e10, lex=0x7fd254003138, result_output=0x7fd254006a68, unit_for_union=0x0)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/engine_execute.cpp:472
#10 0x0000000002cea825 in Tianmu::core::Engine::HandleSelect (this=0x7218d50, thd=0x7fd254000e10, lex=0x7fd254003138, result=@0x7fd519f6edc8: 0x7fd254006a68, setup_tables_done_option=0,
res=@0x7fd519f6edc4: 0, is_optimize_after_tianmu=@0x7fd519f6edbc: 1, tianmu_free_join=@0x7fd519f6edc0: 1, with_insert=0)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/engine_execute.cpp:243
#11 0x0000000003084b13 in Tianmu::DBHandler::ha_my_tianmu_query (thd=0x7fd254000e10, lex=0x7fd254003138, result_output=@0x7fd519f6edc8: 0x7fd254006a68, setup_tables_done_option=0,
res=@0x7fd519f6edc4: 0, is_optimize_after_tianmu=@0x7fd519f6edbc: 1, tianmu_free_join=@0x7fd519f6edc0: 1, with_insert=0)
at /root/work/stonedb-dev-20230605/storage/tianmu/sql/ha_my_tianmu.cpp:95
#12 0x0000000002427aa8 in execute_sqlcom_select (thd=0x7fd254000e10, all_tables=0x7fd254014a88) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5204
#13 0x0000000002420e1e in mysql_execute_command (thd=0x7fd254000e10, first_level=true) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:2847
#14 0x0000000002428b0d in mysql_parse (thd=0x7fd254000e10, parser_state=0x7fd519f6ff90) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5642
#15 0x000000000241db04 in dispatch_command (thd=0x7fd254000e10, com_data=0x7fd519f70730, command=COM_QUERY) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1495
#16 0x000000000241c945 in do_command (thd=0x7fd254000e10) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1034
#17 0x000000000254eeb5 in handle_connection (arg=0x965db20) at /root/work/stonedb-dev-20230605/sql/conn_handler/connection_handler_per_thread.cc:313
#18 0x0000000002c1e6f4 in pfs_spawn_thread (arg=0x9490860) at /root/work/stonedb-dev-20230605/storage/perfschema/pfs.cc:2197
#19 0x00007fd569c2b1ca in start_thread () from /lib64/libpthread.so.0
#20 0x00007fd566aa1e73 in clone () from /lib64/libc.so.6
函数实现:
Item_tianmufield *MysqlExpression::GetTianmufieldItem(Item_field *ifield) {
auto key = item2varid->find(ifield);
DEBUG_ASSERT(key != item2varid->end());
auto it = tianmu_fields_cache.find(key->second);
Item_tianmufield *tianmufield = nullptr;
if (it != tianmu_fields_cache.end()) {
tianmufield = *it->second.begin();
tianmufield->varID.push_back(key->second);
} else {
tianmufield = new Item_tianmufield(ifield, key->second);
std::set<Item_tianmufield *> s_tmp;
s_tmp.insert(tianmufield);
tianmu_fields_cache.insert(make_pair(key->second, s_tmp));
}
return (tianmufield);
}函数目的分析:
- 由于上层逻辑是在执行Query::AddFields和TransformTree, 所以不能从GetTianmufieldItem中获取具体的业务处理相关的信息
- 那么为什么还要在这个函数挂断点分析执行过程呢? 因为这个函数是在处理Item_tianmufield的其中一个过程
Tianmu::core::Query::Preexecute
执行结果的临时表的case里属性的数据:
(gdb) p output_table[0].attrs[0][0]
$27 = (Tianmu::core::TempTable::Attr) {
<Tianmu::core::PhysicalColumn> = {
<Tianmu::core::Column> = {
ct = {
type = Tianmu::common::ColumnType::BIGINT,
unsigned_flag_ = false,
precision = 19,
scale = 0,
internal_size = 8,
display_size = 20,
collation = {
collation = 0x44664e0 <my_charset_bin>,
derivation = DERIVATION_NONE,
repertoire = 3
},
fmt = Tianmu::common::PackFmt::DEFAULT,
flag = std::bitset
}
},
members of Tianmu::core::PhysicalColumn:
_vptr.PhysicalColumn = 0x44057d0 <vtable for Tianmu::core::TempTable::Attr+16>,
is_unique = false,
is_unique_updated = false
},
members of Tianmu::core::TempTable::Attr:
si = {
separator = "",
order = st_order::ORDER_NOT_RELEVANT
},
buffer = 0x0,
no_obj = 0,
no_power = 16,
no_materialized = 0,
page_size = 1,
alias = 0x7fd254024a70 "case when age IS NOT NULL THEN age else 33 end",
mode = Tianmu::common::ColOperation::LISTING,
distinct = false,
term = {
type = Tianmu::common::ColumnType::UNK,
vc = 0x7fd254eb5a30,
cond_value = std::vector of length 0, capacity 0,
cond_numvalue = std::shared_ptr<Tianmu::utils::Hash64> (empty) = {
get() = 0x0
},
vc_id = 0,
is_vc_owner = false,
item = 0x0
},
--Type <RET> for more, q to quit, c to continue without paging--
dim = -2,
orig_precision = 19,
not_complete = true
}
此函数目的:
- 追踪这个函数是从结果出发, 来分析case列相关的属性和数据信息, 倒推case列属性是如何执行的
- 对output_table[0].attrs[0][0]的数据做分析, 有以下奇怪的地方
- mode的值是Tianmu::common::ColOperation::LISTING, 这个是自然属性, 而并不是case这种计算的属性。这种属性会导致直接取值, 而非取值后的case计算
- buffer = 0x0 缓存列的中间结果, 其值为空, 也就是没有拿到数据, 结合mode是LISTING, 其值符合LISTING属性的预期
- 所以可以分析出, case属性的列, 在执行的时候, 是按照LISTING进行处理的, 而并没有做case运算
Query::AddColumnForMysqlExpression
调用堆栈:
#0 Tianmu::core::Query::AddColumnForMysqlExpression (this=0x7fd519f6e750, mysql_expression=0x7fd254024e70, tmp_table=...,
alias=0x7fd2540065c8 "case when age IS NOT NULL THEN age else 33 end", oper=Tianmu::common::ColOperation::LISTING, distinct=false, group_by=false)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/query_compile.cpp:932
#1 0x0000000002d223ec in Tianmu::core::Query::AddFields (this=0x7fd519f6e750, fields=..., tmp_table=..., base_table=..., group_by_clause=false, num_of_added_fields=@0x7fd519f6e4c4: 0,
ignore_minmax=false, aggregation_used=@0x7fd519f6e40f: false) at /root/work/stonedb-dev-20230605/storage/tianmu/core/query_compile.cpp:507
#2 0x0000000002d25efd in Tianmu::core::Query::Compile (this=0x7fd519f6e750, compiled_query=0x7fd519f6e680, selects_list=0x7fd254004bc0, last_distinct=0x0, res_tab=0x0, ignore_limit=false,
left_expr_for_subselect=0x0, oper_for_subselect=0x0, ignore_minmax=false, for_subq_in_where=false) at /root/work/stonedb-dev-20230605/storage/tianmu/core/query_compile.cpp:1210
#3 0x0000000002ceb69c in Tianmu::core::Engine::Execute (this=0x7218d50, thd=0x7fd254000e10, lex=0x7fd254003138, result_output=0x7fd254006a68, unit_for_union=0x0)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/engine_execute.cpp:472
#4 0x0000000002cea825 in Tianmu::core::Engine::HandleSelect (this=0x7218d50, thd=0x7fd254000e10, lex=0x7fd254003138, result=@0x7fd519f6edc8: 0x7fd254006a68, setup_tables_done_option=0,
res=@0x7fd519f6edc4: 0, is_optimize_after_tianmu=@0x7fd519f6edbc: 1, tianmu_free_join=@0x7fd519f6edc0: 1, with_insert=0)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/engine_execute.cpp:243
#5 0x0000000003084b13 in Tianmu::DBHandler::ha_my_tianmu_query (thd=0x7fd254000e10, lex=0x7fd254003138, result_output=@0x7fd519f6edc8: 0x7fd254006a68, setup_tables_done_option=0,
res=@0x7fd519f6edc4: 0, is_optimize_after_tianmu=@0x7fd519f6edbc: 1, tianmu_free_join=@0x7fd519f6edc0: 1, with_insert=0)
at /root/work/stonedb-dev-20230605/storage/tianmu/sql/ha_my_tianmu.cpp:95
#6 0x0000000002427aa8 in execute_sqlcom_select (thd=0x7fd254000e10, all_tables=0x7fd254014a88) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5204
#7 0x0000000002420e1e in mysql_execute_command (thd=0x7fd254000e10, first_level=true) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:2847
#8 0x0000000002428b0d in mysql_parse (thd=0x7fd254000e10, parser_state=0x7fd519f6ff90) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5642
#9 0x000000000241db04 in dispatch_command (thd=0x7fd254000e10, com_data=0x7fd519f70730, command=COM_QUERY) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1495
#10 0x000000000241c945 in do_command (thd=0x7fd254000e10) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1034
#11 0x000000000254eeb5 in handle_connection (arg=0x965db20) at /root/work/stonedb-dev-20230605/sql/conn_handler/connection_handler_per_thread.cc:313
#12 0x0000000002c1e6f4 in pfs_spawn_thread (arg=0x9490860) at /root/work/stonedb-dev-20230605/storage/perfschema/pfs.cc:2197
#13 0x00007fd569c2b1ca in start_thread () from /lib64/libpthread.so.0
#14 0x00007fd566aa1e73 in clone () from /lib64/libc.so.6
函数实现:
Query::AddFields
QueryRouteTo Query::AddFields(List<Item> &fields, TabID const &tmp_table, TabID const &base_table,
bool const group_by_clause, int &num_of_added_fields, bool ignore_minmax,
bool &aggregation_used) {
List_iterator_fast<Item> li(fields);
Item *item;
int added = 0;
item = li++;
while (item) {
WrapStatus ws;
common::ColOperation oper;
bool distinct;
if (QueryRouteTo::kToMySQL == OperationUnmysterify(item, oper, distinct, group_by_clause))
return QueryRouteTo::kToMySQL;
if (IsAggregationItem(item))
aggregation_used = true;
// in case of transformed subquery sometimes we need to revert back
// transformation to MIN/MAX
if (ignore_minmax && (oper == common::ColOperation::MIN || oper == common::ColOperation::MAX))
oper = common::ColOperation::LISTING;
// select PHYSICAL COLUMN or AGGREGATION over PHYSICAL COLUMN
if ((IsFieldItem(item) || IsAggregationOverFieldItem(item)) &&
(IsLocalColumn(item, tmp_table) || (!base_table.IsNullID() && IsLocalColumn(item, base_table))))
AddColumnForPhysColumn(item, tmp_table, base_table, oper, distinct, false, item->item_name.ptr());
// REF to FIELD_ITEM
else if (item->type() == Item::REF_ITEM) {
item = UnRef(item);
continue;
}
// if ((UnRef(item)->type() == Item_tianmufield::enumTIANMUFiledItem::TIANMUFIELD_ITEM ||
// UnRef(item)->type() == Item_tianmufield::FIELD_ITEM) &&
// IsLocalColumn(UnRef(item), tmp_table))
// AddColumnForPhysColumn(UnRef(item), tmp_table, oper, distinct, false, false);
// else {
// //
// }
else if (IsAggregationItem(item) && (((Item_sum *)item)->get_arg(0))->type() == Item::REF_ITEM &&
(UnRef(((Item_sum *)item)->get_arg(0))->type() == Item_tianmufield::get_tianmuitem_type() ||
(UnRef(((Item_sum *)item)->get_arg(0))->type() == Item_tianmufield::FIELD_ITEM)) &&
IsLocalColumn(UnRef(((Item_sum *)item)->get_arg(0)), tmp_table))
// AGGR on REF to FIELD_ITEM
AddColumnForPhysColumn(UnRef(((Item_sum *)item)->get_arg(0)), tmp_table, TabID(), oper, distinct, false,
item->item_name.ptr());
else if (IsAggregationItem(item)) {
// select AGGREGATION over EXPRESSION
Item_sum *item_sum = (Item_sum *)item;
if (item_sum->get_arg_count() > 1 || HasAggregation(item_sum->get_arg(0)))
return QueryRouteTo::kToMySQL;
if (IsCountStar(item_sum)) { // count(*) doesn't need any virtual column
AttrID at;
cq->AddColumn(at, tmp_table, CQTerm(), oper, item_sum->item_name.ptr(), false);
field_alias2num[TabIDColAlias(tmp_table.n, item_sum->item_name.ptr())] = at.n;
} else {
MysqlExpression *expr;
ws = WrapMysqlExpression(item_sum->get_arg(0), tmp_table, expr, false, false);
if (ws == WrapStatus::FAILURE)
return QueryRouteTo::kToMySQL;
AddColumnForMysqlExpression(expr, tmp_table,
ignore_minmax ? item_sum->get_arg(0)->item_name.ptr() : item_sum->item_name.ptr(),
oper, distinct);
}
} else if (item->type() == Item::SUBSELECT_ITEM) {
CQTerm term;
AttrID at;
if (Item2CQTerm(item, term, tmp_table,
/*group_by_clause ? HAVING_FILTER :*/ CondType::WHERE_COND) == QueryRouteTo::kToMySQL)
return QueryRouteTo::kToMySQL;
cq->AddColumn(at, tmp_table, term, common::ColOperation::LISTING, item->item_name.ptr(), distinct);
field_alias2num[TabIDColAlias(tmp_table.n, item->item_name.ptr())] = at.n;
} else {
// select EXPRESSION
if (HasAggregation(item)) {
oper = common::ColOperation::DELAYED;
aggregation_used = true;
}
MysqlExpression *expr(nullptr);
ws = WrapMysqlExpression(item, tmp_table, expr, false, oper == common::ColOperation::DELAYED);
if (ws == WrapStatus::FAILURE)
return QueryRouteTo::kToMySQL;
if (!item->item_name.ptr()) {
Item_func_conv_charset *item_conv = dynamic_cast<Item_func_conv_charset *>(item);
if (item_conv) {
Item **ifunc_args = item_conv->arguments();
AddColumnForMysqlExpression(expr, tmp_table, ifunc_args[0]->item_name.ptr(), oper, distinct);
} else {
AddColumnForMysqlExpression(expr, tmp_table, item->item_name.ptr(), oper, distinct);
}
} else
AddColumnForMysqlExpression(expr, tmp_table, item->item_name.ptr(), oper, distinct);
}
added++;
item = li++;
}
num_of_added_fields = added;
return QueryRouteTo::kToTianmu;
}(gdb) p item
$30 = (Item_func_case *) 0x7fd254005840
此函数目的:
- case的列属性构建tianmu的查询序列, mode设置为LISTING导致了case运算的缺失, 那么就需要分析构建查询序列的列属性的过程
- 分析该函数有这么几个目的:
- 尝试将tianmu的列属性与mysql/sql的列属性保持一致, 补充确实的case运算
- 如果tianmu引擎无法计算该属性, 直接拿到item的信息进行处理
Tianmu::core::Query::AddColumnForMysqlExpression
调用堆栈:
#0 Tianmu::core::Query::AddColumnForMysqlExpression (this=0x7fb151126750, mysql_expression=0x7fae4401c470, tmp_table=...,
alias=0x7fae44010808 "case when age IS NOT NULL THEN age else 33 end", oper=Tianmu::common::ColOperation::LISTING, distinct=false, group_by=false)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/query_compile.cpp:932
#1 0x0000000002d223ec in Tianmu::core::Query::AddFields (this=0x7fb151126750, fields=..., tmp_table=..., base_table=..., group_by_clause=false, num_of_added_fields=@0x7fb1511264c4: 0,
ignore_minmax=false, aggregation_used=@0x7fb15112640f: false) at /root/work/stonedb-dev-20230605/storage/tianmu/core/query_compile.cpp:507
#2 0x0000000002d25efd in Tianmu::core::Query::Compile (this=0x7fb151126750, compiled_query=0x7fb151126680, selects_list=0x7fae44005520, last_distinct=0x0, res_tab=0x0, ignore_limit=false,
left_expr_for_subselect=0x0, oper_for_subselect=0x0, ignore_minmax=false, for_subq_in_where=false) at /root/work/stonedb-dev-20230605/storage/tianmu/core/query_compile.cpp:1210
#3 0x0000000002ceb69c in Tianmu::core::Engine::Execute (this=0x79ded50, thd=0x7fae44000e10, lex=0x7fae44003138, result_output=0x7fae440120e8, unit_for_union=0x0)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/engine_execute.cpp:472
#4 0x0000000002cea825 in Tianmu::core::Engine::HandleSelect (this=0x79ded50, thd=0x7fae44000e10, lex=0x7fae44003138, result=@0x7fb151126dc8: 0x7fae440120e8, setup_tables_done_option=0,
res=@0x7fb151126dc4: 0, is_optimize_after_tianmu=@0x7fb151126dbc: 1, tianmu_free_join=@0x7fb151126dc0: 1, with_insert=0)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/engine_execute.cpp:243
#5 0x0000000003084b13 in Tianmu::DBHandler::ha_my_tianmu_query (thd=0x7fae44000e10, lex=0x7fae44003138, result_output=@0x7fb151126dc8: 0x7fae440120e8, setup_tables_done_option=0,
res=@0x7fb151126dc4: 0, is_optimize_after_tianmu=@0x7fb151126dbc: 1, tianmu_free_join=@0x7fb151126dc0: 1, with_insert=0)
at /root/work/stonedb-dev-20230605/storage/tianmu/sql/ha_my_tianmu.cpp:95
#6 0x0000000002427aa8 in execute_sqlcom_select (thd=0x7fae44000e10, all_tables=0x7fae44013648) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5204
#7 0x0000000002420e1e in mysql_execute_command (thd=0x7fae44000e10, first_level=true) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:2847
#8 0x0000000002428b0d in mysql_parse (thd=0x7fae44000e10, parser_state=0x7fb151127f90) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5642
#9 0x000000000241db04 in dispatch_command (thd=0x7fae44000e10, com_data=0x7fb151128730, command=COM_QUERY) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1495
#10 0x000000000241c945 in do_command (thd=0x7fae44000e10) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1034
#11 0x000000000254eeb5 in handle_connection (arg=0x9e23f40) at /root/work/stonedb-dev-20230605/sql/conn_handler/connection_handler_per_thread.cc:313
#12 0x0000000002c1e6f4 in pfs_spawn_thread (arg=0x9c38930) at /root/work/stonedb-dev-20230605/storage/perfschema/pfs.cc:2197
#13 0x00007fb15b44a1ca in start_thread () from /lib64/libpthread.so.0
#14 0x00007fb1582c0e73 in clone () from /lib64/libc.so.6
函数实现:
vcolumn::VirtualColumn *Query::CreateColumnFromExpression(std::vector<MysqlExpression *> const &exprs,
TempTable *temp_table, int temp_table_alias,
MultiIndex *mind) {
DEBUG_ASSERT(exprs.size() > 0);
if (exprs.size() != 1) {
DEBUG_ASSERT(0);
}
vcolumn::VirtualColumn *vc = nullptr;
Item *item = exprs[0]->GetItem();
if (exprs[0]->IsDeterministic() && (exprs[0]->GetVars().empty())) {
ColumnType type(exprs[0]->EvalType());
vc = new vcolumn::ConstColumn(*(exprs[0]->Evaluate()), type, true);
} else if (vcolumn::VirtualColumn::IsConstExpression(exprs[0], temp_table_alias, &temp_table->GetAliases()) &&
exprs[0]->IsDeterministic()) {
if (IsFieldItem(item)) {
// a special case when a naked column is a parameter
// without this column type would be a seen by mysql, not TIANMU.
// e.g. timestamp would be string 19
TabID tab;
AttrID col;
tab.n = exprs[0]->GetVars().begin()->tab;
col.n = exprs[0]->GetVars().begin()->col;
col.n = col.n < 0 ? -col.n - 1 : col.n;
ColumnType ct = ta[-tab.n - 1]->GetColumnType(col.n);
vc = new vcolumn::ConstExpressionColumn(exprs[0], ct, temp_table, temp_table_alias, mind);
} else
vc = new vcolumn::ConstExpressionColumn(exprs[0], temp_table, temp_table_alias, mind);
} else {
if (tianmu_control_.isOn()) {
if ((item->type()) == Item::FUNC_ITEM) {
Item_func *ifunc = down_cast<Item_func *>(item);
tianmu_control_.lock(mind->m_conn->GetThreadID())
<< "Unoptimized expression near '" << ifunc->func_name() << "'" << system::unlock;
}
}
vc = new vcolumn::ExpressionColumn(exprs[0], temp_table, temp_table_alias, mind);
if (static_cast<vcolumn::ExpressionColumn *>(vc)->GetStringType() == MysqlExpression::StringType::STRING_TIME &&
vc->TypeName() != common::ColumnType::TIME) { // common::CT::TIME is already as int64_t
vcolumn::TypeCastColumn *tcc = new vcolumn::String2DateTimeCastColumn(vc, ColumnType(common::ColumnType::TIME));
temp_table->AddVirtColumn(vc);
vc = tcc;
}
}
MysqlExpression::SetOfVars params = vc->GetParams();
MysqlExpression::TypOfVars types;
for (auto &iter : params) {
types[iter] = ta[-iter.tab - 1]->GetColumnType(iter.col < 0 ? -iter.col - 1 : iter.col);
}
vc->SetParamTypes(&types);
return vc;
}此函数目的:
- 利用MysqlExpression中的信息，生成 VirtualColumn
TempTable::Materialize
调用堆栈:
#0 Tianmu::core::TempTable::Materialize (this=0x7fae44eaa1c0, in_subq=false, sender=0x796f3a0, lazy=false) at /root/work/stonedb-dev-20230605/storage/tianmu/core/temp_table.cpp:1955
#1 0x0000000002cebc6e in Tianmu::core::Engine::Execute (this=0x79ded50, thd=0x7fae44000e10, lex=0x7fae44003138, result_output=0x7fae44006a88, unit_for_union=0x0)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/engine_execute.cpp:518
#2 0x0000000002cea825 in Tianmu::core::Engine::HandleSelect (this=0x79ded50, thd=0x7fae44000e10, lex=0x7fae44003138, result=@0x7fb151126dc8: 0x7fae44006a88, setup_tables_done_option=0,
res=@0x7fb151126dc4: 0, is_optimize_after_tianmu=@0x7fb151126dbc: 1, tianmu_free_join=@0x7fb151126dc0: 1, with_insert=0)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/engine_execute.cpp:243
#3 0x0000000003084b13 in Tianmu::DBHandler::ha_my_tianmu_query (thd=0x7fae44000e10, lex=0x7fae44003138, result_output=@0x7fb151126dc8: 0x7fae44006a88, setup_tables_done_option=0,
res=@0x7fb151126dc4: 0, is_optimize_after_tianmu=@0x7fb151126dbc: 1, tianmu_free_join=@0x7fb151126dc0: 1, with_insert=0)
at /root/work/stonedb-dev-20230605/storage/tianmu/sql/ha_my_tianmu.cpp:95
#4 0x0000000002427aa8 in execute_sqlcom_select (thd=0x7fae44000e10, all_tables=0x7fae44f39648) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5204
#5 0x0000000002420e1e in mysql_execute_command (thd=0x7fae44000e10, first_level=true) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:2847
#6 0x0000000002428b0d in mysql_parse (thd=0x7fae44000e10, parser_state=0x7fb151127f90) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5642
#7 0x000000000241db04 in dispatch_command (thd=0x7fae44000e10, com_data=0x7fb151128730, command=COM_QUERY) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1495
#8 0x000000000241c945 in do_command (thd=0x7fae44000e10) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1034
#9 0x000000000254eeb5 in handle_connection (arg=0x9e23f40) at /root/work/stonedb-dev-20230605/sql/conn_handler/connection_handler_per_thread.cc:313
#10 0x0000000002c1e6f4 in pfs_spawn_thread (arg=0x9c38930) at /root/work/stonedb-dev-20230605/storage/perfschema/pfs.cc:2197
#11 0x00007fb15b44a1ca in start_thread () from /lib64/libpthread.so.0
#12 0x00007fb1582c0e73 in clone () from /lib64/libc.so.6
函数实现:
void TempTable::Materialize(bool in_subq, ResultSender *sender, bool lazy) {
MEASURE_FET("TempTable::Materialize()");
if (sender)
sender->SetAffectRows(no_obj);
CreateDisplayableAttrP();
CalculatePageSize();
int64_t offset = 0; // controls the first object to be materialized
int64_t limit = -1; // if(limit>=0) -> for(row = offset; row < offset +
// limit; row++) ....
bool limits_present = mode.top;
bool exists_only = mode.exists;
if (limits_present) {
offset = (mode.param1 >= 0 ? mode.param1 : 0);
limit = (mode.param2 >= 0 ? mode.param2 : 0);
// applied => reset to default values
mode.top = false;
mode.param2 = -1;
mode.param1 = 0;
}
int64_t local_offset = 0; // controls the first object to be materialized in a given algorithm
int64_t local_limit = -1;
if (materialized && (order_by.size() > 0 || limits_present) && no_obj) {
// if TempTable is already materialized but some additional constraints were
// specified, e.g., order by or limit this is typical case for union, where
// constraints are specified for the result of union after materialization
if (limits_present) {
local_offset = offset;
local_limit = std::min(limit, (int64_t)no_obj - offset);
local_limit = local_limit < 0 ? 0 : local_limit;
} else
local_limit = no_obj;
if (exists_only) {
if (local_limit == 0) // else no change needed
no_obj = 0;
return;
}
if (order_by.size() != 0 && no_obj > 1) {
std::shared_ptr<TempTable> temporary_source_table =
CreateMaterializedCopy(true, in_subq); // true: translate definition of ordering
OrderByAndMaterialize(order_by, local_limit, local_offset);
DeleteMaterializedCopy(temporary_source_table);
} else if (limits_present)
ApplyOffset(local_limit, local_offset);
order_by.clear();
return;
}
if ((materialized && !this->lazy) || (this->lazy && no_obj == no_materialized)) {
return;
}
bool table_distinct = this->mode.distinct;
bool distinct_on_materialized = false;
for (uint i = 0; i < NumOfAttrs(); i++)
if (attrs[i]->mode != common::ColOperation::LISTING)
group_by = true;
if (table_distinct && group_by) {
distinct_on_materialized = true;
table_distinct = false;
}
if (order_by.size() > 0 || group_by || this->mode.distinct || force_full_materialize)
lazy = false;
this->lazy = lazy;
bool no_rows_too_large = filter.mind_->TooManyTuples();
no_obj = -1; // no_obj not calculated yet - wait for better moment
VerifyAttrsSizes(); // resize attr[i] buffers basing on the current
// multiindex state
// the case when there is no grouping of attributes, check also DISTINCT
// modifier of TT
if (!group_by && !table_distinct) {
DEBUG_ASSERT(!distinct_on_materialized); // should by false here, otherwise must be
// added to conditions below
if (limits_present) {
if (no_rows_too_large && order_by.size() == 0)
no_obj = offset + limit; // offset + limit in the worst case
else
no_obj = filter.mind_->NumOfTuples();
if (no_obj <= offset) {
no_obj = 0;
materialized = true;
order_by.clear();
return;
}
local_offset = offset;
local_limit = std::min(limit, (int64_t)no_obj - offset);
local_limit = local_limit < 0 ? 0 : local_limit;
} else {
no_obj = filter.mind_->NumOfTuples();
local_limit = no_obj;
}
if (exists_only) {
order_by.clear();
return;
}
output_mind.Clear();
output_mind.AddDimension_cross(local_limit); // an artificial dimension for result
CalculatePageSize(); // recalculate, as no_obj might changed
// perform order by: in this case it can be done on source tables, not on
// the result
bool materialized_by_ordering = false;
if (CanOrderSources())
// false if no sorting used
materialized_by_ordering = this->OrderByAndMaterialize(order_by, local_limit, local_offset, sender);
if (!materialized_by_ordering) { // not materialized yet?
// materialize without aggregations. If ordering then do not send result
if (order_by.size() == 0)
FillMaterializedBuffers(local_limit, local_offset, sender, lazy);
else // in case of order by we need to materialize all rows to be next
// ordered
FillMaterializedBuffers(no_obj, 0, nullptr, lazy);
}
} else {
// GROUP BY or DISTINCT - compute aggregations
if (limits_present && !distinct_on_materialized && order_by.size() == 0) {
local_offset = offset;
local_limit = limit;
if (exists_only)
local_limit = 1;
}
if (HasHavingConditions() && in_subq)
having_conds[0].tree->Simplify(true);
ResultSender *local_sender = (distinct_on_materialized || order_by.size() > 0 ? nullptr : sender);
AggregationAlgorithm aggr(this);
aggr.Aggregate(table_distinct, local_limit, local_offset,
local_sender); // this->tree (HAVING) used inside
if (no_obj == 0) {
order_by.clear();
return;
}
output_mind.Clear();
output_mind.AddDimension_cross(no_obj); // an artificial dimension for result
}
local_offset = 0;
local_limit = -1;
// DISTINCT after grouping
if (distinct_on_materialized && no_obj > 1) {
if (limits_present && order_by.size() == 0) {
local_offset = std::min((int64_t)no_obj, offset);
local_limit = std::min(limit, (int64_t)no_obj - local_offset);
local_limit = local_limit < 0 ? 0 : local_limit;
if (no_obj <= offset) {
no_obj = 0;
materialized = true;
order_by.clear();
return;
}
} else
local_limit = no_obj;
if (exists_only)
local_limit = 1;
std::shared_ptr<TempTable> temporary_source_table = CreateMaterializedCopy(false, in_subq);
ResultSender *local_sender = (order_by.size() > 0 ? nullptr : sender);
AggregationAlgorithm aggr(this);
aggr.Aggregate(true, local_limit, local_offset,
local_sender); // true => select-level distinct
DeleteMaterializedCopy(temporary_source_table);
output_mind.Clear();
output_mind.AddDimension_cross(no_obj); // an artificial dimension for result
} // end of distinct part
// ORDER BY, if not sorted until now
if (order_by.size() != 0) {
if (limits_present) {
local_offset = std::min((int64_t)no_obj, offset);
local_limit = std::min(limit, (int64_t)no_obj - local_offset);
local_limit = local_limit < 0 ? 0 : local_limit;
if (no_obj <= offset) {
no_obj = 0;
materialized = true;
order_by.clear();
return;
}
} else
local_limit = no_obj;
if (no_obj > 1 && !exists_only) {
std::shared_ptr<TempTable> temporary_source_table =
CreateMaterializedCopy(true, in_subq); // true: translate definition of ordering
OrderByAndMaterialize(order_by, local_limit, local_offset, sender);
DeleteMaterializedCopy(temporary_source_table);
}
order_by.clear();
output_mind.Clear();
output_mind.AddDimension_cross(no_obj); // an artificial dimension for result
}
materialized = true;
}此函数执行完后的case列属性中的数据
(gdb) p result.attrs[0][0]
$5 = (Tianmu::core::TempTable::Attr) {
<Tianmu::core::PhysicalColumn> = {
<Tianmu::core::Column> = {
ct = {
type = Tianmu::common::ColumnType::BIGINT,
unsigned_flag_ = false,
precision = 19,
scale = 0,
internal_size = 8,
display_size = 20,
collation = {
collation = 0x44664e0 <my_charset_bin>,
derivation = DERIVATION_NONE,
repertoire = 3
},
fmt = Tianmu::common::PackFmt::DEFAULT,
flag = std::bitset
}
},
members of Tianmu::core::PhysicalColumn:
_vptr.PhysicalColumn = 0x44057d0 <vtable for Tianmu::core::TempTable::Attr+16>,
is_unique = false,
is_unique_updated = false
},
members of Tianmu::core::TempTable::Attr:
si = {
separator = "",
order = st_order::ORDER_NOT_RELEVANT
},
buffer = 0x7fae44eaa480,
no_obj = 1,
no_power = 16,
no_materialized = 1,
page_size = 1,
alias = 0x7fae44016fd0 "case when age IS NOT NULL THEN age else 33 end",
mode = Tianmu::common::ColOperation::LISTING,
distinct = false,
term = {
type = Tianmu::common::ColumnType::UNK,
vc = 0x7fae44011e50,
cond_value = std::vector of length 0, capacity 0,
cond_numvalue = std::shared_ptr<Tianmu::utils::Hash64> (empty) = {
get() = 0x0
},
vc_id = 0,
is_vc_owner = false,
item = 0x0
},
--Type <RET> for more, q to quit, c to continue without paging--
dim = -2,
orig_precision = 19,
not_complete = false
}
此函数目的:
- 执行完APPLY_CONS过滤了条件之后, 就开始执行该函数
- 函数内部的逻辑自然是对conds条件处理完成之后的临时表, 做处理, 包含聚合, 排序
- 对比正确的case里属性, 在此函数处理过后, 就获取到了正确的值！
- 具体的给case列属性赋值, 是在属性了聚合运算之后
AggregationAlgorithm::MultiDimensionalGroupByScan
调用堆栈:
#0 Tianmu::core::AggregationAlgorithm::MultiDimensionalGroupByScan (this=0x7fb1511264c0, gbw=..., limit=@0x7fb151126118: 1, offset=@0x7fb151126548: 0, sender=0x796f3a0,
limit_less_than_no_groups=false) at /root/work/stonedb-dev-20230605/storage/tianmu/optimizer/aggregation_algorithm.cpp:373
#1 0x0000000002fb80c3 in Tianmu::core::AggregationAlgorithm::Aggregate (this=0x7fb1511264c0, just_distinct=false, limit=@0x7fb151126540: -1, offset=@0x7fb151126548: 0, sender=0x796f3a0)
at /root/work/stonedb-dev-20230605/storage/tianmu/optimizer/aggregation_algorithm.cpp:209
#2 0x0000000002d404d6 in Tianmu::core::TempTable::Materialize (this=0x7fae44eaa1c0, in_subq=false, sender=0x796f3a0, lazy=false)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/temp_table.cpp:2084
#3 0x0000000002cebc6e in Tianmu::core::Engine::Execute (this=0x79ded50, thd=0x7fae44000e10, lex=0x7fae44003138, result_output=0x7fae44006a88, unit_for_union=0x0)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/engine_execute.cpp:518
#4 0x0000000002cea825 in Tianmu::core::Engine::HandleSelect (this=0x79ded50, thd=0x7fae44000e10, lex=0x7fae44003138, result=@0x7fb151126dc8: 0x7fae44006a88, setup_tables_done_option=0,
res=@0x7fb151126dc4: 0, is_optimize_after_tianmu=@0x7fb151126dbc: 1, tianmu_free_join=@0x7fb151126dc0: 1, with_insert=0)
at /root/work/stonedb-dev-20230605/storage/tianmu/core/engine_execute.cpp:243
#5 0x0000000003084b13 in Tianmu::DBHandler::ha_my_tianmu_query (thd=0x7fae44000e10, lex=0x7fae44003138, result_output=@0x7fb151126dc8: 0x7fae44006a88, setup_tables_done_option=0,
res=@0x7fb151126dc4: 0, is_optimize_after_tianmu=@0x7fb151126dbc: 1, tianmu_free_join=@0x7fb151126dc0: 1, with_insert=0)
at /root/work/stonedb-dev-20230605/storage/tianmu/sql/ha_my_tianmu.cpp:95
#6 0x0000000002427aa8 in execute_sqlcom_select (thd=0x7fae44000e10, all_tables=0x7fae44f39648) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5204
#7 0x0000000002420e1e in mysql_execute_command (thd=0x7fae44000e10, first_level=true) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:2847
#8 0x0000000002428b0d in mysql_parse (thd=0x7fae44000e10, parser_state=0x7fb151127f90) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:5642
#9 0x000000000241db04 in dispatch_command (thd=0x7fae44000e10, com_data=0x7fb151128730, command=COM_QUERY) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1495
#10 0x000000000241c945 in do_command (thd=0x7fae44000e10) at /root/work/stonedb-dev-20230605/sql/sql_parse.cc:1034
#11 0x000000000254eeb5 in handle_connection (arg=0x9e23f40) at /root/work/stonedb-dev-20230605/sql/conn_handler/connection_handler_per_thread.cc:313
#12 0x0000000002c1e6f4 in pfs_spawn_thread (arg=0x9c38930) at /root/work/stonedb-dev-20230605/storage/perfschema/pfs.cc:2197
#13 0x00007fb15b44a1ca in start_thread () from /lib64/libpthread.so.0
#14 0x00007fb1582c0e73 in clone () from /lib64/libc.so.6
对比当派生中有值且聚合能非NULL的结果时的数据:
此函数目的:
- 不但执行聚合运算, 并且还要将结果temp_table和sender
- 之所以在执行聚合时还要将结果填充, 最根本的原因是tianmu的执行并非是mysql/sql层的火山模型, 每一个操作符做清晰的划分, 数据自下而上的流动。而是使用类似monetdb的物化模型, 尽可能将中间结果进行物化
- 第373行表明了为什么当派生表为空时, case列属性为NULL, 而不进行case计算
- 对其他列的数据的填充, 是以聚合运算结果的大小为依据
- 当聚合运算为空时, 对其他列不做赋值, 导致也不执行case运算
分析该查询出错的问题总结:
- 在定位查询出错的时候，大部分的情况是对相关代码逻辑并不熟悉, 需要根据线索逐步的定位出问题
- 能否定位并解决问题, 是相对于设计一个流程，更为复杂的过程, 因为这个过程并不是正向的，也就是说需要根据现有结果, 来反推当前代码的设计意图，并找到对于当前场景未考虑到的因素
tianmu引擎对于case列属性以及聚合处理的设计意图:
- 将设计意图的分析放在最后, 是因为设计这个东西最容易引起误解,大部分人都是照本宣科的某种教条来编写代码, 而忽视了代码或者说解决方案所要解决的问题
- tianmu引擎的查询执行非常明显的区别于mysql/sql层的查询执行, 对于操作符的处理尤其不同，这中间有一些数据库模型上的差异
- mysql/sql和innodb的配合是以行存为基础, 以表的访问为最基本的单元。而tianmu是以列为单位, 最小的处理单元是列, 更上层的所谓表的概念是不同的列的组合, 由此抽象出相当复杂的维度表的概念。
- myql/sql层的查询优化和查询执行采取了经典的火山模型, 操作符或者说算子相对独立
- tianmu采取了monetdb的思想, 采用了物化模型, 在执行时操作符之间的处理相对物化因素来说不是那么清晰, 例如在聚合处理的时候, 聚合的列与其他列的影响，以及将聚合后的结果或者说中间结果进行物化处理, 比较明显的就是对于temp_table的处理。
修复该查询错误的问题:
- 限于篇幅, 随后会对如何修复该场景下的查询问题, 如果重新设计相关逻辑和代码架构, 做更为详细的处理
- 这里需要指出, 只有当为了解决问题时, 所谓的代码设计和更宏观的架构才有意义, 而非某种死板的教条。代码架构必定是为了不但要解决当前的问题更是为了避免更为长远的场景下所出现的问题。
