工作中真实踩坑,一个or让sql变慢1000倍
- 1.情况说明
- 2.解释计划
- 3.or改成union
- 4.总结
1.情况说明
- 测试环境,有两张表,分别是讲师表
t_train_lecturer
(后面简称B表),和讲师的授课时长表t_train_activity
(后面简称A表) - B表有1w多条数据,A表10w条数据,但是A是个混合表,其中和B有关系的数据也是1w左右
- B表讲师分为内部和外部的讲师(
坑就坑在这了
),在A表中用lecturer_source的0和1表示,其中内部则关联lecturer_num
工号,外部则关联telephone_num
手机号. - A表lecturer_id,B表的lecturer_num,telephone_num均为索引
于是就有了这么一条sql,分组聚合查出讲师的授课时长:
select
a.lecturer_id,
sum(a.course_hour) totalCourseHour
from
t_train_lecturer l
left join t_train_activity a on
a.lecturer_id is not null
-- 内部
and ((a.lecturer_id = l.lecturer_num
and a.lecturer_source = 0)
-- 外部
or (a.lecturer_id = l.telphone_num
and a.lecturer_source = 1))
where
l.delete_flag = 0
group by
a.lecturer_id
如上,我在left join的条件中用or将内部外部一起查,执行,结果sql达到10s
2.解释计划
解释计划如下: 可以看到,两个表全表扫描了
再用dbeaver的解释计划看执行成本
3.or改成union
为了避免其他地方改动的影响,这里就单单将or的条件拆成两个sql后union
select
a.lecturer_id,
sum(a.course_hour) totalCourseHour
from
t_train_lecturer l
left join t_train_activity a on
a.lecturer_id is not null
-- 内部
and (a.lecturer_id = l.lecturer_num and a.lecturer_source = 0)
where
l.delete_flag = 0
group by
a.lecturer_id
union all
select
a.lecturer_id,
sum(a.course_hour) totalCourseHour
from
t_train_lecturer l
left join t_train_activity a on
a.lecturer_id is not null
-- 外部
and (a.lecturer_id = l.telphone_num and a.lecturer_source = 1)
where
l.delete_flag = 0
group by
a.lecturer_id
执行耗时: 1.5s
解释计划: 可以看到用到了A表的key,因为是left join所以B表是全表
查看dbeaver的执行成本
4.总结
虽然一直都知道or会让索引失效,但是真正遇到的时候才发现可怕之处,这才测试环境1w到10w的数据,就已经是10s的查询时间,正式环境可想而知。
当然最终的sql肯定也不是1.5s, 因为只是需要讲师和它的时长,比如下面这个,先单独聚合A表得到时长,再分别通过exist来关联内部和外部B表数据筛选
select
lecturer_id,
sum(course_hour)
from
t_train_activity a
where
lecturer_id is not null
and lecturer_source = 0
and exists (
select
1
from
t_train_lecturer l
where
l.delete_flag = 0
and l.lecturer_num = a.lecturer_id)
group by
lecturer_id
union
select
lecturer_id,
sum(course_hour)
from
t_train_activity a
where
lecturer_id is not null
and lecturer_source = 1
and exists (
select
1
from
t_train_lecturer l
where
l.delete_flag = 0
and l.telphone_num = a.lecturer_id)
group by
lecturer_id
执行时间20ms