对于大表操作,本来离线数据需要分批处理,刚开始只会用具体日期去做,通过复制多分转换和作业来处理。当日期范围大了后,这是个苦力活儿,kettle里面有参数化传递功能,多动手实操,懂得灵活变通自然工作也就轻松很多。
Mysql取行号
SELECT @row_number:=@row_number+1 AS num FROM tmp, (SELECT @row_number:=0) AS t
MySQL日期参数处理
select * from (
select num,DATE_FORMAT(DATE_SUB(NOW(),INTERVAL num-1 month), '%Y-%m-1 00:00:00') dt
,DATE_FORMAT(DATE_SUB(NOW(),INTERVAL num-1 month), '%Y-%m-1 00:00:00') startDate
,DATE_FORMAT(DATE_SUB(NOW(),INTERVAL (num-2) month), '%Y-%m-1 00:00:00') endDate
from(
SELECT @row_number:=@row_number+1 AS num FROM tmp, (SELECT @row_number:=0) AS t
)A limit 20
)B where startDate>='2021-01-01'
从结果获取参数,设置变量