【MySQL】踩坑笔记——保存带有换行符等特殊字符的数据，需要进行转义保存

news2024/12/28 3:59:59

问题描述

从DBeaver中导出了部分业务数据的 insert sql，明明在开发、测试环境都可以一把执行通过，却在预发环境执行前的语法检查失败了，提示有SQL语法错误。

这条SQL长这样，default_sql是要在odps上执行的sql语句，提前配置好放数据库里，方便后续读取。

INSERT INTO t_profile_attribute (`attribute`, default_sql) VALUES('province', 'WITH t AS (
SELECT b.analysis_dim1
,b.analysis_dim2
,a.uid
FROM (
SELECT uid
FROM {kun_table}
WHERE ds = MAX_PT("{kun_table}")
) a
LEFT JOIN (
SELECT uid, COALESCE(sale_province, ''未知'') AS analysis_dim1, analysis_dim2
FROM usr_user_group_insight_df
LATERAL VIEW OUTER EXPLODE(SPLIT({split_field}, '','')) t1 AS analysis_dim2
WHERE ds = MAX_PT("usr_user_group_insight_df")
) b ON a.uid = b.uid
)
SELECT
analysis_dim2,
analysis_dim1,
COUNT(1) AS cnt
FROM t
WHERE analysis_dim1 NOT IN ('''', ''未知'') and analysis_dim2 NOT IN ('''', ''未知'')
GROUP BY analysis_dim2, analysis_dim1
ORDER BY cnt DESC;');

原因分析：

MySQL数据库使用了不同的转义规则来处理特殊字符。
对于换行符,MySQL将其视为一个特殊字符,需要进行转义处理才能正确保存。
如果我们没有正确处理换行符,MySQL将无法正确解析文本中的换行符,就可能会导致数据保存失败。

解决方案：

替换特殊符号，需要将\n替换成\\n，SQL才能正常执行。
我是直接在IDEA文件中，使用ctrl+r, 将\n 替换成\\n 。
在这里插入图片描述

原DBeaver导出SQL:

INSERT INTO t_profile_attribute (`attribute`, default_sql) VALUES('province', 'WITH t AS (
SELECT b.analysis_dim1
,b.analysis_dim2
,a.uid
FROM (
SELECT uid
FROM {kun_table}
WHERE ds = MAX_PT("{kun_table}")
) a
LEFT JOIN (
SELECT uid, COALESCE(sale_province, ''未知'') AS analysis_dim1, analysis_dim2
FROM usr_user_group_insight_df
LATERAL VIEW OUTER EXPLODE(SPLIT({split_field}, '','')) t1 AS analysis_dim2
WHERE ds = MAX_PT("usr_user_group_insight_df")
) b ON a.uid = b.uid
)
SELECT
analysis_dim2,
analysis_dim1,
COUNT(1) AS cnt
FROM t
WHERE analysis_dim1 NOT IN ('''', ''未知'') and analysis_dim2 NOT IN ('''', ''未知'')
GROUP BY analysis_dim2, analysis_dim1
ORDER BY cnt DESC;');

替换完成后的SQL：

INSERT INTO t_profile_attribute ( `attribute`, default_sql, ) VALUES
('province', 'WITH t AS (\nSELECT b.analysis_dim1\n,b.analysis_dim2\n,a.uid\nFROM (\nSELECT uid\nFROM {kun_table}\nWHERE ds = MAX_PT("{kun_table}")\n) a\nLEFT JOIN (\nSELECT uid, COALESCE(sale_province, ''未知'') AS analysis_dim1, analysis_dim2\nFROM user_group_insight_df\nLATERAL VIEW OUTER EXPLODE(SPLIT({split_field}, '','')) t1 AS analysis_dim2\nWHERE ds = MAX_PT("user_group_insight_df")\n) b ON a.uid = b.uid\n)\nSELECT\nanalysis_dim2,\nanalysis_dim1,\nCOUNT(1) AS cnt\nFROM t\nWHERE analysis_dim1 NOT IN ('''', ''未知'') and analysis_dim2 NOT IN ('''', ''未知'')\nGROUP BY analysis_dim2, analysis_dim1\nORDER BY cnt DESC;')