Hi I’m Shendi
今天收到消息说所有软件不能用了,网页都打不开,遇到了问题,于是在这里记录一下
记一次Mysql数据库宕机This could be because you hit a bug.
起因
为了节省成本,对于小公司而言服务器数量通常不会太多,而我这,就一台。
于是要将数据库、程序、所有的东西都放到这台服务器上,当然为了节省开销,容器也是没有使用的,所以这样就会导致,一个部位出错,整体瘫痪。
今天就遇到了这样的问题,是我还在睡梦中大抵是被电话闹醒,后面才看到十来个未接电话,简单地说就是告知我服务器挂了。
最近编写了一个新的 SpringBoot 程序,基于若依开发,放到服务器上偶然遇到了一次这样的问题 ——数据库宕机,当时不清楚什么情况,并且查找数据库日志也没有找到什么,猜测可能是误关数据库或者…所以就没有再管,就这样,我将数据库启动后,几天后,就是今天,问题又复现了。
环境描述
系统:windows server
数据库:Mysql
问题描述
因为我没有在 my.ini 中配置日志地址,那么默认是和data目录在一起的,我在数据库日志中找到了错误信息
11:08:34 UTC - mysqld got exception 0xc0000005 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
Attempting to collect some information that could help diagnose the problem.
As this is a crash and something is definitely wrong, the information
collection process might fail.
key_buffer_size=8388608
read_buffer_size=131072
max_used_connections=49
max_threads=200
thread_count=48
connection_count=47
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 86998 K bytes of memory
Hope that's ok; if not, decrease some variables in the equation.
Thread pointer: 0x9b37908c70
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
7ff743231ce6 mysqld.exe!json_binary::parse_binary()[json_binary.cc:1082]
7ff7430b34c6 mysqld.exe!Field_json::val_json()[field.cc:7645]
7ff7430e692b mysqld.exe!get_json_wrapper()[item_json_func.cc:822]
7ff74329ddf3 mysqld.exe!Table_function_json::fill_result_table()[table_function.cc:788]
7ff743253297 mysqld.exe!join_materialize_table_function()[sql_executor.cc:2652]
7ff7432540fb mysqld.exe!QEP_TAB::prepare_scan()[sql_executor.cc:1576]
7ff743257936 mysqld.exe!sub_select()[sql_executor.cc:1438]
7ff743250589 mysqld.exe!do_select()[sql_executor.cc:1171]
7ff743252897 mysqld.exe!JOIN::exec()[sql_executor.cc:294]
7ff7430d35af mysqld.exe!subselect_single_select_engine::exec()[item_subselect.cc:2954]
7ff7430d2dde mysqld.exe!Item_subselect::exec()[item_subselect.cc:772]
7ff7430d8c7b mysqld.exe!Item_singlerow_subselect::val_int()[item_subselect.cc:1312]
7ff742f1ae27 mysqld.exe!Item::evaluate()[item.cc:6603]
7ff742f287d4 mysqld.exe!Item::update_null_value()[item.cc:6556]
7ff7430d490e mysqld.exe!Item_subselect::is_null()[item_subselect.h:185]
7ff742e96e57 mysqld.exe!Item_func_isnotnull::val_int()[item_cmpfunc.cc:5277]
7ff742f4292c mysqld.exe!eval_const_cond()[item_func.cc:153]
7ff7430fa0ee mysqld.exe!internal_remove_eq_conds()[sql_optimizer.cc:9418]
7ff7430f9d55 mysqld.exe!internal_remove_eq_conds()[sql_optimizer.cc:9295]
7ff7430f9d55 mysqld.exe!internal_remove_eq_conds()[sql_optimizer.cc:9295]
7ff7430ff8bc mysqld.exe!remove_eq_conds()[sql_optimizer.cc:9504]
7ff7430fd478 mysqld.exe!optimize_cond()[sql_optimizer.cc:9260]
7ff7430fc455 mysqld.exe!JOIN::optimize()[sql_optimizer.cc:295]
7ff7430c0d26 mysqld.exe!SELECT_LEX::optimize()[sql_select.cc:1483]
7ff7430bdb9f mysqld.exe!Sql_cmd_dml::execute_inner()[sql_select.cc:640]
7ff7430bd9a6 mysqld.exe!Sql_cmd_dml::execute()[sql_select.cc:554]
7ff743022d38 mysqld.exe!mysql_execute_command()[sql_parse.cc:4147]
7ff743023816 mysqld.exe!mysql_parse()[sql_parse.cc:4927]
7ff74301d6b8 mysqld.exe!dispatch_command()[sql_parse.cc:1607]
7ff74301e5e5 mysqld.exe!do_command()[sql_parse.cc:1233]
7ff742e9c538 mysqld.exe!handle_connection()[connection_handler_per_thread.cc:308]
7ff743fc3e87 mysqld.exe!pfs_spawn_thread()[pfs.cc:2839]
7ff743bdb1dc mysqld.exe!win_thread_start()[my_thread.cc:52]
7ffba9f2f4a0 ucrtbase.DLL!_o__realloc_base()
7ffbbb5c13f2 KERNEL32.DLL!BaseThreadInitThunk()
7ffbbd9d54f4 ntdll.dll!RtlUserThreadStart()
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort.
Query (9b3424c5b8): select count(0) from (
SELECT * FROM table WHERE state!=2 AND pqid=105 AND hasimg=0 AND (type!=2 OR (select 1 from JSON_TABLE(param, '$.answer[*]' COLUMNS( content VARCHAR(512) PATH '$.content', isCorrect BOOLEAN PATH '$.isCorrect' ) ) AS jt WHERE jt.isCorrect=TRUE AND jt.content='姝g‘') IS NOT NULL)
) tmp_count
Connection ID (thread ID): 8026
Status: NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
这也是我头一次遇到这种情况,错误信息也没有更详细的说明了,或许是我不会看,其中只有一个程序中的SQL语句,这个语句也是我第一次使用 JSON_TABLE
,所以问题基本上就是这里了,可能是这种复杂的语句使内存占用过多导致宕机,而我的数据格式使用的JSON,又不好更改,没有找到解决办法。
我在 Windows 事件管理器中看到了 mysqld 的错误事件,如下
解决办法
没有找到解决办法,但这个问题总得解决,首先问题可能是内存不够,于是将内存想办法增加,其次可能还会继续遇到数据库宕机的问题,既然无法避免,那么就增加一个定时任务,隔一段时间启动数据库,这样即使宕机了也能在一定时间内恢复,而在Windows中,可以直接增加定时任务,最快是1分钟一次,这里同样记录下方法
-
编写启动脚本,xxx.bat,内容为
net start mysql
-
打开任务计划程序,可以按win+R输入taskschd.msc回车打开
-
任务计划程序库中,右键创建任务,输入名称描述等
-
点击触发器,新建,开始任务可以选择按预定计划或者发生事件时最好建两个触发器,将两个都弄上,事件id是1000,按预定计划需要勾选重复任务间隔,例如一分钟,持续一天,而上面的设置每隔一天一次
-
操作中新建,程序和脚本选择最开始编写的脚本即可。
总结
- 在实际使用中对于复杂的结构应避免JSON字段的使用,否则就会像我这样,为了实现某个需求不得不使用JSON_TABLE又出现宕机的问题。
- 挂个脚本当程序挂掉后自动重启,增加保障,减少负面影响。
END