系列文章目录
文章目录
- 系列文章目录
- 前言
- 一、定位CPU的流程是什么?
- 二、实操
- 1.测试代码样例
- 2.定位问题实践过程
- 总结
前言
一、定位CPU的流程是什么?
项目上线,CPU飙高不下,触发报警,如何定位排查问题。有两种办法1、通过堆栈 2、通过火焰图(本文略)
1、top查看进程占用率最高的进程
2、ps -mp pid定位到进程中cpu占用最高的线程
ps -mp 1153 -o THREAD,tid,pid
3、pstack 打印堆栈调用
二、实操
1.测试代码样例
#include <iostread>
#include <thread>
void test1()
{
while(1){
sleep(0.1);
}
}
void test2()
{
while(1){
sleep(10);
}
}
void dbdemo()
{
std::thread(test1).detach();
std::thread(test2).detach();
}
int main()
{
dbdemo();
return 0;
}
2.定位问题实践过程
1、top查出占用率高的进程DBclient的Pid 5896
2、ps -mp 5869 -o THREAD,tid,pid 查出DBclient进程中占用率过高的Tid,5900,转为16进制0x170C
3、pstack 5896,找到对应5900的线程,即可发现调用栈
发现线程4调用关系,test1中sleep函数导致cpu占用过高,
#0 0x00007fa889545626 in sleep () from /lib64/libc.so.6
#1 0x000000000042dd35 in test1() ()
[root@localhost ~]# pstack 5896
Thread 6 (Thread 0x7fa88616e700 (LWP 5897)):
#0 0x00007fa88954585d in nanosleep () from /lib64/libc.so.6
#1 0x00007fa889576134 in usleep () from /lib64/libc.so.6
#2 0x00007fa88ab51422 in bvar::detail::SamplerCollector::run (this=0x16d9b40) at /opt/data/code/vplatform_thirdparty/BRPC-1.0.0-rc02/src/bvar/detail/sampler.cpp:180
#3 0x00007fa88ab51f49 in bvar::detail::SamplerCollector::sampling_thread (arg=<optimized out>) at /opt/data/code/vplatform_thirdparty/BRPC-1.0.0-rc02/src/bvar/detail/sampler.cpp:110
#4 0x00007fa88a075ea5 in start_thread () from /lib64/libpthread.so.0
#5 0x00007fa88957e96d in clone () from /lib64/libc.so.6
Thread 5 (Thread 0x7fa88596d700 (LWP 5898)):
#0 0x00007fa889573c3d in poll () from /lib64/libc.so.6
#1 0x00007fa887a33948 in mongoc_socket_poll () from /home/vagrant/Mycode/DBClient/../libs/gcc_lib/libmongoc-1.0.so.0
#2 0x00007fa887a36627 in _mongoc_stream_socket_poll () from /home/vagrant/Mycode/DBClient/../libs/gcc_lib/libmongoc-1.0.so.0
#3 0x00007fa887a35815 in mongoc_stream_poll () from /home/vagrant/Mycode/DBClient/../libs/gcc_lib/libmongoc-1.0.so.0
#4 0x00007fa887a31cbb in _server_monitor_awaitable_ismaster_recv () from /home/vagrant/Mycode/DBClient/../libs/gcc_lib/libmongoc-1.0.so.0
#5 0x00007fa887a32208 in mongoc_server_monitor_check_server () from /home/vagrant/Mycode/DBClient/../libs/gcc_lib/libmongoc-1.0.so.0
#6 0x00007fa887a32c12 in _server_monitor_thread () from /home/vagrant/Mycode/DBClient/../libs/gcc_lib/libmongoc-1.0.so.0
#7 0x00007fa88a075ea5 in start_thread () from /lib64/libpthread.so.0
#8 0x00007fa88957e96d in clone () from /lib64/libc.so.6
Thread 4 (Thread 0x7fa88516c700 (LWP 5900)):
#0 0x00007fa889545626 in sleep () from /lib64/libc.so.6
#1 0x000000000042dd35 in test1() ()
#2 0x0000000000433e73 in void std::_Bind_simple<void (*())()>::_M_invoke<>(std::_Index_tuple<>) ()
#3 0x0000000000433dcd in std::_Bind_simple<void (*())()>::operator()() ()
#4 0x0000000000433d66 in std::thread::_Impl<std::_Bind_simple<void (*())()> >::_M_run() ()
#5 0x00007fa889e1b330 in ?? () from /lib64/libstdc++.so.6
#6 0x00007fa88a075ea5 in start_thread () from /lib64/libpthread.so.0
#7 0x00007fa88957e96d in clone () from /lib64/libc.so.6
Thread 3 (Thread 0x7fa88496b700 (LWP 5901)):
#0 0x00007fa88954585d in nanosleep () from /lib64/libc.so.6
#1 0x00007fa8895456f4 in sleep () from /lib64/libc.so.6
#2 0x000000000042dd45 in test2() ()
#3 0x0000000000433e73 in void std::_Bind_simple<void (*())()>::_M_invoke<>(std::_Index_tuple<>) ()
#4 0x0000000000433dcd in std::_Bind_simple<void (*())()>::operator()() ()
#5 0x0000000000433d66 in std::thread::_Impl<std::_Bind_simple<void (*())()> >::_M_run() ()
#6 0x00007fa889e1b330 in ?? () from /lib64/libstdc++.so.6
#7 0x00007fa88a075ea5 in start_thread () from /lib64/libpthread.so.0
#8 0x00007fa88957e96d in clone () from /lib64/libc.so.6
Thread 2 (Thread 0x7fa87ffff700 (LWP 5902)):
#0 0x00007fa88a079de2 in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00007fa887a32b69 in mongoc_server_monitor_wait () from /home/vagrant/Mycode/DBClient/../libs/gcc_lib/libmongoc-1.0.so.0
#2 0x00007fa887a32dc2 in _server_monitor_rtt_thread () from /home/vagrant/Mycode/DBClient/../libs/gcc_lib/libmongoc-1.0.so.0
#3 0x00007fa88a075ea5 in start_thread () from /lib64/libpthread.so.0
#4 0x00007fa88957e96d in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7fa88bd6ab80 (LWP 5896)):
#0 0x00007fa88954585d in nanosleep () from /lib64/libc.so.6
#1 0x00007fa8895456f4 in sleep () from /lib64/libc.so.6
#2 0x000000000042deba in main ()
[root@localhost ~]#
总结
如果你对线上进程cpu飙高不知道如何定位,可以通过本文尝试一下,希望对你有所帮助。
老铁,如果觉得不错,请点赞收藏。