大纲
- 环境
- 分析过程
- 函数是否真的未定义
- 是否有完整实现
- 被谁编译
- 代码是否被编译到静态库
- 链接出现了什么问题
- 原因猜想
- 解决方案
- 参考资料
在《Opentelemetry-Language APIs & SDKs-C+±Getting Started》一文中,介绍了如果编译一个可以发出Trace遥测数据的C++项目。虽然过程很详细,但是在我的环境下,编译出现了问题。本文将介绍分析并解决该问题的过程。
环境
我的环境是
cat /proc/version
Linux version 5.15.0-102-generic (buildd@lcy02-amd64-080) (gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, GNU ld (GNU Binutils for Ubuntu) 2.38) #112-Ubuntu SMP Tue Mar 5 16:50:32 UTC 2024
cmake --version
cmake version 3.29.2
CMake suite maintained and supported by Kitware (kitware.com/cmake).
Opentelemetry-cpp的编译需要3.20以上的cmake。如果操作系统比较新,直接apt安装最新的cmake基本能满足需求;如果比较老,软件安装包里也没有符合的cmake。则可以参考这篇文章《正确的方式升级ubuntu的cmake》。
g++ --version
g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright © 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
分析过程
在执行完《Opentelemetry-Language APIs & SDKs-C+±Getting Started》中最后一条编译指令后,会报出如下错误:
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(tracer_provider.cc.o): in function `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetLogHandler()':
tracer_provider.cc:(.text._ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler13GetLogHandlerEv[_ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler13GetLogHandlerEv]+0x9): undefined reference to `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(tracer_provider.cc.o): in function `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetLogLevel()':
tracer_provider.cc:(.text._ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler11GetLogLevelEv[_ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler11GetLogLevelEv]+0x9): undefined reference to `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(random_id_generator.cc.o): in function `opentelemetry::v1::sdk::trace::RandomIdGenerator::GenerateSpanId()':
random_id_generator.cc:(.text+0x41): undefined reference to `opentelemetry::v1::sdk::common::Random::GenerateRandomBuffer(opentelemetry::v1::nostd::span<unsigned char, 18446744073709551615ul>)'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(random_id_generator.cc.o): in function `opentelemetry::v1::sdk::trace::RandomIdGenerator::GenerateTraceId()':
random_id_generator.cc:(.text+0xc7): undefined reference to `opentelemetry::v1::sdk::common::Random::GenerateRandomBuffer(opentelemetry::v1::nostd::span<unsigned char, 18446744073709551615ul>)'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/exporters/ostream/libopentelemetry_exporter_ostream_span.a(span_exporter.cc.o): in function `opentelemetry::v1::exporter::trace::OStreamSpanExporter::OStreamSpanExporter(std::ostream&)':
span_exporter.cc:(.text+0x19a): undefined reference to `opentelemetry::v1::sdk::trace::SpanExporter::SpanExporter()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/exporters/ostream/libopentelemetry_exporter_ostream_span.a(span_exporter.cc.o): in function `opentelemetry::v1::exporter::trace::OStreamSpanExporter::~OStreamSpanExporter()':
span_exporter.cc:(.text._ZN13opentelemetry2v18exporter5trace19OStreamSpanExporterD2Ev[_ZN13opentelemetry2v18exporter5trace19OStreamSpanExporterD5Ev]+0x36): undefined reference to `opentelemetry::v1::sdk::trace::SpanExporter::~SpanExporter()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/exporters/ostream/libopentelemetry_exporter_ostream_span.a(span_exporter.cc.o):(.data.rel.ro._ZTIN13opentelemetry2v18exporter5trace19OStreamSpanExporterE[_ZTIN13opentelemetry2v18exporter5trace19OStreamSpanExporterE]+0x10): undefined reference to `typeinfo for opentelemetry::v1::sdk::trace::SpanExporter'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/resource/libopentelemetry_resources.a(resource_detector.cc.o): in function `opentelemetry::v1::sdk::resource::OTELResourceDetector::Detect()':
resource_detector.cc:(.text+0x5f): undefined reference to `opentelemetry::v1::sdk::common::GetStringEnvironmentVariable(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)'
/usr/bin/ld: resource_detector.cc:(.text+0x7e): undefined reference to `opentelemetry::v1::sdk::common::GetStringEnvironmentVariable(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)'
collect2: error: ld returned 1 exit status
gmake[2]: *** [CMakeFiles/dice-server.dir/build.make:102: dice-server] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/dice-server.dir/all] Error 2
gmake: *** [Makefile:91: all] Error 2
问题比较多,我们先定位和关注第一个问题:
函数是否真的未定义
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(tracer_provider.cc.o): in function `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetLogHandler()':
tracer_provider.cc:(.text._ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler13GetLogHandlerEv[_ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler13GetLogHandlerEv]+0x9): undefined reference to `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()'
它的意思是找不到opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()的定义。
我们到工程中定位该文件,然后看它编译到哪个项目里去了。
是否有完整实现
这个函数定义在opentelemetry-cpp/sdk/src/common/global_log_handler.cc中。可以看到该函数是有完整实现的。
被谁编译
我们在opentelemetry-cpp/sdk/src/common/global_log_handler.cc所在目录下找到CMakeLists.txt文件
其中编译相关的指令是
……
set(COMMON_SRCS random.cc core.cc global_log_handler.cc env_variables.cc
base64.cc)
……
add_library(opentelemetry_common ${COMMON_SRCS})
set_target_properties(opentelemetry_common PROPERTIES EXPORT_NAME common)
set_target_version(opentelemetry_common)
target_link_libraries(
opentelemetry_common PUBLIC opentelemetry_api opentelemetry_sdk
Threads::Threads)
可见这个函数被编译到opentelemetry_common这个静态库中。
这个文件位于opentelemetry-cpp/build/sdk/src/common/libopentelemetry_common.a。
代码是否被编译到静态库
nm libopentelemetry_common.a | grep GetHandlerAndLevel
0000000000000048 b _ZGVZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler18GetHandlerAndLevelEvE17handler_and_level
000000000000025a T _ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler18GetHandlerAndLevelEv
0000000000000020 b _ZZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler18GetHandlerAndLevelEvE17handler_and_level
U _ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler18GetHandlerAndLevelEv
第二行的符号_ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler18GetHandlerAndLevelEv指向的就是opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()方法(因为这是C++项目,所以符号表是经过处理的),它的状态是T,即
The symbol is in the text (code) section.
这说明这个方法的实现是存在于libopentelemetry_common.a中的。
所以,这只能说明roll-dice的链接过程,没有找到opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()方法,而方法的确在静态库libopentelemetry_common.a中。进而有两个可能:
- roll-dice没有链接libopentelemetry_common.a
- roll-dice由于种种原因没有在libopentelemetry_common.a中找到opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()的实现。
链接出现了什么问题
我们先研究上述1的可能性,即roll-dice是否没有链接libopentelemetry_common.a?
在roll-dice/build/CMakeFiles/dice-server.dir/link.txt文件中,我们看到如下内容
/usr/bin/c++ -rdynamic "CMakeFiles/dice-server.dir/main.cpp.o"
-o dice-server /home/fangliang/otel-cpp-starter/oatpp/build/src/liboatpp.a
/home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/common/libopentelemetry_common.a
/home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a
/home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/exporters/ostream/libopentelemetry_exporter_ostream_span.a
/home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/resource/libopentelemetry_resources.a
可以发现,roll-dice项目链接了libopentelemetry_common.a 。
那只能去研究为什么roll-dice没在libopentelemetry_common.a中找到opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()方法。
我们回到最开的错误提示,需要梳理下它们的关系
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(tracer_provider.cc.o): in function `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetLogHandler()':
tracer_provider.cc:(.text._ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler13GetLogHandlerEv[_ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler13GetLogHandlerEv]+0x9): undefined reference to `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(tracer_provider.cc.o): in function `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetLogLevel()':
tracer_provider.cc:(.text._ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler11GetLogLevelEv[_ZN13opentelemetry2v13sdk6common12internal_log16GlobalLogHandler11GetLogLevelEv]+0x9): undefined reference to `opentelemetry::v1::sdk::common::internal_log::GlobalLogHandler::GetHandlerAndLevel()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(random_id_generator.cc.o): in function `opentelemetry::v1::sdk::trace::RandomIdGenerator::GenerateSpanId()':
random_id_generator.cc:(.text+0x41): undefined reference to `opentelemetry::v1::sdk::common::Random::GenerateRandomBuffer(opentelemetry::v1::nostd::span<unsigned char, 18446744073709551615ul>)'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/trace/libopentelemetry_trace.a(random_id_generator.cc.o): in function `opentelemetry::v1::sdk::trace::RandomIdGenerator::GenerateTraceId()':
random_id_generator.cc:(.text+0xc7): undefined reference to `opentelemetry::v1::sdk::common::Random::GenerateRandomBuffer(opentelemetry::v1::nostd::span<unsigned char, 18446744073709551615ul>)'
这些都提示libopentelemetry_trace.a依赖于libopentelemetry_common.a,但是没找到libopentelemetry_common.a中的的确存在的一系列方法。
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/exporters/ostream/libopentelemetry_exporter_ostream_span.a(span_exporter.cc.o): in function `opentelemetry::v1::exporter::trace::OStreamSpanExporter::OStreamSpanExporter(std::ostream&)':
span_exporter.cc:(.text+0x19a): undefined reference to `opentelemetry::v1::sdk::trace::SpanExporter::SpanExporter()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/exporters/ostream/libopentelemetry_exporter_ostream_span.a(span_exporter.cc.o): in function `opentelemetry::v1::exporter::trace::OStreamSpanExporter::~OStreamSpanExporter()':
span_exporter.cc:(.text._ZN13opentelemetry2v18exporter5trace19OStreamSpanExporterD2Ev[_ZN13opentelemetry2v18exporter5trace19OStreamSpanExporterD5Ev]+0x36): undefined reference to `opentelemetry::v1::sdk::trace::SpanExporter::~SpanExporter()'
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/exporters/ostream/libopentelemetry_exporter_ostream_span.a(span_exporter.cc.o):(.data.rel.ro._ZTIN13opentelemetry2v18exporter5trace19OStreamSpanExporterE[_ZTIN13opentelemetry2v18exporter5trace19OStreamSpanExporterE]+0x10): undefined reference to `typeinfo for opentelemetry::v1::sdk::trace::SpanExporter'
这些都提示libopentelemetry_exporter_ostream_span.a依赖于libopentelemetry_trace.a,但是没找到libopentelemetry_trace.a中的的确存在的一系列方法。
/usr/bin/ld: /home/fangliang/otel-cpp-starter/opentelemetry-cpp/build/sdk/src/resource/libopentelemetry_resources.a(resource_detector.cc.o): in function `opentelemetry::v1::sdk::resource::OTELResourceDetector::Detect()':
resource_detector.cc:(.text+0x5f): undefined reference to `opentelemetry::v1::sdk::common::GetStringEnvironmentVariable(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)'
/usr/bin/ld: resource_detector.cc:(.text+0x7e): undefined reference to `opentelemetry::v1::sdk::common::GetStringEnvironmentVariable(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)'
这些都提示libopentelemetry_resources.a依赖于libopentelemetry_common.a,但是没找到libopentelemetry_common.a中的的确存在的一系列方法。
这些我们在CMakelists.txt中也存在也会得到印证:
- opentelemetry-cpp/sdk/src/trace/CMakeLists.txt
target_link_libraries(opentelemetry_trace PUBLIC opentelemetry_common
opentelemetry_resources)
- opentelemetry-cpp/exporters/ostream/CMakeLists.txt
target_link_libraries(opentelemetry_exporter_ostream_span
PUBLIC opentelemetry_trace)
- opentelemetry-cpp/sdk/src/resource/CMakeLists.txt
target_link_libraries(opentelemetry_resources opentelemetry_common)
而roll-dice的链接指令顺序是
原因猜想
这个顺序似乎符合一种猜想:
- 链接opentelemetry_common时不知道opentelemetry_trace需要什么,导致后续链接opentelemetry_trace时找不到依赖opentelemetry_common中的方法。
- 链接opentelemetry_trace时不知道opentelemetry_exporter_ostream_span需要什么,导致后续链接opentelemetry_exporter_ostream_span时找不到依赖opentelemetry_trace中的方法。
- 链接opentelemetry_common时不知道opentelemetry_resources需要什么,导致后续链接opentelemetry_resources时找不到依赖opentelemetry_common中的方法。
这个猜想和之前发现的报错信息吻合。
解决方案
那么我们将链接顺序做个调整:
- opentelemetry_common被依赖最多,最后链接
- opentelemetry_resources只依赖于opentelemetry_common,但是被opentelemetry_trace依赖,所以要位于opentelemetry_trace之后
- opentelemetry_trace被opentelemetry_exporter_ostream_span依赖,所以它要在opentelemetry_exporter_ostream_span之后链接,而在依赖项opentelemetry_resources和opentelemetry_common之前。
我们只需要修改roll-dice/CMakeLists.txt文件即可
# target_link_libraries(dice-server PRIVATE ${OATPP_LIB} ${OPENTELEMETRY_COMMON_LIB} ${OPENTELEMETRY_TRACE_LIB} ${OPENTELEMETRY_EXPORTER_LIB} ${OPENTELEMETRY_RESOURCE_LIB})
target_link_libraries(dice-server PRIVATE ${OATPP_LIB} ${OPENTELEMETRY_EXPORTER_LIB} ${OPENTELEMETRY_TRACE_LIB} ${OPENTELEMETRY_RESOURCE_LIB} ${OPENTELEMETRY_COMMON_LIB})
清理项目,然后重新生成
cmake ..
cmake --build .
然后我们就看到编译成功了。
[100%] Linking CXX executable dice-server
[100%] Built target dice-server
参考资料
- https://praveenv253.github.io/logs/2014/03/15/log-message-1.html
- https://stackoverflow.com/questions/45135/why-does-the-order-in-which-libraries-are-linked-sometimes-cause-errors-in-gcc
- https://www.cnblogs.com/Maker-Liu/p/16550381.html
- https://linux.die.net/man/1/nm