坑惨啦！！！——符号冲突案例分析

背景

前段时间在北汽项目中，遇到了一个奇怪现象：程序启动之后，偶现运行一段时间后，crash，复现频率较高。困扰了大家较长时间。最终在和同事的不懈努力下，找到的根因，并找到了解决方法。过程中也学习到了很多。在此，记录并分享，希望能够帮助大家。

问题描述

作为OTA服务的提供方，我们提供方式一般为将自己的代码编译成动态库（libsysi4dpc.so），提供给设备厂家，让他们进行集成，并调试。

从控和主控之间通过MQTT协议实现进程间通信。MQTT client相关接口被封装在libupc.a静态库中。也就是说，libsysi4dpc.so 库中其实是包含libupc.a 静态库的所有代码段和数据段（需要的都会加载进去，以.o为单位），并不再依赖其他MQTT 动态库。如下图所示：

但是当客户集成libsysi4dpc.so 库，生成可执行程序hal-process。当出现crash时，通过gdb进行函数调用栈跟踪。会发现，虽然出错线程的底层调用函数是我们的adm_ups_run 接口（进行MQTT 重连的线程），但是其栈进入到了libmqtt-paho.so库中（这应该也是支持MQTT协议的库）。这就让我们很奇怪，因为我们代码中的MQTT协议接口应该是在libupc.a 中，为什么会走到libmqtt-paho.so 中呢？

通过查询hal-process 动态库依赖，如下：

由上可知，libmqtt-paho.so 是hal-process进行链接的。（细心的同事可能会觉得奇怪，为什么没有看到libsysi4dpc.so，那是因为hal-process并不直接依赖我们的库，而是通过libqzm_dpc.so 间接依赖）。

到此，问题应该就明确了，crash 的原因：是因为adm_ups_run 接口，并没有走到我们自己的mqtt库中，而是运行到了客户提供的mqtt库中，由于mqtt版本的不同，其中兼容问题，导致了crash。

项目的模块依赖关系大致如下：

上述的问题其实有一个专业的名词，叫做同名符号冲突。

符号冲突

符号冲突，作为开发人员，我们应该并不少见。比如编译器提示如下错误:

multiple definition of `xxx';

其原因，就是同一作用域内，存在相同的符号。可参考C语言中弱符号与弱引用的实际应用_c语言中的弱引用_谢艺华的博客-CSDN博客文章。

但是为什么上面的场景中，客户在编译hal-process程序时，编译器没有提示相关错误呢？因为符号冲突存在两种形式：显式符号冲突，隐式符号冲突。

显示符号冲突

显示符号冲突，就是我们常见的一种错误，它可以在编译阶段直接报错。比如：

yihua@ubuntu:~/symbol$ cat strong_symbol.c
#include<stdio.h>
int symbol()
{
        printf("strong symbol\n");
}
yihua@ubuntu:~/symbol$ cat week_symbol.c
#include<stdio.h>
int symbol()
{
        printf("week symbol\n");
}
yihua@ubuntu:~/symbol$ cat main.c
#include<stdio.h>
#include<stdlib.h>
extern int symbol();
int main()
{
        symbol();
        return 0;
}
yihua@ubuntu:~/symbol$ gcc main.c strong_symbol.c week_symbol.c -o main
/tmp/ccSU1PtA.o: In function `symbol':
week_symbol.c:(.text+0x0): multiple definition of `symbol'
/tmp/cc1gLIic.o:strong_symbol.c:(.text+0x0): first defined here
collect2: error: ld returned 1 exit status

但有时也存在这样的场景。两个静态库拥有相同的函数符号symbol，并且主函数也调用了symbol,编译阶段会提示错误吗？

还是上述的代码，我们进行如下验证

生成libstrong.a和libweek.a静态库

C
yihua@ubuntu:~/symbol$ gcc -c strong_symbol.c
yihua@ubuntu:~/symbol$ gcc -c week_symbol.c
yihua@ubuntu:~/symbol$ ar -rcs libstrong.a strong_symbol.o
yihua@ubuntu:~/symbol$ ar -rcs libweek.a week_symbol.o

2. main.c链接两个静态库，生成可执行程序

由上可知，并没有提示错误，并且输出week symbol。

分析：

我们可以在编译选项中增加-Wl,-–verbose选项（将编译过程中的详细信息进行输出）。输出如下：

yihua@ubuntu:~/symbol$ gcc main.c -L./ -Wl,--verbose -lweek -lstrong -o main
GNU ld (GNU Binutils) 2.30
Supported emulations:
   elf_x86_64
   elf32_x86_64
   elf_i386
   elf_iamcu
   i386linux
   elf_l1om
   elf_k1om
using internal linker script:
==================================================
/* Script for -pie -z combreloc -z now -z relro: position independent executable, combine & sort relocs */
/* Copyright (C) 2014-2018 Free Software Foundation, Inc.
   Copying and distribution of this script, with or without modification,
   are permitted in any medium without royalty provided the copyright
   notice and this notice are preserved. */
OUTPUT_FORMAT("elf64-x86-64", "elf64-x86-64",
              "elf64-x86-64")
OUTPUT_ARCH(i386:x86-64)
ENTRY(_start)
SEARCH_DIR("/usr/local/x86_64-pc-linux-gnu/lib64"); SEARCH_DIR("/usr/local/lib64"); SEARCH_DIR("/lib64"); SEARCH_DIR("/usr/lib64"); SEARCH_DIR("/usr/local/x86_64-pc-linux-gnu/lib"); SEARCH_DIR("/usr/local/lib"); SEARCH_DIR("/lib"); SEARCH_DIR("/usr/lib");
SECTIONS
{
/* Read-only sections, merged into text segment: */
PROVIDE (__executable_start = SEGMENT_START("text-segment", 0)); . = SEGMENT_START("text-segment", 0) + SIZEOF_HEADERS;
.interp         : { *(.interp) }
.note.gnu.build-id : { *(.note.gnu.build-id) }
.hash           : { *(.hash) }
.gnu.hash       : { *(.gnu.hash) }
.dynsym         : { *(.dynsym) }
.dynstr         : { *(.dynstr) }
.gnu.version    : { *(.gnu.version) }
.gnu.version_d : { *(.gnu.version_d) }
.gnu.version_r : { *(.gnu.version_r) }
.rela.dyn       :
    {
      *(.rela.init)
      *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*)
      *(.rela.fini)
      *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*)
      *(.rela.data .rela.data.* .rela.gnu.linkonce.d.*)
      *(.rela.tdata .rela.tdata.* .rela.gnu.linkonce.td.*)
      *(.rela.tbss .rela.tbss.* .rela.gnu.linkonce.tb.*)
      *(.rela.ctors)
      *(.rela.dtors)
      *(.rela.got)
      *(.rela.bss .rela.bss.* .rela.gnu.linkonce.b.*)
      *(.rela.ldata .rela.ldata.* .rela.gnu.linkonce.l.*)
      *(.rela.lbss .rela.lbss.* .rela.gnu.linkonce.lb.*)
      *(.rela.lrodata .rela.lrodata.* .rela.gnu.linkonce.lr.*)
      *(.rela.ifunc)
    }
.rela.plt       :
    {
      *(.rela.plt)
      PROVIDE_HIDDEN (__rela_iplt_start = .);
      *(.rela.iplt)
      PROVIDE_HIDDEN (__rela_iplt_end = .);
    }
.init           :
{
    KEEP (*(SORT_NONE(.init)))
}
.plt            : { *(.plt) *(.iplt) }
.plt.got        : { *(.plt.got) }
.plt.sec        : { *(.plt.sec) }
.text           :
{
    *(.text.unlikely .text.*_unlikely .text.unlikely.*)
    *(.text.exit .text.exit.*)
    *(.text.startup .text.startup.*)
    *(.text.hot .text.hot.*)
    *(.text .stub .text.* .gnu.linkonce.t.*)
    /* .gnu.warning sections are handled specially by elf32.em. */
    *(.gnu.warning)
}
.fini           :
{
    KEEP (*(SORT_NONE(.fini)))
}
PROVIDE (__etext = .);
PROVIDE (_etext = .);
PROVIDE (etext = .);
.rodata         : { *(.rodata .rodata.* .gnu.linkonce.r.*) }
.rodata1        : { *(.rodata1) }
.eh_frame_hdr : { *(.eh_frame_hdr) *(.eh_frame_entry .eh_frame_entry.*) }
.eh_frame       : ONLY_IF_RO { KEEP (*(.eh_frame)) *(.eh_frame.*) }
.gcc_except_table   : ONLY_IF_RO { *(.gcc_except_table
.gcc_except_table.*) }
.gnu_extab   : ONLY_IF_RO { *(.gnu_extab*) }
/* These sections are generated by the Sun/Oracle C++ compiler. */
.exception_ranges   : ONLY_IF_RO { *(.exception_ranges
.exception_ranges*) }
/* Adjust the address for the data segment. We want to adjust up to
     the same address within the page on the next page up. */
. = DATA_SEGMENT_ALIGN (CONSTANT (MAXPAGESIZE), CONSTANT (COMMONPAGESIZE));
/* Exception handling */
.eh_frame       : ONLY_IF_RW { KEEP (*(.eh_frame)) *(.eh_frame.*) }
.gnu_extab      : ONLY_IF_RW { *(.gnu_extab) }
.gcc_except_table   : ONLY_IF_RW { *(.gcc_except_table .gcc_except_table.*) }
.exception_ranges   : ONLY_IF_RW { *(.exception_ranges .exception_ranges*) }
/* Thread Local Storage sections */
.tdata          : { *(.tdata .tdata.* .gnu.linkonce.td.*) }
.tbss           : { *(.tbss .tbss.* .gnu.linkonce.tb.*) *(.tcommon) }
.preinit_array     :
{
    PROVIDE_HIDDEN (__preinit_array_start = .);
    KEEP (*(.preinit_array))
    PROVIDE_HIDDEN (__preinit_array_end = .);
}
.init_array     :
{
    PROVIDE_HIDDEN (__init_array_start = .);
    KEEP (*(SORT_BY_INIT_PRIORITY(.init_array.*) SORT_BY_INIT_PRIORITY(.ctors.*)))
    KEEP (*(.init_array EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o ) .ctors))
    PROVIDE_HIDDEN (__init_array_end = .);
}
.fini_array     :
{
    PROVIDE_HIDDEN (__fini_array_start = .);
    KEEP (*(SORT_BY_INIT_PRIORITY(.fini_array.*) SORT_BY_INIT_PRIORITY(.dtors.*)))
    KEEP (*(.fini_array EXCLUDE_FILE (*crtbegin.o *crtbegin?.o *crtend.o *crtend?.o ) .dtors))
    PROVIDE_HIDDEN (__fini_array_end = .);
}
.ctors          :
{
    /* gcc uses crtbegin.o to find the start of
       the constructors, so we make sure it is
       first. Because this is a wildcard, it
       doesn't matter if the user does not
       actually link against crtbegin.o; the
       linker won't look for a file to match a
       wildcard. The wildcard also means that it
       doesn't matter which directory crtbegin.o
       is in. */
    KEEP (*crtbegin.o(.ctors))
    KEEP (*crtbegin?.o(.ctors))
    /* We don't want to include the .ctor section from
       the crtend.o file until after the sorted ctors.
       The .ctor section from the crtend file contains the
       end of ctors marker and it must be last */
    KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o ) .ctors))
    KEEP (*(SORT(.ctors.*)))
    KEEP (*(.ctors))
}
.dtors          :
{
    KEEP (*crtbegin.o(.dtors))
    KEEP (*crtbegin?.o(.dtors))
    KEEP (*(EXCLUDE_FILE (*crtend.o *crtend?.o ) .dtors))
    KEEP (*(SORT(.dtors.*)))
    KEEP (*(.dtors))
}
.jcr            : { KEEP (*(.jcr)) }
.data.rel.ro : { *(.data.rel.ro.local* .gnu.linkonce.d.rel.ro.local.*) *(.data.rel.ro .data.rel.ro.* .gnu.linkonce.d.rel.ro.*) }
.dynamic        : { *(.dynamic) }
.got            : { *(.got.plt) *(.igot.plt) *(.got) *(.igot) }
. = DATA_SEGMENT_RELRO_END (0, .);
.data           :
{
    *(.data .data.* .gnu.linkonce.d.*)
    SORT(CONSTRUCTORS)
}
.data1          : { *(.data1) }
_edata = .; PROVIDE (edata = .);
. = .;
__bss_start = .;
.bss            :
{
   *(.dynbss)
   *(.bss .bss.* .gnu.linkonce.b.*)
   *(COMMON)
   /* Align here to ensure that the .bss section occupies space up to
      _end. Align after .bss to ensure correct alignment even if the
      .bss section disappears because there are no input sections.
      FIXME: Why do we need it? When there is no .bss section, we don't
      pad the .data section. */
   . = ALIGN(. != 0 ? 64 / 8 : 1);
}
.lbss   :
{
    *(.dynlbss)
    *(.lbss .lbss.* .gnu.linkonce.lb.*)
    *(LARGE_COMMON)
}
. = ALIGN(64 / 8);
. = SEGMENT_START("ldata-segment", .);
.lrodata   ALIGN(CONSTANT (MAXPAGESIZE)) + (. & (CONSTANT (MAXPAGESIZE) - 1)) :
{
    *(.lrodata .lrodata.* .gnu.linkonce.lr.*)
}
.ldata   ALIGN(CONSTANT (MAXPAGESIZE)) + (. & (CONSTANT (MAXPAGESIZE) - 1)) :
{
    *(.ldata .ldata.* .gnu.linkonce.l.*)
    . = ALIGN(. != 0 ? 64 / 8 : 1);
}
. = ALIGN(64 / 8);
_end = .; PROVIDE (end = .);
. = DATA_SEGMENT_END (.);
/* Stabs debugging sections. */
.stab          0 : { *(.stab) }
.stabstr       0 : { *(.stabstr) }
.stab.excl     0 : { *(.stab.excl) }
.stab.exclstr 0 : { *(.stab.exclstr) }
.stab.index    0 : { *(.stab.index) }
.stab.indexstr 0 : { *(.stab.indexstr) }
.comment       0 : { *(.comment) }
/* DWARF debug sections.
     Symbols in the DWARF debugging sections are relative to the beginning
     of the section so we begin them at 0. */
/* DWARF 1 */
.debug          0 : { *(.debug) }
.line           0 : { *(.line) }
/* GNU DWARF 1 extensions */
.debug_srcinfo 0 : { *(.debug_srcinfo) }
.debug_sfnames 0 : { *(.debug_sfnames) }
/* DWARF 1.1 and DWARF 2 */
.debug_aranges 0 : { *(.debug_aranges) }
.debug_pubnames 0 : { *(.debug_pubnames) }
/* DWARF 2 */
.debug_info     0 : { *(.debug_info .gnu.linkonce.wi.*) }
.debug_abbrev   0 : { *(.debug_abbrev) }
.debug_line     0 : { *(.debug_line .debug_line.* .debug_line_end ) }
.debug_frame    0 : { *(.debug_frame) }
.debug_str      0 : { *(.debug_str) }
.debug_loc      0 : { *(.debug_loc) }
.debug_macinfo 0 : { *(.debug_macinfo) }
/* SGI/MIPS DWARF 2 extensions */
.debug_weaknames 0 : { *(.debug_weaknames) }
.debug_funcnames 0 : { *(.debug_funcnames) }
.debug_typenames 0 : { *(.debug_typenames) }
.debug_varnames 0 : { *(.debug_varnames) }
/* DWARF 3 */
.debug_pubtypes 0 : { *(.debug_pubtypes) }
.debug_ranges   0 : { *(.debug_ranges) }
/* DWARF Extension. */
.debug_macro    0 : { *(.debug_macro) }
.debug_addr     0 : { *(.debug_addr) }
.gnu.attributes 0 : { KEEP (*(.gnu.attributes)) }
/DISCARD/ : { *(.note.GNU-stack) *(.gnu_debuglink) *(.gnu.lto_*) }
}

==================================================
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o succeeded
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o succeeded
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crti.o
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o succeeded
/usr/lib/gcc/x86_64-linux-gnu/7/crtbeginS.o
attempt to open /tmp/ccipcMTK.o succeeded
/tmp/ccipcMTK.o
attempt to open .//libweek.so failed
attempt to open .//libweek.a succeeded
(.//libweek.a)week_symbol.o
attempt to open .//libstrong.so failed
attempt to open .//libstrong.a succeeded
attempt to open .//libgcc.so failed
attempt to open .//libgcc.a failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/libgcc.so failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/libgcc.a succeeded
attempt to open .//libgcc_s.so failed
attempt to open .//libgcc_s.a failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so succeeded
opened script file /usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so
opened script file /usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so
attempt to open libgcc_s.so.1 failed
attempt to open .//libgcc_s.so.1 failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1 succeeded
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
attempt to open .//libgcc.so failed
attempt to open .//libgcc.a failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/libgcc.so failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/libgcc.a succeeded
attempt to open .//libc.so failed
attempt to open .//libc.a failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/libc.so failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/libc.a failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libc.so succeeded
opened script file /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libc.so
opened script file /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/libc.so
attempt to open /lib/x86_64-linux-gnu/libc.so.6 succeeded
/lib/x86_64-linux-gnu/libc.so.6
attempt to open /usr/lib/x86_64-linux-gnu/libc_nonshared.a succeeded
(/usr/lib/x86_64-linux-gnu/libc_nonshared.a)elf-init.oS
attempt to open /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2 succeeded
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
attempt to open .//libgcc.so failed
attempt to open .//libgcc.a failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/libgcc.so failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/libgcc.a succeeded
attempt to open .//libgcc_s.so failed
attempt to open .//libgcc_s.a failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so succeeded
opened script file /usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so
opened script file /usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so
attempt to open libgcc_s.so.1 failed
attempt to open .//libgcc_s.so.1 failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1 succeeded
libgcc_s.so.1 (/usr/lib/gcc/x86_64-linux-gnu/7/libgcc_s.so.1)
attempt to open .//libgcc.so failed
attempt to open .//libgcc.a failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/libgcc.so failed
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/libgcc.a succeeded
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o succeeded
/usr/lib/gcc/x86_64-linux-gnu/7/crtendS.o
attempt to open /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o succeeded
/usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/crtn.o
ld-linux-x86-64.so.2 needed by /lib/x86_64-linux-gnu/libc.so.6
found ld-linux-x86-64.so.2 at /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2

我们知道静态其实就是目标文件.o的集合体，只有当可执行程序需要时，才会从静态库中提取出相关依赖的.o文件，进行编译。这也是为什么hello world程序依赖libc.a，但是最终可执行程序却比libc.a小的多的原因，因为可执行程序仅将其中的printf.o进行编译。

attempt to open .//libweek.so failed
attempt to open .//libweek.a succeeded
(.//libweek.a)week_symbol.o
attempt to open .//libstrong.so failed
attempt to open .//libstrong.a succeeded

从上述的红色字体中，我们可以知道编译过程: