Android Tombstone 分析

1.什么是tombstone

Tombstone是指在分布式系统中用于标记数据已被删除的记录，通常包含删除操作的时间戳和相关信息。

当一个动态库（native程序）开始执行时，系统会注册一些连接到 debuggerd 的signal handlers。当系统发生崩溃时，会保存一个tombstone文件到/data/tombstones目录下，并在Logcat中提供相应信息。这个tombstone文件类似于墓碑，记录了已终止进程的基本信息，包括进程号、线程号、崩溃地址等。此外，tombstone文件还会详细记录崩溃时的现场情况，包括一系列的堆栈调用信息，帮助开发人员分析问题并进行调试。

2.tombstone文件长什么样

在这里插入图片描述

一个tombstone文件大概包含以下信息

--------- beginning of crash
F/libc    (  244): invalid address or address of corrupt block 0xb82f54a0 passed to dlfree
I/libc    (  244): debuggerd_signal_handler called: signal=11, fn=0xb6fbdaa1
F/libc    (  244): Fatal signal 11 (SIGSEGV), code 1, fault addr 0xdeadbaad in tid 244 (mediaserver)
I/libc    (  244): exit from debuggerd_signal_handler
W/NativeCrashListener(  916): Couldn't find ProcessRecord for pid 244
I/DEBUG   (  241): *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
E/DEBUG   (  241): AM write failure (32 / Broken pipe)
I/DEBUG   (  241): Build fingerprint: XXXXXXXXX
I/DEBUG   (  241): Revision: '0'
I/DEBUG   (  241): ABI: 'arm'
I/DEBUG   (  241): pid: 244, tid: 244, name: mediaserver  >>> /system/bin/mediaserver <<<
I/DEBUG   (  241): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0xdeadbaad
I/art     ( 3078): now dumpable=1
I/DEBUG   (  241): Abort message: 'invalid address or address of corrupt block 0xb82f54a0 passed to dlfree'
I/DEBUG   (  241):     r0 00000000  r1 b6f20dec  r2 deadbaad  r3 00000000
I/DEBUG   (  241):     r4 b82f54a0  r5 b6f220f8  r6 00000000  r7 42424242
I/DEBUG   (  241):     r8 ffffffff  r9 b82f5460  sl 00000030  fp 00000000
I/DEBUG   (  241):     ip 00000000  sp beb2c020  lr b6ef1fa7  pc b6ef1fa8  cpsr 600e0030
I/DEBUG   (  241):     d0  0000000000000000  d1  6f2073736572646c
I/DEBUG   (  241):     d2  707572726f632066  d3  206b636f6c622072
I/DEBUG   (  241):     d4  4242424242424242  d5  4242424242424242
I/DEBUG   (  241):     d6  4242424242424242  d7  3ecccccd42424242
I/DEBUG   (  241):     d8  0000000000000000  d9  0000000000000000
I/DEBUG   (  241):     d10 0000000000000000  d11 0000000000000000
I/DEBUG   (  241):     d12 0000000000000000  d13 0000000000000000
I/DEBUG   (  241):     d14 0000000000000000  d15 0000000000000000
I/DEBUG   (  241):     d16 0000000000000000  d17 3ff0000000000000
I/DEBUG   (  241):     d18 7e37e43c8800759c  d19 bfd5f3f082400000
I/DEBUG   (  241):     d20 3e66376972bea4d0  d21 bf66b12699b6468f
I/DEBUG   (  241):     d22 3fc54aa75950670f  d23 bfd73498f0a5ef3a
I/DEBUG   (  241):     d24 3fe0000000000000  d25 bfaaf3ec933c988f
I/DEBUG   (  241):     d26 0000000000000000  d27 4000000000000000
I/DEBUG   (  241):     d28 4002e6931e14bde7  d29 3faaf3ec9198f99c
I/DEBUG   (  241):     d30 3ff0000000000000  d31 3fd29572efd86cee
I/DEBUG   (  241):     scr 20000010
I/DEBUG   (  241): 
I/DEBUG   (  241): backtrace:
I/DEBUG   (  241):     #00 pc 00028fa8  /system/lib/libc.so (dlfree+1239)
I/DEBUG   (  241):     #01 pc 0000f2cb  /system/lib/libc.so (free+10)
I/DEBUG   (  241):     #02 pc 0000a1cb  /system/lib/libstagefright_foundation.so (_ZN7android7ABufferD2Ev+42)
I/DEBUG   (  241):     #03 pc 0000a211  /system/lib/libstagefright_foundation.so (_ZN7android7ABufferD0Ev+4)
I/DEBUG   (  241):     #04 pc 0000d68d  /system/lib/libutils.so (_ZNK7android7RefBase9decStrongEPKv+40)
I/DEBUG   (  241):     #05 pc 0005adfd  /system/lib/libstagefright.so (_ZN7android2spINS_13GraphicBufferEED2Ev+10)
I/DEBUG   (  241):     #06 pc 0007cd0f  /system/lib/libstagefright.so (_ZN7android14MPEG4Extractor10parseChunkEPxi+634)
I/DEBUG   (  241):     #07 pc 0007d43d  /system/lib/libstagefright.so (_ZN7android14MPEG4Extractor10parseChunkEPxi+2472)
I/DEBUG   (  241):     #08 pc 0007e873  /system/lib/libstagefright.so (_ZN7android14MPEG4Extractor12readMetaDataEv+58)
I/DEBUG   (  241):     #09 pc 0007eaa1  /system/lib/libstagefright.so (_ZN7android14MPEG4Extractor11countTracksEv+4)
I/DEBUG   (  241):     #10 pc 000acf9d  /system/lib/libstagefright.so (_ZN7android13ExtendedUtils29MediaExtractor_CreateIfNeededENS_2spINS_14MediaExtractorEEERKNS1_INS_10DataSourceEEEPKc+60)
I/DEBUG   (  241):     #11 pc 0008e3f5  /system/lib/libstagefright.so (_ZN7android14MediaExtractor6CreateERKNS_2spINS_10DataSourceEEEPKc+624)
I/DEBUG   (  241):     #12 pc 0006ace9  /system/lib/libstagefright.so (_ZN7android13AwesomePlayer15setDataSource_lERKNS_2spINS_10DataSourceEEE+12)
I/DEBUG   (  241):     #13 pc 0006c0dd  /system/lib/libstagefright.so (_ZN7android13AwesomePlayer13setDataSourceEixx+228)
I/DEBUG   (  241):     #14 pc 0003d647  /system/lib/libmediaplayerservice.so (_ZN7android18MediaPlayerService6Client13setDataSourceEixx+362)
I/DEBUG   (  241):     #15 pc 0005ea03  /system/lib/libmedia.so (_ZN7android13BnMediaPlayer10onTransactEjRKNS_6ParcelEPS1_j+478)
I/DEBUG   (  241):     #16 pc 00017fad  /system/lib/libbinder.so (_ZN7android7BBinder8transactEjRKNS_6ParcelEPS1_j+60)
I/DEBUG   (  241):     #17 pc 0001cfdb  /system/lib/libbinder.so (_ZN7android14IPCThreadState14executeCommandEi+562)
I/DEBUG   (  241):     #18 pc 0001d12f  /system/lib/libbinder.so (_ZN7android14IPCThreadState20getAndExecuteCommandEv+38)
I/DEBUG   (  241):     #19 pc 0001d171  /system/lib/libbinder.so (_ZN7android14IPCThreadState14joinThreadPoolEb+48)
I/DEBUG   (  241):     #20 pc 00001721  /system/bin/mediaserver
I/DEBUG   (  241):     #21 pc 0000f411  /system/lib/libc.so (__libc_init+44)
I/DEBUG   (  241):     #22 pc 00001998  /system/bin/mediaserver
I/DEBUG   (  241): 
I/DEBUG   (  241): stack:
I/DEBUG   (  241):          beb2bfe0  00000000  
I/DEBUG   (  241):          beb2bfe4  29ec038f  
I/DEBUG   (  241):          beb2bfe8  0009eb34  
I/DEBUG   (  241):          beb2bfec  b82f54a0  [heap]
I/DEBUG   (  241):          beb2bff0  b6f220f8  
I/DEBUG   (  241):          beb2bff4  00000000  
I/DEBUG   (  241):          beb2bff8  42424242  
I/DEBUG   (  241):          beb2bffc  b6edb3d1  /system/lib/libc.so (__libc_fatal_no_abort+16)
I/DEBUG   (  241):          beb2c000  b6f12f97  /system/lib/libc.so
I/DEBUG   (  241):          beb2c004  beb2c014  [stack]
I/DEBUG   (  241):          beb2c008  b6f167be  /system/lib/libc.so
I/DEBUG   (  241):          beb2c00c  b6ef1fa7  /system/lib/libc.so (dlfree+1238)
I/DEBUG   (  241):          beb2c010  b6f12f97  /system/lib/libc.so
I/DEBUG   (  241):          beb2c014  b82f54a0  [heap]
I/DEBUG   (  241):          beb2c018  b6f167be  /system/lib/libc.so
I/DEBUG   (  241):          beb2c01c  b82f54b0  [heap]
I/DEBUG   (  241):     #00  beb2c020  b82f5460  [heap]
......

它包含了发生问题的进程ID信息

I/DEBUG   (  241): pid: 244, tid: 244, name: mediaserver  >>> /system/bin/mediaserver <<<

当 tid （线程ID）== pid （进程ID）时，问题发生在父进程，反之问题发生在子进程，从上面的日志信息可以看出发生问题的进程是mediaserver的子进程。

Terminated signal 和 fault address 信息

F/libc    (  244): Fatal signal 11 (SIGSEGV), code 1, fault addr 0xdeadbaad in tid 244 (mediaserver)

这里的信息说明出现进程 Crash 的原因是因为程序产生了段错误的信号，访问了非法的内存空间，而访问的非法地址是 0xdeadbaad。

信号机制是 Linux 进程间通信的一种重要方式，Linux 信号一方面用于正常的进程间通信和同步，如任务控制(SIGINT, SIGTSTP,SIGKILL, SIGCONT，……)；另一方面，它还负责监控系统异常及中断。当应用程序运行异常时， Linux 内核将产生错误信号并通知当前进程。当前进程在接收到该错误信号后，可以有三种不同的处理方式。

（1）忽略该信号。
（2）捕捉该信号并执行对应的信号处理函数(signal handler)。
（3）执行该信号的缺省操作(如 SIGSEGV，其缺省操作是终止进程)。

当 Linux 应用程序在执行时发生严重错误，一般会导致程序 crash。其中，Linux 专门提供了一类 crash 信号，在程序接收到此类信号时，缺省操作是将 crash 的现场信息记录到 core 文件，然后终止进程。

crash 信号列表：

Signal	Description
SIGSEGV	Invalid memory reference.
SIGBUS	Access to an undefined portion of a memory object.
SIGFPE	Arithmetic operation error, like divide by zero.
SIGILL	Illegal instruction, like execute garbage or a privileged instruction
SIGSYS	Bad system call.
SIGXCPU	CPU time limit exceeded.
SIGXFSZ	File size limit exceeded.

定义在prebuilts/gcc/linux-x86/host/x86_64-linux-glibc2.11-4.8/sysroot/usr/include/bits/signum.h

/* Signals.  */
#define SIGHUP          1       /* Hangup (POSIX).  */
#define SIGINT          2       /* Interrupt (ANSI).  */
#define SIGQUIT         3       /* Quit (POSIX).  */
#define SIGILL          4       /* Illegal instruction (ANSI).  */
#define SIGTRAP         5       /* Trace trap (POSIX).  */
#define SIGABRT         6       /* Abort (ANSI).  */
#define SIGIOT          6       /* IOT trap (4.2 BSD).  */
#define SIGBUS          7       /* BUS error (4.2 BSD).  */
#define SIGFPE          8       /* Floating-point exception (ANSI).  */
#define SIGKILL         9       /* Kill, unblockable (POSIX).  */
#define SIGUSR1         10      /* User-defined signal 1 (POSIX).  */
#define SIGSEGV         11      /* Segmentation violation (ANSI).  */
#define SIGUSR2         12      /* User-defined signal 2 (POSIX).  */
#define SIGPIPE         13      /* Broken pipe (POSIX).  */
#define SIGALRM         14      /* Alarm clock (POSIX).  */
#define SIGTERM         15      /* Termination (ANSI).  */
#define SIGSTKFLT       16      /* Stack fault.  */
#define SIGCLD          SIGCHLD /* Same as SIGCHLD (System V).  */
#define SIGCHLD         17      /* Child status has changed (POSIX).  */
#define SIGCONT         18      /* Continue (POSIX).  */
#define SIGSTOP         19      /* Stop, unblockable (POSIX).  */
#define SIGTSTP         20      /* Keyboard stop (POSIX).  */
#define SIGTTIN         21      /* Background read from tty (POSIX).  */
#define SIGTTOU         22      /* Background write to tty (POSIX).  */
#define SIGURG          23      /* Urgent condition on socket (4.2 BSD).  */
#define SIGXCPU         24      /* CPU limit exceeded (4.2 BSD).  */
#define SIGXFSZ         25      /* File size limit exceeded (4.2 BSD).  */
#define SIGVTALRM       26      /* Virtual alarm clock (4.2 BSD).  */
#define SIGPROF         27      /* Profiling alarm clock (4.2 BSD).  */
#define SIGWINCH        28      /* Window size change (4.3 BSD, Sun).  */
#define SIGPOLL         SIGIO   /* Pollable event occurred (System V).  */
#define SIGIO           29      /* I/O now possible (4.2 BSD).  */
#define SIGPWR          30      /* Power failure restart (System V).  */
#define SIGSYS          31      /* Bad system call.  */
#define SIGUNUSED       31

#define _NSIG           65      /* Biggest signal number + 1
                                   (including real-time signals).  */

3.怎么分析tombstone文件

我们主要关注 backtrace 下面的内容，它保存了发生 crash 时候的函数调用关系，但是需要注意的是它的调用顺序是从下向上执行的（#XX pc -->#00 pc）,通过这些函数调用关系，我们就可以大概定位出问题发生的地方，在本次 tombstone 日志中，我们通过

I/DEBUG   (  241):     #00 pc 00028fa8  /system/lib/libc.so (dlfree+1239)
I/DEBUG   (  241):     #01 pc 0000f2cb  /system/lib/libc.so (free+10)
I/DEBUG   (  241):     #02 pc 0000a1cb  /system/lib/libstagefright_foundation.so (_ZN7android7ABufferD2Ev+42)
I/DEBUG   (  241):     #03 pc 0000a211  /system/lib/libstagefright_foundation.so (_ZN7android7ABufferD0Ev+4)
I/DEBUG   (  241):     #04 pc 0000d68d  /system/lib/libutils.so (_ZNK7android7RefBase9decStrongEPKv+40)

可以分析出问题是在调用free函数时发生了指针错误，还可以看出问题发生的原因是libstagefright_foundation.so中释放了两次ABuffer引用,接着就去分析是谁谁释放的AUbffer强指针。

I/DEBUG   (  241):     #05 pc 0005adfd  /system/lib/libstagefright.so (_ZN7android2spINS_13GraphicBufferEED2Ev+10)
I/DEBUG   (  241):     #06 pc 0007cd0f  /system/lib/libstagefright.so (_ZN7android14MPEG4Extractor10parseChunkEPxi+634)
I/DEBUG   (  241):     #07 pc 0007d43d  /system/lib/libstagefright.so (_ZN7android14MPEG4Extractor10parseChunkEPxi+2472)
I/DEBUG   (  241):     #08 pc 0007e873  /system/lib/libstagefright.so (_ZN7android14MPEG4Extractor12readMetaDataEv+58)

可以看出来在 libstagefright 动态库中的MPEG4Extractor.cpp 的 parseChunk函数出现的错误。

4.一些分析工具

虽然通过 tombstone 的日志文件我们就可以大致定位出引发 crash 的代码的位置，但是通过借助一些分析工具，可以大大的提高工作效率和准确性，下面就来介绍以下这些工具。

（1）addr2line

addr2line 是用来获得指定动态链接库文件或者可执行文件中指定地址对应的源代码信息的工具

它的各种参数如下所示（这个是google aosp android M 中带的）：

~/source/google_android/prebuilts/gcc/linux-x86/x86/x86_64-linux-android-4.9/bin$ ./x86_64-linux-android-addr2line -h
Usage: ./x86_64-linux-android-addr2line [option(s)] [addr(s)]
 Convert addresses into line number/file name pairs.
 If no addresses are specified on the command line, they will be read from stdin
 The options are:
  @<file>                Read options from <file>
  -a --addresses         Show addresses
  -b --target=<bfdname>  Set the binary file format
  -e --exe=<executable>  Set the input file name (default is a.out)
  -i --inlines           Unwind inlined functions
  -j --section=<name>    Read section-relative offsets instead of addresses
  -p --pretty-print      Make the output easier to read for humans
  -s --basenames         Strip directory names
  -f --functions         Show function names
  -C --demangle[=style]  Demangle function names
  -h --help              Display this information
  -v --version           Display the program's version
./x86_64-linux-android-addr2line: supported targets: elf64-x86-64 elf32-i386 elf32-x86-64 a.out-i386-linux pei-i386 pei-x86-64 elf64-l1om elf64-k1om elf64-little elf64-big elf32-little elf32-big plugin srec symbolsrec verilog tekhex binary ihex
Report bugs to <http://source.android.com/source/report-bugs.html>

addr2line 的基本用法如下所示：

./prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.8/bin/arm-linux-androideabi-addr2line -f -e out/debug/target/product/XXXX/symbols/system/lib/libstagefright.so  0007cd0f 
_ZN7android14MPEG4Extractor10parseChunkEPxi
/home/XXX/source/XXX/LINUX/android/frameworks/av/media/libstagefright/MPEG4Extractor.cpp:2180 (discriminator 1)

这里需要注意的是不能直接使用out/debug/target/product/XXX/system/lib/libstagefright.so，会出现运行上面命令之后显示

??
??:0

因为这个动态库是最后要打包到最后生成的system.ing中的，所以它不包含调试符号信息。

（2）ndk-stack

Android NDK 自从版本 r6开始, 提供了一个工具 ndk-stack。这个工具能自动分析 tombstone 文件, 能将崩溃时的调用内存地址和 c++ 代码一行一行对应起来.

它的使用方法为

./ndk-stack 
Usage:
   ndk-stack -sym <path> [-dump <path>]
      -sym  Contains full path to the root directory for symbols.
      -dump Contains full path to the file containing the crash dump.
            This is an optional parameter. If ommited, ndk-stack will
            read input data from stdin
   See docs/NDK-STACK.html in your NDK installation tree for more details.

dump 参数很容易理解, 即 dump 下来的 log 文本文件. ndk-stack会分析此文件。
sym 参数就是你android项目下，编译成功之后，obj目录下的文件（android系统源码o 中带有符号信息的文件）。

我们可以使用它来分析我们的log文件

ndk-stack -sym xxx.so -dump logfile

所以我们在调试android系统源码的时候也可以直接分析log中的crash信息。

adb shell logcat | ndk-stack -sym out/debug/target/product/XXXX/symbols/system/lib/xxx.so

（3）stack.py

stack.py工具就是要把backtrace通过addr2line工具一次性把addr对应到代码

#!/usr/bin/python2.4 -E
import getopt
import os
import re
import string
import sys
import getpass
import urllib
import subprocess
def PrintUsage():
  print
  print "  usage: " + sys.argv[0] + " [options] [FILE]"
  print
  print "  --symbols-dir=path"
  print "       the path to a symbols dir, such as =/tmp/out/target/product/dream/symbols"
  print
  print "  --symbols-zip=path"
  print "       the path to a symbols zip file, such as =dream-symbols-12345.zip"
  print
  print "  --auto"
  print "       attempt to:"
  print "         1) automatically find the build number in the crash"
  print "         2) if it's an official build, download the symbols "
  print "            from the build server, and use them"
  print
  print "  FILE should contain a stack trace in it somewhere"
  print "       the tool will find that and re-print it with"
  print "       source files and line numbers.  If you don't"
  print "       pass FILE, or if file is -, it reads from"
  print "       stdin."
  print
  sys.exit(1)
def FindSymbolsDir():
  cmd = "CALLED_FROM_SETUP=true make -f build/core/envsetup.mk " \
      + "dumpvar-abs-TARGET_OUT_UNSTRIPPED"
  stream = os.popen(cmd)
  str = stream.read()
  stream.close()
  return str.strip()
# returns a list containing the function name and the file/lineno
def CallAddr2Line(lib, addr):
  uname = os.uname()[0]
  if uname == "Darwin":
    proc = os.uname()[-1]
    if proc == "i386":
      uname = "darwin-x86"
    else:
      uname = "darwin-ppc"
  if lib != "":
    #cmd = "./prebuilt/" + uname + "/toolchain-eabi-4.2.1/bin/arm-eabi-addr2line" \
    #cmd = "./prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.7/bin/arm-linux-androideabi-addr2line" \
    cmd = " arm-eabi-addr2line" \
        + " -f -e " + SYMBOLS_DIR + lib \
        + " 0x" + addr
    stream = os.popen(cmd)
    lines = stream.readlines()
    list = map(string.strip, lines)
  else:
    list = []
  if list != []:
    # Name like "move_forward_type<JavaVMOption>" causes troubles
    mangled_name = re.sub('<', '\<', list[0]);
    mangled_name = re.sub('>', '\>', mangled_name);
    #cmd = "./prebuilt/" + uname + "/toolchain-eabi-4.2.1/bin/arm-eabi-c++filt "\
    cmd = "./prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.7/bin/arm-linux-androideabi-c++filt "\
          + mangled_name
    stream = os.popen(cmd)
    list[0] = stream.readline()
    stream.close()
    list = map(string.strip, list)
  else:
    list = [ "(unknown)", "(unknown)" ]
  return list
class SSOCookie(object):
  """
  creates a cookie file so we can download files from the build server
  """
  def __init__(self, cookiename=".sso.cookie", keep=False):
    self.sso_server = "login.corp.google.com"
    self.name = cookiename
    self.keeper = keep
    self.tmp_opts = ".curl.options"
    if not os.path.exists(self.name):
      user = os.environ['USER']
      print "\n%s, to access the symbols, please enter your LDAP " % user,
      password = getpass.getpass()
      params = urllib.urlencode({"u": user, "pw": password})
      fd = os.open(self.tmp_opts, os.O_RDWR | os.O_CREAT, 0600)
      os.write(fd, '-b "%s"\n' % self.name)
      os.write(fd, '-c "%s"\n' % self.name)
      os.write(fd, '-s"\n-L\n-d "%s"\n' % params)
      os.write(fd, 'url = "https://%s/login?ssoformat=CORP_SSO"\n' % 
               self.sso_server)
      # login to SSO
      response = os.popen("/usr/bin/curl -K %s" % self.tmp_opts)
      response.close()
      if os.path.exists(self.tmp_opts):
        os.remove(self.tmp_opts)
      if os.path.exists(self.name):
        os.chmod(self.name, 0600)
      else:
        print "Could not log in to SSO"
        sys.exit(1)
  def __del__(self):
      """clean up"""
      if not self.keeper:
        os.remove(self.name)
class NoBuildIDException(Exception):
  pass
def FindBuildFingerprint(lines):
  """
  Searches the given file (array of lines) for the build fingerprint information
  """
  fingerprint_regex = re.compile("^.*Build fingerprint:\s'(?P<fingerprint>.*)'")
  for line in lines:
    fingerprint_search = fingerprint_regex.match(line.strip())
    if fingerprint_search:
      return fingerprint_search.group('fingerprint')
      
  return None  # didn't find the fingerprint string, so return none
  
class SymbolDownloadException(Exception):
  pass
DEFAULT_SYMROOT = "/tmp/symbols"
def DownloadSymbols(fingerprint, cookie):
  """
  Attempts to download the symbols from the build server, extracts them,
  and returns the path.  Takes the fingerprint from the pasted stack trace
  and the SSOCookie
  """
  if fingerprint is None:
    return (None, None)
  symdir = "%s/%s" % (DEFAULT_SYMROOT, hash(fingerprint))
  if not os.path.exists(symdir):
    os.makedirs(symdir)
  # build server figures out the branch based on the CL
  params = {
              'op': "GET-SYMBOLS-LINK",
              'fingerprint': fingerprint,
           }
  url = urllib.urlopen("http://android-build/buildbot-update?",
                            urllib.urlencode(params)).readlines()[0]
  if url == "":
    raise SymbolDownloadException, "Build server down? Failed to find syms..."
  regex_str = (r'(?P<baseURL>http\:\/\/android-build\/builds\/.*\/[0-9]+' + 
           r'\/)(?P<img>.*)')
  url_regex = re.compile(regex_str)
  url_match = url_regex.match(url)
  if url_match is None:
    raise SymbolDownloadException, "Unexpected results from build server URL..."
  
  baseURL = url_match.group('baseURL')
  img =  url_match.group('img')
  symbolfile = img.replace("-img-", "-symbols-")
  symurl = baseURL + symbolfile
  localsyms = symdir + symbolfile
  if not os.path.exists(localsyms):
    print "downloading %s ..." % symurl
    curlcmd = ("""/usr/bin/curl -b %s -sL -w %%{http_code} -o %s %s""" % 
                          (cookie.name, localsyms, symurl))
    (fi,fo,fe) = os.popen3(curlcmd)
    fi.close()
    code = fo.read()
    err = fe.read()
    if err != "":
      raise SymbolDownloadException, "stderr from curl download: %s" % err
    if code != "200":
      raise SymbolDownloadException, "Faied to download %s" % symurl
  else:
    print "using existing cache for symbols"
  print "extracting %s..." % symbolfile
  saveddir = os.getcwd()
  os.chdir(symdir)
  unzipcode = subprocess.call(["unzip", "-qq", "-o", localsyms])
  if unzipcode > 0:
    raise SymbolDownloadException, ("failed to extract symbol files (%s)." 
                             % localsyms)
  os.chdir(saveddir)
  
  return (symdir, "%s/out/target/product/dream/symbols" % symdir)
def UnzipSymbols(symbolfile):
  """Unzips a file to DEFAULT_SYMROOT and returns the unzipped location.
  Args:
    symbolfile: The .zip file to unzip
  Returns:
    A tuple containing (the directory into which the zip file was unzipped,
    the path to the "symbols" directory in the unzipped file).  To clean
    up, the caller can delete the first element of the tuple.
  Raises:
    SymbolDownloadException: When the unzip fails.
  """
  symdir = "%s/%s" % (DEFAULT_SYMROOT, hash(symbolfile))
  if not os.path.exists(symdir):
    os.makedirs(symdir)
  print "extracting %s..." % symbolfile
  saveddir = os.getcwd()
  os.chdir(symdir)
  unzipcode = subprocess.call(["unzip", "-qq", "-o", symbolfile])
  if unzipcode > 0:
    raise SymbolDownloadException, ("failed to extract symbol files (%s)." 
                             % symbolfile)
  os.chdir(saveddir)
  
  return (symdir, "%s/out/target/product/dream/symbols" % symdir)
def PrintTraceLines(traceLines):
    maxlen = max(map(lambda tl: len(tl[1]), traceLines))
    print
    print "Stack Trace:"
    print "  ADDR      " + "FUNCTION".ljust(maxlen) + "  FILE:LINE"
    for tl in traceLines:
      print "  " + tl[0] + "  " + tl[1].ljust(maxlen) + "  " + tl[2]
    return
def PrintValueLines(valueLines):
    print
    print "Stack Data:"
    print "  ADDR      VALUE     " + "FILE:LINE/FUNCTION"
    for vl in valueLines:
      print "  " + vl[1] + "  " + vl[2] + "  " + vl[4]
      if vl[4] != "":
        print "                      " + vl[3]
    return
def ConvertTrace(lines):
  PROCESS_INFO_LINE = re.compile("(pid: [0-9]+, tid: [0-9]+.*)")
  SIGNAL_LINE = re.compile("(signal [0-9]+ \(.*\).*)")
  REGISTER_LINE = re.compile("(([ ]*[0-9a-z]{2} [0-9a-f]{8}){4})")
  TRACE_LINE = re.compile("(.*)\#([0-9]+)  (..) ([0-9a-f]{3})([0-9a-f]{5})  ([^\r\n \t]*)")
  VALUE_LINE = re.compile("(.*)([0-9a-f]{2})([0-9a-f]{6})  ([0-9a-f]{3})([0-9a-f]{5})  ([^\r\n \t]*)")
  THREAD_LINE = re.compile("(.*)(\-\-\- ){15}\-\-\-")
  traceLines = []
  valueLines = []
  for line in lines:
    header = PROCESS_INFO_LINE.search(line)
    if header:
      print header.group(1)
      continue
    header = SIGNAL_LINE.search(line)
    if header:
      print header.group(1)
      continue
    header = REGISTER_LINE.search(line)
    if header:
      print header.group(1)
      continue
    if TRACE_LINE.match(line):
      match = TRACE_LINE.match(line)
      groups = match.groups()
      if groups[5] == "<unknown>" or groups[5] == "[heap]" or groups[5] == "[stack]":
        traceLines.append((groups[3]+groups[4], groups[5], groups[5]))
      else:
        info = CallAddr2Line(groups[5], groups[4])
        traceLines.append((groups[3]+groups[4], info[0], info[1]))
    if VALUE_LINE.match(line):
      match = VALUE_LINE.match(line)
      groups = match.groups()
      if groups[5] == "<unknown>" or groups[5] == "[heap]" or groups[5] == "[stack]" or groups[5] == "":
        valueLines.append((groups[0], groups[1]+groups[2], groups[3]+groups[4], groups[5], ""))
      else:
        info = CallAddr2Line(groups[5], groups[4])
        valueLines.append((groups[0], groups[1]+groups[2], groups[3]+groups[4], info[0], info[1]))
    header = THREAD_LINE.search(line)
    if header:
      if len(traceLines) > 0:
        PrintTraceLines(traceLines)
      if len(valueLines) > 0:
        PrintValueLines(valueLines)
      traceLines = []
      valueLines = []
      print
      print "-----------------------------------------------------\n"
  if len(traceLines) > 0:
    PrintTraceLines(traceLines)
  if len(valueLines) > 0:
    PrintValueLines(valueLines)
SYMBOLS_DIR = FindSymbolsDir()
if __name__ == '__main__':
  try:
    options, arguments = getopt.getopt(sys.argv[1:], "",
                             ["auto", "symbols-dir=", "symbols-zip=", "help"])
  except getopt.GetoptError, error:
    PrintUsage()
  
  AUTO = False
  zipArg = None
  for option, value in options:
    if option == "--help":
      PrintUsage()
    elif option == "--symbols-dir":
      SYMBOLS_DIR = value
    elif option == "--symbols-zip":
      zipArg = value
    elif option == "--auto":
      AUTO = True
  
  if len(arguments) > 1:
    PrintUsage()
  if AUTO:
    cookie = SSOCookie(".symbols.cookie")
  
  if len(arguments) == 0 or arguments[0] == "-":
    print "Reading native crash info from stdin"
    f = sys.stdin
  else:
    print "Searching for native crashes in %s" % arguments[0]
    f = open(arguments[0], "r")
  lines = f.readlines()
  rootdir = None
  if AUTO:
    fingerprint = FindBuildFingerprint(lines)
    print "fingerprint:", fingerprint
    rootdir, SYMBOLS_DIR = DownloadSymbols(fingerprint, cookie)
  elif zipArg is not None:
    rootdir, SYMBOLS_DIR = UnzipSymbols(zipArg)
  
  print "Reading symbols from", SYMBOLS_DIR
  lines = ConvertTrace(lines)
  
  if rootdir is not None:
    # be a good citizen and clean up...os.rmdir and os.removedirs() don't work
    cmd = "rm -rf \"%s\"" % rootdir
    print "\ncleaning up (%s)" % cmd
    os.system(cmd)
  
  # vi: ts=2 sw=2

使用方法：

python stack.py --symbols-dir=out/target/profuct/XXX/sysbols/  tombstone-00(tombstone文件)