什么是tombstone

tombstone中文称墓碑,顾名思义,墓碑会记录死者的一些生平信息。同样Android中也有个tombstone,它会记录一个进程死亡时的死亡现场,方便开发人员调试bug.那这个tombstone是谁创建的呢,这点要涉及到Android系统监控进程异常退出的机制。当一个native程序开始执行时,系统会注册一些连接到debuggerd的signal handlers,debuggerd是Android中的一个守护进程。当程序出现异常状态时,Linux kernel会发送相应的signal给异常进程,debuggerd捕获这些signal,在/data/tombstones目录下生成一个tombstone。Tombstone文件以tombstone_XX形式命名,该文件个数上限可以进行设置,当超过上限时则每次覆盖时间最老的文件。Tombstone文件记录了崩溃的进程的基本信息,堆栈调用信息,内存信息等等。

分析tombstone

下面就以一个tombstone文件为例,解释一下其内容的含义。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
Revision: '0'
ABI: 'arm'
Timestamp: 2021-10-27 07:41:26+0000
pid: 6488, tid: 6488, name: com.android.tv  >>> com.android.tv <<<
uid: 1000
signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
Abort message: 'Attempted to retrieve value from failed HIDL call: Status(EX_TRANSACTION_FAILED): 'DEAD_OBJECT: ''
    r0  00000000  r1  00001958  r2  00000006  r3  ff800510
    r4  ff800524  r5  ff800508  r6  00001958  r7  0000016b
    r8  ff800510  r9  ff800520  r10 ff800540  r11 ff800530
    ip  00001958  sp  ff8004e0  lr  f1fcf999  pc  f1fcf9ac

backtrace:
      #00 pc 000389ac  /apex/com.android.runtime/lib/bionic/libc.so (abort+172) (BuildId: 9c1376f68909498526ac951ceeeb272a)
      #01 pc 003fd73d  /apex/com.android.art/lib/libart.so (art::Runtime::Abort(char const*)+1812) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #02 pc 0000da15  /system/lib/libbase.so (android::base::SetAborter(std::__1::function<void (char const*)>&&)::$_3::__invoke(char const*)+48) (BuildId: 3537a03427bc64f05ae9032437c9fec6)
      #03 pc 0000d337  /system/lib/libbase.so (android::base::LogMessage::~LogMessage()+226) (BuildId: 3537a03427bc64f05ae9032437c9fec6)
      #04 pc 0003b42d  /system/lib/libhidlbase.so (android::hardware::details::return_status::onValueRetrieval() const+140) (BuildId: b890d44f0d4135b1e1e93cb6054cc954)
      #05 pc 0000ae05  /system/lib/libtvcompany_jni.so (JUsbhdd::UsbHddHotplgReq(char const*, int)+76) (BuildId: 82ab44171ed423e256d010e246dcadf9)
      #06 pc 0000da8d  /system/lib/libtvcompany_jni.so (com_socionext_tv_company_TVcompany_UsbHddHotplgReq(_JNIEnv*, _jobject*, _jstring*, int)+96) (BuildId: 82ab44171ed423e256d010e246dcadf9)
      #07 pc 000403a1  /system/product/priv-app/LiveTv/oat/arm/LiveTv.odex (art_jni_trampoline+96) (BuildId: a669f868ef55ec117c02189583c295081a9ed231)
      #08 pc 000d33d5  /apex/com.android.art/lib/libart.so (art_quick_invoke_stub_internal+68) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #09 pc 004eb941  /apex/com.android.art/lib/libart.so (art_quick_invoke_stub+280) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #10 pc 0012c6e7  /apex/com.android.art/lib/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+142) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #11 pc 0023ef43  /apex/com.android.art/lib/libart.so (art::interpreter::ArtInterpreterToCompiledCodeBridge(art::Thread*, art::ArtMethod*, art::ShadowFrame*, unsigned short, art::JValue*)+254) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #12 pc 00237375  /apex/com.android.art/lib/libart.so (bool art::interpreter::DoCall<false, false>(art::ArtMethod*, art::Thread*, art::ShadowFrame&, art::Instruction const*, unsigned short, art::JValue*)+756) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #13 pc 004df1cd  /apex/com.android.art/lib/libart.so (MterpInvokeDirect+500) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #14 pc 000cdf14  /apex/com.android.art/lib/libart.so (mterp_op_invoke_direct+20) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #15 pc 003641c8  /system/product/priv-app/LiveTv/LiveTv.apk (offset 0xe25000) (com.socionext.tv.company.TVcompany.UsbHddHotplgReq)
      #16 pc 004dd2f5  /apex/com.android.art/lib/libart.so (MterpInvokeVirtual+1368) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #17 pc 000cde14  /apex/com.android.art/lib/libart.so (mterp_op_invoke_virtual+20) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #18 pc 00185a2a  /system/product/priv-app/LiveTv/LiveTv.apk (offset 0xe25000) (com.android.tv.usb.service.UsbHddService.doUsbInfo+462)
      #19 pc 004df415  /apex/com.android.art/lib/libart.so (MterpInvokeDirect+1084) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #20 pc 000cdf14  /apex/com.android.art/lib/libart.so (mterp_op_invoke_direct+20) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #21 pc 0018548c  /system/product/priv-app/LiveTv/LiveTv.apk (offset 0xe25000) (com.android.tv.usb.service.UsbHddService.access$200)
      #22 pc 004dfc19  /apex/com.android.art/lib/libart.so (MterpInvokeStatic+1028) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #23 pc 000cdf94  /apex/com.android.art/lib/libart.so (mterp_op_invoke_static+20) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #24 pc 0018535a  /system/product/priv-app/LiveTv/LiveTv.apk (offset 0xe25000) (com.android.tv.usb.service.UsbHddService$2.handleMessage+70)
      #25 pc 002304f5  /apex/com.android.art/lib/libart.so (art::interpreter::Execute(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame&, art::JValue, bool, bool) (.llvm.16447052521471672853)+244) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #26 pc 00236ad7  /apex/com.android.art/lib/libart.so (art::interpreter::EnterInterpreterFromEntryPoint(art::Thread*, art::CodeItemDataAccessor const&, art::ShadowFrame*)+114) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #27 pc 004cea57  /apex/com.android.art/lib/libart.so (artQuickToInterpreterBridge+690) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #28 pc 000d7f61  /apex/com.android.art/lib/libart.so (art_quick_to_interpreter_bridge+32) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #29 pc 005685d1  /system/framework/arm/boot-framework.oat (android.os.Handler.dispatchMessage+136) (BuildId: a7a3361a75d453f82fdf557c0f66b997ccd37e25)
      #30 pc 0056b067  /system/framework/arm/boot-framework.oat (android.os.Looper.loop+1542) (BuildId: a7a3361a75d453f82fdf557c0f66b997ccd37e25)
      #31 pc 003a4d29  /system/framework/arm/boot-framework.oat (android.app.ActivityThread.main+752) (BuildId: a7a3361a75d453f82fdf557c0f66b997ccd37e25)
      #32 pc 000d33d5  /apex/com.android.art/lib/libart.so (art_quick_invoke_stub_internal+68) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #33 pc 004eba71  /apex/com.android.art/lib/libart.so (art_quick_invoke_static_stub+276) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #34 pc 0012c6f9  /apex/com.android.art/lib/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+160) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #35 pc 003f7a91  /apex/com.android.art/lib/libart.so (art::InvokeMethod(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jobject*, _jobject*, unsigned int)+832) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #36 pc 0039521f  /apex/com.android.art/lib/libart.so (art::Method_invoke(_JNIEnv*, _jobject*, _jobject*, _jobjectArray*)+30) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #37 pc 00083637  /apex/com.android.art/javalib/arm/boot.oat (art_jni_trampoline+110) (BuildId: 92246b26fb6c8434372c71f0b6b8216531e46b98)
      #38 pc 0074cf11  /system/framework/arm/boot-framework.oat (com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run+112) (BuildId: a7a3361a75d453f82fdf557c0f66b997ccd37e25)
      #39 pc 007542df  /system/framework/arm/boot-framework.oat (com.android.internal.os.ZygoteInit.main+1934) (BuildId: a7a3361a75d453f82fdf557c0f66b997ccd37e25)
      #40 pc 000d33d5  /apex/com.android.art/lib/libart.so (art_quick_invoke_stub_internal+68) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #41 pc 004eba71  /apex/com.android.art/lib/libart.so (art_quick_invoke_static_stub+276) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #42 pc 0012c6f9  /apex/com.android.art/lib/libart.so (art::ArtMethod::Invoke(art::Thread*, unsigned int*, unsigned int, art::JValue*, char const*)+160) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #43 pc 003f6c21  /apex/com.android.art/lib/libart.so (art::JValue art::InvokeWithVarArgs<art::ArtMethod*>(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, art::ArtMethod*, std::__va_list)+348) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #44 pc 003f6ed7  /apex/com.android.art/lib/libart.so (art::JValue art::InvokeWithVarArgs<_jmethodID*>(art::ScopedObjectAccessAlreadyRunnable const&, _jobject*, _jmethodID*, std::__va_list)+42) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #45 pc 0031dd53  /apex/com.android.art/lib/libart.so (art::JNI<true>::CallStaticVoidMethodV(_JNIEnv*, _jclass*, _jmethodID*, std::__va_list)+478) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #46 pc 0006684d  /system/lib/libandroid_runtime.so (_JNIEnv::CallStaticVoidMethod(_jclass*, _jmethodID*, ...)+28) (BuildId: c4ea39e62c1ad20a4339e21bc9bf2e1f)
      #47 pc 0006c5bd  /system/lib/libandroid_runtime.so (android::AndroidRuntime::start(char const*, android::Vector<android::String8> const&, bool)+680) (BuildId: c4ea39e62c1ad20a4339e21bc9bf2e1f)
      #48 pc 00002f0d  /system/bin/app_process32 (main+1060) (BuildId: edfc2186549531426409eded57f047ef)
      #49 pc 000331df  /apex/com.android.runtime/lib/bionic/libc.so (__libc_init+66) (BuildId: 9c1376f68909498526ac951ceeeb272a)
1
Timestamp: 2021-10-27 07:41:26+0000

发生native crash的时间,有了这个时间可以对照logcat的日志,找到准确的crash位置。

1
pid: 6488, tid: 6488, name: com.android.tv  >>> com.android.tv <<<

发生crash的进程ID,线程ID,因为进程 ID 和线程 ID 一致。第一个名称是线程名称,»> 和 «< 中间的名称是进程名称.对于应用,进程名称通常是完全限定的软件包名称(如 com.android.tv)这个是应用LiveTv的包名。

1
2
3
4
5
6
signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
Abort message: 'Attempted to retrieve value from failed HIDL call: Status(EX_TRANSACTION_FAILED): 'DEAD_OBJECT: ''
    r0  00000000  r1  00001958  r2  00000006  r3  ff800510
    r4  ff800524  r5  ff800508  r6  00001958  r7  0000016b
    r8  ff800510  r9  ff800520  r10 ff800540  r11 ff800530
    ip  00001958  sp  ff8004e0  lr  f1fcf999  pc  f1fcf9ac

这种错误一般是程序主动调用Abort函数导致的,abort调用会向发起调用的线程发出 SIGABRT 信号。后面Abort message 展示了中断的原因:“企图接收一个从失败的HIDL调用的返回值”,再下面收到信号时 CPU 寄存器的内容。(本区段在各 ABI 之间变化很大。)这些内容的有用程度取决于确切的崩溃问题。

  • 下面这一段就是crash的backtrace了
1
2
3
4
5
6
7
8
backtrace:
      #00 pc 000389ac  /apex/com.android.runtime/lib/bionic/libc.so (abort+172) (BuildId: 9c1376f68909498526ac951ceeeb272a)
      #01 pc 003fd73d  /apex/com.android.art/lib/libart.so (art::Runtime::Abort(char const*)+1812) (BuildId: 856f40a584d43ed435db0d13c421e0c7)
      #02 pc 0000da15  /system/lib/libbase.so (android::base::SetAborter(std::__1::function<void (char const*)>&&)::$_3::__invoke(char const*)+48) (BuildId: 3537a03427bc64f05ae9032437c9fec6)
      #03 pc 0000d337  /system/lib/libbase.so (android::base::LogMessage::~LogMessage()+226) (BuildId: 3537a03427bc64f05ae9032437c9fec6)
      #04 pc 0003b42d  /system/lib/libhidlbase.so (android::hardware::details::return_status::onValueRetrieval() const+140) (BuildId: b890d44f0d4135b1e1e93cb6054cc954)
      #05 pc 0000ae05  /system/lib/libtvcompany_jni.so (JUsbhdd::UsbHddHotplgReq(char const*, int)+76) (BuildId: 82ab44171ed423e256d010e246dcadf9)
      #06 pc 0000da8d  /system/lib/libtvcompany_jni.so (com_socionext_tv_company_TVcompany_UsbHddHotplgReq(_JNIEnv*, _jobject*, _jstring*, int)+96) (BuildId: 82ab44171ed423e256d010e246dcadf9)

通过backtrace可以发现代码发生crash的具体位置。

  1. 第一列是帧号,#00是最底下的帧,也是最后出问题的地方。
  2. 第二列和第三列是pc值和共享库的地址, 这个地方我们在利用addr2line定位函数调用的位置时,会用到。
  3. 第三列是映射区域的名称(通常是共享库或可执行文件,但可能不适用于经过 JIT 编译的代码)
  4. 第四列,如果有符号,则会显示与 PC 值对应的符号以及到该符号的偏移量(以字节为单位)

使用addr2line工具

上一小节介绍了tombstone中内容的一些含义,但是阅读起来还是有点费劲,下面就借助一些工具来解析tombstone文件。

addr2line 是 用来获得指定动态链接库文件或者可执行文件中指定地址对应的源代码信息的工具

1
2
3
$ addr2line -fC -e libtvcompany_jni.so 0000ae05
JTuner::channelInfoToJavaObject(_JNIEnv*, vendor::company::hardware::company::V1_0::companyTvChannelInfo&)
libnativehelper/include_jni/jni.h:726

其中0000ae05就是上面backtrace的第三列,这样就能定位到具体的源码位置了。

需要注意的是这个地方的so库需要使用有符号的,就是在目录

1
out/target/product/cc_company/symbols/system/lib

下的so库,如果直接在设备里安装的so库是不带符号的,是不能用来定位的。

使用stack工具

在aosp里,android自带了一个解析tombstone的脚本stack,他的目录在

1
development/scripts/stack

如果你已经lunch过,可以直接使用stack命令。

stack解析tombstone的语法如下:

1
stack < tombstone-00 > result.txt

总结

面对Native Crash (简称(NC)脑残问题), 其实并没有那么可怕,恐惧来源无知,慢慢熟悉它,了解它,征服它,才是正确之道。相信通过这个过程,妈妈再也不用担心我的Native Crash问题了,^V^!

参考资料

关于我

  • 公众号: CodingDev