The tool "llvm-symbolizer" is supposed to turn memory addresses into line numbers and paths to source code files. Whenever there is a crash trace, it's supposed to activate and convert everything in the stack trace into readable source code locations. No matter what I do, I always get the following message on crash traces and llvm-symbolizer never fires:
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
I've looked everywhere - StackOverflow, Apple Developer Forums, etc., but haven't found anything that works. There are plenty of stack traces posted online with the error message, so it's a very common problem that most people seem to ignore. I set both PATH and LLVM_SYMBOLIZER_PATH to the llvm-symbolizer generated from compiling the Swift toolchain, and it didn't work. I also tried the llvm-symbolizer acquired from downloading LLVM via Homebrew.
The closest solution I found was this GitHub issue, but I'm not using Asan and their solution didn't change anything. I'm running tests using a locally built Swift compiler toolchain (macOS arm64 + Ninja), and reproducing the crash in this file caused by PR-32302.
It's a bit difficult to acquire the stack trace, because of how the test is set up. I can't just copy and paste the source code into a file and then run, because then nothing happens. I must run it through lit.py and remove not --crash and XFAIL: * from the file's comment header. This is what I get:
What I got
GenericTypeParamDecl has incorrect depth
Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project and the crash backtrace.
Stack dump:
0. Program arguments: /Users/philipturner/Documents/Swift-Compiler/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/bin/swift-frontend -frontend -merge-modules -emit-module /var/folders/qn/86czb43d3pv03bfnxvb3x66h0000gn/T/pr32302-autodiff-generictypeparamdecl-has-incorrect-depth-c2cae5.swiftmodule -parse-as-library -disable-diagnostic-passes -disable-sil-perf-optzns -target arm64-apple-macosx10.9 -Xllvm -aarch64-use-tbi -warn-on-potentially-unavailable-enum-case -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -F /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -F /Users/philipturner/Documents/Swift-Compiler/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/lib -module-cache-path /Users/philipturner/Documents/Swift-Compiler/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/swift-test-results/arm64-apple-macosx10.9/clang-module-cache -swift-version 5 -define-availability "SwiftStdlib 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999" -define-availability "SwiftStdlib 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2" -define-availability "SwiftStdlib 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0" -define-availability "SwiftStdlib 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4" -define-availability "SwiftStdlib 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0" -define-availability "SwiftStdlib 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5" -define-availability "SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0" -define-availability "SwiftStdlib 5.6:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999" -target-sdk-version 12.1 -emit-module-doc-path /Users/philipturner/Documents/Swift-Compiler/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/test-macosx-arm64/AutoDiff/compiler_crashers/Output/pr32302-autodiff-generictypeparamdecl-has-incorrect-depth.swift.tmp/pr32302.swiftdoc -emit-module-source-info-path /Users/philipturner/Documents/Swift-Compiler/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/test-macosx-arm64/AutoDiff/compiler_crashers/Output/pr32302-autodiff-generictypeparamdecl-has-incorrect-depth.swift.tmp/pr32302.swiftsourceinfo -module-name pr32302 -o /Users/philipturner/Documents/Swift-Compiler/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/test-macosx-arm64/AutoDiff/compiler_crashers/Output/pr32302-autodiff-generictypeparamdecl-has-incorrect-depth.swift.tmp/pr32302.swiftmodule
1. Swift version 5.6-dev (LLVM 4f1fbc3fa858c4b, Swift 5a9abb2119ab051)
2. Compiling with the current language version
3. While verifying GenericTypeParamDecl 'τ_1_0' (in module 'pr32302')
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0 swift-frontend 0x0000000106a59ae0 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56
1 swift-frontend 0x0000000106a58c10 llvm::sys::RunSignalHandlers() + 128
2 swift-frontend 0x0000000106a5a144 SignalHandler(int) + 304
3 libsystem_platform.dylib 0x00000001bb5304e4 _sigtramp + 56
4 libsystem_pthread.dylib 0x00000001bb518eb0 pthread_kill + 288
5 libsystem_c.dylib 0x00000001bb456314 abort + 164
6 swift-frontend 0x0000000106f90b58 (anonymous namespace)::Verifier::walkToDeclPost(swift::Decl*) (.cold.7) + 0
7 swift-frontend 0x00000001032405b0 (anonymous namespace)::Verifier::walkToDeclPost(swift::Decl*) + 5280
8 swift-frontend 0x0000000103247628 (anonymous namespace)::Traversal::doIt(swift::Decl*) + 140
9 swift-frontend 0x0000000103247590 swift::Decl::walk(swift::ASTWalker&) + 32
10 swift-frontend 0x0000000103235310 swift::verify(swift::Decl*) + 204
11 swift-frontend 0x00000001027091a4 swift::ModuleFile::verify() const + 100
12 swift-frontend 0x000000010277c4b0 swift::SerializedModuleLoaderBase::verifyAllModules() + 32
13 swift-frontend 0x00000001031b011c swift::ASTContext::verifyAllLoadedModules() const + 84
14 swift-frontend 0x00000001022701a0 performEndOfPipelineActions(swift::CompilerInstance&) + 556
15 swift-frontend 0x000000010226f168 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) + 6492
16 swift-frontend 0x00000001021b5ea8 swift::mainEntry(int, char const**) + 484
17 dyld 0x0000000118ad50f4 start + 520
<unknown>:0: error: unable to execute command: Abort trap: 6
<unknown>:0: error: merge-module command failed due to signal 6 (use -v to see invocation)
On that same file, there is a crash trace below, generated by @astrotuna201 on 5.6-dev (August 17, 2021). It has all of the line numbers shown alongside addresses, and it doesn't have the warning saying llvm-symbolizer isn't found.
What I need to get
// GenericTypeParamDecl has incorrect depth
// Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project and the crash backtrace.
// Stack dump:
// 0. Program arguments: /work/software/swift-stocktoolchain/build/ds/swift-linux-x86_64/bin/swift-frontend -frontend -merge-modules -emit-module /tmp/pr32302-autodiff-generictypeparamdecl-has-incorrect-depth-acc95c.swiftmodule -parse-as-library -disable-diagnostic-passes -disable-sil-perf-optzns -target x86_64-unknown-linux-gnu -warn-on-potentially-unavailable-enum-case -disable-objc-interop -module-cache-path /work/software/swift-stocktoolchain/build/ds/swift-linux-x86_64/swift-test-results/x86_64-unknown-linux-gnu/clang-module-cache -swift-version 5 -define-availability "SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0" -emit-module-doc-path /work/software/swift-stocktoolchain/build/ds/swift-linux-x86_64/test-linux-x86_64/AutoDiff/compiler_crashers/Output/pr32302-autodiff-generictypeparamdecl-has-incorrect-depth.swift.tmp/pr32302.swiftdoc -emit-module-source-info-path /work/software/swift-stocktoolchain/build/ds/swift-linux-x86_64/test-linux-x86_64/AutoDiff/compiler_crashers/Output/pr32302-autodiff-generictypeparamdecl-has-incorrect-depth.swift.tmp/pr32302.swiftsourceinfo -module-name pr32302 -o /work/software/swift-stocktoolchain/build/ds/swift-linux-x86_64/test-linux-x86_64/AutoDiff/compiler_crashers/Output/pr32302-autodiff-generictypeparamdecl-has-incorrect-depth.swift.tmp/pr32302.swiftmodule
// 1. Swift version 5.6-dev (LLVM ba0b85f590c1ba2, Swift 319b3e64aaeb252)
// 2. Compiling with the current language version
// 3. While verifying GenericTypeParamDecl 'τ_1_0' (in module 'pr32302')
// #0 0x000000000632fb13 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /work/software/swift-stocktoolchain/llvm-project/llvm/lib/Support/Unix/Signals.inc:563:13
// #1 0x000000000632dda0 llvm::sys::RunSignalHandlers() /work/software/swift-stocktoolchain/llvm-project/llvm/lib/Support/Signals.cpp:72:18
// #2 0x000000000632fe95 SignalHandler(int) /work/software/swift-stocktoolchain/llvm-project/llvm/lib/Support/Unix/Signals.inc:0:3
// #3 0x00007ffbfc0ab3c0 __restore_rt (/lib/x86_64-linux-gnu/libpthread.so.0+0x153c0)
// #4 0x00007ffbfbb3618b raise (/lib/x86_64-linux-gnu/libc.so.6+0x4618b)
// #5 0x00007ffbfbb15859 abort (/lib/x86_64-linux-gnu/libc.so.6+0x25859)
// #6 0x0000000002417d95 llvm::MutableArrayRef<swift::GenericTypeParamDecl*>::operator[](unsigned long) const /work/software/swift-stocktoolchain/llvm-project/llvm/include/llvm/ADT/ArrayRef.h:425:7
// #7 0x0000000002417d95 (anonymous namespace)::Verifier::verifyCheckedAlways(swift::GenericTypeParamDecl*) /work/software/swift-stocktoolchain/swift/lib/AST/ASTVerifier.cpp:2866:11
// #8 0x0000000002417d95 swift::GenericTypeParamDecl* (anonymous namespace)::Verifier::dispatchVisitPost<swift::GenericTypeParamDecl*>(swift::GenericTypeParamDecl*) /work/software/swift-stocktoolchain/swift/lib/AST/ASTVerifier.cpp:426:9
// #9 0x0000000002417d95 (anonymous namespace)::Verifier::walkToDeclPost(swift::Decl*) /work/software/swift-stocktoolchain/swift/include/swift/AST/DeclNodes.def:159:7
// #10 0x00000000024214c7 (anonymous namespace)::Traversal::doIt(swift::Decl*) /work/software/swift-stocktoolchain/swift/lib/AST/ASTWalker.cpp:1275:12
// #11 0x0000000002421433 swift::Decl::walk(swift::ASTWalker&) /work/software/swift-stocktoolchain/swift/lib/AST/ASTWalker.cpp:1909:3
// #12 0x000000000240c6dd swift::verify(swift::Decl*) /work/software/swift-stocktoolchain/swift/lib/AST/ASTVerifier.cpp:3761:1
// #13 0x0000000001285157 swift::ModuleFile::verify() const /work/software/swift-stocktoolchain/swift/lib/Serialization/ModuleFile.cpp:1244:3
// #14 0x00000000011ead9a swift::SerializedModuleLoaderBase::verifyAllModules() /work/software/swift-stocktoolchain/swift/lib/Serialization/SerializedModuleLoader.cpp:1268:39
// #15 0x000000000236546a swift::ASTContext::verifyAllLoadedModules() const /work/software/swift-stocktoolchain/swift/lib/AST/ASTContext.cpp:1794:21
// #16 0x000000000058259d performEndOfPipelineActions(swift::CompilerInstance&) /work/software/swift-stocktoolchain/swift/lib/FrontendTool/FrontendTool.cpp:933:37
// #17 0x000000000057faa7 performCompile(swift::CompilerInstance&, int&, swift::FrontendObserver*) /work/software/swift-stocktoolchain/swift/lib/FrontendTool/FrontendTool.cpp:1252:10
// #18 0x000000000057ef21 swift::performFrontend(llvm::ArrayRef<char const*>, char const*, void*, swift::FrontendObserver*) /work/software/swift-stocktoolchain/swift/lib/FrontendTool/FrontendTool.cpp:2154:8
// #19 0x00000000004b36e7 run_driver(llvm::StringRef, llvm::ArrayRef<char const*>) /work/software/swift-stocktoolchain/swift/lib/DriverTool/driver.cpp:196:7
// #20 0x00000000004b3312 swift::mainEntry(int, char const**) /work/software/swift-stocktoolchain/swift/lib/DriverTool/driver.cpp:402:5
// #21 0x00000000004b2c72 main /work/software/swift-stocktoolchain/swift/tools/driver/driver.cpp:20:3
// #22 0x00007ffbfbb170b3 __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b3)
// #23 0x00000000004b2b7e _start (/work/software/swift-stocktoolchain/build/ds/swift-linux-x86_64/bin/swift-frontend+0x4b2b7e)
It's still possible to locate the symbol names using Cmd + f in Xcode, but that's a very laborious process and I'm going to be tracing a lot of stack traces to fix compiler bugs. Has anyone been able to get llvm-symbolizer to fire on crash logs?