Android CrossCompilation on macOS: swiftc fails due missed symbol swift_addNewDSOImage

android
cross-compilation

(Vlad Gorlov) #1

Hello!

Need your help in order to understand the reason of missed swift_addNewDSOImage symbol error when using Swift compiler built on macOS cross-compile host. Swift compiler build with same sources on Ubuntu 18.04.1 (via Vagrant) works without such error. Seems pure CMake configuration issue.

Here is an error I am getting:

swiftrt.o:SwiftRT-ELF.cpp:function swift_image_constructor(): error: undefined reference to 'swift_addNewDSOImage'

From what I understand by exploring CMake files that symbol swift_addNewDSOImage should be inside libswiftCore.so shared object. And it seems inside:

$ nm /.../android-on-swift/Install/armv7a-macos/swift/usr/lib/swift/android/libswiftCore.so | grep swift_addNewDSOImage 
003c01a4 T swift_addNewDSOImage

Undefined reference also present in swiftrt.o file.

$ nm /.../android-on-swift/Install/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o | grep swift_addNewDSOImage 
U swift_addNewDSOImage

I am trying to understand purpose of swiftrt.o file and a reason of dedicated configuration for ImageInspectionELF.cpp file in swift/stdlib/public/runtime/CMakeLists.txt CMake file.

Thank you!


Here is a detail log.

mkdir -p "/Users/vova/Repositories/GitHub/Projects/android-on-swift/Build/armv7a/Hello"
cd /Users/vova/Repositories/GitHub/Projects/android-on-swift/Build/armv7a/Hello 
PATH=/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/swift/usr/bin:$PATH \
swiftc \
-v -tools-directory /Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin \
-target armv7-none-linux-androideabi \
-Xcc -I/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/sysroot/usr/include \
-Xcc -DDEPLOYMENT_TARGET_ANDROID \
-Xcc -DDEPLOYMENT_RUNTIME_SWIFT \
-Xlinker -v /Users/vova/Repositories/GitHub/Projects/android-on-swift/Projects/Hello/hello.swift
    
Swift version 5.0-dev (LLVM f63b283c71, Clang 41ac4c4262, Swift 8e38b67d66)
Target: armv7-none-linux-android
/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/swift/usr/bin/swift
-frontend -c
-primary-file /Users/vova/Repositories/GitHub/Projects/android-on-swift/Projects/Hello/hello.swift
-target armv7-none-linux-android -disable-objc-interop
-Xcc -I/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/sysroot/usr/include
-Xcc -DDEPLOYMENT_TARGET_ANDROID
-Xcc -DDEPLOYMENT_RUNTIME_SWIFT -color-diagnostics
-module-name hello
-o /var/folders/7l/skdbvw8s5jx0g9vs5_qrkync0000gt/T/hello-16dd94.o
 /Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/swift/usr/bin/swift-autolink-extract /var/folders/7l/skdbvw8s5jx0g9vs5_qrkync0000gt/T/hello-16dd94.o
-o /var/folders/7l/skdbvw8s5jx0g9vs5_qrkync0000gt/T/hello-d5644e.autolink

/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin/clang++
-fuse-ld=gold -B /Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin
-pie -target armv7-none-linux-androideabi /Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o /var/folders/7l/skdbvw8s5jx0g9vs5_qrkync0000gt/T/hello-16dd94.o
@/var/folders/7l/skdbvw8s5jx0g9vs5_qrkync0000gt/T/hello-d5644e.autolink
-L /Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/swift/usr/lib/swift/android
-lswiftCore --target=armv7-none-linux-android -v -Xlinker -v -o hello

Android (4751641 based on r328903) clang version 7.0.2 (https://android.googlesource.com/toolchain/clang 003100370607242ddd5815e4a043907ea9004281) (https://android.googlesource.com/toolchain/llvm 1d739ffb0366421d383e04ff80ec2ee591315116) (based on LLVM 7.0.2svn)
    Target: armv7-none-linux-android
    Thread model: posix
    InstalledDir: /Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin
    Found candidate GCC installation: /Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin/../lib/gcc/arm-linux-androideabi/4.9.x
    Selected GCC installation: /Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin/../lib/gcc/arm-linux-androideabi/4.9.x
    Candidate multilib: thumb;@mthumb
    Candidate multilib: armv7-a;@march=armv7-a
    Candidate multilib: armv7-a/thumb;@march=armv7-a@mthumb
    Candidate multilib: .;
    Selected multilib: armv7-a;@march=armv7-a

/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld.gold
--sysroot=/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin/../sysroot
-pie -X --enable-new-dtags --eh-frame-hdr -m armelf_linux_eabi -dynamic-linker /system/bin/linker
-o hello /Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin/../sysroot/usr/lib/../lib/crtbegin_dynamic.o
-L/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/swift/usr/lib/swift/android
-L/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/lib64/clang/7.0.2/lib/linux/arm
-L/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin/../lib/gcc/arm-linux-androideabi/4.9.x/armv7-a
-L/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/lib/../lib/armv7-a
-L/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin/../sysroot/usr/lib/../lib
-L/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/lib/armv7-a
-L/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin/../sysroot/usr/lib
/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o
/var/folders/7l/skdbvw8s5jx0g9vs5_qrkync0000gt/T/hello-16dd94.o -lswiftSwiftOnoneSupport -lswiftCore -lswiftCore
-v -lstdc++ -lm -lgcc -lgcc -ldl -lc -lgcc -lgcc -ldl /Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/ndk/bin/../sysroot/usr/lib/../lib/crtend_android.o
/Users/vova/Repositories/GitHub/Projects/android-on-swift/Install/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o:SwiftRT-ELF.cpp:function swift_image_constructor(): error: undefined reference to 'swift_addNewDSOImage'

GNU gold (GNU Binutils 2.27.0.20170315) 1.12
clang70++: error: linker command failed with exit code 1 (use -v to see invocation)

btw: Not sure if Swift officially support Android compilation on macOS Cross-Compile host. I am using this Automated workflow to build Swift for Android on macOS: https://github.com/vgorloff/Android-On-Swift


(Saleem Abdulrasool) #2

The swift_addDSOImage is used for image registration with the swift runtime. swiftrt.o is the image registrar and is embedded into the final image to ensure that the image is registered with the runtime upon load. It sounds like you are not linking against the right runtime?


(Vlad Gorlov) #3

It sounds like you are not linking against the right runtime?

Is it means that I have to explicitly specify some libraries (via -l arguments) from NDK for Darwin. Or swiftrt.o, libswiftCore.so, libswiftSwiftOnoneSupport.so, ... compiled with wrong settings. Or maybe Swift compiler itself compiled with wrong settings?


Here is what I have when inspecting files:

$ file /EDITED/armv7a-macos/swift/usr/bin/swift 
/EDITED/armv7a-macos/swift/usr/bin/swift: Mach-O 64-bit executable x86_64

$ file /EDITED/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o
/EDITED/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o: ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (SYSV), not stripped

$ file /EDITED/armv7a-macos/swift/usr/lib/swift/android/libswiftCore.so
/EDITED/armv7a-macos/swift/usr/lib/swift/android/libswiftCore.so: ELF 32-bit LSB shared object ARM, EABI5 version 1 (SYSV), dynamically linked, with debug_info, not stripped

$ file /EDITED/armv7a-macos/swift/usr/lib/swift/android/libswiftSwiftOnoneSupport.so
/EDITED/armv7a-macos/swift/usr/lib/swift/android/libswiftSwiftOnoneSupport.so: ELF 32-bit LSB shared object ARM, EABI5 version 1 (SYSV), dynamically linked, with debug_info, not stripped

SO-files definitely dynamically linked and seems for correct architecture. swiftrt.o – ELF 32-bit LSB relocatable. But i don't really understand what is relocatable means :smile:


The only significant arguments I am passing to swiftc seems:

  • -tools-directory /.../armv7a-macos/ndk/bin – path to Android NDK for Darwin.
  • -target armv7-none-linux-androideabi – target triple which informs that result executable should be targeted Android (i.e. ELF 32-bit LSB pie executable ARM, EABI5 ...)

Also few flags passed to avoid clang errors "due wrong/missed headers".

  • -Xcc -DDEPLOYMENT_TARGET_ANDROID
  • -Xcc -DDEPLOYMENT_RUNTIME_SWIFT

My guess was that clang++ and ld.gold from NDK doing rest of work. At least same swiftc invocation works well under Ubuntu (Vagrant).

Here is a command line used:

PATH=/EDITED/armv7a-macos/swift/usr/bin:$PATH swiftc -v \
-tools-directory /EDITED/armv7a-macos/ndk/bin \
-target armv7-none-linux-androideabi \
-Xcc -I/EDITED/armv7a-macos/ndk/sysroot/usr/include \
-Xcc -DDEPLOYMENT_TARGET_ANDROID \
-Xcc -DDEPLOYMENT_RUNTIME_SWIFT \
-Xlinker -v /EDITED/Projects/Hello/hello.swift

(Saleem Abdulrasool) #4

Uh, I don't see a -use-ld=gold or -use-ld=lld there. That definitely seems like you're not using the right tools.


(Vlad Gorlov) #5

Thank you!

I isolated issue but can't understand why it is working in one case and not working in other case :0

So, first we can lookup symbols to ensure that swift_addNewDSOImage symbol really exists (in libswiftCore.so.):

nm -a swiftrt.o
         U 
00000000 t 
00000000 t $a.0
00000130 t $d.1
00000000 a SwiftRT-ELF.cpp
00000000 t _ZL23swift_image_constructorv
00000000 b _ZN12_GLOBAL__N_18sectionsE
         U __start_swift5_assocty
         U __start_swift5_fieldmd
         U __start_swift5_protocol_conformances
         U __start_swift5_protocols
         U __start_swift5_reflstr
         U __start_swift5_replace
         U __start_swift5_type_metadata
         U __start_swift5_typeref
         U __stop_swift5_assocty
         U __stop_swift5_fieldmd
         U __stop_swift5_protocol_conformances
         U __stop_swift5_protocols
         U __stop_swift5_reflstr
         U __stop_swift5_replace
         U __stop_swift5_type_metadata
         U __stop_swift5_typeref
         U swift_addNewDSOImage

nm -a libswiftSwiftOnoneSupport.so
         U swift_addNewDSOImage

nm -a libswiftCore.so
003c01a4 T swift_addNewDSOImage

Then we can compile Swift source code:

/EDITED/armv7a-macos/swift/usr/bin/swift -frontend -c \
-primary-file /EDITED/Temp/hello.swift \
-target armv7-none-linux-android -disable-objc-interop \
-Xcc -I/EDITED/armv7a-macos/ndk/sysroot/usr/include \
-Xcc -DDEPLOYMENT_TARGET_ANDROID -Xcc -DDEPLOYMENT_RUNTIME_SWIFT \
-color-diagnostics -module-name hello \
-o /EDITED/Temp/hello.o

file /EDITED/Temp/hello.o

# Prints
/EDITED/Temp/hello.o: ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (SYSV), not stripped

Then we can run swift-autolink-extract tool.

/EDITED/armv7a-macos/swift/usr/bin/swift-autolink-extract \
/EDITED/Temp/hello.o \
-o /EDITED/Temp/hello.autolink

cat /EDITED/Temp/hello.autolink

# Prints
-lswiftSwiftOnoneSupport
-lswiftCore

Then we can launch linker.

/EDITED/armv7a-macos/ndk/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld.gold \
--sysroot=/EDITED/armv7a-macos/ndk/bin/../sysroot \
-pie -X --enable-new-dtags --eh-frame-hdr -m armelf_linux_eabi -dynamic-linker /system/bin/linker \
-o /EDITED/Temp/hello \
/EDITED/armv7a-macos/ndk/bin/../sysroot/usr/lib/../lib/crtbegin_dynamic.o \
-L/EDITED/armv7a-macos/ndk/lib64/clang/7.0.2/lib/linux/arm \
-L/EDITED/armv7a-macos/ndk/bin/../lib/gcc/arm-linux-androideabi/4.9.x/armv7-a \
-L/EDITED/armv7a-macos/ndk/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/lib/../lib/armv7-a \
-L/EDITED/armv7a-macos/ndk/bin/../sysroot/usr/lib/../lib \
-L/EDITED/armv7a-macos/ndk/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/lib/armv7-a \
-L/EDITED/armv7a-macos/ndk/bin/../sysroot/usr/lib \
/EDITED/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o \
/EDITED/Temp/hello.o \
-L/EDITED/armv7a-macos/swift/usr/lib/swift/android \
-lswiftSwiftOnoneSupport -lswiftCore \
-v -lstdc++ -lm -lgcc -lgcc -ldl -lc -lgcc -lgcc -ldl \
/EDITED/armv7a-macos/ndk/bin/../sysroot/usr/lib/../lib/crtend_android.o

# As result error:
/EDITED/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o:SwiftRT-ELF.cpp:function swift_image_constructor(): error: undefined reference to 'swift_addNewDSOImage'

But if we specifying full path to SO-files, then liker works well.

/EDITED/armv7a-macos/ndk/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld.gold \
--sysroot=/EDITED/armv7a-macos/ndk/bin/../sysroot \
-pie -X --enable-new-dtags --eh-frame-hdr -m armelf_linux_eabi -dynamic-linker /system/bin/linker \
-o /EDITED/Temp/hello \
/EDITED/armv7a-macos/ndk/bin/../sysroot/usr/lib/../lib/crtbegin_dynamic.o \
-L/EDITED/armv7a-macos/ndk/lib64/clang/7.0.2/lib/linux/arm \
-L/EDITED/armv7a-macos/ndk/bin/../lib/gcc/arm-linux-androideabi/4.9.x/armv7-a \
-L/EDITED/armv7a-macos/ndk/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/lib/../lib/armv7-a \
-L/EDITED/armv7a-macos/ndk/bin/../sysroot/usr/lib/../lib \
-L/EDITED/armv7a-macos/ndk/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/lib/armv7-a \
-L/EDITED/armv7a-macos/ndk/bin/../sysroot/usr/lib \
/EDITED/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o \
/EDITED/Temp/hello.o \
/EDITED/armv7a-macos/swift/usr/lib/swift/android/libswiftCore.so \
/EDITED/armv7a-macos/swift/usr/lib/swift/android/libswiftSwiftOnoneSupport.so \
-v -lstdc++ -lm -lgcc -lgcc -ldl -lc -lgcc -lgcc -ldl \
/EDITED/armv7a-macos/ndk/bin/../sysroot/usr/lib/../lib/crtend_android.o

file /EDITED/Temp/hello

# Prints
/EDITED/Temp/hello: ELF 32-bit LSB pie executable ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /system/bin/linker, with debug_info, not stripped

Difference in two linker invocations is following:

--- untitled
+++ (clipboard)
@@ -11,7 +11,7 @@
 -L/EDITED/armv7a-macos/ndk/bin/../sysroot/usr/lib \
 /EDITED/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o \
 /EDITED/Temp/hello.o \
-/EDITED/armv7a-macos/swift/usr/lib/swift/android/libswiftCore.so \
-/EDITED/armv7a-macos/swift/usr/lib/swift/android/libswiftSwiftOnoneSupport.so \
+-L/EDITED/armv7a-macos/swift/usr/lib/swift/android \
+-lswiftSwiftOnoneSupport -lswiftCore \
 -v -lstdc++ -lm -lgcc -lgcc -ldl -lc -lgcc -lgcc -ldl \
 /EDITED/armv7a-macos/ndk/bin/../sysroot/usr/lib/../lib/crtend_android.o

What can be the reason why ld.gold linker from Android NDK can't find symbols if Shared Objects specified not with a full path?

UPDATE: Linker shows same error even if options -lswiftSwiftOnoneSupport -lswiftCore not passed at all. Looks like linker not really using Shared Objects specified with -l option.

UPDATE 2: Order of passing libraries matters.

# Works
cd /EDITED/armv7a/Hello && /EDITED/armv7a-macos/ndk/bin/clang++ -fuse-ld=gold \
-B /EDITED/armv7a-macos/ndk/bin -pie -target armv7-none-linux-androideabi \
/EDITED/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o \
/EDITED/armv7a/Hello/hello-main.o \
-l/EDITED/armv7a-macos/swift/usr/lib/swift/android/libswiftCore.so \
-l/EDITED/armv7a-macos/swift/usr/lib/swift/android/libswiftSwiftOnoneSupport.so \
--target=armv7-none-linux-android \
-o /EDITED/armv7a/Hello/hello

# Not working
cd /EDITED/armv7a/Hello && /EDITED/armv7a-macos/ndk/bin/clang++ -fuse-ld=gold \
-B /EDITED/armv7a-macos/ndk/bin -pie -target armv7-none-linux-androideabi \
/EDITED/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o \
/EDITED/armv7a/Hello/hello-main.o \
-l/EDITED/armv7a-macos/swift/usr/lib/swift/android/libswiftSwiftOnoneSupport.so \
-l/EDITED/armv7a-macos/swift/usr/lib/swift/android/libswiftCore.so \
--target=armv7-none-linux-android \
-o /EDITED/armv7a/Hello/hello

/EDITED/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o:SwiftRT-ELF.cpp:function swift_image_constructor(): error: undefined reference to 'swift_addNewDSOImage'
clang70++: error: linker command failed with exit code 1 (use -v to see invocation)

Super strange why libswiftCore.so needs to be passed before libswiftSwiftOnoneSupport.so.


(Vlad Gorlov) #6

UPDATE 3:

After updating to Swift 5 and NDK 19 the error is different: error: undefined reference to '_swift_FORCE_LOAD_$_swiftGlibc'

hello-main.o:hello-main.o:function main: error: undefined reference to '$ss27_allocateUninitializedArrayySayxG_BptBwlFyp_Tg5'
hello-main.o:hello-main.o:function main: error: undefined reference to '$ss27_allocateUninitializedArrayySayxG_BptBwlFyp_Tg5'
hello-main.o:hello-main.o:function main: error: undefined reference to '$ss27_allocateUninitializedArrayySayxG_BptBwlFyp_Tg5'
hello-main.o:hello-main.o:function main: error: undefined reference to '$ss27_allocateUninitializedArrayySayxG_BptBwlFyp_Tg5'
hello-main.o:hello-main.o:$s5hello6PersonL_V10CodingKeysON: error: undefined reference to '_swift_FORCE_LOAD_$_swiftGlibc'

(Saleem Abdulrasool) #7

Sounds like you aren't linking against swiftGlibc. With auto-link-extract, -lswiftGlibc should be added implicitly, I would suggest that you figure out what the actual linker invocation looks like and why -lswiftGlibc is missing.


(Vlad Gorlov) #8

The actual liker invocation looks like below.
Swift auto-linker not used (due another link problems). -lswiftGlibc flag is present.

cd /android-on-swift/Build/armv7a/Hello && /android-on-swift/Install/armv7a-macos/swift/usr/bin/swift -frontend -c \
-primary-file /android-on-swift/Projects/Hello/hello.swift \
-target armv7-none-linux-android -disable-objc-interop -color-diagnostics \
-module-name hello -o /android-on-swift/Build/armv7a/Hello/hello-main.o \
-Xcc -I/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include \
-I/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/include/arm-linux-androideabi \
-Xcc -DDEPLOYMENT_TARGET_ANDROID -Xcc -DDEPLOYMENT_TARGET_LINUX -Xcc -DDEPLOYMENT_RUNTIME_SWIFT

### Checking that Swift compiled sample source code file.
file /android-on-swift/Build/armv7a/Hello/hello-main.o
/android-on-swift/Build/armv7a/Hello/hello-main.o: ELF 32-bit LSB relocatable, ARM, EABI5 version 1 (SYSV), not stripped

cd /android-on-swift/Build/armv7a/Hello && /android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/armv7a-linux-androideabi21-clang \
-fuse-ld=gold -pie -v \
-B /android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin \
/android-on-swift/Install/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o \
/android-on-swift/Build/armv7a/Hello/hello-main.o \
-L /android-on-swift/Build/armv7a/Hello -lswiftCore -lswiftGlibc -lswiftSwiftOnoneSupport -lswiftDispatch -lBlocksRuntime -lc++_shared -lFoundation \
-L /android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/lib/gcc/arm-linux-androideabi/4.9.x \
-o /android-on-swift/Build/armv7a/Hello/hello

Android (5058415 based on r339409) clang version 8.0.2 (https://android.googlesource.com/toolchain/clang 40173bab62ec746213857d083c0e8b0abb568790) (https://android.googlesource.com/toolchain/llvm 7a6618d69e7e8111e1d49dc9e7813767c5ca756a) (based on LLVM 8.0.2svn)
Target: armv7a-unknown-linux-android21
Thread model: posix
InstalledDir: /android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin
Found candidate GCC installation: /android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x
Selected GCC installation: /android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x
Candidate multilib: thumb;@mthumb
Candidate multilib: armv7-a;@march=armv7-a
Candidate multilib: armv7-a/thumb;@march=armv7-a@mthumb
Candidate multilib: .;
Selected multilib: armv7-a;@march=armv7-a

 "/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/bin/ld.gold" \
 -pie -X --enable-new-dtags --eh-frame-hdr -m armelf_linux_eabi -dynamic-linker /system/bin/linker \
 -o /android-on-swift/Build/armv7a/Hello/hello \
 /android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/lib/arm-linux-androideabi/21/crtbegin_dynamic.o \
 -L/android-on-swift/Build/armv7a/Hello \
 -L/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/lib/gcc/arm-linux-androideabi/4.9.x \
 -L/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/lib64/clang/8.0.2/lib/linux/arm \
 -L/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/armv7-a \
 -L/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/lib/../lib/armv7-a \
 -L/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/lib/arm-linux-androideabi/21 \
 -L/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/lib/arm-linux-androideabi \
 -L/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/lib/../lib \
 -L/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/lib/arm-linux-androideabi/../../lib \
 -L/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/../lib/gcc/arm-linux-androideabi/4.9.x/../../../../arm-linux-androideabi/lib/armv7-a \
 -L/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/lib /android-on-swift/Install/armv7a-macos/swift/usr/lib/swift/android/armv7/swiftrt.o \
 /android-on-swift/Build/armv7a/Hello/hello-main.o -lswiftCore -lswiftGlibc -lswiftSwiftOnoneSupport -lswiftDispatch -lBlocksRuntime -lc++_shared -lFoundation \
 -lgcc -ldl -lc -lgcc -ldl \
 /android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/../sysroot/usr/lib/arm-linux-androideabi/21/crtend_android.o
 
/android-on-swift/Build/armv7a/Hello/hello-main.o:hello-main.o:function main: error: undefined reference to '$ss27_allocateUninitializedArrayySayxG_BptBwlFyp_Tg5'
/android-on-swift/Build/armv7a/Hello/hello-main.o:hello-main.o:function main: error: undefined reference to '$ss27_allocateUninitializedArrayySayxG_BptBwlFyp_Tg5'
/android-on-swift/Build/armv7a/Hello/hello-main.o:hello-main.o:function main: error: undefined reference to '$ss27_allocateUninitializedArrayySayxG_BptBwlFyp_Tg5'
/android-on-swift/Build/armv7a/Hello/hello-main.o:hello-main.o:function main: error: undefined reference to '$ss27_allocateUninitializedArrayySayxG_BptBwlFyp_Tg5'
/android-on-swift/Build/armv7a/Hello/hello-main.o:hello-main.o:$s5hello6PersonL_V10CodingKeysON: error: undefined reference to '_swift_FORCE_LOAD_$_swiftGlibc'
clang: error: linker command failed with exit code 1 (use -v to see invocation)

(Vlad Gorlov) #9

Picking and replacing files libswiftGlibc.so and libswiftCore.so from same toolchain build on Linux fixes link errors. Seems build settings for Cross Compilation for that two libraries incorrect. Other libraries (like libswiftSwiftOnoneSupport.so, libswiftDispatch.so, libFoundation.so, libdispatch.so and libBlocksRuntime.so) seems built correctly.

Here is minimum required linker options:

/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/arm-linux-androideabi/bin/ld.gold \
-o /android-on-swift/Build/armv7a/Hello/hello \
/android-on-swift/Build/armv7a/Hello/hello-main.o \
-L /android-on-swift/Build/armv7a/Hello \
-lswiftCore -lswiftGlibc -lswiftSwiftOnoneSupport -lswiftDispatch -lBlocksRuntime -lFoundation \
-L /android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/sysroot/usr/lib/arm-linux-androideabi/21 \
-lc

(Vlad Gorlov) #10

Seems issue in wrong soname due autogenerated rules.ninja by Cmake.

#############################################
# Rule for linking CXX shared library.

rule CXX_SHARED_LIBRARY_LINKER__swiftGlibc-android-armv7
  command = $PRE_LINK && /android-on-swift/Build/armv7a-macos/llvm/./bin/clang++ $LANGUAGE_COMPILE_FLAGS $ARCH_FLAGS -shared -Wl,-headerpad_max_install_names $LINK_FLAGS -o $TARGET_FILE  $in $LINK_PATH $LINK_LIBRARIES && $POST_BUILD
  description = Linking CXX shared library $TARGET_FILE
  restat = $RESTAT

So, we passing linker option -Wl with value -headerpad_max_install_names

After that ld.gold resolves unknown option -headerpad_max_install_names as below:

  -h FILENAME, -soname FILENAME
                              Set shared library name

Literally soname became eaderpad_max_install_names :0

After reading ELF data readelf --all Build/armv7a/Hello/libswiftCore.so we can see following:

Dynamic section at offset 0x46aed4 contains 32 entries:
  Tag        Type                         Name/Value
...
 0x00000004 (HASH)                       0xac13c
 0x00000001 (NEEDED)                     Shared library: [libm.so]
 0x00000001 (NEEDED)                     Shared library: [libdl.so]
 0x00000001 (NEEDED)                     Shared library: [liblog.so]
 0x00000001 (NEEDED)                     Shared library: [libicudataswift.so.64]
 0x00000001 (NEEDED)                     Shared library: [libicui18nswift.so.64]
 0x00000001 (NEEDED)                     Shared library: [libicuucswift.so.64]
 0x00000001 (NEEDED)                     Shared library: [libc++_shared.so]
 0x00000001 (NEEDED)                     Shared library: [libc.so]
 0x0000000e (SONAME)                     Library soname: [eaderpad_max_install_names]
 0x0000001a (FINI_ARRAY)                 0x461460

While ELF data of library build on Linux as below:

Dynamic section at offset 0x46aed4 contains 32 entries:
  Tag        Type                         Name/Value
...
 0x00000004 (HASH)                       0xac130
 0x00000001 (NEEDED)                     Shared library: [libm.so]
 0x00000001 (NEEDED)                     Shared library: [libdl.so]
 0x00000001 (NEEDED)                     Shared library: [liblog.so]
 0x00000001 (NEEDED)                     Shared library: [libicudataswift.so.64]
 0x00000001 (NEEDED)                     Shared library: [libicui18nswift.so.64]
 0x00000001 (NEEDED)                     Shared library: [libicuucswift.so.64]
 0x00000001 (NEEDED)                     Shared library: [libc++_shared.so]
 0x00000001 (NEEDED)                     Shared library: [libc.so]
 0x0000000e (SONAME)                     Library soname: [libswiftCore.so]
 0x0000001a (FINI_ARRAY)                 0x461460

(Jordan Rose) #11

It looks like CMake is adding -headerpad_max_install_names because we don't use CMake's cross-compilation support to build the standard library. You can probably manually hack it out of CMAKE_CXX_LINK_FLAGS where it's being added.


(Vlad Gorlov) #12

Thanks!
Hacking file rules.ninja does the job.

line = line.gsub('$SONAME_FLAG $INSTALLNAME_DIR$SONAME', '-Wl,-soname,$SONAME')
line = line.gsub('-Wl,-headerpad_max_install_names', '')

Strange that for instance in swift-corelibs-libdispatch repository rules.ninja don't require hacking. Correct rules generated by cmake out of box.

#############################################
# Rule for linking C shared library.

rule C_SHARED_LIBRARY_LINKER__BlocksRuntime
  command = $PRE_LINK && /android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --gcc-toolchain=/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64 --sysroot=/android-on-swift/Sources/ndk-macos-r19b/toolchains/llvm/prebuilt/darwin-x86_64/sysroot -fPIC $LANGUAGE_COMPILE_FLAGS $ARCH_FLAGS $LINK_FLAGS -shared $SONAME_FLAG$SONAME -o $TARGET_FILE $in $LINK_PATH $LINK_LIBRARIES && $POST_BUILD
  description = Linking C shared library $TARGET_FILE
  restat = $RESTAT

So, non-official CrossCompile toolchain on macOS Host, which builds Swift (including Foundation and Dispatch libraries) for Android platform seems works.

Sample run:

adb shell ls -l /data/local/tmp/hello
total 44792
-rwxrwxrwx 1 shell shell    60504 2019-03-12 02:35 hello
-rwxrwxrwx 1 shell shell   117492 2019-03-12 02:35 libBlocksRuntime.so
-rw-rw-rw- 1 shell shell 14697312 2019-03-12 02:35 libFoundation.so
-rwxrwxrwx 1 shell shell  4842992 2019-03-12 02:35 libc++_shared.so
-rwxrwxrwx 1 shell shell  2619056 2019-03-12 02:35 libcrypto.so.1.1
-rwxrwxrwx 1 shell shell   500140 2019-03-12 02:35 libcurl.so
-rwxrwxrwx 1 shell shell  1268968 2019-03-12 02:35 libdispatch.so
-rwxrwxrwx 1 shell shell     5256 2019-03-12 02:35 libicudataswift.so.64
-rwxrwxrwx 1 shell shell  4522788 2019-03-12 02:35 libicui18nswift.so.64
-rwxrwxrwx 1 shell shell  2739748 2019-03-12 02:35 libicuucswift.so.64
-rwxrwxrwx 1 shell shell   587076 2019-03-12 02:35 libssl.so.1.1
-rw-rw-rw- 1 shell shell  7715976 2019-03-12 02:35 libswiftCore.so
-rw-rw-rw- 1 shell shell   402012 2019-03-12 02:35 libswiftDispatch.so
-rw-rw-rw- 1 shell shell   691028 2019-03-12 02:35 libswiftGlibc.so
-rw-rw-rw- 1 shell shell  1193304 2019-03-12 02:35 libswiftRemoteMirror.so
-rw-rw-rw- 1 shell shell   652704 2019-03-12 02:35 libswiftSIMDOperators.so
-rw-rw-rw- 1 shell shell  1035376 2019-03-12 02:35 libswiftSwiftOnoneSupport.so
-rwxrwxrwx 1 shell shell  2140764 2019-03-12 02:35 libxml2.so

Starting execution of "/data/local/tmp/hello/hello"...
adb shell LD_LIBRARY_PATH=/data/local/tmp/hello /data/local/tmp/hello/hello
SA - SwiftCore: Works!
SA - DispatchQueue: Works!
SA - BlockOperation: Works!
SA - JSONSerialization/JSONDecoder: Works!
SA - URLSession: Currently disabled. Will fail with `Segmentation fault`. Seems something in Foundation classes needs to be fixed.

Execution of "/data/local/tmp/hello/hello" completed.