Great! I’ll try your 5.10 instructions for starters.
An update on the Android port - the new Android overlay that Alex and Saleem proposed was merged into Swift 6 earlier this month and will ship with that toolchain's stable release in a couple months. That just means the Swift 6 compiler supports the new overlay: you will still have to use my prebuilt Android SDK then or build one yourself, using the OSS patches and instructions in my doc there.
In the meantime, my CI puts out builds of Swift 6.0 "devel" and Swift 6.1 trunk SDKs for Android (scroll down for the links to the zip file "Artifacts" that have the SDKs), which you can use with the official snapshot toolchains for linux by following the same procedure for 5.10 outlined in my doc. Simply download the sdk-devel-aarch64 zip file if you want to build for Android AArch64 API 24 with the Swift 6.0 July 19 snapshot toolchain, or similarly, sdk-trunk-x86_64 if targeting Android x86_64 24 with the 6.1 July 15 snapshot toolchain.
I use those particular snapshot toolchain dates because the Foundation rewrite in Swift was merged into the Swift 6 branches this week, and that does not build for Android yet, so I had to disable updating those Swift 6 source snapshots past those dates on my daily Android CI yesterday. I will look into getting the Foundation rewrite building on Android in the coming weeks (@Alex_L, do you have any of it working?).
Also, I want to put out an Android SDK bundle, which is more convenient to cross-compile with, and get the Swift compiler validation suite running for Android on the official CI in the coming months. Hopefully, that will culminate with an official Android SDK bundle on swift.org in the coming year.
12 Likes
swift-foundation already builds for Android now. There are some problems left to address with the macros builds that @bnbarham, @jmschonfeld, @etcwilde, and I are discussing. We have a set of required changes for swift-corelibs-foundation that enable re-coring that to swift-foundation on Android as well. There will be some more changes needed to the CI scripts for Windows when this is merged, but we are pretty close to having a newer version of Foundation for Android as well.
5 Likes
OK, I had not seen your recent pulls adding the Android overlay to that repo.
Yes, that is what I referred to above, building swift-corelibs-foundation in the Swift toolchain after the recent merge of swift-foundation doesn't work.
Great, let me know when you submit pulls for that and I will test it out.
This is very exciting progress! We've been working with a Swift 5.10 Android toolchain on macOS, but haven't yet gotten going with Swift 6. We are also looking forward to switching from swift-corelibs-foundation to swift-foundation, since we see that as a more flexible way to support Android-specific customizations that will be needed for our use case.
I know @Finagolfin has been working on the Android-hosted toolchain and @compnerd and co. have been working on the Windows-hosted toolchain, but is anyone actively working on a complete macOS-hosted toolchain for Android? We would be interested in contributing to this effort.
Yes, we will be looking into the macOS hosted toolchain with Android SDK support soon. The first part of this will be getting macOS toolchains built as the SDK is tied to the toolchain due to the swiftmodules. This work will be at The Browser Company's fork of swift-build on GitHub.
If you would like to aid in this effort for the official toolchain releases, it would be interesting to port some of the work that we have done in build.ps1 to build the Android SDK to be replicated into build-script.
You build this yourself?
I currently distribute two different Swift artifacts: a native Swift toolchain that runs on Android devices, which you mention, and a Swift Android SDK, that can be used with a Swift toolchain for linux to cross-compile for Android. Some people have tried that Android SDK with the official Apple toolchain also, but it hasn't always worked, likely because of proprietary additions to the Xcode toolchain. I think my Android SDK should work well with the Homebrew OSS Swift toolchain for macOS, but I don't think that has been tried much.
None that is up to date that I know of, but with the recent addition of the SDK bundles feature and the Musl static linux SDK bundle in particular, the hope is that these SDK bundles will be interchangeable among all Swift toolchain platforms. Once I have an Android SDK bundle, I plan to try it with the macOS and Windows toolchains also and set up Android cross-compilation CI for those platforms too, instead of only linux atm.
Such as? The official build-script used on non-Windows platforms has supported cross-compiling a Swift SDK for Android for many years now, not sure what else you'd like to add to that. In fact, I use it to cross-compile the entire Swift toolchain for Android too, though that does require some additional patches.
Of course, there are always small improvements that can be made, eg the build-script only supports building for one Android architecture at a time now, so I have to invoke it three times to build an SDK for all the currently supported Android architectures in Swift: AArch64, armv7, and x86_64.
1 Like
The pieces that I was thinking of is a non-toolchain (SDK only) build that also packages in a similar layout to the Windows SDK build so that we can re-use the artifacts across Windows, Linux, and potentially macOS.
1 Like
I've gotten the Android SDK working with the official macOS toolchains for 5.10.1, devel/6.0, and trunk. I can contribute to your CI action so it can be continuously validated alongside your Ubuntu runner, if you are interested?
2 Likes
Sounds good. I don't use macOS, which is why I haven't bothered so far.
The Android SDKs for 5.10, 6.0, and trunk that you tried are built by you or someone else?
Great! I've filed a PR against your repo, adding a workflow job that cross-compiles from macOS to Android using the official toolchains. We can continue discussing in the PR.
I had previously built 5.10 myself using a variety of patches from other toolchain projects, but I've found that your work on 6.0 has been keeping up so well with ongoing changes (and frequent breakage) that I've just started using that.
Also, I figure there will be a lot of work to do to switch from swift-corelibs-foundation to swift-foundation, so it might make sense to pool our efforts.
2 Likes
I just ran the Swift compiler validation suite for Android on linux x86_64 and thought I'd share the results. I built the latest Aug. 3 trunk source snapshot with this small patch to use the just-released Android NDK 27:
diff -ruw swift-swift-DEVELOPMENT-SNAPSHOT-2024-08-03-a/include/swift/Runtime/PrebuiltStringMap.h swift/include/swift/Runtime/PrebuiltStringMap.h
--- swift-swift-DEVELOPMENT-SNAPSHOT-2024-08-03-a/include/swift/Runtime/PrebuiltStringMap.h 2024-08-03 22:08:32.000000000 +0000
+++ swift/include/swift/Runtime/PrebuiltStringMap.h 2024-08-04 21:38:52.629572967 +0000
@@ -18,6 +18,7 @@
#include <cstdint>
#include <cstring>
#include <optional>
+#include <utility>
namespace swift {
diff -ruw swift-swift-DEVELOPMENT-SNAPSHOT-2024-08-03-a/stdlib/cmake/modules/AddSwiftStdlib.cmake swift/stdlib/cmake/modules/AddSwiftStdlib.cmake
--- swift-swift-DEVELOPMENT-SNAPSHOT-2024-08-03-a/stdlib/cmake/modules/AddSwiftStdlib.cmake 2024-08-03 22:08:32.000000000 +0000
+++ swift/stdlib/cmake/modules/AddSwiftStdlib.cmake 2024-08-05 03:18:58.078621954 +0000
@@ -557,7 +557,7 @@
if(NOT CMAKE_HOST_SYSTEM MATCHES Windows)
# The Android resource dir is specified from build.ps1 on windows.
if(NOT "${SWIFT_ANDROID_NDK_PATH}" STREQUAL "")
- if("${SWIFT_ANDROID_NDK_PATH}" MATCHES "r26")
+ if("${SWIFT_ANDROID_NDK_PATH}" MATCHES "r26|r27")
file(GLOB RESOURCE_DIR ${SWIFT_SDK_ANDROID_ARCH_${LFLAGS_ARCH}_PATH}/../lib/clang/*)
else()
file(GLOB RESOURCE_DIR ${SWIFT_SDK_ANDROID_ARCH_${LFLAGS_ARCH}_PATH}/../lib64/clang/*)
diff -ruw swift-swift-DEVELOPMENT-SNAPSHOT-2024-08-03-a/stdlib/public/Platform/android.modulemap swift/stdlib/public/Platform/android.modulemap
--- swift-swift-DEVELOPMENT-SNAPSHOT-2024-08-03-a/stdlib/public/Platform/android.modulemap 2024-08-03 22:08:32.000000000 +0000
+++ swift/stdlib/public/Platform/android.modulemap 2024-08-05 03:15:47.855946150 +0000
@@ -556,6 +556,12 @@
header "bits/sa_family_t.h"
export *
}
+
+module _bits_sockaddr_storage [system] {
+ header "bits/sockaddr_storage.h"
+ export *
+}
+
module _bits_stdatomic [system] {
// Note: this module is not part of 'stdatomic'
// as it depends on libc++ and forcing it to
I then ran this command, as shown in the Android doc but with NDK 27 (and added flags to turn assertions off, use --host-test mode because I don't have an Android device connected for native Android executable tests to run, and skip running the tests for the linux x86_64 host too):
> ./swift/utils/build-script -RA --android --android-ndk ~/android-ndk-r27 --android-arch aarch64 --android-api-level 24 -T --host-test --skip-test-linux
I see only 20 tests failing from that compiler validation suite run, mostly 14 CAS tests that don't work when cross-compiling?
<testcase classname="Swift(android-aarch64).CAS" name="Xcc_args.swift" time="0.40">
<failure><![CDATA[Script:
--
: 'RUN: at line 1'; rm -rf "/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp" && mkdir -p "/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp"
: 'RUN: at line 2'; split-file /home/finagolfin/swift/test/CAS/Xcc_args.swift /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp
: 'RUN: at line 4'; /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/bin/swift-frontend -target aarch64-unknown-linux-android -sdk /home/finagolfin/android-ndk-r27/toolchains/llvm/prebuilt/linux-x86_64/sysroot -resource-dir /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/lib/swift -module-cache-path /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/swift-test-results/aarch64-unknown-linux-android/clang-module-cache -swift-version 4 -define-availability 'SwiftStdlib 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -define-availability 'SwiftStdlib 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2' -define-availability 'SwiftStdlib 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0' -define-availability 'SwiftStdlib 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4' -define-availability 'SwiftStdlib 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0' -define-availability 'SwiftStdlib 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5' -define-availability 'SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -define-availability 'SwiftStdlib 5.6:macOS 12.3, iOS 15.4, watchOS 8.5, tvOS 15.4' -define-availability 'SwiftStdlib 5.7:macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0' -define-availability 'SwiftStdlib 5.8:macOS 13.3, iOS 16.4, watchOS 9.4, tvOS 16.4' -define-availability 'SwiftStdlib 5.9:macOS 14.0, iOS 17.0, watchOS 10.0, tvOS 17.0' -define-availability 'SwiftStdlib 5.10:macOS 14.4, iOS 17.4, watchOS 10.4, tvOS 17.4, visionOS 1.1' -define-availability 'SwiftStdlib 6.0:macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0' -define-availability 'SwiftStdlib 6.1:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999, visionOS 9999' -typo-correction-limit 10 -scan-dependencies -module-name Test -module-cache-path /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/clang-module-cache -O -disable-implicit-string-processing-module-import -disable-implicit-concurrency-module-import /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/test.swift -o /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/deps.json -swift-version 5 -cache-compile-job -cas-path /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/cas -Xcc -D_VERSION=1 -Xcc -fmodule-map-file=/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/include/module.modulemap -Xcc -ivfsoverlay -Xcc /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/empty.yaml -Xcc -I/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/empty.hmap -module-load-mode prefer-serialized
: 'RUN: at line 10'; /usr/bin/python3.12 /home/finagolfin/swift/test/CAS/Inputs/BuildCommandExtractor.py /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/deps.json clang:SwiftShims > /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/shims.cmd
: 'RUN: at line 11'; '/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/bin/swift-frontend' @/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/shims.cmd
: 'RUN: at line 13'; /usr/bin/python3.12 /home/finagolfin/swift/test/CAS/Inputs/BuildCommandExtractor.py /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/deps.json clang:_Macro > /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/Macro.cmd
: 'RUN: at line 14'; '/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/bin/swift-frontend' @/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/Macro.cmd
: 'RUN: at line 16'; /usr/bin/python3.12 /home/finagolfin/swift/test/CAS/Inputs/GenerateExplicitModuleMap.py /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/deps.json > /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/map.json
: 'RUN: at line 17'; llvm-cas --cas /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/cas --make-blob --data /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/map.json > /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/map.casid
: 'RUN: at line 19'; /usr/bin/python3.12 /home/finagolfin/swift/test/CAS/Inputs/BuildCommandExtractor.py /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/deps.json Test > /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/MyApp.cmd
: 'RUN: at line 20'; /usr/bin/python3.12 /home/finagolfin/swift/utils/PathSanitizingFileCheck --allow-unused-prefixes --sanitize BUILD_DIR=/home/finagolfin/build/Ninja-Release/swift-linux-x86_64 --sanitize SOURCE_DIR=/home/finagolfin/swift --use-filecheck /home/finagolfin/build/Ninja-Release/llvm-linux-x86_64/bin/FileCheck --color /home/finagolfin/swift/test/CAS/Xcc_args.swift --input-file=/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/MyApp.cmd
: 'RUN: at line 28'; /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/bin/swift-frontend -target aarch64-unknown-linux-android -sdk /home/finagolfin/android-ndk-r27/toolchains/llvm/prebuilt/linux-x86_64/sysroot -resource-dir /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/lib/swift -module-cache-path /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/swift-test-results/aarch64-unknown-linux-android/clang-module-cache -swift-version 4 -define-availability 'SwiftStdlib 9999:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999' -define-availability 'SwiftStdlib 5.0:macOS 10.14.4, iOS 12.2, watchOS 5.2, tvOS 12.2' -define-availability 'SwiftStdlib 5.1:macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0' -define-availability 'SwiftStdlib 5.2:macOS 10.15.4, iOS 13.4, watchOS 6.2, tvOS 13.4' -define-availability 'SwiftStdlib 5.3:macOS 11.0, iOS 14.0, watchOS 7.0, tvOS 14.0' -define-availability 'SwiftStdlib 5.4:macOS 11.3, iOS 14.5, watchOS 7.4, tvOS 14.5' -define-availability 'SwiftStdlib 5.5:macOS 12.0, iOS 15.0, watchOS 8.0, tvOS 15.0' -define-availability 'SwiftStdlib 5.6:macOS 12.3, iOS 15.4, watchOS 8.5, tvOS 15.4' -define-availability 'SwiftStdlib 5.7:macOS 13.0, iOS 16.0, watchOS 9.0, tvOS 16.0' -define-availability 'SwiftStdlib 5.8:macOS 13.3, iOS 16.4, watchOS 9.4, tvOS 16.4' -define-availability 'SwiftStdlib 5.9:macOS 14.0, iOS 17.0, watchOS 10.0, tvOS 17.0' -define-availability 'SwiftStdlib 5.10:macOS 14.4, iOS 17.4, watchOS 10.4, tvOS 17.4, visionOS 1.1' -define-availability 'SwiftStdlib 6.0:macOS 15.0, iOS 18.0, watchOS 11.0, tvOS 18.0, visionOS 2.0' -define-availability 'SwiftStdlib 6.1:macOS 9999, iOS 9999, watchOS 9999, tvOS 9999, visionOS 9999' -typo-correction-limit 10 -typecheck -cache-compile-job -cas-path /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/cas -swift-version 5 -disable-implicit-swift-modules -disable-implicit-string-processing-module-import -disable-implicit-concurrency-module-import -module-name Test -explicit-swift-module-map-file @/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/map.casid /home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/test.swift @/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/test-android-aarch64/CAS/Output/Xcc_args.swift.tmp/MyApp.cmd
--
Exit Code: 1
Command Output (stderr):
--
<unknown>:0: error: builtin headers belong to system modules, and _Builtin_ modules are ignored for cstdlib headers was enabled in PCH file but is currently disabled
<unknown>:0: error: module file SwiftShims.pcm cannot be loaded due to a configuration mismatch with the current compilation
<unknown>:0: error: missing required module 'SwiftShims'
--
]]></failure>
</testcase>
Those CAS tests pass when natively compiled on my Android phone, where I built the July 15 trunk source snapshot last month and only 20 tests failed there too. That was a larger subset of tests though, as the 1k+ executable tests were also run there.
Three tests failed in both test runs, test/DebugInfo/enum.swift, test/Interop/Cxx/class/closure-thunk-irgen.swift, and test/Interop/Cxx/class/function-call-irgen.swift, while three more failed only when cross-compiled, test/IRGen/lto_autolink.swift, test/SymbolGraph/Symbols/Kinds/DistributedActor.swift, and test/Interop/Cxx/stdlib/android-and-std-module.swift. The first two of the latter triad probably pass in the wrong flags when cross-compiling, but haven't seen this error before from cross-compiling android-and-std-module.swift:
Command Output (stderr):
--
<unknown>:0: error: cannot open file '/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/lib/swift/shims/module.modulemap': Too many open files
<module-includes>:1:10: note: in file included from <module-includes>:1:
1 | #include "__std_clang_module"
| `- note: in file included from <module-includes>:1:
2 |
<unknown>:0: error: cannot open file '/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/lib/swift/shims/module.modulemap': Too many open files
<unknown>:0: error: cannot open file '/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/lib/swift/shims/module.modulemap': Too many open files
<unknown>:0: error: cannot open file '/home/finagolfin/build/Ninja-Release/swift-linux-x86_64/lib/swift/shims/module.modulemap': Too many open files
/home/finagolfin/android-ndk-r27/toolchains/llvm/prebuilt/linux-x86_64/sysroot/usr/include/c++/v1/__std_clang_module:157:12: error: could not build module 'std_semaphore'
155 | #include <scoped_allocator>
156 | #if !defined(_LIBCPP_HAS_NO_THREADS)
157 | # include <semaphore>
| `- error: could not build module 'std_semaphore'
158 | #endif
159 | #include <set>
@Alex_L, are you seeing similar test results when cross-compiling the compiler validation suite for Android? I should note that these are the least number of failing tests I've seen for Android in years, largely because of Alex's work modularizing the Android modulemap, which fixed dozens of C++ Interop tests on Android that I've now enabled in the compiler validation suite again.
I plan to fix or disable those 20 failing cross-compiled tests and submit Android to the official CI, then get the Android emulator running in that CI and after doing the same for the remaining 17 failing executable tests, getting the 1k+ executable tests running in the emulator too in the coming months.
4 Likes