@Max_Desiatov currently for our Swift Android setup we must specify sdkRoot as the NDK’s sysroot. That path is not “owned” by us so there is no possibility to put the swift libs etc in there.
It’s possible we can work out how to do this another way but I couldn’t get things compiling without specifying the sdkRoot as the “C” sysroot inside the NDK, and then specifying resourceDir as the Swift “sysroot” (folder containing /usr/lib/swift/…)
I would prefer that these were called “sysroot” and “sdkRoot” respectively (i.e. the other way around, and removing the resourceDir concept entirely), but the compiler doesn’t work that way at the moment.
I know this works a bit differently for SwiftWasm, which is great, but I don’t know if that setup is generalisable. That’s probably because SwiftWasm “owns” the entire toolchain, ie. the sdkRoot, which in the case of SwiftWasm also contains the C sysroot.
A part of the problem is probably the difficulties with relative paths in clang modules - we can’t really set a base path for the Libc modulemaps, so it must be specified either relative to the modulemap file itself or as an absolute path. Unless we dynamically generate those modulemaps (which has its own problems regarding build system dependency caching) we do need a way to specify two base paths I think.
Is it because Android NDK is installed elsewhere on the system as a separate "package"? Would it be possible to include the NDK as a part of a destination bundle as a workaround? I imagine specific versions of Swift toolchain and SDK are tied to specific Android NDK versions, so wouldn't it be preferable to "vendor" the latter in the bundle if corresponding licenses allow that?
The thing is, the NDK is a system package installed in a standard location by eg. Android Studio and/or by developers themselves via an installer bundle (which does not allow customising the install path). It ends up in a standardised location on the dev’s system.
AFAIK it’s important for the same NDK version to be used when building various native dependencies, so it makes sense to be in the spot Android Studio etc. expects it, to avoid duplicates and incompatibilities.
I’m also not sure whether licensing would allow us to distribute a version nested inside another toolchain. I’ve never heard of this happening for other programming languages. Instead, my experience has always been that they require the dev to point to a pre-installed NDK path via a command line flag or environment variable. Would we require devs to have two copies of the NDK in the case where they also have e.g. Rust deps? The NDK is ~1GB when compressed, so that wouldn’t be ideal.
In short, that might be a solution, but I’m not certain it’s a good one. It seems to pushing some complexity down the chain a little bit, and it would work against precedent from other langs.
Maybe there’s scope to just tease out the headers and libraries from the sysroot (ie. without all the clang binaries etc). That might reduce the amount of duplication in terms of file size but I’m not sure it’s a good idea in terms of correctness.
Leaving aside the Android NDK, there certainly exist proprietary sysroots that we cannot redistribute, so I hope this pitch will maintain working with those in arbitrary locations, perhaps by providing a way for the programmer to manually specify that absolute path for a sysroot when setting up an otherwise complete destination bundle.
One question: what does the newly added runtimeDir in your new JSON format refer to in terms of existing Swift compiler flags, something like the -resource-dir flag but with the platform appended, ie <resourceDir>/<OS>/<arch>? If so, we could use that info to configure the compiler to use a non-overlapping C sysroot and Swift resource directory, as @kateinoigakukun and @Geordie_J want.
This is an "extension point" for future directions, a directory for tools and resources needed not to build, but to run cross-compiled binaries. When in a some future pitch we reach a consensus how swift run and swift test work with destinations, runtimeDir could enable support for remote running and remote testing.
OK, I see that you talked about runtimeDir last month, which I had forgotten about.
My only criticism of this pitch is the same as theirs: it seems to assume that we can always redistribute platform C sysroots, ie your sdkRootDir, whereas my understanding is that is sometimes not an option.
Since you're passing those in extra-swiftc-flags, is there any downside to passing it in the newly proposed extraSwiftCFlags? The only difference is capitalization of the dictionary key really.
Yes, we could keep the Swift resource directory configured in the same way, but your sdkRootDir would need to be modified to not just handle relative paths in the bundle but also an absolute path to an external C sysroot, or at least be completely overridden by a -sdk absolute path passed into extraSwiftCFlags.
I agree that could be an option, yes. But if we continue to do that, it wouldn’t feel like the kind of “cross-compilation feature to rule them all” that I’d hope for, given the relatively large (“major version bump”) overhaul this pitch is presenting.
Edit: actually I’m not sure if this is a fair assessment. I think it could still be a major improvement, but it still seems like we’d be unable to use this with a simple swift build —destination android-arm64, for example, without some customisation of the artifact bundle based on eg the NDK path, either manually or via a script. My bar for being truly +1 on this proposal would be that the above just works.
Is there some kind of tool (similar to xcrun for example) that would allow us to automatically discover the NDK path? Otherwise, I'm not sure how to make something "just work" with an arbitrary absolute filesystem path that we don't control and don't know about prior to bundle installation.
Is there some specific feedback you'd like to provide WRT to these platforms? Since these are currently not supported by Swift, I'm not sure I'd be able to address that I'm afraid.
Is there some kind of tool (similar to xcrun for example) that would allow us to automatically discover the NDK path?
No.
Otherwise, I'm not sure how to make something "just work" with an arbitrary absolute filesystem path that we don't control and don't know about prior to bundle installation.
I think it is too ambitious to go from absolute paths for the sdk and toolchain so far to assuming that all platform sysroots/tools can be redistributed as cross-compilation bundles, ie using only relative paths in your proposed JSON format.
My suggestion is to give bundlers the option of specifying all three relative paths as nil or some such token and require that the user pass in those absolute paths manually, ie if the sdk and toolchain are set to nil, the user must also pass in --sdk and --toolchain flags when installing the bundle, after which it uses those paths from then on, or it errors and prompts the user to specify those paths.
To clarify a bit more, it looks like for Android the toolchain-bin-dir path and path passed as -tools-directory are different. Could you clarify what exactly the latter is expected to contain when cross-compiling to Android? Do you need clang, linker, or anything else from the Android NDK for things to work? What's the reason for preferring those to the tools present in the Swift toolchain?
I know they're not officially supported - although you can get a patched toolchain working on it of course. But I think that the infrastructure you propose could make it easier for such patches to live outside and/or alongside of the official releases and improve the ease of adopting new toolchains by end-users by a lot.
I think that the ZephyrOS toolchain, WASM toolchain and Android toolchains are some great examples. I also think that no-OS toolchains, which have been discussed and even seen working (by @compnerd ) are possibilities.
I know it's hard to officially support these kinds of use cases, but I think this could lead to reduced friction when people try to improve upon the existing Swift ecosystem. Whether it gets merged in main or not.
To clarify a bit more, it looks like for Android the toolchain-bin-dir path and path passed as -tools-directory are different.
It is not particularly important for this new format, as I pass the latter in through extra swiftc flags, which you are keeping the same in your new JSON format.
Could you clarify what exactly the latter is expected to contain when cross-compiling to Android? Do you need clang, linker, or anything else from the Android NDK for things to work? What's the reason for preferring those to the tools present in the Swift toolchain?
There was some issue back when I first distributed my Android SDK a couple years ago, where linking wasn't working with the Swift clang or linker, so I switched to the NDK version instead, as those tools are patched slightly differently for Android. I don't remember the exact issue and it is possible that flag is no longer needed, as some of those patches have since been upstreamed.
-tools-directory path is absolute in your configuration though, pointing to a directory within the Android NDK. Thus it isn't a non-escaping relative path within a destination bundle. Looks like we need to introduce a separate JSON property and a destination install option to override that.