Since you're passing those in extra-swiftc-flags, is there any downside to passing it in the newly proposed extraSwiftCFlags? The only difference is capitalization of the dictionary key really.
Yes, we could keep the Swift resource directory configured in the same way, but your sdkRootDir would need to be modified to not just handle relative paths in the bundle but also an absolute path to an external C sysroot, or at least be completely overridden by a -sdk absolute path passed into extraSwiftCFlags.
I agree that could be an option, yes. But if we continue to do that, it wouldn’t feel like the kind of “cross-compilation feature to rule them all” that I’d hope for, given the relatively large (“major version bump”) overhaul this pitch is presenting.
Edit: actually I’m not sure if this is a fair assessment. I think it could still be a major improvement, but it still seems like we’d be unable to use this with a simple swift build —destination android-arm64, for example, without some customisation of the artifact bundle based on eg the NDK path, either manually or via a script. My bar for being truly +1 on this proposal would be that the above just works.
Is there some kind of tool (similar to xcrun for example) that would allow us to automatically discover the NDK path? Otherwise, I'm not sure how to make something "just work" with an arbitrary absolute filesystem path that we don't control and don't know about prior to bundle installation.
I applaud support for cross-compiling to platforms like WASM and Android. I do want to chime in with embedded (no-OS or Zephyr) setups as well.
1 Like
Would it be reasonable for an installer of the artefact bundle to point to the NDK path during setup?
Alternatively the bundle could point to a standardised path like ~/NDK and the dev must provide a symlink at that location?
1 Like
Is there some specific feedback you'd like to provide WRT to these platforms? Since these are currently not supported by Swift, I'm not sure I'd be able to address that I'm afraid.
Is there some kind of tool (similar to
xcrunfor example) that would allow us to automatically discover the NDK path?
No.
Otherwise, I'm not sure how to make something "just work" with an arbitrary absolute filesystem path that we don't control and don't know about prior to bundle installation.
I think it is too ambitious to go from absolute paths for the sdk and toolchain so far to assuming that all platform sysroots/tools can be redistributed as cross-compilation bundles, ie using only relative paths in your proposed JSON format.
My suggestion is to give bundlers the option of specifying all three relative paths as nil or some such token and require that the user pass in those absolute paths manually, ie if the sdk and toolchain are set to nil, the user must also pass in --sdk and --toolchain flags when installing the bundle, after which it uses those paths from then on, or it errors and prompts the user to specify those paths.
Thanks, I was leaning in this direction myself before you posted this answer. Sounds good!
1 Like
To clarify a bit more, it looks like for Android the toolchain-bin-dir path and path passed as -tools-directory are different. Could you clarify what exactly the latter is expected to contain when cross-compiling to Android? Do you need clang, linker, or anything else from the Android NDK for things to work? What's the reason for preferring those to the tools present in the Swift toolchain?
I know they're not officially supported - although you can get a patched toolchain working on it of course. But I think that the infrastructure you propose could make it easier for such patches to live outside and/or alongside of the official releases and improve the ease of adopting new toolchains by end-users by a lot.
I think that the ZephyrOS toolchain, WASM toolchain and Android toolchains are some great examples. I also think that no-OS toolchains, which have been discussed and even seen working (by @compnerd ) are possibilities.
I know it's hard to officially support these kinds of use cases, but I think this could lead to reduced friction when people try to improve upon the existing Swift ecosystem. Whether it gets merged in main or not.
To clarify a bit more, it looks like for Android the
toolchain-bin-dirpath and path passed as-tools-directoryare different.
It is not particularly important for this new format, as I pass the latter in through extra swiftc flags, which you are keeping the same in your new JSON format.
Could you clarify what exactly the latter is expected to contain when cross-compiling to Android? Do you need clang, linker, or anything else from the Android NDK for things to work? What's the reason for preferring those to the tools present in the Swift toolchain?
There was some issue back when I first distributed my Android SDK a couple years ago, where linking wasn't working with the Swift clang or linker, so I switched to the NDK version instead, as those tools are patched slightly differently for Android. I don't remember the exact issue and it is possible that flag is no longer needed, as some of those patches have since been upstreamed.
-tools-directory path is absolute in your configuration though, pointing to a directory within the Android NDK. Thus it isn't a non-escaping relative path within a destination bundle. Looks like we need to introduce a separate JSON property and a destination install option to override that.
-tools-directorypath is absolute in your configuration though, pointing to a directory within the Android NDK. Thus it isn't a non-escaping relative path within a destination bundle.
Yep.
Looks like we need to introduce a separate JSON property and a
destination installoption to override that.
Maybe it would be best to just have a way to edit or override specific properties of the destination JSON file that comes with a cross-compilation bundle, ie either let users edit the one they downloaded or add their own local JSON that overrides the properties they want from the bundle, eg to add absolute paths only they know?
That may prove more usable than trying to anticipate every extra clang/swiftc flag that requires an absolute path and adding all those into the JSON format as separate properties now.
1 Like
I think having a local/override destination JSON file seems like a good idea.
1 Like
An overrides mechanism is what I'm prototyping right now. Want to see whether we can do it nicely as an overriding CLI option or we need a way to override the whole JSON altogether.
1 Like
An updated version of the pitch has been published, which I hope addresses most if not all of the recent feedback. I've updated nomenclature to use "build-time triple" and "run-time triple" instead of "host target" and "target platform", as we think that "target" is too ambiguous in the context of the pitch, especially as it already mentions SwiftPM binary targets multiple times.
Properties of destination.json file have been reworked to support arrays of paths, together with a new swiftResourcesPaths property.
Additionally, in this version we're introducing a concept of toolset files, which are somewhat inspired by CMake toolchain files. We also propose a new swift destination configure subcommand for providing toolsets, header search paths, and library search paths that may come outside of a destination bundles.
Many thanks to everyone providing feedback and suggestions for this pitch! Specifically, @etcwilde and @compnerd came up with ideas for specifying search paths as arrays and toolsets as separate files, and are now credited as co-authors.
3 Likes
This has moved to the proposal stage, here's a link to the review thread: SE-0387: Cross-Compilation Destination Bundles
2 Likes
I'm a bit late, but I have got to say that I find the new naming extremely confusing since the entire process of cross-compilation is happening at build-time, so either triple is technically a "build-time triple". In fact, the only things running (if we don't take into account testing which the proposal isn't covering) are on the host, so technically the triples used for choosing host tools would be more appropriately named "run-time triples".
2 Likes
I personally think "host"and "target" are still the clearest terminology, the "host' is where the build runs (eg localhost) and the "target" is the platform you are building for.
8 Likes