Re-examining meaning of global options in cross-compilation contexts

Development Package Manager
1

Re-examining meaning of global options in cross-compilation contexts

Since the host/target split is new for SwiftPM I think it's a good time to re-visit meaning and structure of affected command options such as --sdk , --toolchain , --triple and various -X options. This post is also inspired by plugins: Global compiler flags should not be used for plugin dependencies by euanh · Pull Request #7549 · apple/swift-package-manager · GitHub (authored by euanh), which was aimed at fixing a case hit when attempting to cross compile.

First regarding terminology - I use "host" and "target" platforms because that's the terminology that has been agreed upon during the review of the cross-compilation sdk proposal. In particular the following sentence from the acceptance announcement:

We elected to use host and target when specifying cross compilation and host, build and target when using Canadian Cross.

Cross compilation complicates the existing options - a module can now be built for both the host and the target depending on the context, completely transparently to the user. For example, a target could have a macro dependency which would mean that it has to be built for the host. Some other non-macro/plugin module could depend on that target at the same time, so it would then have to be built for the target platform as well. Note that plugins, macros, and test targets that directly depend on macros, are always built for the host.

Let’s start with -X options. Currently only Swift language options support host/target separation by means of -Xbuild-tools-swiftc prefix although how that actually behaves is not well-documented at the moment. Currently -Xswiftc applies to all builds and -Xbuild-tools-swiftc applies to manifest, plugins, but not macros. At the same time, such capability doesn't exist at all for cc , cxx, and linker flags.

--sdk/--toolchain options were originally added as an early form of cross-compilation and were hidden from the help output. When switching to swift-argument-parser, that was no longer the case (no hidden was added), but they still have no documentation. Both only apply to target.

--triple applies to the target only and can be useful in cross-compiling scenarios where a separate SDK/toolchain isn’t needed.

I’d like to propose leaving --sdk/--toolchain as is for now (and perhaps deprecating for --swift-sdk in the future), --triple being renamed to --target-triple (deprecating --triple), and renaming the other options through either:

  1. -X{cc, cxx, swiftc, linker} are applicable to all contexts
    1. -Xhost-{cc, cxx, swiftc, linker} , only applicable to host platform related builds (including macros) and override -X options.
    2. -Xtarget-{cc, cxx, swiftc, linker}, only applicable to target platform related builds (not macros) and override -X options.
  2. -X{cc, cxx, swiftc, linker} are applicable only to target context (not macros)
    1. -Xhost-{cc, cxx, swiftc, linker} could be used to specify host specific values if required (including macros).
    2. Optional: Deprecate -X and --triple in favor of verbose -Xtarget-{cc, cxx, swiftc, linker}.

With my preference being towards option 1, since it seems like the most intuitive and backward compatible approach.

Please share your thoughts!

8 Likes
2

option 1 absolutely! this would be so so so so helpful for embedded swift

2 Likes
3

I really like option 1) for how understandable it is. The host and target prefixing makes a lot of sense and is pretty intuitive.

In general I like the hierarchy it introduces in "what is affected", and option 2 is somewhat fuzzy still tbh, I'm pretty sure I'd forget that -X (in option 2) does NOT affect host context, it seems to me like "it obviously should" and then I'd have to remember it doesn't :sweat_smile:

2 Likes
4

One thing I've been looking at is how we can support multiple targets in a build. I can imagine some interesting Swift Everywhere architectures that involve building a distributed system across desktop, web, server, and embedded with libraries shared between those platforms. If you make changes to those libraries, you would want to build for everything that uses it to check for errors.

To support this, I'm not sure these command line switches make sense. I think they need to be brought into the manifest or some other build config file or make it easier to make SDKs that can be referenced by the manifests. More investigation and design to be done to make sure I'm not over thinking things. But I'd like to keep those doors open for now.

1 Like
5

Only seeing this thread now that it's recently discussed again, but I definitely think this should be fixed, as I pointed out that this seemed to be a problem with Rust macros that Swift should try to avoid years ago. Either option seems fine to me, as SwiftPM only started building so much code for the host alongside cross-compilation recently, so you could argue 2. is more intuitive because most people only expected these flags to apply to the target.

You'd probably want to use SDK bundles for that? One deficiency with SwiftPM is that it seems to support building for multiple architectures with one invocation when targeting macOS, but not for other target platforms.

Not sure what you're suggesting here, that we remove these CLI flags altogether or change them to support multiple targets also. The benefit of CLI flags is that they allow quickly trying or overriding flags without pulling in more "build config files" and other heavier config. Multiple targets seem such a specialized use case that I'm not sure it needs CLI support yet, while these flags are definitely useful for the much more common host and single cross-compilation target scenarios.

6

Not sure what you're suggesting here...

I think I covered that in my last line

More investigation and design to be done to make sure I'm not over thinking things.

Multi-target is going to make things more complex and I'm not sure the command line args are the right approach. But I'm not sure yet.

That said, we are running into issues where applying -Xcc to all targets is causing problems in cross scenarios. So we may need to do something in the short term. That's covered here: Swift on STM32C011

7

If I can add a +1...I like option #1 from here as well. It is pretty clear and should not cause too much additional confusion. Do we need to come up with a proposal for this change?