Toolchain Management / Hygiene in multi-toolchain / multi-sdk environments

Using Swift
, , ,
1

Hi all,

Over the past couple of days I've been struggling with some toolchain... confusion? chicanery? mazurkas? ... in my local environment, and I think it's highlighting a disconnect between my understanding of how the toolchains and SDKs and environment "managers" (swiftly vs XCode, quotes for the latter) relate to are isolated from one another, and the consistency of this relationship across different platforms.

For context, my desktop environment is a mac running macos 26.6. It has the following toolchains installed:

  • XCode17 beta last updated on 22 July
  • The most recent XCode16
  • swiftly 1.1.3 and the open-source toolchains:
    1. 6.3.3
    2. 6.3.0
    3. 6.2.4

I'm also cross compiling, targeting a raspi, so I have a 6.3.3-RELEASE_debian_bookworm_aarch64 SDK installed for that purpose. The overall matrix of combos I'm trying is something like:

Host OS Target SDK Toolchain Manager Result Notes
MacOS MacOS default for tools 6.4.X XCode :white_check_mark:
MacOS Rasp (Debian 12) 6.3.3-RELEASE_debian_bookworm_aarch64 6.3.3 swiftly :white_check_mark:
Linux (Docker) Linux(Docker) default for tools 6.3.3 Provided in image (swift:6.3-noble) :white_check_mark:
MacOS MacOS ? 6.3.3 swiftly :cross_mark: See Below

So, what I'm seeing is success across all of those combinations, except for when I attempt to use the swiftly-provided open-source 6.3.3 MacOS toolchain. The specific failure that I see is in swift-crypto, which was originally filed as #449: CryptoExtras: SHA512256Digest fails ContiguousBytes conformance on the macOS 27 SDK (MemberImportVisibility + missing import Foundation) (which I missed when I duped it in #451.

I understand the differences between the toolchains and SDKs. I also understand the legacy of XCode bundling toolchains, and xcode-select managing the blessing of bundled toolchains to be used as the system toolchain. What surprised me was that the failure I saw in the swift-crypto build related to a missing protocol conformance requirement (the withBytes(…) method of ContiguousBytes. That declaration is marked "beta" in the docs, so I assume it's new for 6.4. For the build that failed, I was trying to take special care that the toolchain is not contaminated with any 6.4 bits, and that I was using clean, open-source 6.3.3 soup-to-nuts:

  • I was using the 6.3.3 toolchain installed via swiftly, with 6.3.3 configured in my .swift-version in the associated project. 6.3.3 was also the global default.
  • My Package.swift is configured for swift-tools-version 6.3.
  • While I had compiled the tree using the xcode-supplied toolchain, I've never opened the project with Xcode. It's only ever been opened or edited with vi or Zed.
  • I completely blew away my .build/ and Package.resolved before building.
  • I compiled using swiftly run swift build.

My expectation, was that aside from the tools running on macos, I'd be using a consistent 6.3.3 toolchain/sdk combo analgous to what's used in the Docker build. Based upon what I saw, the only explanation that came to mind was that somehow the 6.4 sdk was leaking into the build environment.

My confusion is probably mine alone, so if you've read this far and come to that conclusion, then we're on the same page, and I just want to understand this better to manage my own expectations. I haven't found anything so far that spells this out, but I suspect that's just because I haven't dug deeply enough. I also haven't found any tools beyond what's mentioned above to help diagnose these configurations.

As I mentioned in #449, I'd hoped that by writing this and making myself actually think, I'd answer my own question, but doesn't seem I have. So, clicking the button. I'd appreciate any tips or resources that come to mind.

Thanks!

And thanks especially to the folks who have been fielding my questions/reports elsewhere. Your exemplary bedside manner is noted.

1 Like
2

Ok. The awesome @lukasa has answered my original question far more thoroughly than I expected, in this response to the aforementioned #449 on swift-crypto. I'm not going to quote the whole thing here, as it's responding to a question that was formulated a bit differently than the above, but it's a really helpful read if one needs clarification like I did.

The net-net for the failure I had, however, was that for targeting Apple platforms, the SDK is always provided by the currently-selected Xcode. So, while the following results in the breakage I note above:

$ sudo xcode-select -s /Applcations/Xcode-beta.app
$ swiftly use 6.3.3
$ swiftly run swift build

updating the Xcode pointer picks up the correct SDK version.

$ sudo xcode-select -s /Applcations/Xcode-beta.app
$ swiftly run swift build # still using 6.3.3

And that does indeed fix my build.

But definitely read the response, as it contains a ton of helpful detail.

1 Like
3

I believe Swift requires the compiler and SDK to "align". Having a compiler with a "newer" SDK is an unsupported configuration. I raised a couple issues recently to try and make this "misalignment" more apparent.

1 Like
4

@bkhouri yup, the issue comment I linked to above goes into this in great detail. The issues you filed, if accepted, would be a big help in clarifying this going forward.

TBH my confusion was less about sdk/compiler alignment, which I expected since there's only so much that can be done with conditional compilation when both language and API evolve, especially with an expressive language like Swift.

My mistake was I expected that when installing, say, 6.3.3 via swiftly, that there was a complete, isolated, parallel open-source toolchain and sdk installed, separately from what XCode provides, which is of course incorrect.

5

I think you're mixing up SDK and Swift SDKs here, which are distinct concepts that operate on different levels and are implemented on different levels. swift build --sdk and swift build --swift-sdk are separate distinct flags and features. -sdk is a Swift Driver/Clang feature that swift build historically happens to support, while Swift SDK is an exclusively SwiftPM feature. 6.3.3-RELEASE_debian_bookworm_aarch64 is a Swift SDK, not "SDK". Mixing these up only leads to confusion and build errors.

As mentioned, both Swift SDK and SDK versions need to exactly match the toolchain version to be usable. Note that Xcode Swift toolchain and swift.org toolchains are always of versions distinct from each other:

  • swift.org toolchains use 3-component semver for identification for releases (e.g. swift-6.3.2-RELEASE) or dated snapshots matching Git tags visible at Tags · swiftlang/swift · GitHub.
  • Xcode Swift toolchains use 5-component version, e.g. swiftlang-6.3.3.1.3 that don't have corresponding tags on open source repositories, since Xcode and its components are closed source.

As for Swift 6.3, Xcode Swift toolchains are not suitable for cross-compilation to non-Darwin platforms. When cross-compiling to non-Darwin you have to ensure that you've selected swift.org toolchain and have exactly matching Swift SDK version. As far as I know, there are no SDKs generally available that allow cross-compiling to non-Darwin platforms with swift build, you need a Swift SDK for that.

A table I would use to explain it all would look like this:

versioning convention version example SDKs Swift SDKs
swift.org toolchain 3 component semver or dated snapshots swift-6.3.3-RELEASE Supplied by Xcode, only usable for macOS Available for non-Darwin platforms
Xcode toolchain 5 components swiftlang-6.3.3.1.3 Supplied by Xcode, usable for Darwin platforms Generally unavailable

TL;DR summary

  • If you target Darwin platforms: use Xcode and Xcode toolchains and SDKs.
  • If you target non-Darwin platforms: use swift.org toolchains and Swift SDKs.
  • SDK and Swift SDK versions need to exactly match the corresponding toolchain version that you're using.
2 Likes
6

Thanks @Max_Desiatov . Looks like this was more conflated in my mind than I thought, but in my build scripts, where I've mentioned "SDK", I've been referring to the argument to --swift-sdk. I had forgotten about the distinct --sdk param, but I've just been adapting the commands I've found on posts from those more knowledgable than me, so I can't take credit for getting that part right when I did.

With the context I have now, I'm am seeing bits and pieces of it in various places (for example in the Prerequisites section of Getting Started with the Static Linux SDK ). In cramming a bunch of stuff together (doing something server-side with swift, targeting linux, cross-compiling), I clearly missed some important details. That things kinda worked until they didn't was a bummer. But also, probably not a common use case.

Just a guess, but I'll bet that this info as a standalone doc would be helpful to a more general audience. Again, a lot of it is at least hinted at in GSWTSLSdk, but starting as I did with web searches for cross-compiling for a raspberry pi and bootstrapping from the examples I saw, I didn't read that until a bit later in the game. I did quickly end up at the Swiftly docs, so a See Also there would probably be a discoverable spot.

But again, thank you and everyone else for the details. Super, super helpful.

1 Like