I have a CLI tool I'm writing in Swift, and I want to ship builds for macOS and Linux on several different architectures.
It looks like one way to build for Linux is swift sdk install. Selecting this SDK seems to build some of the objects, but then it gives an error:
/Users/awright/tool/main.swift:1:8: error: compiled module was created by a different version of the compiler ''; rebuild 'Foundation' and try again: /Users/awright/Library/org.swift.swiftpm/swift-sdks/swift-6.2.1-RELEASE_static-linux-0.0.1.artifactbundle/swift-6.2.1-RELEASE_static-linux-0.0.1/swift-linux-musl/musl-1.2.5.sdk/x86_64/usr/lib/swift_static/linux-static/Foundation.swiftmodule/x86_64-swift-linux-musl.swiftmodule
1 | import Foundation;
| `- error: compiled module was created by a different version of the compiler ''; rebuild 'Foundation' and try again: /Users/awright/Library/org.swift.swiftpm/swift-sdks/swift-6.2.1-RELEASE_static-linux-0.0.1.artifactbundle/swift-6.2.1-RELEASE_static-linux-0.0.1/swift-linux-musl/musl-1.2.5.sdk/x86_64/usr/lib/swift_static/linux-static/Foundation.swiftmodule/x86_64-swift-linux-musl.swiftmodule
2 | import FSM;
3 |
First, shouldn't swift build just do it? I'm not sure how I rebuild Foundation. Why is that my job?
Second, it looks like this technique only produces builds for amd64/x86_64. How do I also build for arm64?
you’re probably using the swift toolchain that comes with xcode on macOS. for cross-compiling you need to install the open source toolchain that matches your linux sdk (in this case 6.2.1) from Install Swift | Swift.org
for building for arm64 architecture, I think you need to build your own SDK. I suggest checking out this blog post that walks you through generating the SDK for your specific target
So what is meant by "toolchain" here? Normally I would run different toolchains for each combination of programming language and target architecture. e.g. I run cc-i386-linux and it will build x86 code from C. What is Swift doing differently?
Ideally, I would just like to install swiftc-amd64-linux, swiftc-arm64-linux, swiftc-macos and then run SWIFTC=swiftc-amd64-linux make and have that just work.
The model you've described is something that GCC uses. Swift, same as Clang, doesn't require you to have different copies of the compiler for each architecture. After all the parser, the type checker, the platform-independent parts of the optimizer are all the same for all (non-Darwin) platforms, so why would you want copies of these components duplicated in hypothetical swiftc-amd64-linux and swiftc-arm64-linux?
What is different for all of those platforms is the standard library, libc, and other core libraries. Those are bundled in Swift SDKs that you install for the platform you need to cross-compile to. Artifact bundles can contain multiple Swift SDKs to support multiple platforms that can be installed with a single command. By using swift build from swift.org and a corresponding Swift SDK it will "just work".
Why wouldn't two different toolchains be able to share resources? Obviously you can share header files, parsers, utilities, etc.
My question is why can't I do this in Swift? The current documentation has essentially no guidance on how to produce multiple builds of a utility for different architectures. So I'm requesting more comprehensive documentation on this subject.
They would be able to share headers etc, but code in these headers not stripped away by optimizations would be duplicated in each binary using those headers. Then why not a single compiler binary that avoids this duplication in the first place and can cross-compile to any supported architecture, allowing users to select an architecture via a CLI option instead?
Have you read the official instructions for the Static Linux SDK that you linked above? It specifically shows you how to build a command-line executable using that SDK, even showing you how to build for different architectures, and says that you cannot use the Swift toolchain from Xcode.
I suggest you read it through carefully, and let us know if you are still having problems with it. It may seem a bit long-winded, but it has most of the info you need.
Sorry if I'm a little bit scattered, but there's multiple issues that are confounding and so things aren't always entirely clear. I think some minor improvements to the tooling and documentation would go a long way.
And Swift is still buggy in many parts and that doesn't help. Swift 6.2.1 won't compile my code in release mode, and I'm not sure when this fix is going to be merged.
On a Linux host, I've been able to get swift build to work and swift build --swift-sdk x86_64-swift-linux-musl but sometimes I can't switch between the two. If I try running both, I get an error until I delete the whole .build directory and try again. Swift is definitely not tracking all of the build dependencies somewhere. But sometimes it works and I can compile both!
Isn't that what shared objects are for? And I need some way to install new architectures. Eventually I will want to target wasm, for example. Or bare-metal code, like a bootloader. I like the LLVM toolchain but Swift seems to abstract it all away and now I have no clue what I'm doing.
First, it's still not clear how I build for another architecture. How do I build for i386, armv6 (e.g. Raspberry Pi), or arm64? How do I know what architectures are supported targets? How do I build for a particular version of glibc, if I want to target a particular Debian?
I'm able to call swift build --swift-sdk x86_64-swift-linux-musl on Linux, and that builds.
And if I call swift build by itself, it builds for the host platform as I would expect, and puts it into a directory called x86_64-unknown-linux-gnu. But if I try to run swift build --swift-sdk x86_64-unknown-linux-gnu that errors (even if I rm -rf .build first).
For releasing builds, how do I ensure I get an x86_64 glibc build, regardless of host architecture?
it says Use swift sdk list command to see available Swift SDKs. but if I run swift sdk list that doesn't list any architectures whatsoever:
I don't think anybody has actually tried following the instructions in the document. This code:
#else
#error(Unknown platform)
#endif
This isn't even valid Swift. The error directive will cause a parse error and fail the build.
(Is this even a good idiom? Literally all I need is the socket function. It would be nice if all of these interfaces implemented a common protocol, and then I select a concrete implementation at link time.)
I'm still unclear what is meant by "toolchain" in this context? I don't think Swift is using the word the same way it is in normal gcc or even llvm contexts. What is the deficiency of the Xcode toolchain that will cause problems, exactly?
Shared objects introduce a performance overhead on indirect calls. Specifically in this case they would be a solution for a preventable problem. Swift, like Clang, avoids it altogether by having a single compiler binary for all target architectures.
You don't need to install new architectures. These are all supported in swift.org toolchains.
That's where Swift SDKs come in. These components are independent of the compiler and they're distributed as Swift SDKs for corresponding platforms.
Platforms and CPU architectures are independent from each other. The compiler knows how to emit code for a given CPU architecture, but it needs a Swift SDK that provides the actual libraries to link with. Swift toolchain distributed from swift.org includes support for all CPU architectures that Swift officially supports. If you need to cross-compile to a certain platform, install a Swift SDK that contains libraries compiled for that platform that your package will link with during cross-compilation.
Xcode and a lot of components included in Xcode are closed-source products meant to be used specifically when targeting Darwin platforms. Closed-source products are in general off-topic on this forum. Xcode toolchains don't support CPU architectures that aren't supported by Darwin platforms. It also doesn't have necessary utilities for cross-compiling to non-Darwin platforms (linkers, librarian tools, debugger support etc).
swift.org toolchains are open-source products that target non-Darwin platforms and are meant to be used for cross-compilation to those.
There's no deficiency, these are different tools for different needs.
I still don't know which architectures are supported on my system, or how to build for one in particular.
Since not all swift toolchains support all architectures (e.g. how Xcode doesn't support Linux), in my build script, if I call out to swift, how do I know in advance that the target architecture/platform is supported or not supported?
That issue appears to have been fixed months ago, but you can only check that with the trunk snapshots that have the fix. Unfortunately, there haven't been any trunk Static Linux SDK snapshots shipped since then for you to use with that trunk compiler, so you could only verify it by compiling for macOS or directly on linux, ie without cross-compiling using the Static Linux SDK.
What you may be seeing if you downloaded snapshot SDKs too is that the version of the compiler and the SDK must exactly match, and the tools currently don't track that for you. Otherwise, if you are seeing random flakes, try running swift package clean and starting over: report an issue if you see an error fairly consistently.
You build for the different architectures by supplying different target triples, that is clear in the doc. The tools do a bad job of telling you what architectures are supported by each SDK: I have an open pull to fix that. Swift SDKs do not currently support fine-grained versioning of Glibc and Debian and so on, at least if they use the same target triple and you want some way to specify those other versions to the compiler.
The tool currently assumes that --swift-sdk is only invoked when cross-compiling, not when natively compiling.
Based on that above assumption, there isn't a single command you could use: you'd have to factor in the host arch and pass in different flags based on that.
Good point, if you replace that error with #error("Unknown platform"), it will work.
It isn't, but it was the easiest to implement. I believe most cross-platform languages that expose the platform C library punt on this problem and do not provide a cross-platform C API or wrapper, because it's a lot of work to do otherwise. There has been some discussion about providing something like what you want, but it hasn't led to anything you can use.
Some of this stuff you list is clear in the tutorial, some of it is bugs and errors, some of it is specific to Swift's insular world. Swift is still going cross-platform, and you'll have to pardon our dust while we make these SDKs more robust.
Keep reporting your problems in this forum and on GitHub: they will help us improve these SDKs and tools. And since most of it is open source, you can always pitch in and help fix them, of course.
Thanks for the response @Finagolfin, I was starting to wonder if I was in the right place...
This is inconvenient for me because Xcode and Xcode Cloud don't use snapshot builds. The bug appeared a few months ago, and normally I would expect a fix to appear in the very next bugfix release, e.g. reported in 6.2.0, fixed in 6.2.1. I've been unable to deploy an optimized build for several months now!
Further, the Swift Git repository has 620 branches. A normal developer like me has no way of contributing quick one-off patches to this repository. It is highly dissuading. (This need not be a function of project size, Linux has one public branch, and I can always see if my patch has been pulled into that branch or not.)
They're both 6.2.1, is there some other version I need to know about?
Could you please quote from the document what you're referring to? The word "target" only appears once in the whole doc; "triple" doesn't appear at all. I'm guessing that's programming language, host, and target?
swift build --help shows a --triple argument but it has no description (neither does --sdk for that matter):
BUILD OPTIONS:
-c, --configuration <configuration>
Build with configuration (values: debug, release)
-Xcc <Xcc> Pass flag through to all C compiler invocations.
-Xswiftc <Xswiftc> Pass flag through to all Swift compiler invocations.
-Xlinker <Xlinker> Pass flag through to all linker invocations.
-Xcxx <Xcxx> Pass flag through to all C++ compiler invocations.
--triple <triple>
--sdk <sdk>
--toolchain <toolchain>
--swift-sdk <swift-sdk> Filter for selecting a specific Swift SDK to build with.
--sanitize <sanitize> Turn on runtime checks for erroneous behavior, possible values: address, thread, undefined, scudo, fuzzer. (values: address, thread, undefined, scudo, fuzzer)
The man page for swift has no content either. I would expect a man 1 swift-build page detailing this.
The documentation simply does not provide any guidance whatsoever on how to build for a particular architecture—so far I haven't seen any way to distinguish "the command line arguments were written incorrectly" from "Architecture i386 is unrecognized, here's the list of supported values".
Am I supposed to know the difference between --sdk and --swift-sdk?
Would it be fair to request more documentation aimed at those Swift developers not hacking on the internals? Surely I'm not the only developer who's going to produce builds per platform.
Where do I request this feature more formally? Given that I'm building on a couple different architectures and platforms, I would like my script to just work, or at least for as many platforms as the swift executable supports.
It is definitely a complicated problem and I appreciate the thought going into it; I was thinking something more along the lines of a macro: "If you see this function signature (in the style of Darwin), substitute in this for Glibc, and that for FreeBSD libc" etc. It wouldn't be functionally different than how you can have multiple concrete implementations for a string literal.
This clarifies things a little bit, thanks for the link.
This touches on another learning curve problem I have: Is a normal (non-internal) Swift developer supposed to be reading through Swift Evolution proposals? If the proposal says "Status: Implemented", I would expect that to include documentation, too.
So in short: I would like to write a script to produce a build per platform. I should be able to run this on any of my boxes (that supports zsh or make or whatever is reasonable). If a particular platform can't be targeted, I'd like to report a good excuse why. Is this sort of thing just uncommon, or is this normally done a completely different way?
Check the output of swift --version. Xcode Swift will have a non-semver full version string with multiple (usually four or five) components together with the corresponding Clang version, e.g.
> swift --version
swift-driver version: 1.127.14.1 Apple Swift version 6.2.1 (swiftlang-6.2.1.4.8 clang-1700.4.4.1)
While swift.org toolchain version string will be equivalent to the Git tag it was produced from:
> swift --version
Apple Swift version 6.2 (swift-6.2-RELEASE)
Additionally for automation you can use -print-target-info output. Specifically for Swift SDKs I use this snippet for versions to always exactly match:
swift run --swift-sdk "$(swiftc -print-target-info | jq -r '.swiftCompilerTag')_static-linux-0.0.1"
No, --sdk on swift build is not intended for end users, it still shows up in SwiftPM's CLI help output for historical reasons, but I'd argue it needs to be hidden or removed. There is a clear distinction between SDKs (Xcode, Clang and Swift Driver concept) and Swift SDKs (SwiftPM concept), in 99% of the cases end users of SwiftPM should not need to use or care about the --sdk option.
I have to clarify that the document linked is an accepted Swift SDK evolution proposal, not an "SDK spec". Again worth reiterating, there's a very clear and important distinction between SDKs and Swift SDKs, and mixing that up only leads to errors and confusion.
This is the same Swift compiler, with some proprietary modifications, used for Xcode and so on, so the release branch managers are extremely conservative about what they allow in, ie you were meant to be dissuaded from adding "quick one-off patches" to a release branch.
As Max explained to you on that pull, you could try asking Slava or nominating that fix to the release/6.2 branch yourself, but merging it will depend on a risk assessment by the relevant branch managers.
The Getting Started doc doesn't use that more technical language, but it explicitly shows you how to build for different architectures by supplying different flags like --swift-sdk aarch64-swift-linux-musl. That is the target triple.
This line in the linked proposal, from the short paragraph on actually using it, explains that a bit:
We'd also like to make `--swift-sdk` option flexible enough to recognize target triples when there's only a single Swift SDK installed for such triple:
"swift build --swift-sdk x86_64-unknown-linux-gnu"
It is uncommon, eg Swift on Windows doesn't even support cross-compiling with these SDK bundles yet. It is an interesting idea, may be worth filing an issue and see if anyone wants to implement it.
Some of the issues you raised are known holes in the devex with Swift SDK bundles, which we hope to fix soon.
To follow up on this, probably what you were looking for is clang -print-targets command. This will only list CPU architectures supported by Clang that's installed together with Swift, as they both share the LLVM backend that supports those CPU architectures. Ensure that the clang binary in that invocation is a peer of swiftc binary.
If what you're looking for is platforms you can cross-compile to when using swift build, run swift sdk list.
No, he's right, as I pointed out in this thread to him above and to you on GitHub, SwiftPM currently does not tell you exactly which architectures are supported by a particular Swift SDK. I hope to finally get the pull I submitted fixing that problem updated this week.
I believe most cross-platform languages that expose the platform C library punt on this problem and do not provide a cross-platform C API or wrapper, because it's a lot of work to do otherwise. There has 