On macOS 13.5, with Xcode 14.3.1, trying to build a Swift toolchain (to then use within Xcode), based on the the instructions in the readme: GitHub - apple/swift: The Swift Programming Language I hit a problem:
ld: warning: ignoring file /opt/homebrew/lib/libzstd.dylib, building for macOS-x86_64 but attempting to link with file built for macOS-arm64
Undefined symbols for architecture x86_64:
"_ZSTD_compress", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef<unsigned char>, llvm::SmallVectorImpl<unsigned char>&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_compressBound", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef<unsigned char>, llvm::SmallVectorImpl<unsigned char>&, int) in libLLVMSupport.a(Compression.cpp.o)
"_ZSTD_isError", referenced from:
llvm::compression::zstd::compress(llvm::ArrayRef<unsigned char>, llvm::SmallVectorImpl<unsigned char>&, int) in libLLVMSupport.a(Compression.cpp.o)
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
[2137/4537][ 47%][493.962s] Building C...les/clang-tblgen.dir/NeonEmitter.cpp.o
ninja: build stopped: subcommand failed.
This is because my libzstd in homebrew is thin (ARM64), but the toolchain build is building a fat toolchain, and needs an x86_64 version of this library. I've checked homebrew, but this formula doesn't have a "universal" variant.
How are other people avoiding this? Can I build a "thin" toolchain? Is there an easy way to get a fat libzstd? Should I just not be building a toolchain at all?
Which exact build-script invocation are you using to build it?
I just did ./swift/utils/build-toolchain some.bundle.id
This is not the command you should be using to build locally. Please follow instructions specified in the GettingStarted.md document.
Should the main README.md be updated then? "Building a toolchain as a one-off" uses the build-toolchain command.
1 Like
Thanks for the help.
Yeah, the README really positions build-toolchain as an alternative to the other build styles that can then be installed into Xcode, so I thought that was what I needed.
In fact, although following the instructions you linked has worked, I now don't know how to use this compiler within Xcode. All the docs I've found in the Swift repo either say "use build-toolchain to use your Swift within Xcode" or "use your new compiler on the command-line"...
Correct, installing a newly built toolchain for usage in Xcode is not a common use case. .pkg files that are distributed on Swift.org - Download Swift are not built locally, but on CI and are code-signed. When building locally you're much more likely to use it from command-line, run lit tests, and integration tests. After proving that your toolchain works with those test runs, you can submit a PR and build a new toolchain on CI with that PR.
Nevertheless, if you still want to get it recognized by Xcode, the file system layout of a freshly built toolchain with build-script is quite close to what you get after installing one from a .pkg file, so maybe you can get away with manually creating a directory for it in ~/Library/Developer/Toolchains/ and copying there build artifacts you get from your build-script run. I haven't tried this method though, but this is where I'd start looking if there was such a need.
You can also pass --installable-package=$(pwd)/installable-package.tar.gz --install-destdir=$(pwd)/installable-package to build-script, which will create a tarball for you to unpack into your freshly created ~/Library/Developer/Toolchains/<new_toolchain>.xctoolchain directory. Make sure that the new toolchain directory layout matches that are already installed and proven to work.
Thanks.
I was unable to find sufficient similarity between the output structure of the normal build and what's expected of a toolchain, nor to make --installable-package work at all, so I reverted to a worse solution:
- I built libzstd myself from source, fat (it supports this natively, so long as you don't want lz4 or xz support, neither of which seem to have native fat support nor universal options in homebrew)
- I installed it to /usr/local
- I symlinked it into /opt/homebrew/lib
At this point the build-toolchain script did work (consuming 184GB of disk space, which, I, wow), and I was able to install the toolchain in such a way that Xcode recognizes it.
I was able to build our app for "any iOS device (arm64)" using this toolchain, up to the point of linking an executable, at which point it failed: File not found: /Library/Developer/Toolchains/swift-LOCAL-2023-09-04-a.xctoolchain/usr/lib/clang/13.0.0/lib/darwin/libclang_rt.profile_ios.a. I copied this file from Xcode 14.3.1 (which oddly uses a later clang version, which might later be an issue?) but got me past this problem temporarily.
Unfortunately, this "main branch" Swift crashes when compiling some of our code, so I still don't really have a working toolchain... another problem for another day.
Can you elaborate on this? Did this produce an error message? If so, which exact invocation did you use to get to it? This should be filed as a bug so that we can improve the build process.
Sorry, I didn't pay much attention to it :/ I added your --installable-package=$(pwd)/installable-package.tar.gz --install-destdir=$(pwd)/installable-package to the end of my previously-working build-script invocation, and it failed trying to interact with a file inside $(pwd)/installable-package that it hadn't actually created.
Wrapping up this thread: After trying to build a few tags and failing either to build, or to produce a toolchain that could actually compile our code (eg. due to debug assertions), and given the extreme disk space requirements (I have a 500GB SSD, and I ran out of disk space for other things with over 150GB of Swift-related stuff hanging around. Someone on a 256GB disk doesn't stand a chance), I've given up on being able to contribute to Swift 
Would you be able to share the exact error messages that you stumbled upon?
Did you build with --debug? This is not a configuration I'd recommend, especially for first-time contributors precisely for the reason you've listed. Most of the time you don't need debug builds, especially enabled in the whole LLVM and Swift toolchain stack all at once. You should only be enabling debugging information in a single component that you certainly know you will be attaching a debugger to, and build-script has plenty of options for fairly granular control of this.
Have you tried running release builds at all?