Hello everyone!
The work on the new build system is making good progress. I would like to thank @compnerd, @edymtt, and @justice-adams-apple for their hard work in helping me with this project.
The ultimate goal of the project is that we can easily produce Swift toolchains and SDKs for platforms without needing to make major changes to the build system. Once we are done fixing the runtime build, we will refocus our efforts on the compiler build.
The goals for the new runtime build include
- producing Swift SDKs that overlay an existing system sysroot
- producing Swift SDKs that integrate into a system sysroot
- reduce the challenge of cross-compiling the runtimes to new platforms
- avoid using custom solutions when off-the-shelf solutions exist in CMake
The new runtime build uses pure, documented, CMake, leveraging the support for Swift that has been landing in CMake itself. This makes it easier to cross-compile the project for any platform that CMake supports while substantially reducing the maintenance burden of the build system while also leveraging future improvements in CMake's ability to work with Swift code.
The new runtime build lives in the Runtimes directory of the Swift compiler repository. The sources for the standard library still live in the stdlib/ directory. There is a Resync.cmake script to copy files from the old standard library into the appropriate locations in the new build. To synchronize the directories, run cmake -P Resync.cmake from the Runtimes/ directory. Once the migration to the new build system is done, we will remove the script and move the sources to the appropriate location in the Runtimes/ directory, preserving git history.
How do I try it?
Note: This information is included in the readme files for the
RuntimesandRuntimes/Core.
Synchronize Sources:
Start by synchronizing the standard library sources from their current location to their new locations.
cd Runtimes
cmake -P Resync.cmake
Configure Standard Library:
Note: The standard library must be built with a Swift compiler that is at least as new as the sources in the standard library. You will likely need to pre-build the compiler as you would normally. The
<swiftc>symbol indicates the path to the just-built Swift compiler.
We'll start with a basic bare-bones configuration that resulting in building a static SwiftCore library without optimizations:
cmake -G Ninja -B build -S Core -DCMAKE_Swift_COMPILER=<swiftc>
If we want to reconfigure our build to emit a dynamic library instead, we can run:
cmake -B build -DBUILD_SHARED_LIBS=YES
The new build system offers several knobs for enabling and disabling features including vector support, runtime type printing, command-line support, and more, depending on what the needs of the situation are. The default configuration is not representative of any specific configuration, it is meant to be a bare-bones runtime that builds with minimal effort. The ccmake tool makes it easier to customize the build configuration to your needs:
ccmake build
Once the build is configured to your goals, run:
cmake --build build
Reproducing a Build
Having the knobs to configure builds as needed is useful for engineering and one-off situations, but can make it difficult to reproduce a specific build configuration.
To reproduce specific build configurations, we provide CMake cache files. Currently this directory only contains caches for the Apple platforms under Runtimes/Core/cmake/caches/Vendor/Apple to compile the Apple OS runtimes.
To build one of these configurations, from the Runtimes directory run the following:
SDKROOT=iphoneos cmake \
-G 'Ninja' \
-B build-iOS \
-S Core \
-DCMAKE_Swift_COMPILER=<swiftc> \
--toolchain cmake/caches/Vendors/Apple/Darwin.toolchain.cmake \
-DCMAKE_OSX_DEPLOYMENT_TARGET=18 \
-C Core/cmake/caches/Vendors/Apple/arm64-iPhoneOS.cmake
cmake --build build-iOS
With these two commands, we compiled the standard library for iOS 18. This is the same configuration used by the OS, so the resulting build matches how the runtimes are built for the iPhone.
Developer Experience
The new build system takes advantage of the support for Swift that is built into CMake. If you have CMake 3.29 or newer, the developer experience improvements become apparent.
Semantic Editing
CMake can generate compile-commands that SourceKit-LSP uses to provide semantic editing support for your editor of choice, whether that is VSCode with the Swift VSCode plugin or Neovim.
cmake -B build -DCMAKE_EXPORT_COMPILE_COMMANDS=YES
CMake places compile_commands.json file in the build/ directory. Some editors know to look for this file <source-root>/build, but many often don't.
On Unix-like environments, create a symlink from the that file into the source-root directory. This build configuration is for the Core portion of the project so we should symlink it into that subdirectory.
cd Core
ln -s ../build/compile_commands.json
You should notice that features like hover-information, go-to-definition, and semantic highlighting are working on the standard library sources after running a build to generate the swift modules.
Incremental Builds
CMake is able to express the behavior of a given Swift compilation command to Ninja more clearly than the arbitrary custom commands that we were using to build Swift sources. This means that Ninja won't attempt to rebuild things that have already been built. Modifying a Swift source in SwiftOnoneSupport and then re-running Ninja will only rebuild SwiftOnoneSupport instead of trying to recompile the whole project. Additionally, running a build more than once correctly reports ninja: no work to do.
What's Next?
There is still a lot of work to do regarding testing and validation, enabling the new build on all of the supported platforms, validating that cross-compilation works, and migrating more libraries:
- Make all tests run against output from new build system
- Enable new build system on Linux
- Add CI validating that cross-compiling the runtimes works
- Install new runtimes into nightly toolchains for folks to live on them
- Bring up the platform overlay libraries
- Bring up the supplemental libraries
If you're interested in helping with this effort, please let me know.
