Compiler Build System Roadmap

Compiler Build System Roadmap

This is the third update to the vision roadmap for improving the build system. This update is focused on the compiler. The first step worked through splitting the runtime libraries into a separate build under the Runtimes/ directory. The runtime work is largely done with the exception of building the embedded runtimes. The second step splits the test suite, enabling us to build the compiler and runtimes independently of building and testing the runtime in order to help reduce test times and support compiling the runtime tests for specific platforms. In this step, we adopt native CMake support for Swift compilation and migrate off of the custom functions used in the compiler build today, fully decouple the runtime build from the compiler and tool build, integrate with the LLVM build graph, and unify duplicated code between SwiftCompilerSources and other parts of the compiler.

The build system has to serve two uses. First, it has to build the software. It has to represent the build dependency graph and enforce the requirements of the software. Second, it has to customize the build of that software to the needs of the vendor. Different distributors may have vastly different requirements for shipping a piece of software. Many Linux distributions require software libraries ship as dynamic libraries so that they can patch and swap out an individual library if a vulnerability is discovered without needing to rebuild and relink the entire OS. It also helps amortize the binary-size cost of a library across the executables that depend on it. At the same time, there are applications, like in server ecosystems, where a single statically linked executable makes sense. In these environments, the executable may not get to choose which distribution it is running on, and Linux doesn't guarantee library-level ABI stability, but does guarantee a stable syscall table. A fully statically linked, hermetically sealed, binary makes distribution easy without needing to worry about installing and managing different versions of dependencies. Filesystem hierarchies may also differ between different operating systems. Some distributions define a full Filesystem Hierarchy Standard (FHS) with the classic /usr/{bin/,include/,lib/,libexec/,local/} structure, to minor differences like using usr/lib64/ for 64-bit binaries instead of usr/lib/, all of the way to distributions that make no guarantees about library locations like on Windows and NixOS. The build system has to take the requirements of the software vendor and of the software it is building and determine whether the configuration is possible and either produce the built binary or fail fast and emit an error indicating why the requested configuration is not possible.

Today, the compiler and build system are largely hard-coded to specific configurations for each platform. Creating new configurations has historically required adding a new "platform" to the build, as was seen during the development of the static Linux SDK. The new runtime build migrated to native CMake support for Swift and, as a result, is able to rely on the platform support that CMake has built up over decades of cross-platform development to support cross-compiling the runtimes to new platforms without requiring major changes to the build system itself. Now we want to bring that flexibility and support to the compiler build, allowing building the compiler with different configurations. The ultimate goal of this work is to support building the compiler against an SDK similar to Python's manylinux distribution to produce a generic "Linux" toolchain on swift.org, while also supporting the needs of each specific Linux distro, without requiring major changes to the build system itself. In order for this to work, we need to provide the appropriate RPATH search path control when building the compiler and toolchain tools. We also need to support fully cross-compiling the compiler and toolchain to new platforms. This requirement means that parts of the toolchain cannot assume that the just-built compiler is available.

Luckily, fixing the compiler build is one of the easier parts of the puzzle to fix. Once split from the runtime, the compiler build is largely the same as building any other command line executable. The build needs to know the sysroot, target triple, toolchain, RPATH information, and optimization flags to use. Compiler-specific options include what targets the built compiler can produce outputs for as well as any default search paths to find interfaces, libraries, or macro plugins. Unlike the runtime libraries, these options don't impact ABI, security properties, or performance characteristics of other binaries, which can be a lot harder to verify. We can leverage the compiler, execution, and validation tests and source compatibility suite to ship the compiler with reasonable confidence that the compiler isn't broken.

Definitions

  • Swift SDK: The collection of interfaces, Swift modules, macro libraries, and executables needed to compile Swift code into objects, and the runtime libraries needed to link against in order to produce executable binaries. Each Swift SDK enables cross-compiling to a specific platform.

  • Sysroot: A collection of headers and libraries representing the system being compiled to. The term originates from the Unix 'System Root' directory. Depending on the platform, the layout may be important. Note that not all systems follow the Filesystem Hierarchy Standard with the standard /usr/{bin,lib,include}/ directory layout. Even though the platform may not have a sysroot in the sense of the classic Unix sysroot, we still refer to the collection of libraries and interfaces needed for interacting with that platform as the sysroot. This may include headers and libraries for interacting with the kernel, C and C++ runtimes, and other platform runtime libraries.

  • Toolchain: The collection of primitive tools needed to produce usable binary artifacts. In C, this might be the preprocessor, the C driver, C frontend, an assembler, archiver, and linker. Note that this may include the runtime libraries that the tools need. These libraries are not necessarily intended to be compiled against and may not have ABI alignment with what the compiler produces.

Proposal

This is a proposal to use best practices in CMake and leverage the tools CMake supports in place of creating custom abstractions for building the compiler, while giving examples of the kinds of changes to expect. Functions like add_swift_host_library, add_swift_host_tool, and other similar variants were created before a lot of the native support for Swift was implemented in CMake, and some before Swift supported anything other than Apple platforms. This is reflected in many of the assumptions baked into their design about the layout and expectations for the shape of an SDK and sysroot, with support for other platforms tacked on. CMake now has much stronger support for compiling Swift and even supports mixing Swift, C, and C++ sources in a single target, while supporting a vast number of platforms from its decades of cross-platform development.

The current design has several issues with how targets compute and propagate flags, outdated practices for compiling Swift code, complicated legacy code that no longer works but is intertwined deeply with the system, and convoluted logic building up compile flags and definitions into strings, attempting to reconstruct information that is frequently already contained in the build graph.

More specifically, the flags computed for a given target come from a combination of global mutable state and the state of the directory calling the function, making it difficult to determine where and why specific targets build with certain flags. Many of these flags area part of the library interface of one of the dependencies. Attaching interface flags to a target with calls to functions like target_compile_options, target_compile_definitions, and target_link_options, ensures that the flags are propagated to all targets that depend on it directly, and to any other libraries that expose a given target as part of its public interface. This avoids having to manually recompute the flags for each target based on its dependencies.

Much of the complexity is associated with computing the appropriate flags for each target. There is also complexity in other places like linking, bootstrapping, and handling Swift sources that is no longer necessary, but hasn't been cleaned up yet. As part of this effort, we will remove a lot of the old workarounds that frequently resulted in abstraction leakage and remove the now defunct bootstrapping code, resulting in an easier codebase to navigate.

Many of these changes should be fairly uncontentious migrations to use improved support for Swift in CMake. There are some changes that may cause more concern that we would like to discuss in this proposal as well. These topics include how the toolchain interacts with the runtime build, how the compiler builds against LLVM, Swift Syntax, and the Swift driver, and unifying how the Swift sources that are already part of the compiler project are structured and built.

Runtime

The compiler build and the old runtime build are intertwined. In order to compile some of the Swift sources, the build would first build the runtime libraries, fish out the host runtime, and then use that just-built runtime to build the rest of the tools. Other parts of the build, like in ASTGen, rely on an existing install of Swift in order to build those portions of the compiler. On platforms where we don't have ABI stability, we are mixing new and old runtime ABI versions in the same build, which is bad.

LLVM-based toolchains have historically required that the toolchain build against an existing set of runtimes. Several companies have structured their builds around this assumption, and some platforms expect the compiler to backdeploy to older environments. Keeping in alignment with these requirements, and relying on the fact that the Swift compiler now requires an existing Swift installation and SDK to build, we propose that all toolchain tools are built from the same, existing SDK. In practice, this should simplify the build because CMake already wires through the tools from the existing toolchain and runtimes from the sysroot by default. The runtime that the compiler builds and executes against does not need to be the same runtime that the applications the compiler produces run against. This split will resolve SR-5975: [SR-5975] Fix implicit requirements on swift-stdlib-* targets · Issue #48534 · swiftlang/swift · GitHub.

Note that the Windows toolchains already separate the toolchain runtime DLLs from the SDK runtime libraries. The bootstrap staging is due to issues with testing, where the execution tests would both invoke the compiler and then execute the output of the compiler, requiring that the compiler run on the same runtime as the output of the compiler. Xcode also separates these, although it doesn't ship the implementation of the runtime libraries in the SDK as they are already installed on device.

LLVM

Build-Script builds LLVM, the Swift compiler, and LLDB independently. This segments the build graph based on project. As a result, each stage builds the entire project before proceeding to the next. A single change in an unrelated part of LLVM results in a complete rebuild of LLVM, Swift, and LLDB because the tools are unable to track the impact of that file changing across the project boundaries.

We propose migrating build-script to the integrated build with LLVM_EXTERNAL_PROJECTS to build the Swift compiler and toolchain tools. This integrates the compiler build with the LLVM build graph for more granular dependency tracking. Furthermore, it makes it easier to ensure that dependencies are up-to-date without causing a full rebuild. Folks working on LLDB will likely feel the impact of this change the most. Currently, any change to LLVM results in a full rebuild of LLVM, clang, Swift, the Swift runtime libraries, before rebuilding LLDB. By merging the build graph, only the targets that are affected by the change are affected, and if a file is changed that doesn't impact the binary, nothing is rebuilt.

Note that the Windows toolchain builds already uses this mechanism today. Experiments are showing that merging the builds reduces the compiler build time by about 20% without significant changes to the build system. Further improvements to the CMake scripts and simplified build graph could further improve build times.

Swift Syntax

The Swift Syntax build serves both the compiler and macros. SwiftPM has its own mechanisms for providing the libraries for package users but folks compiling Swift in other build systems need to compile Swift Syntax in order to build their macros. As such, it is important that the Swift Syntax build exposes the necessary knobs to control installation and RPATH search paths for both kinds of consumers.

These values are hard-coded today, which doesn't offer flexibility required by various vendors, often requiring patching to meet their needs. Instead, the vendor of the library should decide where and how the library installs. Exposing these values largely removes the need for the two codepaths through the Swift Syntax build to support macro and compiler build needs. The compiler project will be expected to provide the appropriate flags for its use-case.

Additional changes will include adopting CMP0195 in CMake builds that support it to emit the nested swiftmodule directory structure instead of computing it manually and emitting a SwiftSyntaxConfig.cmake file so that CMake-based projects can use the build products with find_package to avoid rebuilding Swift Syntax.

The integration with the compiler repository is correct, using FetchContent to make the Swift Syntax build graph and targets available to the compiler build. Changes will primarily focus on cleaning up hard-coded assumptions, making use of newer CMake features to improve the build, and making the build products accessible to external build systems.

Swift Driver

The full compiler build currently has two Swift driver builds, the "early swift driver", and the "swift driver". The early swift driver is built using the toolchain and SDK that the rest of the tools are built with. After building the early Swift driver, we build the compiler and toolchain, and then the runtime libraries, SwiftPM, and then use the new SwiftPM and new runtime libraries to build the Swift Driver that is shipped.

On platforms that do not guarantee ABI stability, this results in an irreconcilable runtime requirement where the compiler requires a runtime with the original SDK ABI and the driver driving that compiler requires the new ABI. From a build perspective, this requires that the compiler is not cross-compiled, and at least one of the built runtimes will work on the build machine in order to run SwiftPM.

All toolchain tools need to run against the same version of the runtime and build with the same set of tools to ensure ABI compatibility while driving down build times. The Windows toolchains today only build the Swift driver once, against the runtimes that the toolchain tools run against. This change will be applied to the other platforms. For consistency, the compiler build will use FetchContent to merge the driver build graph into the compiler build graph.

Swift Compiler Sources

There are currently two separate directories containing compiler sources, the original lib/ and SwiftCompilerSources/. The Swift compiler sources was originally an experiment adding Swift to the Swift compiler, written before CMake supported building mixed Swift and C++ targets. With newer versions of CMake, Swift, C, and C++ may be compiled as part of a single target and sources that are logically part of the same library can sit and build next to each other. CMake supports generating the compile_commands.json compilation database for Swift sources, which works well with SourceKit-LSP and the swift-xcodegen tool, providing a full semantic editing experience that aligns with the configuration you are building in all three languages.

The ASTGen component also contains Swift sources. Both ASTGen/ and SwiftCompilerSources/ have implemented separate copies of a Swift overlay for lib/AST and lib/Basic, duplicating effort and code.

The current state of scattered sources across different directories, multiple re-implementations of the same libraries, and separate package files results in a segmented ecosystem with entirely different processes for developing the compiler. The plan is to start migrating the SwiftCompilerSources/ back into the lib/ directory structure. This process will be incremental in nature and will necessarily need to start by reconciling the separate implementations of the Swift AST and Swift Basic libraries. With build support for mixed language targets, the Swift sources can move to the corresponding lib/AST and lib/Basic directories, forming a single target exposing both a Swift and C++ API.

Conclusion

Ultimately, the goal is to follow best practices to enable vendors to control how Swift integrates with their platform. This is done by adopting newer features and exposing the necessary controls to the vendor of the toolchain and runtimes.

EDIT: Remove 80-col line wrapping.

3 Likes