1

Hello everyone,

It's been two years since I originally suggested that we start down the path of requiring Swift on the host when building a Swift compiler and toolchain. Since then, the gap between the "builds only with C++ code" and "builds with a host Swift toolchain" has grown considerably: macros, Embedded Swift, driver features, and many new optimization passes are written in Swift. And on the other hand, we're not really testing the "builds only with C++ code" path much, because the official CI builds for Windows, Linux, and Darwin are all built with host Swift toolchains.

The details of the "how" haven't changed much since my August, 2022 post on the matter, but I think it's time to remove the ability to build the Swift compiler without a host Swift toolchain. I think there are roughly three steps:

  1. Introduce a CMake error when building the Swift compiler without a host Swift toolchain, only on main. This will flush out remaining places where we aren't building with a host Swift toolchain.
  2. Remove the ability to disable the use of swift-syntax in the compiler build.
  3. Start deleting the C++ implementations that are only used when there is no host Swift toolchain, because this code is now dead. Give a little cheer for each such case, because our maintenance effort has been reduced.

After this, there's still a long road for the various C++ implementations to be replaced by their newer Swift counterparts. The driver is fairly close, but the parser requires a lot more engineering effort to switch. However, by taking the steps above, it gives us a path back to only having a single implementation for these compiler components... one that's written in Swift, with all we've learned over the years.

Thoughts and concerns?

Doug

34 Likes
2

My reflex is to be all for this. However, i'm concerned about the bootstrap process for new platforms and architectures. Is it possible to cross-build the compiler for a new platform if someone wants to start adding support for it?

8 Likes
3

What's the plan for bringing Swift up on a new platform in this new always-having-a-toolchain world? Cross-compiling the compiler first, presumably?

[Edit: @QuietMisdreavus You beat me to it by a few seconds :-)]

4 Likes
4

Great news. As more and more of the compiler is rewritten in Swift, has there been any serious benchmarking of these Swift components? If so, how is Swift performance looking compared to the C++ versions, in this admittedly niche domain of implementing the compiler itself?

You'd probably want to start with porting the stdlib first, and once the C++ Driver is removed, you'd need to port libdispatch and swift-corelibs-foundation next, as the swift-driver uses them. Finally, you would cross-compile the rest of the compiler for the new platform.

I believe the CI cross-compiles the toolchain for different architectures of macOS regularly, and I do the same from linux x86_64 to Android with a little patching, so most of the support for cross-compiling the toolchain is there.

What isn't there is a way to easily run the cross-compiled tests for the stdlib and corelibs on the new platform while porting. For example, @Brian_Gesiak added support 8 years ago for pushing cross-compiled executable tests from the compiler validation suite to Android using its adb debug bridge tool and running them remotely, but most platforms don't have a tool like that.

In sum, a lot of the support is there, but it's not easy to get going, particularly with running cross-compiled tests.

1 Like
5

To be fair, bootstrapping is how C is ported to new platforms and architectures too.

6

At least in my experience, 5.10 was the last compiler that would cleanly build without a Swift compiler preinstalled, so this doesn't change my experience that much.

That said, I'd like to ask that we amend the requirement of what the version of Swift the Swift source files in the Swift compiler must build with to 5.10 so that we can avoid the multi-hour multi-stage toolchain-build-hopping dance, and that anything that doesn't compile with the 5.10 compiler can be reverted aggressively. At least until we have a reliable cross-compiling story and can bring up new compilers on new platforms from one that has an existing compiler easily.

3 Likes
7

Totally. I just wanted to be clear that the implication is that bringing up new platforms involves cross-compiling, which makes it important that the Swift build process has reasonable cross-compilation support (right now, I think if we're honest, it needs some work).

8 Likes