Implementing Parts of the Swift Compiler in Swift

[Douglas_Gregor]

@Finagolfin

Hey Doug, I've been hearing recently from you and other Swift devs that the plan is to get rid of the bootstrap mode detailed here a couple years ago and make a prebuilt Swift compiler a requirement to build the Swift compiler, with the new swift-syntax parser written in Swift becoming a mandatory part of the compiler.

Yes, that's correct. We'd like to get to the point where we can remove the existing (C++) parser and driver in favor of the Swift implementations, which means it will no longer be possible to build a Swift compiler without a Swift compiler.

@Finagolfin

I am fine with that new plan, but that means the only way to build the Swift compiler for new platforms on which a prebuilt Swift compiler doesn't exist will be to cross-compile it initially, ie the first cross-compilation scenario you laid out above.

Right.

@Finagolfin

Unfortunately, cross-compilation CMake config for the new Swift portions of the Swift compiler doesn't exist. @Erik_Eckstein added support to SwiftCompilerSources/ a couple years ago , but that has not been extended to the swift-syntax parser. I have filed an issue for this and submitted some pulls for non-Darwin platforms, but that has been met with silence and some strange assertions in the pulls.

I think there's some confusion about how cross-compilation should work going forward. Everything in SwiftCompilerSources is based on custom commands for building Swift code, because it predated having sufficient Swift support in CMake. That also meant that it needed its own bespoke support for cross-compilation.

Now, we have improved support for Swift-in-CMake, which is how the swift-syntax-based parts of the compiler (e.g., ASTGen and macros support) are built. SwiftCompilerSources will be moving over to this approach as well, because it greatly simplifies the CMake build and lets us use the normal CMake feature set vs. having our own hand-rolled versions. For cross-compilation, I believe this means we should be using toolchain files consistently, not continuing with our own BOOTSTRAPPING configuration. The end result should, I think, make it easier to cross-compile because CMake is handling it for us.

@Finagolfin

Can you lay out the plan for when you want to drop this bootstrap process, before the 6.0 branch?

Yes, I would like us to get to the point where SwiftCompilerSources is building with Swift-in-CMake (rather than the bootstrapping code) before the 6.0 branch. Aside from the technical reasons above, this goal is partly motivated by Windows, where the bootstrapping code isn't working so certain optimization passes are disabled. All of ASTGen + macros are building fine on Windows with the Swift-in-CMake approach, and it's where we want to end up anyway, so I'd like us to cut over to it so we have a consistent set of compiler features and optimization passes on all of the host platforms.

Now, with the Swift 6.0 branch it will still be possible to build a working---albeit somewhat limited---Swift compiler without having a Swift host compiler beforehand. Beyond the Swift 6.0 branch, we won't promise that any more for main.

@Finagolfin

And will there be work done on supporting cross-compiling the Swift portions of the Swift compiler before that?

I expect there will, but not through the BOOTSTRAPPING code. However, I don't have a solid handle on exactly what needs to be done to make the Swift-in-CMake approach cross-compile easily, and will have to defer to others (e.g., @etcwilde, @bnbarham, @compnerd).

Doug