Dropping the requirement for C++-only bootstrapping

Development Compiler
1

Dear compiler developers,

I'm happy to announce that from now on "mandatory" parts of the compiler can be implemented in Swift (on the main branch).
So far we only used Swift as an implementation language in parts of the compiler which are not strictly necessary for correct code generation of the standard library, like some optimization passes.
Now, we can use Swift in all parts of the compiler, including the parser, AST, type checker or mandatory SIL passes.

We are dropping the requirement to bootstrap the compiler with a pure C++ host toolchain.
Instead we are requiring that a Swift toolchain must be present to build the compiler.

How to bring the Swift compiler to new platforms where no Swift host toolchain is available, yet?
This can be done by bootstrapping with the 6.2 compiler. The 6.2 compiler is the last release which can be bootstrapped without a Swift host compiler. On the new platform the 6.2 compiler can be built and used as a Swift host toolchain.

57 Likes
2

This is great news, thanks to all who made is possible!

Does this mean that certain parts of the compiler, like the C++ parser, will be removed soon? Can new changes to the language grammar be implemented only in swift-syntax instead of both parsers as has been the case up until now?

Is there a documented policy for which host compilers will be supported going forward (e.g., "Swift 6.X can be built with a Swift 6.(X – 2) compiler" or something similar)?

3 Likes
3

Does this mean that certain parts of the compiler, like the C++ parser, will be removed soon?

No, this just means that parts can be replaced :slight_smile:

@Douglas_Gregor may know some details on the parser.

Is there a documented policy for which host compilers will be supported going forward (e.g., "Swift 6.X can be built with a Swift 6.(X – 2) compiler" or something similar)?

not yet
cc @etcwilde

3 Likes
4

Alternatively, you can cross-compile a bootstrap compiler on one of the well-supported host platforms, then use that on the new platform to build the Swift compiler natively. That is how @networkextension appears to have gotten FreeBSD AArch64 going.

The linux CI builds the trunk Swift toolchain with Swift 5.9.2, ensuring no later Swift features creep into the Swift code used in the Swift compiler itself, but we're currently looking at upping that to Swift 6.2.4.

5

Does that mean a cross-compiled bootstrap will be the primary way to bring the Swift compiler to new platforms?

6

That has been the plan for years now, but given that Erik said the next 6.4last 6.2 release still supports natively bootstrapping, most will likely choose that route instead. Last week, @etcwilde listed the build steps he used to natively bootstrap Swift 6.2 on FreeBSD x86_64 last year. Alternately, the few build patches I use to cross-compile the 6.2 stdlib and compiler for Android are available here, along with the build-script command I invoke.

Of course, most of the work is in porting the Swift standard library to a new platform, these build config issues are secondary.

2 Likes
7

This can be done by bootstrapping with the 6.4 compiler. The 6.4 compiler is the last release which can be bootstrapped without a Swift host compiler.

We disabled the C++ bootstrapping verification bot most of the way through the 6.3 development and I stopped maintaining the C++ bootstrap when we did so. Given how frequently passes in the SIL optimizer were crashing during the 6.3 development timeframe and how often we would have source breaks because there were mandatory passes written in Swift that I couldn't use, I would be very surprised if we can successfully bootstrap with 6.4 without crashing. Have you verified this claim?

Did Optimizer: replace PredictableMemoryAccessOptimizations with a "mandatory" redundant load elimination pass by eeckstein · Pull Request #79186 · swiftlang/swift · GitHub get reverted in 6.4? This was a consistent source of breakage in the stdlib/compiler sources. Without these mandatory passes which were re-written in Swift, the stage-0 compiler would report that functions were missing returns if they had something that looked like an infinite loop but returned in the body of the loop, so I had to sprinkle fatalError throughout the codebase to keep that working. e.g. Fix Bootstrap: UTF8EncoddingError by etcwilde · Pull Request #80934 · swiftlang/swift · GitHub

Other bits that likely need looking at are in the optimizer itself. Passes were increasingly relying on the results of Swift-only passes during 6.3, and would crash or trip the SIL verifier. I had to keep track of those and disable them: e.g. SILOptimizer: Disable invalid passes in C++-only compiler by etcwilde · Pull Request #80920 · swiftlang/swift · GitHub

I haven't been doing the work of maintaining that, so unless someone has been behind the scenes, I don't have a lot of confidence that 6.4 will bootstrap from C++.

The last version I feel 100% confident saying will bootstrap to a fully passing state successfully is 6.2, and that's because we have CI verifying that things are working when patches are made against release/6.2: [release/6.2] OSS - Swift Bootstrap - Amazon Linux 2023 [Jenkins]. I feel confident that a little work could make 6.3 work, though I think we dropped the bootstrap testing while the compiler was crashing somewhere in the SIL optimizer.

I think the options right now are either keeping things working with 6.2 and using that to do the bootstrap, or cross-compiling.

As @Finagolfin notes:

The linux CI builds the trunk Swift toolchain with Swift 5.9.2, ensuring no later Swift features creep into the Swift code used in the Swift compiler itself, but we're currently looking at upping that to Swift 6.2.4.

2 Likes
8

The last version I feel 100% confident saying will bootstrap to a fully passing state successfully is 6.2

Right, I didn't think of that we don't test C++ bootstrapping with > 6.2.

I corrected the version number

9

I know the older releases have been closed for a while, and this request is a long shot, but it might be nice to consider keeping a 6.2 branch (or a branch of whatever might be considered the Last Release in the future) open strictly for bootstrap related issues that might arise when bringing Swift to new platforms in the future, to slightly ameliorate maintaining local side patches. I can also see arguments not to do this, but the thought has been rattling around in my head for a while…

1 Like
10

Erik,

Congrats to all who participated! This achievement will pay all sorts of great dividends in the future!

1 Like
11

Not a long shot. This is exactly the plan. I don't have a branch name yet since we haven't had to bring one up, but it'll probably be something like 6.2/bootstrap.

2 Likes