The Problem
While Swift codegen (IRGen and SILGen) is fairly similar on all platforms, C++ codegen is not. Even on the "same" platform, codegen might be wildly different depending on, for example, the selected ABI. Unfortunately, many of these differences leak into the Swift tests for C++ interop. For example, we might have three or more names for the same function, depending on what ABI/CPU/OS is being targeted. This makes it difficult to write tests locally because you either have to fire up a few VMs or run the tests on Jenkins, knowing that they're going to fail.
This also means platforms with less testing, such as iOS or Android, sometimes have bugs that are only caught days after a change lands.
The Solution
I propose that we start cross-compiling some of the C++ interop codegen tests. We could then check the codegen locally on any number of platforms before the code runs on the CI or causes problems down the road. I think this would be a great productivity boost for developers writing these tests, and it would allow us to write better tests because we would have as much information as possible on our local machines.
Implementability
There are a few challenges with this, namely, cross-compiling the standard library. I propose that rather than cross-compiling the full standard library, we use a micro standard library, such as @compnerd's uswift. Unfortunately, a standard library like this means that we won't be able to run execution tests or certain codegen tests that rely on stdlib features. I think this is OK because the execution tests don't have the problems described above, and if there are any codegen tests that rely on the full stdlib, we can still use the old method of testing.
This weekend, I wrote a proof of concept for this testing strategy where I show that we can run many of the existing IRGen tests on three different targets (macOS, Android, and Windows). The basic idea is:
- We add uswift as a dependency.
- The build script creates a uswift build directory for each target.
- The uswift builds are copied into the swift build directory.
- A lit substitution invokes swift-frontend for each target.
Here's a link to the diff. (Note: this is just proof that my proposal can be implemented with relatively few changes, this isn't meant to be a polished PR.) Also, thanks for the help with this @compnerd!
Drawbacks
- This will increase compile time slightly. Running the codegen tests X times makes them X times slower.
- This adds some complexity to the build script and test suite.
- This adds another dependency to Swift.