Working on the Swift Compiler - Some Notes


(John Holdsworth) #1

I thought I’d put together some notes on working with the Swift compiler as I don’t see too many summaries with all this information in one place in particular the git side of things. Much of what follows could be described as coping strategies of someone who “doesn’t really know what they are doing” but lets hope it's of use. You’ll need about 40gb free on your computer.

To work on swift, first step is to make your own fork of the https://github.com/apple/swift project in github and then clone your fork e.g.:

$ mkdir swift-source
$ cd swift-source
$ git clone https://github.com/<username>/swift

Then, fetch the other associated projects as described in the swift project home page https://github.com/apple/swift:

$ ./swift/utils/update-checkout —clone

At this point there are two routes you can go down. Swift uses cmake files to describe it's build process and cmake can generate input files for two build systems: ninja (a 21st century version of make) or fully fledged .xcodeproj files suitable for Xcode. If you plan on making a relatively small change to the compiler the ninja route is probably the route to take as it iterates slightly more quickly:

$ cd swift
$ utils/build-script --release-debuginfo

This will take a few hours to complete but afterwards you’ll have a directory something like: ../build/Ninja-RelWithDebInfoAssert+stdlib-RelWithDebInfo/swift-macosx-x86_64 you can cd into for the rest of your work.

Before modifying any file create a new branch for your changes and make a point of never checking anything into master. This allows you to reuse your fork for multiple changes you may be working on and makes rebasing easier. Make your changes and type:

$ ../ninja-build/ninja swift && bin/swift

This is the fast iteration route and can take as little as 10 seconds depending on what your recompiling. If you’re working on something on Swift source the stdlib use the following command which takes a minute or two as it has to rebuild the entirety of stdlib.

$ ../ninja-build/ninja swift-stdlib && bin/swift

Once you’ve fleshed out your change you can run all tests to see if you broken anything with the following command:

$ cmake --build . -- check-swift-macosx-x86_64

For more information testing see https://github.com/apple/swift/blob/master/docs/Testing.md

If you are planning on more major surgery on the compiler you may want to go down the Xcode route which gives you full debugger capability on the compiler itself. To go down this route use the command:

$ utils/build-script --release-debuginfo —xcode

Your milage may vary and you might have you retry the build a few times and create a symbolic link or two but it’s worth the effort as you can find your way around the source far more easily. There’s a very good talk about this approach https://www.youtube.com/watch?v=oGJKsp-pZPk&feature=youtu.be. Afterwards you have a file ../build/Xcode-RelWithDebInfoAssert/swift-macosx-x86_64/Swift.xcodeproj which you can open in Xcode. It has 800 or so targets and when it asks you if you want to manually manage schemes, say yes and add the swift target. Change the run scheme’s build configuration to “RelWithDebInfo” to avoid have to rebuild everything again and you should be able to run and debug the compiler. You can also add targets for stdlib, testing etc.

If you are working on a syntax change that you’d like to test inside Xcode you can build a entire toolchain with the following command:

$ ../../../swift/utils/build-toolchain com.example

This takes a fair old while and creates tar files you can extract into ~ that can then be selected from inside Xcode as a toolchain. If your patch is crashing somewhere in SourceKit and you need symbolication extract the -symbols archive as well.

So, once you have you change ready. check it into the branch and you can open a PR on apple/swift.

If, as inevitably happens you have to wait a while before your PR gets merged you might end up with conflicts and need to rebase. The recipe for this that I use is typing the following with your brach checked out in your clone:

$ git remote add apple https://github.com/apple/swift.git
$ git fetch apple
$ git rebase apple/master

This will remove your changes, bring your repo up to date with apple’s repo then attempt to remake the changes on your branch. If there are still conflicts you’ll have to edit the file looking for the <<< markers then stage and:

$ git rebase —continue

At this point you can’t just pull/push/commit your changes as you would normally but must use the following command:

$ git push -f origin 

To start a new project you can rebase (fast forward) your master branch in this way and force push it then create a new branch to work on as a starting point. You may need to update your other clones with another:

$  ./swift/utils/update-checkout —clone

Hopefully I’ve not lead anybody too far astray with my half understanding of git. If so feel free to put me in the right.

I’ve just noticed there are some more very handy notes here: What should I learn if I want to contribute to the Swift compiler?