Need a workflow advice

Hello guys, I'm currently playing around with some internals of the Swift compiler and would like to check if the changes take effect on a real Swift file. Can someone advice me how and where should I create a test file and convince Xcode only to compile that particular Swift file only (ideally by also excluding the stdlib if that's possible).

I'm a completely newcomer on that field and the project codebase is really overwhelming. Gladly Xcode 9 provides some refactoring tooling that helped me change a few things quickly without breaking everything (mostly just some renaming). The most interesting change I made affected swiftAST.

1 Like

There are lots of options here and I'm not really sure what you're trying to do. If you just want to do something quick to try it out, you can go to the build log for some project in Xcode, expand the details of a Swift compile command and copy it. You can edit that command to invoke your modified compiler instead of the one from Xcode, and then run it from a terminal.

If you want something more than that, you can use the utils/build-toolchain script in the Swift repo to create a toolchain that Xcode can use. You should be able to install that custom toolchain in the same way as a development toolchain downloaded from

Well let's say for example you'd rename the var keyword to variable and would like to test it. Again I'm playing around with the compiler, learning very small bits. Then I'd like to create a swift file and check if the compiler now accepts variable instead of var.

I've heard that building a whole toolchain can take hours, this seems not that efficient to me to check small changes like this. Would you mind to provide a small example on the former if that's not a complicated or time intensive for you? I literally have no idea where to start.

1 Like

I'm pretty sure you can incrementally compile things after the first time you compile a toolchain. I don't remember what the details are, but you can do it. That speeds up things considerably :slight_smile:

Where do I find documentations on how to create a toolchain actually? I still feel lost in the humongous project infrastructure.

The main repo has instructions in the Readme. Have you tried those?

Remember to:

  1. Install the dependencies (clang and such), specially the right versions. I've forgotten to do so before and finding the problem was a pain in the ass.
  2. Ask the update-checkout script to download a specific tag, like a release one. That way you are guaranteed to have a more stable version of the toolchain than what is in the master branches of the repos.

Sorry FĂ©lix but I do not follow. What I did before is building the swift project like it's described in the README but I don't remember which flags I used actually (build the project a few weeks ago, but had no time to play around with it).

The build folder has two sub-folders:

  • Ninja-DebugAssert
  • Xcode-RelWithDebInfoAssert+swift-DebugAssert (~22GB)

I don't see anything related to toolchains in the README file except for a reference to the docs with dozens of different files. I really would like to know where I should start reading.

I'm currently looking into this file, because in this video @codafi used a tool called lit to test a specific file. So I get the impression it's what I was looking for, I can test a specific swift file without the need to compile a toolchain right?

Ohh. That's a good question. I don't know if you can test a tool w/o compiling the toolchain at least once. But if Robert said it can be done, then it's true.

Regarding the compilation steps, I will try to give you a walkthrough when I come back home. Atm I'm on the phone and that's just not good enough xD

Thank you FĂ©lix, take the time you need, I'm not in the rush here, since I won't become a compiler engineer over night. :smiley:

1 Like

If you just want to run the compiler on a simple testcase, you don't need to build a toolchain. Just use utils/build-script and run the swiftc binary that gets created. It's easiest to do this from a shell instead of trying to get it set up to work from Xcode.

If you go into the ninja folder you should see swift-macosx-x86_64/ go into that and you can run ninja commands in there to do pretty easy incremental builds (ninja swift note this does not build the stdlib, that’s ninja swift-stdlib). From there you can use bin/swiftc to compile swift files, there’s also the REPL you can use (bin/swift). I made an environment variable to get easy access to this folder (cd ${SWIFT_BUILD}). As for testing, llvm lit tests are by far that fastest (I also setup an environment var ${SWIFT_TEST} —filter=testnamehere). I can go more in depth to my workflow when I get home however. Also, I don’t use Xcode, so if you’re really keen on using Xcode then I can’t help you out here.

1 Like

You absolutely do not need a toolchain to test!

You say that you have a Ninja-DebugAssert build and an Xcode-RelWithDebInfoAssert+swift-DebugAssert build, so (assuming you have a mac - you can replace the macosx-x86_64 with the arch of your choice as long as you've got it built)


$ cd ./build/Ninja-DebugAssert/swift-macosx-x86_64/ # Commands Assume You Run Me First!

Full build

$ ninja

Incremental build

$ ninja <target1> <target2> ... <targetn> # Use `ninja -t targets` to list these 

Running Swift

$ ./bin/swift /Path/To/test.swift

Getting a Development REPL

$ ./bin/swift

Running Tests

$ cd /Path/To/Swift/Source/.. # Commands Assume You Run Me First! (Note the ..)

Running All The Tests

$ ./llvm/utils/lit/ ./build/Ninja-DebugAssert/swift-macosx-x86_64/test-macosx-x86_64/

Running The Tests Without Filling Your Shell

$ ./llvm/utils/lit/ -sv ./build/Ninja-DebugAssert/swift-macosx-x86_64/test-macosx-x86_64/

Oh Crap, I Broke This Test And Need To See All The Output

$ ./llvm/utils/lit/ -a ./build/Ninja-DebugAssert/swift-macosx-x86_64/test-macosx-x86_64/Path/To/Test.swift

Oh Crap, I Broke The Validation Tests (Wait, We Have Validation Tests??)

$ ./llvm/utils/lit/ ./build/Ninja-DebugAssert/swift-macosx-x86_64/validation-test-macosx-x86_64/

Why Am I Running All These Tests, Can't I Just Run A Subset?

$ ./llvm/utils/lit/ ./build/Ninja-DebugAssert/swift-macosx-x86_64/test-macosx-x86_64/ --filter=<REGEX>

Caveat: Lit drops multiple --filter flags and only takes the last one

Writing Tests

$ cd /Path/To/Swift/Source/ # Commands Assume You Run Me First!

Open an existing file under ./test. If you really can't find one/want to play around, write one and put it in the right folder.

RUN Lines: Tell Lit what to do (they're comments so Swiftc doesn't barf - don't name anything RUN: or REQUIRES:, etc. or you will have a bad time)

// RUN: <shell commands & Lit Substitutions>
// RUN: <more shell commands & Lit Substitutions>
// ...
// RUN: etc.
// See ./test/lit.cfg (look for config.substitutions)


// These Are Equivalent (Now you see why we use the substitutions)
// RUN: %target-typecheck-verify-swift
// RUN: /build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swiftc -frontend -target x86_64-apple-macosx10.9  -module-cache-path '/build/Ninja-DebugAssert/swift-macosx-x86_64/swift-test-results/x86_64-apple-macosx10.9/clang-module-cache' -sdk /Applications/ -swift-version 3  -typecheck -verify -disable-objc-attr-requires-foundation-module /swift/test/Path/to/file.swift -module-name MyModule

REQUIRES Lines: Sometimes you've gotta gate your test

// REQUIRES: objc_interop # I need Objective-C
// REQUIRES: executable_test # Test needs to build and run a binary 
// REQUIRES: OS=macosx # Only macOS
// REQUIRES: CPU=x86_64 # Only Intel x86_64
// REQUIRES: rdar777777 # I'm disabling/need to fix this test later, sorry, here's a radar number
// See ./test/lit.cfg for more

I was gonna post like, two of those lines.

But Robert blew it out of the park :D

Also, that answer is really, really good. Can we pin it somewhere? It's like a tl;dr for the whole toolchain debugging workflow. If we don't have it posted in the official repo, maybe we should. Or maybe linking this answer from the Readme.

IMO The natural place to put this is either in debugging the compiler or in a file next to it in the docs directory.