Need a workflow advice

You absolutely do not need a toolchain to test!

You say that you have a Ninja-DebugAssert build and an Xcode-RelWithDebInfoAssert+swift-DebugAssert build, so (assuming you have a mac - you can replace the macosx-x86_64 with the arch of your choice as long as you've got it built)

Ninja-DebugAssert

$ cd ./build/Ninja-DebugAssert/swift-macosx-x86_64/ # Commands Assume You Run Me First!

Full build

$ ninja

Incremental build

$ ninja <target1> <target2> ... <targetn> # Use `ninja -t targets` to list these 

Running Swift

$ ./bin/swift /Path/To/test.swift

Getting a Development REPL

$ ./bin/swift

Running Tests

$ cd /Path/To/Swift/Source/.. # Commands Assume You Run Me First! (Note the ..)

Running All The Tests

$ ./llvm/utils/lit/lit.py ./build/Ninja-DebugAssert/swift-macosx-x86_64/test-macosx-x86_64/

Running The Tests Without Filling Your Shell

$ ./llvm/utils/lit/lit.py -sv ./build/Ninja-DebugAssert/swift-macosx-x86_64/test-macosx-x86_64/

Oh Crap, I Broke This Test And Need To See All The Output

$ ./llvm/utils/lit/lit.py -a ./build/Ninja-DebugAssert/swift-macosx-x86_64/test-macosx-x86_64/Path/To/Test.swift

Oh Crap, I Broke The Validation Tests (Wait, We Have Validation Tests??)

$ ./llvm/utils/lit/lit.py ./build/Ninja-DebugAssert/swift-macosx-x86_64/validation-test-macosx-x86_64/

Why Am I Running All These Tests, Can't I Just Run A Subset?

$ ./llvm/utils/lit/lit.py ./build/Ninja-DebugAssert/swift-macosx-x86_64/test-macosx-x86_64/ --filter=<REGEX>

Caveat: Lit drops multiple --filter flags and only takes the last one

Writing Tests

$ cd /Path/To/Swift/Source/ # Commands Assume You Run Me First!

Open an existing file under ./test. If you really can't find one/want to play around, write one and put it in the right folder.

RUN Lines: Tell Lit what to do (they're comments so Swiftc doesn't barf - don't name anything RUN: or REQUIRES:, etc. or you will have a bad time)

// RUN: <shell commands & Lit Substitutions>
// RUN: <more shell commands & Lit Substitutions>
// ...
// RUN: etc.
// See ./test/lit.cfg (look for config.substitutions)

e.g.

// These Are Equivalent (Now you see why we use the substitutions)
// RUN: %target-typecheck-verify-swift
// RUN: /build/Ninja-ReleaseAssert/swift-macosx-x86_64/bin/swiftc -frontend -target x86_64-apple-macosx10.9  -module-cache-path '/build/Ninja-DebugAssert/swift-macosx-x86_64/swift-test-results/x86_64-apple-macosx10.9/clang-module-cache' -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk -swift-version 3  -typecheck -verify -disable-objc-attr-requires-foundation-module /swift/test/Path/to/file.swift -module-name MyModule

REQUIRES Lines: Sometimes you've gotta gate your test

// REQUIRES: objc_interop # I need Objective-C
// REQUIRES: executable_test # Test needs to build and run a binary 
// REQUIRES: OS=macosx # Only macOS
// REQUIRES: CPU=x86_64 # Only Intel x86_64
// REQUIRES: rdar777777 # I'm disabling/need to fix this test later, sorry, here's a radar number
// See ./test/lit.cfg for more
28 Likes