What is the development workflow for Swift stdlib changes?

I have a few questions related to swift stdlib development workflow -

  1. What is the workflow for testing out stdlib changes? For the compiler related changes, I believe the workflow is to make the relevant changes, re-build to generate newer swift-frontend or swiftc executable and try to compile a Swift source file to generate an executable or intermediate obj-file. But recompiling stdlib wouldn't really regenerate any executable. So, how do I use the newly compiled stdlib? For example, plug it into a playground or pass it as an argument or flag when trying to run a swift file which has some test code with new behavior?

  2. When I setup my local environment based on https://github.com/apple/swift/blob/main/docs/HowToGuides/GettingStarted.md and finish the Xcode setup, when I try to search for Unicode in the source tree in Xcode (left pane, filter bar at the bottom), there are a lot of reds, indicating that source files are missing or not generated. For example - UnicodeExtendedGraphemeClusters.cpp.gyb is a file that exists, but the corresponding UnicodeExtendedGraphemeClusters.cpp is marked as red in Xcode. I see this for the files listed below, and wanted to check if it is just me or is anyone else also seeing that:

  • UnicodeExtendedGraphemeClusters.cpp
  • UnicodeExtendedGraphemeClusters.cpp.rule
  • UnicodeGraphemeBreakTest.cpp
  • UnicodeGraphemeBreakTest.cpp.rule
  • StdlibUnicodeUnittest
  • StdlibUnicodeUnittest.o.rule

There are probably better ways to do this, but when I made changes to the stdlib I found an appropriate test file in test/Interpreter and modified it, then ran that specific test with a command like ./utils/run-test --build-dir <path to the build> <path to the specific test file>, for example: ./utils/run-test --build-dir /Users/benjaminpious/Documents/swift-project/build/Ninja-RelWithDebInfoAssert /Users/benjaminpious/Documents/swift-project/swift/test/Interpreter/keypath.swift.

You can also build the whole toolchain and integrate it into Xcode, which I expect is the most time consuming option, but will give you the best developer experience afterwards, b/c it's the same developer experience you have with the toolchain that ships with Xcode.

To the second point; I'm guessing these aren't generated till you run the build (in case you're unaware, "gyb" is an acronym for "gen your boilerplate," and these files are responsible for outputting the files you've listed. If the xcode project generation command generated these, they'd probably be marked as uneditable, because they'd be overwritten during the build process anyway.

2 Likes

You can use the DYLD_LIBRARY_PATH environment variable to point your executable to a new stdlib, but it's worth noting that for faster iteration and convenience it's often easier to make a standalone project and use --parse-stdlib and --disable-access-control (typed from memory, flag names may not be 100% accurate) to simulate being in the stdlib. Then once you're pretty sure it works, you can move it into the real stdlib.

4 Likes

Thank you @benpious and @David_Smith. I incorporated some of your suggestions and this is what I have for now:

  1. I make code changes to the stdlib and build through xcode
  2. I have a test swift file that I use to write swift code that makes use of the changes I made - say Test.swift
  3. I use the DYLD_LIBRARY_PATH flag as follows:
    cd ../build/Ninja-*/swift-macosx-*/bin
    `DYLD_LIBRARY_PATH='../lib/swift/macosx' ./swift -g -sdk $(xcrun -sdk macosx -show-sdk-path) <path_to_Test.swift>`
    

Similarly swiftc can also be invoked to generate an intermediate .o file or an actual executable. That executable can then be run as:
DYLD_LIBRARY_PATH='../lib/swift/macosx' ./Test

This is reasonably fast for me at the moment, especially as a newbie. Maybe there are better workflows that I may come across with time.

Thanks again for the pointers!