New contributor feedback & development workflow advice

Development Compiler
, ,
1

hello all – i relatively recently have been trying to become more comfortable navigating the compiler and stdlib, so that i can contribute to them in various ways. i feel i've made decent progress in this endeavor, but there are many things that i still find confusing and difficult, so i'm looking for any advice people more experienced with this domain are willing to offer. i also have some general thoughts/reflections to share on how the process has seemed to me thus far. so, in no particular order:

update-checkout

how often do you run the update-checkout script (or is this even what you use?)? i feel i've come to have a sort of 'learned fear' of using it frequently because it seems most times i run it, i seem to be unable to rebuild whatever component i was working on incrementally (via ninja). i then end up wiping my build directory and rebuilding everything, which can take ~30-50 minutes.

IDE configuration

  • what IDE setup do you use (or would you recommend trying) for working on the compiler?
  • is your recommendation the same for working with the C++ components vs the Swift ones?
  • do you use a setup different from those outlined in the getting started guide?
    • if so, how does it differ?

so far i have used both Xcode and VSCode, but don't feel like my workflows are particularly good in either. more specifically:

Xcode:

i've not actually used the recommend Xcode-with-ninja setup that much, but it seems like there is potential there. i find the scheme management a bit confusing, and the symbol indexing has been hit-or-miss sometimes (e.g. right now the Xcode UI reports ~60% done indexing on the workspace project i generated... months ago?). i still haven't figured out how to get cmd+b to build the things i want (i think this is covered in the guide, so perhaps my own shortcoming), and generally just run build invocations from the command line. autocomplete (in both C++ and Swift contexts) seems worse than i typically experience when doing generic iOS development – often times it seems like sourcekit gets confused about whether a symbol is or is not available in a given context, and when autocomplete doesn't work, it is really a drag on my muscle memory & productivity.

VSCode:

i've probably used VSCode the most so far, and it has been just okay. i have Microsoft's C/C++ extension installed, which offers decent symbol jumping and documentation surfacing capabilities. i'm definitely more comfortable/familiar with Xcode though, so would prefer to switch if i can figure out a better setup. autocomplete also feels rather middling in this tool.

debugging

this is probably the area i feel most limited by with respect to what i'm used to in the Xcode-based-iOS-developer context. when working with the C++ portions of the compiler in particular i've found myself almost always resorting to print debugging, which is quite painful as it requires recompilation, and even the syntax i've settled on for emitting debug print statements is pretty awkward to use (mainly by passing things to llvm::errs() in various ways). i have had some limited success running lldb via the command line and then invoking the compiler on a file after having tried to set some breakpoints. issues i've run into here though are:

  1. configuring the breakpoints is way more tedious than setting them via the UI in an IDE
  2. often times things i'd like to inspect in the debugger have been optimized away, or it's not clear to me how to access/interpret the relevant bits via the debugger

i assume both of these could probably be improved by some workflow changes (maybe i'm not building with the right debug configuration?). i'm curious though if anyone has a setup that enables using breakpoints via an IDE though, as that's what i'm currently most familiar with. so if you have to debug the compiler, how do you do it?

C++

i know Doug Gregor has recently published a series of blog posts 'Swift for C++ Practitioners', but i feel i could use the converse :slight_smile:. various LLMs have been pretty good at helping explain things, but if anyone knows of a decent primer that would be useful for the patterns that exist in the compiler, i'd be happy to hear them.

kudos

lest it seem like i'm mostly complaining in this post, i wanted to explicitly call out some things that have improved my experience thus far:

  • the getting started guide was extremely useful in getting off the ground at all. without something like this, i probably would have given up on this endeavor as there are a lot of things i'm not particularly familiar with regarding the build systems involved. thank you to the maintainers of this (and related) documents!
  • all the reviewers involved in my PRs have been very welcoming, helpful, and supportive – thank you!
  • the swift.org community more broadly has been quite helpful in generally helping to resolve my questions – thanks!
2 Likes
2

Whenever I make a new branch based on upstream/main and it fails to build the compiler due to some swift-driver, swift-syntax, or LLVM change I will run update-checkout to get those new sources. After running this command, it is expected that you'd need to rebuild those dependencies and the compiler/stdlib itself.

2 Likes
3

thanks for the feedback Alejandro. to clarify – after you run update-checkout, do you then generally do an incremental build via Ninja somehow, or do you re-run the build-script utility and rebuild everything (or just some particular targets)? i have generally found after updating my local branch, incremental builds typically end up failing for me with build errors i usually do not understand (sometimes they seem to build for a bit, but then linking fails or somesuch). when i fall back to re-running build-script it can take between ~40-90 minutes depending on the config options used (on a 2021 M1 Max MBP). this is tolerable, but i'd like to ensure i'm minimizing the re-build times to the extent that can be done.

4

This shouldn't be necessary most of the time. In my experience it is usually sufficient to update the Swift repo and then rebuild with ninja. If ASTGen fails to build, then swift-syntax needs to be updated. If something in the ClangImporter fails, probably LLVM. The remaining components don't change enough for it to matter.

I wonder what issues you're having that cause you to regularly wipe your build directory -- I probably end up doing this once a month at most.

This might be controversial but I just use a text editor (Sublime Text right now, emacs and vi in the past).

Just run ninja bin/swift-frontend or ninja swift-stdlib in the build directory.

git grep :-)

I sometimes use lldb to get a backtrace and print variables after an assertion failure or some such, for the most part I just use print debugging. It works quite well in the Swift frontend, because many data types provide dump(), print() and operator<< methods. You can also wrap the print statements in an LLVM_DEBUG so that they can be switched on and off with a command line flag. Finally, if you find yourself often setting a breakpoint to check the state of some thing, add an assertion instead. Rebuilding after making a small change should only take a few seconds (unless you change a header file that's included in many places).

There is a "Source Code Reference" sections at the end of Chapter 3 and 4 of https://download.swift.org/docs/assets/generics.pdf describes a handful of common patterns, like the isa, cast, and dyn_cast template functions for example.

You might also find this book about Clang interesting since Clang and Swift share a similar coding style: Google Books

But really, it's just about reading and writing code and spending time in the code base.

8 Likes