Getting started with tinkering with the standard library

#1

TL;DR: I want to contribute to the standard library, how can I do that without needing to build the entire Swift project?

I've enjoyed discussing ideas on here, but I would like to actually implement things. As a simple starter, I'd like to implement an extension on Optional (from this pitch). Even if it gets rejected in evolution, it seems like a simple feature to start with.

Is there a way to develop only the standard library? I don't have the 70 GB of free disk space that is required for building the Swift project. I also don't see myself overcoming the hurdles of jumping into C++ development (at least until the far, faaaaaaaaar future), so I don't really need to have my own build of the compiler, if it can be avoided.

Also, I've been rummaging a lot through the source code of Swift standard library. Naturally, ctrl-f and GitHub search are quite tedious. It would be much nicer to be able to open this up in Xcode and "jump to definition" all over the place. Is there a way to pull in the standard library's source into my personal projects, so if I command click on a standard library API, it'll take me to the real source, rather than the generated interface?

#2

There is a C++ compiler, clang++, or c++, that is already part of Xcode. You don't have build a separate compiler.

#3

My understanding was that the Swift GitHub repo contains not just the standard library, but also the Swift compiler frontend for LLVM (written in C++). That's what I was referring to. I don't plan to tinker with it, so I don't want to pay the cost (build time, disk space, workflow complexity) of having it.

#4

Ok. I can understand that. However, based on what I've read in the forums, there could be some interconnections between the compiler and the standard library that may affect what you may be working on. You may need to at least reference the source code. Optionals, if I remember correctly, are one of those standard library types that seem to be highly interrelated to the inner workings of the compiler.

1 Like
#5

That makes sense. But assuming my change isn't one that requires touching the compiler, can't the standard library work against the built Swift compiler I'll already have installed (not built myself, but through Xcode or whatever)

#6

NOTE THIS IS OUT OF DATE, SEE BOTTOM OF THIS POST

@AlexanderM I wouldn't attempt to use a shipped Xcode in this case. The stdlib may require features not in that Xcode. Instead I would suggest downloading a development toolchain (https://swift.org/download/#snapshots) from swift.org and using the preset that I am adding in this PR:

(You can take my branch or wait for it to merge, your choice). You will still need to do update-checkout though. But you will not need to build anything else. I just tried this locally and it worked for me.

One caveat: this preset doesn't work with tests (there is a dependency there that we can get rid of). But it may help you get on your way. = ).

EDIT: Note this branch is out of date! I am going to add instructions to the main README.md page to show how to do this. If I haven't done that yet, see the instructions from try! swift here: https://swift.org/quickstart-contribution/#build-swift-stdlib-against-a-downloadable-toolchain. Except I would use the most recent toolchain rather than one I suggested there and just try to build update-checkout against mater.

5 Likes
#7

12 hours ago

Wow, did you cook that up just for my request? Thanks!

#8

Dont worry I had ulterior motives as well. = p.

#9

@Michael_Gottesman I tried to do this using a fresh update-checkout at tag swift-DEVELOPMENT-SNAPSHOT-2019-06-23-a. The preset you refer to in the PR does not seem to exist (--preset=stdlib_RDA,standalone,notest). Setting toolchain_path appears to be understood but disregarded by the script: it promptly declares "build-script: note: Using toolchain default"

What's the right invocation to make it work?

#10

I need to update the documentation (I am juggling a few different things right now). I would use the latest master snapshot with ToT. It should work the majority of the time. I am talking with Mishal Shah to see if there is a way that we can avoid even small issues around inconcistencies.

@glessard tell me if it works plz = ).

#11

I'm unsure what I should try! Here's what I've got:

  • I'm working from a directory that contains all the checkouts below it
  • a fresh update-checkout, $ swift/utils/update-checkout
  • no build/ directory $ rm -rf build/

From there, the invocations I've tried have had the form:
$ swift/utils/build-script --preset={PRESETS_HERE} toolchain_path=/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2019-06-23-a.xctoolchain/usr/bin

The preset cited in https://github.com/apple/swift/pull/25078 (stdlib_RDA,standalone,notest) does not seem to exist. I've tried things that look somewhat like it from "utils/build-presets.ini", such as stdlib_RDA_standalone,build, but haven't gotten far. I feel like I must be taking a wrong turn early!

#12

Yea, I renamed them. It is going to start slow b/c you are compiling Swift.o which messes with the parallelism.

Do not give up hope! If it doesn't finish in 20 minutes you went down the wrong path. On my machine it generally finishes in < 5.

#13

I updated the notes above with a link to some instructions we used a try! swift. I am going to eventually put it into the main README.md. I am just messing with something else right now that I need to finish ASAP ; ).

#14

To be clear, I was referring to this page: https://swift.org/quickstart-contribution/#build-swift-stdlib-against-a-downloadable-toolchain

#15

Oh my, I did take a wrong turn early! Will try when I get back to my machine.

#16

It got farther this time (using stdlib_DA_standalone,build), and it seems to have errored on an actual error (perhaps an inconsistency between expectations and my actual system). I'll try to prod it along this afternoon.

For what it's worth, the beginning stages of the build used clang from the default toolchain, but later when compiling the stdlib it used swiftc from the toolchain I specified.

#17

I got it to (largely) work with --preset=stdlib_DA_standalone,build,test.
I had to tweak SceneKit.swift.gyb, and that caused the corresponding test to fail.
Thanks for working on this, @Michael_Gottesman!

#18

How long did it take you to build?

#19

There were many stops and starts so I'm not sure. Now that I have it with no compilation error, I zapped build/ and restarted so as to provide an answer.

#20

7 ½ minutes for stdlib_DA_standalone,build. This is not on a particularly fast machine (4-core Mac Pro 3,1 with PCIe SSD).

The git checkouts (from scratch) took a comparable amount of time than the configure-and-build.

1 Like
Terms of Service

Privacy Policy

Cookie Policy