Swift mentee, (in the Swift.org mentorship program), having trouble compiling the Swift compiler. Please help!

Hey all,

I'm taking part in the Swift.org mentorship program and have been tasked with compiling the Swift compiler.

I've followed the getting started guide on How to Set Up an Edit-Build-Test-Debug Loop

  • Created a dir.
  • Cloned the sources, via SSH
  • Dbl checked that swift's sibling directories are present llvm-project, swiftpm etc.
  • Installed cmake, ninja, and sccache via home-brew and spot checked their versions.
  • Ran Sccache, sccache --start-server
  • I tried building via Ninja with utils/build-script --skip-build-benchmarks \ --skip-ios --skip-watchos --skip-tvos --swift-darwin-supported-archs "$(uname -m)" \ --sccache --release-debuginfo --swift-disable-dead-stripping --test
  • I've also tried building via the Xcode build-script
  • But my build always fails ->



Failed Tests (11):
Swift(macosx-x86_64) :: Index/Store/output-failure.swift
Swift(macosx-x86_64) :: Interop/Cxx/class/inheritance/fields.swift
Swift(macosx-x86_64) :: Interop/Cxx/class/inheritance/functions.swift
Swift(macosx-x86_64) :: Interop/Cxx/foreign-reference/pod.swift
Swift(macosx-x86_64) :: Interop/Cxx/operators/member-inline.swift
Swift(macosx-x86_64) :: Interpreter/SDK/libc.swift
Swift(macosx-x86_64) :: ModuleInterface/ModuleCache/force-module-loading-mode-archs.swift
Swift(macosx-x86_64) :: ModuleInterface/ModuleCache/force-module-loading-mode-framework.swift
Swift(macosx-x86_64) :: ModuleInterface/ModuleCache/force-module-loading-mode.swift
Swift(macosx-x86_64) :: remote-run/upload-and-download.test-sh
Swift(macosx-x86_64) :: remote-run/upload-stderr.test-sh

Testing Time: 1513.53s
Unsupported : 294
Passed : 8490
Expectedly Failed: 23
Failed : 11

4 warning(s) in tests
FAILED: test/CMakeFiles/check-swift-macosx-x86_64 /Volumes/LaCie/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/test/CMakeFiles/check-swift-macosx-x86_64
cd /Volumes/LaCie/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/test && /usr/local/Cellar/cmake/3.23.2/bin/cmake -E remove_directory /Volumes/LaCie/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/./swift-test-results/x86_64-apple-macosx10.9 && /usr/local/Cellar/cmake/3.23.2/bin/cmake -E make_directory /Volumes/LaCie/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/./swift-test-results/x86_64-apple-macosx10.9 && /usr/local/Cellar/cmake/3.23.2/bin/cmake -E env /Applications/Xcode.app/Contents/Developer/usr/bin/python3 /Volumes/LaCie/swift-project/llvm-project/llvm/utils/lit/lit.py --param bootstrapping_mode=BOOTSTRAPPING-WITH-HOSTLIBS -sv -j 8 --xunit-xml-output=/Volumes/LaCie/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/./swift-test-results/x86_64-apple-macosx10.9/lit-tests.xml --param differentiable_programming --param concurrency --param distributed --param string_processing --param threading=darwin --param swift_test_subset=primary --param swift_test_mode=optimize_none /Volumes/LaCie/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/test-macosx-x86_64
ninja: build stopped: subcommand failed.
ERROR: command terminated with a non-zero exit status 1, aborting

Other notes and specs:

  • macOS Monterey 12.4
  • Apple M1
  • Xcode version 13.4

I've cloned the project to an external SSD cause it ended up being to large for my internal drive.
Resources are slim and I'm not familiar with this toolchain.
I would appreciate any help, feedback, or a point in the right direction!

Thanks!

3 Likes

Try building with

(swift)$ utils/build-script \
                --release  \
                --debug-swift

In most cases, passing --release and --debug-swift options often makes it build the swift compiler with debugging symbols. Though debugging symbols consume space, and the build will take longer, but it can attach a lldb debugger to compiler which will be helpful in knowing what it is doing.
If the build is successful, it will create a swift compiler executable. You can then invoke the executable at the following path:

swift-macosx-arm64/bin/swiftc --version

The above command will then provide you with the version and the target.

Hope that helps!

1 Like

You don't need all the tests to pass, unless building a toolchain. Of the 11 failed tests, it's possible that some have since been fixed, so you could try utils/update-checkout again. Otherwise, you could try utils/build-script without the --test option, and then run only the tests needed (by whatever you're working on). For example:

utils/run-test \
  --build-dir=../build/Ninja-RelWithDebInfoAssert \
  --build=true \
  --mode=optimize_none \
  --subset=all \
  --target=macosx-x86_64 \
  --verbose \
  test/stdlib/CharacterProperties.swift \
  test/stdlib/UnicodeScalarPropertiesTests.swift

It's strange that the failing tests are for macosx-x86_64 on your M1 computer. With the --swift-darwin-supported-archs "$(uname -m)" option, I'd expect only macosx-arm64 tests, unless the Terminal app is using Rosetta.

1 Like

Thanks so much! I'll give it a shot

Thanks for the response! Yeah that's what I was trying to figure out - everything that's created in the build folder after running the build-script ends up being x86_64. I was expecting to generate or ../build/Xcode-RelWithDebInfoAssert/swift-macosx-arm64

I'm getting this build error now:

@Jazek I've seen code signing issues before, and normally they were because there wasn't enough disk space left in the machine for it to be able to perform the operation. So maybe you could check for that.
Checkout system requirements section for info on minimum. Xcode build needs a lot o disk space specially with debug info.

Hey, thanks for the reply! I cloned the repo to an external SSD and am running the build-script on that drive (not enough room on my internal drive). Do you think it's possible I'm getting code-signing errors cause my OS and Xcode are on a separate drive?

I thinks is possible, but couldn't affirm for sure because I never used it like that. But when I did have code signing issues they were related to low disk space, is possible that code sign step is still using your internal drive and failing since there is no enough space, but is also possible that is a different issue. You can look up in the logs and normally there is a more detailed message about the code sign command the failure if I'm not mistaken. One thing it may be worth trying is to run the command while observing disk space to check if while code signing what drive is lowering in space and confirm if that is the case.
Hope that helps =]


So I wiped my internal drive and cloned / compiled there. Still getting a failing build:

1 Like

Looks like the build is OK, testing seems to be failing... are you using --test? This flag together with --xcode is known for causing issues. I would recommend to run --xcode build without --test and run tests on a ninja setup. But also, some of those tests are expected to fail even on ninja build(at least for me they always do locally) but is fine to ignore them if they are unrelated to your changes =]
We could probably open a thread or issue about them at some point...

Okay thanks! I'll try --xcode build without --test and see. Do you have any ideas about why I'm not generating arm64 directories in the build folder? They're all x86_64 and I'm on an M1...d

Also, I'm getting a Link swift-serialize-diagnostics (x86_64) error in the generated swift.xcodeproj project..
Screen Shot 2022-07-31 at 2.54.50 AM

Which command are you running? The --swift-darwin-supported-archs "$(uname -m)" would ensure that if your are on M1 you get arm64. If you run uname -m in your terminal what is the output? Maybe running terminal on rosetta? Not really sure...

This is a problem that I still face as well, it happens because llvm libraries are built it the relwithdebug folder but when linking Xcode build is trying to look up in debug folder ... not really sure why, my attempts to dig into cmake files and figure this out were not successful :sweat_smile:
I'd suggest running everything Debug mode with --debug instead of --release-debuginfo

The full command
./swift/utils/build-script --xcode --debug --skip-build-benchmarks --swift-darwin-supported-archs "$(uname -m)" --sccache --skip-ios --skip-tvos --skip-watchos --skip-test-cmark --bootstrap=off,
It costs a little bit more disk space but that probably works.

1 Like

Thank you everyone for all the help! I was able to successfully build with a few pointers and script from @QuietMisdreavus !

build Xcode projects
utils/build-script --skip-ios --skip-tvos --skip-watchos --skip-build-benchmarks --swift-darwin-supported-``archs=x86_64 --xcode --debug --bootstrapping=off

build Ninja configurations and run tests
utils/build-script --skip-ios --skip-tvos --skip-watchos --skip-build-benchmarks --swift-darwin-supported-``archs=x86_64 --release --test --skip-test-cmark
Screen Shot 2022-08-01 at 3.13.39 PM

3 Likes