Error: no such module 'Foundation' issue with Linux toolchain

[Kyle-Ye (Kyle)]

How can I add Foundation(swift-corelib-foundation) support in a local build toolchain on Linux? I try to pass --foundation to the build-script but it still does not work.

I'm following the doc here swift/docs/HowToGuides/FAQ.md at main · apple/swift · GitHub

Build Script Log: /home/kyle/swift-project/build/.build_script_log
Build Percentage 	 Build Duration (sec) 	 Build Phase
================ 	 ==================== 	 ===========
76.7%             	 78.84                 	 linux-aarch64-foundation-build
15.2%             	 15.66                 	 linux-aarch64-libdispatch-build
7.6%              	 7.78                  	 Building llvm
0.2%              	 0.18                  	 linux-aarch64-swift-build
0.0%              	 0.05                  	 Building cmark
0.0%              	 0.04                  	 linux-aarch64-foundation-test
0.0%              	 0.03                  	 linux-aarch64-swift-test
0.0%              	 0.03                  	 linux-aarch64-libdispatch-test
0.0%              	 0.03                  	 linux-aarch64-swift-install
0.0%              	 0.03                  	 linux-aarch64-libdispatch-install
0.0%              	 0.03                  	 linux-aarch64-foundation-install
0.0%              	 0.03                  	 merged-hosts-lipo-core
0.0%              	 0.03                  	 linux-aarch64-extractsymbols
0.0%              	 0.03                  	 linux-aarch64-package
0.0%              	 0.03                  	 merged-hosts-lipo
Total Duration: 102.82 seconds (1m 42s)

The full command is utils/build-script --release-debuginfo --foundation --skip-early-swift-driver --skip-early-swiftsyntax --sccache (Build against release/5.10 on Ubuntu 22)

And then I use SWIFT_EXEC to build the demo package.

SWIFT_EXEC=xx/Ninja-RelWithDebInfoAssert/swift-linux-aarch64/bin/swiftc swift build
Building for debugging...
DemoKit/Sources/DemoKit.swift:2:8: error: no such module 'Foundation'
import Foundation

[Max_Desiatov]

You most probably need to pass --install-foundation and also corresponding flags to install dependencies of Foundation.

In general, you should only call build-script with a given preset that's known to work (which are the CI presets). Trying to build passing flags manually one-by-one will only lead to frustration most of the time, as some flags are incompatible and some have implicit dependencies on each other.

[Kyle-Ye (Kyle)]

@Max_Desiatov

In general, you should only call build-script with a given preset that's known to work (which are the CI presets).

Thanks. I will give --install-foundation a try.

@Max_Desiatov

You most probably need to pass --install-foundation and also corresponding flags to install dependencies of Foundation.

I did not see such information in GettingStarted documentation. Do I miss something here?

And which preset is recommended? My requirement is debug Swift compiler for a crash on Linux. The CI presets may not fit my needs.

Cannot look up associated type for imported conformance:
(struct_type decl=CDemoKit.(file).CAttribute)
(associated_type_decl "_ObjectiveCType" access=public overridden=)

[Max_Desiatov]

@Kyle-Ye

And which preset is recommended?

There are no generally recommended presets as everyone's needs are different. I personally recommend presets used on CI as those more often than not lead to successful builds and you can get the test suite passing with them.

With toolchain presets you can also replicate what's published on Swift.org - Download Swift, which is important if you're trying to reproduce a bug seen in one of those toolchains.

[Kyle-Ye (Kyle)]

@Kyle-Ye

Thanks. I will give --install-foundation a try.

Still the same results. I'll try to use the CI resets here then. (I guess the CI is using buildbot_linux currently)

[Kyle-Ye (Kyle)]

@Max_Desiatov

With toolchain presets you can also replicate what's published on Swift.org - Download Swift , which is important if you're trying to reproduce a bug seen in one of those toolchains.

utils/build-script --preset buildbot_linux,no_assertions,no_test --sccache install_destdir=/home/kyle/swift-nightly-install installable_package=/home/kyle/null-ubuntu2204.tar.gz
utils/build-script --preset buildbot_linux --sccache install_destdir=/home/kyle/swift-nightly-install installable_package=/home/kyle/null-ubuntu2204.tar.gz

Tried the above presets here, they both failed for the following error message

+ /home/kyle/swift-project/build/cmake-linux-aarch64/bin/cmake --build /home/kyle/swift-project/build/buildbot_linux/swift-linux-aarch64 -- -j10 all swift-stdlib-linux-aarch64 swift-libexec-linux-aarch64
ninja: error: '/home/kyle/.local/lib/swift/linux/aarch64/swiftrt.o', needed by 'bin/swift-frontend', missing and no known rule to make it
ERROR: command terminated with a non-zero exit status 1, aborting

Does it because I'm passing a wrong install_destdir/installable_package here?

kyle@builder:~/.local$ ls
bin  share
kyle@builder:~/.local$ ls bin
clang        docc            lldb-server       sourcekit-lsp           swift-build-sdk-interfaces  swift-help                 swift-test
clang++      ld.lld          llvm-ar           swift                   swift-build-tool            swift-package              swiftc
clang-15     ld64.lld        llvm-cov          swift-api-checker.py    swift-cache-tool            swift-package-collection   swiftly
clang-cache  lld             llvm-profdata     swift-api-digester      swift-demangle              swift-package-registry     wasm-ld
clang-cl     lld-link        plutil            swift-api-extract       swift-driver                swift-plugin-server
clang-cpp    lldb            repl_swift        swift-autolink-extract  swift-experimental-sdk      swift-run
clangd       lldb-argdumper  sdk-module-lists  swift-build             swift-frontend              swift-symbolgraph-extract
kyle@builder:~/.local$

/home/kyle/.local/bin/swift here is installed and managed by swiftly IMO.
I do not know why build-script + preset will use the same location here.

[Max_Desiatov]

build-script currently doesn't support swiftly as the latter puts the compiler and its libraries in locations that build-script doesn't expect. You need to use the same filesystem layout that upstream Swift Docker images use, or ideally and most straightforwardly just use those Docker images directly, as that replicates Swift CI setup almost exactly.

[Kyle-Ye (Kyle)]

@Max_Desiatov

ideally and most straightforwardly just use those Docker images

Forgive me if this sounds to be a silly question.

But I can't find the Docker image you are talking. The Docker images / Dockerfile linked by swift.org seems just downloading the prebuilt toolchain products instead of building them from bottom up.

apple/swift-docker/blob/main/5.9/ubuntu/22.04/Dockerfile

[Max_Desiatov]

That is expected. You need a preinstalled Swift toolchain to build a new toolchain with how bootstrapping works. git clone, update-checkout, and build-script invocations are not included in these Dockerfiles, you're supposed to run it yourself in a container spawned from these images.

The main point is that these images have Swift toolchains preinstalled in correct locations that build-script can handle, which is not the case with swiftly.

[Kyle-Ye (Kyle)]

1. Following the Dockerfile config to set up a preinstalled Swift 5.10 toolchain.

2. Run the following command to successfully build a working toolchain.

utils/build-script --preset buildbot_linux,no_assertions,no_test --sccache install_destdir=/home/kyle/swift-nightly-install installable_package=/home/kyle/null-ubuntu2204.tar.gz

3. Run the test package via my local toolchain and reproduce the bug here.

SWIFT_EXEC=/home/kyle/swift-nightly-install/usr/bin/swiftc swift build

I can now debug it successfully.

Thank you for the comprehensive guidance in helping me set up a Linux toolchain build.

[Kyle-Ye (Kyle)]

Set up lldb to capture the issue, but it does not contain any debug info.

I guess I should rerun a new preset with debug-info for swift/llvm repo.

Following other preset examples, I create a new preset for my purpose, but it will fail here.

Do you know is there any documentation of the build-presets.ini file? @Max_Desiatov

Looks like the release-debuginfo here is conflicting with buildbot_linux's release option.

[preset: buildbot_linux_kyle]
mixin-preset=buildbot_linux,no_test

release-debuginfo
assertions
debug-llvm
debug-swift

kyle@builder:~/swift-project/swift$ utils/build-script --preset buildbot_linux_kyle --sccache install_destdir=/home/kyle/swift-nightly-install installable_package=/home/kyle/null-ubuntu2204.tar.gz
[utils/build-script] NOTE: using preset "buildbot_linux_kyle", which expands to 

utils/build-script --assertions --swift-enable-ast-verifier=0 --no-swift-stdlib-assertions '--swift-install-components=autolink-driver;compiler;clang-resource-dir-symlink;libexec;stdlib;swift-remote-mirror;sdk-overlay;static-mirror-lib;toolchain-tools;license;sourcekit-inproc' '--llvm-install-components=llvm-ar;llvm-cov;llvm-profdata;IndexStore;clang;clang-resource-headers;compiler-rt;clangd;lld;LTO;clang-features-file' --llbuild --swiftpm --swift-driver --xctest --libicu --swiftdocc --build-ninja --install-llvm --install-swift --install-lldb --install-llbuild --install-swiftpm --install-swift-driver --install-swiftsyntax --install-xctest --install-libicu --install-prefix=/usr --install-sourcekit-lsp --install-swiftformat --install-swiftdocc --build-swift-static-stdlib --build-swift-static-sdk-overlay --build-swift-stdlib-unittest-extra --test-installable-package --toolchain-benchmarks --install-destdir=/home/kyle/swift-nightly-install --installable-package=/home/kyle/null-ubuntu2204.tar.gz --relocate-xdg-cache-home-under-build-subdir --build-subdir=buildbot_linux --lldb --release --test --validation-test --long-test --stress-test --test-optimized --foundation --libdispatch --indexstore-db --sourcekit-lsp --swiftdocc '--lit-args=-v --time-tests' --lldb-test-swift-only --install-foundation --install-libdispatch --reconfigure --skip-test-cmark --skip-test-lldb --skip-test-swift --skip-test-llbuild --skip-test-swiftpm --skip-test-swift-driver --skip-test-xctest --skip-test-foundation --skip-test-libdispatch --skip-test-playgroundsupport --skip-test-libicu --skip-test-indexstore-db --skip-test-sourcekit-lsp --skip-test-swiftdocc --release-debuginfo --assertions --debug-llvm --debug-swift --sccache

usage: 
  build-script [-h | --help] [OPTION ...]
  build-script --preset=NAME [SUBSTITUTION ...]
build-script: error: argument -r/--release-debuginfo: not allowed with argument -R/--release
ERROR: command terminated with a non-zero exit status 1, aborting

[Max_Desiatov]

@Kyle-Ye

Do you know is there any documentation of the build-presets.ini file?

There's no documentation on this file that I know of. But it's relatively simple, being a collection of flags passed to build-script. One of the mixed-in presets seems to contain a release flag that you have to remove for release-debuginfo to be allowed. If this is a one-off, you can just run textual search on build-presets.ini and remove all mentions of release. Hopefully this helps.