Using swift-foundation with Linux static SDK

seanmrich · September 16, 2024, 9:56pm

I'm trying to build an executable using the latest Linux Static SDK. I want to use Foundation, but can't seem to get the tooling right. I did install the matching toolchain and static sdk, setting the current toolchain via the environment: export TOOLCHAINS=<snapshot_name>.

I built a demo project first:

swift package init --type executable
swift build --swift-sdk x86_64-swift-linux-musl

This works fine. Then I added import Foundation to main.swift which results in a series of repeated errors: <unknown>:0: error: missing required module '_FoundationCollections'. I was under the impression the nightly 6.0 builds included Foundation automatically? Ok maybe I need to be explicit.

Next I tried adding swift-foundation as a dependency in Package.swift and imported FoundationEssentials instead. This compiles fine. However, my project is also using the App Store Server Library and the types seem incompatible. For example FoundationEssentials.Data is not accepted by the App Store Server Library that expects Foundation.Data.

This seems like it should be obvious (it probably is) but I'm not sure how to use a Foundation dependency on a Linux executable. Searched through all the forum posts, documentation, and how-to's that I could find so I did try to do my own homework but no luck. Appreciate any guidance.

jmschonfeld · September 16, 2024, 10:21pm

Thanks for bringing this up! I believe this indeed should work. Can you confirm which Linux Static SDK you're using? I see that the main one hasn't been updated since early July but the release/6.0 one is quite recent. Any chance you're using the one from July 2nd / main?

Yes Foundation is included in the toolchain by default, no need to specify it in a package dependency. The incompatibility of types is expected behavior since the package build is only used for ease of local development and is not intended to be depended on by clients, so hopefully we should be able to get this working without that package dependency

seanmrich · September 16, 2024, 11:45pm

Glad to hear I'm not totally off base. Here's what I'm using:

swift toolchain 6.0 snapshot from 9/11/2024
linux static sdk from 9/11/2024
swift-foundation 6.0 RELEASE (tagged 9/13/2024)

jmschonfeld · September 16, 2024, 11:46pm

Thanks for the info! Yeah something seems off here (unless my understanding is incorrect too). I pinged some relevant folks on this GitHub issue that describes the problem: Missing required module '_FoundationCollections' when building with the Static Linux SDK · Issue #917 · apple/swift-foundation · GitHub, hopefully we can figure out what might be misconfigured here.

seanmrich · September 16, 2024, 11:50pm

Thanks! I’ll follow along there as well.

doozMen · September 18, 2024, 12:47pm

I'm trying to do the same and encounter the same issue. Seams Foundation is not included in the toolchain.

doozMen · September 18, 2024, 1:11pm

Although I ;like the quick response and reference to the issue I do not thing it is the issue that @seanmrich is stating. The issue is that import Foundation fails due to some weird reason.

The doc of github/apple/swift-foundation state that it should be included in the toolchaing. But as @seanmrich describes they are not.

I took the liberty to report this as a bug in swift-foundation too. It also happens when just doing swift build evan without the --swift-sdk x86_64-swift-linux-musl

See bug report

github.com/apple/swift-foundation

Foundation missing when building with the 6.0 release toolchain

opened 01:11PM - 18 Sep 24 UTC

doozMen

Given you are on macOS 15.0 and installed the toolchain via https://www.swift.or…g/install/macos/ and installed using the package installer the import of Foundation is causing problems when building. The output I will include at the end but how to reproduce is ```bash export TOOLCHAINS=$(plutil -extract CFBundleIdentifier raw /Library/Developer/Toolchains/swift-6.0-RELEASE.xctoolchain/Info.plist) mkdir swift-tool && cd swift-tool swift package init --type executable echo `import Foundation` >> Sources/main.swift swift build ``` this will output the following error ```bash Building for debugging... error: emit-module command failed due to signal 6 (use -v to see invocation) Please submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the crash backtrace. Stack dump: 0. Program arguments: /Library/Developer/Toolchains/swift-6.0-RELEASE.xctoolchain/usr/bin/swift-frontend -frontend -emit-module -experimental-skip-non-inlinable-function-bodies-without-types /Users/stijnwillems/Developer/swift-tool/Sources/main.swift -target arm64-apple-macosx10.13 -Xllvm -aarch64-use-tbi -enable-objc-interop -sdk /Applications/Xcode-16.0.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk -I /Users/stijnwillems/Developer/swift-tool/.build/arm64-apple-macosx/debug/Modules -I /Library/Developer/Toolchains/swift-6.0-RELEASE.xctoolchain/usr/lib/swift/macosx/testing -I /Applications/Xcode-16.0.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib -F /Applications/Xcode-16.0.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -color-diagnostics -enable-testing -g -debug-info-format=dwarf -dwarf-version=4 -module-cache-path /Users/stijnwillems/Developer/swift-tool/.build/arm64-apple-macosx/debug/ModuleCache -swift-version 6 -Onone -D SWIFT_PACKAGE -D DEBUG -entry-point-function-name swift_tool_main -empty-abi-descriptor -plugin-path /Library/Developer/Toolchains/swift-6.0-RELEASE.xctoolchain/usr/lib/swift/host/plugins/testing -resource-dir /Library/Developer/Toolchains/swift-6.0-RELEASE.xctoolchain/usr/lib/swift -enable-anonymous-context-mangled-names -file-compilation-dir /Users/stijnwillems/Developer/swift-tool -Xcc -isysroot -Xcc /Applications/Xcode-16.0.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.0.sdk -Xcc -F -Xcc /Applications/Xcode-16.0.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -Xcc -fPIC -Xcc -g -module-name swift_tool -package-name swift_tool -plugin-path /Library/Developer/Toolchains/swift-6.0-RELEASE.xctoolchain/usr/lib/swift/host/plugins -plugin-path /Library/Developer/Toolchains/swift-6.0-RELEASE.xctoolchain/usr/local/lib/swift/host/plugins -target-sdk-version 15.0 -target-sdk-name macosx15.0 -external-plugin-path /Applications/Xcode-16.0.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib/swift/host/plugins#/Applications/Xcode-16.0.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/bin/swift-plugin-server -external-plugin-path /Applications/Xcode-16.0.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/local/lib/swift/host/plugins#/Applications/Xcode-16.0.0.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/bin/swift-plugin-server -emit-module-doc-path /Users/stijnwillems/Developer/swift-tool/.build/arm64-apple-macosx/debug/Modules/swift_tool.swiftdoc -emit-module-source-info-path /Users/stijnwillems/Developer/swift-tool/.build/arm64-apple-macosx/debug/Modules/swift_tool.swiftsourceinfo -emit-dependencies-path /Users/stijnwillems/Developer/swift-tool/.build/arm64-apple-macosx/debug/swift_tool.build/swift_tool.emit-module.d -o /Users/stijnwillems/Developer/swift-tool/.build/arm64-apple-macosx/debug/Modules/swift_tool.swiftmodule -emit-abi-descriptor-path /Users/stijnwillems/Developer/swift-tool/.build/arm64-apple-macosx/debug/Modules/swift_tool.abi.json 1. Apple Swift version 6.0 (swift-6.0-RELEASE) 2. Compiling with the current language version 3. While evaluating request ExecuteSILPipelineRequest(Run pipelines { Mandatory Diagnostic Passes + Enabling Optimization Passes } on SIL for Darwin) 4. While running pass #2548 SILModuleTransform "MandatorySILLinker". 5. While deserializing SIL function "$s6Darwin14POSIXErrorCodeO8rawValueACSgs5Int32V_tcfC" 6. *** DESERIALIZATION FAILURE *** *** If any module named here was modified in the SDK, please delete the *** *** new swiftmodule files from the SDK and keep only swiftinterfaces. *** module '_errno', builder version '6.0(6.0)/Apple Swift version 6.0 (swift-6.0-RELEASE)', built from swiftinterface, resilient, loaded from '/Users/stijnwillems/Developer/swift-tool/.build/arm64-apple-macosx/debug/ModuleCache/_errno-2GQWKKCJV8S7T.swiftmodule' SILFunction type mismatch for '$s6Darwin14POSIXErrorCodeO8rawValueACSgs5Int32V_tcfC': '$@convention(method) (Int32, @thin POSIXErrorCode.Type) -> Optional<POSIXErrorCode>' != '$@convention(method) (Int32, @thin POSIXErrorCode.Type) -> Optional<POSIXErrorCode>' Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it): 0 swift-frontend 0x0000000109c51e28 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56 1 swift-frontend 0x0000000109c505f4 llvm::sys::RunSignalHandlers() + 112 2 swift-frontend 0x0000000109c52470 SignalHandler(int) + 304 3 libsystem_platform.dylib 0x000000018c638184 _sigtramp + 56 4 libsystem_pthread.dylib 0x000000018c602f70 pthread_kill + 288 5 libsystem_c.dylib 0x000000018c50f908 abort + 128 6 swift-frontend 0x000000010550dadc swift::ModuleFileSharedCore::fatal(llvm::Error) const + 1224 7 swift-frontend 0x00000001054a34dc swift::ModuleFile::getSourceLoc() const + 0 8 swift-frontend 0x00000001054e7a44 swift::SILDeserializer::readSILFunctionChecked(llvm::PointerEmbeddedInt<unsigned int, 31>, swift::SILFunction*, llvm::StringRef, bool, bool) + 1896 9 swift-frontend 0x00000001054f8bbc swift::SILDeserializer::lookupSILFunction(swift::SILFunction*, bool) + 184 10 swift-frontend 0x0000000105597bfc swift::SerializedSILLoader::lookupSILFunction(swift::SILFunction*, bool) + 148 11 swift-frontend 0x00000001055cfd68 swift::SILLinkerVisitor::process() + 484 12 swift-frontend 0x00000001055cfb54 swift::SILLinkerVisitor::processFunction(swift::SILFunction*) + 108 13 swift-frontend 0x000000010564b43c swift::SILModule::linkFunction(swift::SILFunction*, swift::SILModule::LinkingMode) + 108 14 swift-frontend 0x00000001052e71e4 (anonymous namespace)::SILLinker::run() + 128 15 swift-frontend 0x00000001051ad81c swift::SILPassManager::runModulePass(unsigned int) + 856 16 swift-frontend 0x00000001051afbc0 swift::SILPassManager::execute() + 624 17 swift-frontend 0x00000001051aa5ac swift::SILPassManager::executePassPipelinePlan(swift::SILPassPipelinePlan const&) + 72 18 swift-frontend 0x00000001051aa52c swift::ExecuteSILPipelineRequest::evaluate(swift::Evaluator&, swift::SILPipelineExecutionDescriptor) const + 68 19 swift-frontend 0x00000001051e5934 swift::SimpleRequest<swift::ExecuteSILPipelineRequest, std::__1::tuple<> (swift::SILPipelineExecutionDescriptor), (swift::RequestFlags)1>::evaluateRequest(swift::ExecuteSILPipelineRequest const&, swift::Evaluator&) + 28 20 swift-frontend 0x00000001051c5e90 swift::ExecuteSILPipelineRequest::OutputType swift::Evaluator::getResultUncached<swift::ExecuteSILPipelineRequest, swift::ExecuteSILPipelineRequest::OutputType swift::evaluateOrFatal<swift::ExecuteSILPipelineRequest>(swift::Evaluator&, swift::ExecuteSILPipelineRequest)::'lambda'()>(swift::ExecuteSILPipelineRequest const&, swift::ExecuteSILPipelineRequest::OutputType swift::evaluateOrFatal<swift::ExecuteSILPipelineRequest>(swift::Evaluator&, swift::ExecuteSILPipelineRequest)::'lambda'()) + 204 21 swift-frontend 0x00000001051aa788 swift::executePassPipelinePlan(swift::SILModule*, swift::SILPassPipelinePlan const&, bool, swift::irgen::IRGenModule*) + 64 22 swift-frontend 0x00000001051c8358 swift::runSILDiagnosticPasses(swift::SILModule&) + 192 23 swift-frontend 0x00000001049e6ff4 swift::CompilerInstance::performSILProcessing(swift::SILModule*) + 80 24 swift-frontend 0x00000001049f2118 swift::ExplicitModuleInterfaceBuilder::buildSwiftModuleFromInterface(llvm::StringRef, llvm::StringRef, bool, std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>*, llvm::ArrayRef<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, llvm::StringRef) + 2372 25 swift-frontend 0x00000001049f374c std::__1::error_code llvm::function_ref<std::__1::error_code (swift::SubCompilerInstanceInfo&)>::callback_fn<swift::ImplicitModuleInterfaceBuilder::buildSwiftModuleInternal(llvm::StringRef, bool, std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>*, llvm::ArrayRef<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>)::$_3::operator()() const::'lambda'(swift::SubCompilerInstanceInfo&)>(long, swift::SubCompilerInstanceInfo&) + 208 26 swift-frontend 0x00000001049fe760 swift::InterfaceSubContextDelegateImpl::runInSubCompilerInstance(llvm::StringRef, llvm::StringRef, llvm::StringRef, llvm::StringRef, swift::SourceLoc, bool, llvm::function_ref<std::__1::error_code (swift::SubCompilerInstanceInfo&)>) + 2372 27 swift-frontend 0x00000001049f34e0 void llvm::function_ref<void ()>::callback_fn<swift::ImplicitModuleInterfaceBuilder::buildSwiftModuleInternal(llvm::StringRef, bool, std::__1::unique_ptr<llvm::MemoryBuffer, std::__1::default_delete<llvm::MemoryBuffer>>*, llvm::ArrayRef<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>)::$_3>(long) + 268 28 swift-frontend 0x0000000109bc8438 llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) + 164 29 swift-frontend 0x0000000109bc8558 RunSafelyOnThread_Dispatch(void*) + 52 30 swift-frontend 0x0000000109bc8580 void* llvm::thread::ThreadProxy<std::__1::tuple<void (*)(void*), (anonymous namespace)::RunSafelyOnThreadInfo*>>(void*) + 24 31 libsystem_pthread.dylib 0x000000018c6032e4 _pthread_start + 136 32 libsystem_pthread.dylib 0x000000018c5fe0fc thread_start + 8 ``` Trying to mitigate the issue like described swift forum post https://forums.swift.org/t/using-swift-foundation-with-linux-static-sdk/74642 by adding dependency to `swift-foundation` and using `import FoundationEssentials` seams to work but has other issues. This seams to contradict the docs stating that foundation and especially FoundationEssentials is included in the toolchain on any platform?

jmschonfeld · September 18, 2024, 3:52pm

There are two issues in play here: one with the Static Linux SDK and one with using the OSS toolchain on Darwin platforms when importing the Darwin module. The issue that @seanmrich encountered is the former (where there is a missing _FoundationCollections module) and that is actively being resolved by a change in swift-collections (Missing required module '_FoundationCollections' when building with the Static Linux SDK · Issue #420 · apple/swift-collections · GitHub).

The issue you are describing is an issue with the toolchain in general when using the OSS toolchains on Darwin platforms which results in the compiler crash that you reported. @ian-twilightcoder and folks have been looking into this. I noted the GitHub issue that was previously reported that you can follow along with. The reason that import Foundation causes the issue here is because Foundation imports the Darwin module and the issue appears to be with the toolchain's Darwin module.