Can't build 5.9.2 Toolchain on Ubuntu 20.04

Development Compiler
1

Hi there!

I'm trying to build swift toolchain 5.9.2 on Ubuntu 20.04 without prebuilt compiler, so I did the following

mkdir swift-project
cd swift-project/
git clone https://github.com/apple/swift.git swift
cd swift/
utils/update-checkout --clone
utils/update-checkout --tag swift-5.9.2-RELEASE

then I patched buildbot_linux preset with

skip-early-swift-driver
skip-early-swiftsyntax

and launched the compilation

utils/build-script --preset=buildbot_linux,no_test install_destdir=~/swift-install installable_package=~/swift.tar.gz

it expanded to

utils/build-script --no-assertions --no-swift-stdlib-assertions --skip-early-swift-driver --skip-early-swiftsyntax --swift-enable-ast-verifier=0 '--swift-install-components=autolink-driver;compiler;clang-resource-dir-symlink;libexec;stdlib;swift-remote-mirror;sdk-overlay;static-mirror-lib;toolchain-tools;license;sourcekit-inproc' '--llvm-install-components=llvm-ar;llvm-cov;llvm-profdata;IndexStore;clang;clang-resource-headers;compiler-rt;clangd;lld;LTO;clang-features-file' --llbuild --swiftpm --swift-driver --xctest --libicu --swiftdocc --build-ninja --install-llvm --install-swift --install-lldb --install-llbuild --install-swiftpm --install-swift-driver --install-swiftsyntax --install-xctest --install-libicu --install-prefix=/usr --install-sourcekit-lsp --install-swiftformat --install-swiftdocc --build-swift-static-stdlib --build-swift-static-sdk-overlay --build-swift-stdlib-unittest-extra --test-installable-package --toolchain-benchmarks --install-destdir=/home/zabolotniy/swift-install --installable-package=/home/zabolotniy/swift.tar.gz --relocate-xdg-cache-home-under-build-subdir --build-subdir=buildbot_linux --lldb --release --test --validation-test --long-test --stress-test --test-optimized --foundation --libdispatch --indexstore-db --sourcekit-lsp --swiftdocc '--lit-args=-v --time-tests' --lldb-test-swift-only --install-foundation --install-libdispatch --reconfigure --skip-test-cmark --skip-test-lldb --skip-test-swift --skip-test-llbuild --skip-test-swiftpm --skip-test-swift-driver --skip-test-xctest --skip-test-foundation --skip-test-libdispatch --skip-test-playgroundsupport --skip-test-libicu --skip-test-indexstore-db --skip-test-sourcekit-lsp --skip-test-swiftdocc

and finally finished with the error

--- bootstrap: note: Building SwiftPM (with a freshly built swift-bootstrap)
env SWIFTCI_USE_LOCAL_DEPS=1 SWIFTPM_MACOS_DEPLOYMENT_TARGET=10.15 LD_LIBRARY_PATH=/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/bootstrap/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/tsc/lib:/home/zabolotniy/swift-project/build/buildbot_linux/llbuild-linux-x86_64/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/yams/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/swift-argument-parser/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/swift-crypto/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/swift-driver/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/swift-system/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/swift-collections/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/swift-asn1/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/swift-certificates/lib:/home/zabolotniy/swift-install/usr/lib/swift/linux SWIFT_EXEC=/home/zabolotniy/swift-install/usr/bin/swiftc SWIFT_DRIVER_SWIFT_EXEC=/home/zabolotniy/swift-install/usr/bin/swiftc CC=/home/zabolotniy/swift-install/usr/bin/clang SWIFTPM_CUSTOM_LIBS_DIR=/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/bootstrap/pm /home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/bootstrap/bin/swift-bootstrap --build-path /home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64 --configuration release -Xswiftc -module-cache-path -Xswiftc /home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/module-cache -Xmanifest -module-cache-path -Xmanifest /home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/module-cache
[warning]: found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /home/zabolotniy/swift-project/swiftpm/Sources/swift-package-registry/CMakeLists.txt

[warning]: found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /home/zabolotniy/swift-project/swiftpm/Sources/swift-package-collection/CMakeLists.txt

[warning]: found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /home/zabolotniy/swift-project/swiftpm/Sources/PackageRegistryTool/CMakeLists.txt

[warning]: found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /home/zabolotniy/swift-project/swiftpm/Sources/PackageMetadata/CMakeLists.txt

[warning]: found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /home/zabolotniy/swift-project/swiftpm/Sources/PackageDescription/PackageDescription.docc

[warning]: found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /home/zabolotniy/swift-project/swiftpm/Sources/PackageCollectionsTool/CMakeLists.txt

[warning]: found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /home/zabolotniy/swift-project/swiftpm/Sources/PackageCollectionsSigning/CMakeLists.txt

[warning]: found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /home/zabolotniy/swift-project/swiftpm/Sources/PackageCollectionsModel/CMakeLists.txt

[warning]: found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /home/zabolotniy/swift-project/swiftpm/Sources/PackageCollections/CMakeLists.txt

Building for production...
/home/zabolotniy/swift-project/swift-tools-support-core/Sources/TSCBasic/Condition.swift:16:17: warning: stored property '_condition' of 'Sendable'-conforming struct 'Condition' has non-sendable type 'NSCondition'
    private let _condition = NSCondition()
                ^
Foundation.NSCondition:1:12: note: class 'NSCondition' does not conform to the 'Sendable' protocol
open class NSCondition : NSObject, NSLocking {
           ^
/home/zabolotniy/swift-project/swift-tools-support-core/Sources/TSCBasic/Condition.swift:11:1: remark: add '@preconcurrency' to suppress 'Sendable'-related warnings from module 'Foundation'
import Foundation
^
@preconcurrency
/home/zabolotniy/swift-project/swift-tools-support-core/Sources/TSCBasic/FileInfo.swift:37:16: warning: stored property 'fileType' of 'Sendable'-conforming struct 'FileInfo' has non-sendable type 'FileAttributeType'
    public let fileType: FileAttributeType
               ^
Foundation.FileAttributeType:1:15: note: struct 'FileAttributeType' does not conform to the 'Sendable' protocol
public struct FileAttributeType : RawRepresentable, Equatable, Hashable {
              ^
/home/zabolotniy/swift-project/swift-tools-support-core/Sources/TSCBasic/FileInfo.swift:11:1: remark: add '@preconcurrency' to suppress 'Sendable'-related warnings from module 'Foundation'
import Foundation
^
@preconcurrency
/home/zabolotniy/swift-project/swift-tools-support-core/Sources/TSCBasic/FileSystem.swift:412:21: warning: 'init(_:)' is deprecated: use throwing `init(validating:)` variant instead
        return try? AbsolutePath(String(cString: fsr))
                    ^
/home/zabolotniy/swift-project/swift-tools-support-core/Sources/TSCBasic/FileSystem.swift:412:16: warning: no calls to throwing functions occur within 'try' expression
        return try? AbsolutePath(String(cString: fsr))
               ^
/home/zabolotniy/swift-project/swift-tools-support-core/Sources/TSCBasic/FileSystem.swift:337:84: warning: will never be executed
    func hasAttribute(_ name: FileSystemAttribute, _ path: AbsolutePath) -> Bool { false }
                                                                                   ^
/home/zabolotniy/swift-project/swift-tools-support-core/Sources/TSCBasic/FileSystem.swift:337:82: note: 'name' is of type 'FileSystemAttribute' which cannot be constructed because it is an enum with no cases
    func hasAttribute(_ name: FileSystemAttribute, _ path: AbsolutePath) -> Bool { false }
                                                                                 ^
/home/zabolotniy/swift-project/swift-tools-support-core/Sources/TSCBasic/FileSystem.swift:395:9: warning: will never be executed
        return false
        ^
/home/zabolotniy/swift-project/swift-tools-support-core/Sources/TSCBasic/FileSystem.swift:389:82: note: 'name' is of type 'FileSystemAttribute' which cannot be constructed because it is an enum with no cases
    func hasAttribute(_ name: FileSystemAttribute, _ path: AbsolutePath) -> Bool {
                                                                                 ^
/home/zabolotniy/swift-project/swift-tools-support-core/Sources/TSCTestSupport/Product.swift:42:21: warning: 'init(_:relativeTo:)' is deprecated: use throwing `init(validating:relativeTo:)` variant instead
        return try! AbsolutePath(CommandLine.arguments.first!, relativeTo: localFileSystem.currentWorkingDirectory!)
                    ^
/home/zabolotniy/swift-project/swift-tools-support-core/Sources/TSCTestSupport/Product.swift:42:16: warning: no calls to throwing functions occur within 'try' expression
        return try! AbsolutePath(CommandLine.arguments.first!, relativeTo: localFileSystem.currentWorkingDirectory!)
               ^
/home/zabolotniy/swift-project/swiftpm/Sources/PackageCollectionsSigning/Key/Key+RSA.swift:113:59: error: no type named 'RSA' in module 'CCryptoBoringSSL'
    let underlying: UnsafeMutablePointer<CCryptoBoringSSL.RSA>
                                                          ^
/home/zabolotniy/swift-project/swiftpm/Sources/PackageCollectionsSigning/Key/Key+RSA.swift:138:59: error: no type named 'RSA' in module 'CCryptoBoringSSL'
    let underlying: UnsafeMutablePointer<CCryptoBoringSSL.RSA>
                                                          ^

--- bootstrap: error: Command '['env', 'SWIFTCI_USE_LOCAL_DEPS=1', 'SWIFTPM_MACOS_DEPLOYMENT_TARGET=10.15', 'LD_LIBRARY_PATH=/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/bootstrap/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/tsc/lib:/home/zabolotniy/swift-project/build/buildbot_linux/llbuild-linux-x86_64/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/yams/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/swift-argument-parser/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/swift-crypto/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/swift-driver/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/swift-system/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/swift-collections/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/swift-asn1/lib:/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/swift-certificates/lib:/home/zabolotniy/swift-install/usr/lib/swift/linux', 'SWIFT_EXEC=/home/zabolotniy/swift-install/usr/bin/swiftc', 'SWIFT_DRIVER_SWIFT_EXEC=/home/zabolotniy/swift-install/usr/bin/swiftc', 'CC=/home/zabolotniy/swift-install/usr/bin/clang', 'SWIFTPM_CUSTOM_LIBS_DIR=/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/bootstrap/pm', '/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/x86_64-unknown-linux-gnu/bootstrap/bin/swift-bootstrap', '--build-path', '/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64', '--configuration', 'release', '-Xswiftc', '-module-cache-path', '-Xswiftc', '/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/module-cache', '-Xmanifest', '-module-cache-path', '-Xmanifest', '/home/zabolotniy/swift-project/build/buildbot_linux/swiftpm-linux-x86_64/module-cache']' returned non-zero exit status 1.
ERROR: command terminated with a non-zero exit status 1, aborting

ERROR: command terminated with a non-zero exit status 1, aborting

Is there something wrong with CCryptoBoringSSL and how to fix it?

2

swift-crypto doesn't have a swift-5.9.2-RELEASE tag, so most likely you have the latest version of it checked out, while instead you need the version that was used when swift-5.9.2-RELEASE was tagged on the rest of the repositories.

Apparently you need to manually check out swift-crypto 2.5.0.

3

Thanks! I thought that

utils/update-checkout --tag swift-5.9.2-RELEASE

would use the update-checkout-config.json

4

Unfortunately, it can't map a tag to a checkout scheme, especially as schemes are inconsistently named. For example, there are release/5.9 and release/5.9.1 schemes, but no release/5.9.2 scheme.

5

Then, there might be other repos with wrong checkout, should I use this?

utils/update-checkout --scheme release/5.9
utils/update-checkout --tag swift-5.9.2-RELEASE
6

Yes, running those commands in sequence should bring you the closest to the tag as it was built, modulo possible differences in you platform/environment/system dependencies from CI nodes.

1 Like
7

Almost there, looks like there is a problem with tests – gist:06279bb14d0c5201f64c466775ef3a9e · GitHub

make[1]: Leaving directory '/home/zabolotniy/swift-project/build/buildbot_linux/libicu-linux-x86_64/common'
make: *** [Makefile:153: install-recursive] Error 2
ERROR: command terminated with a non-zero exit status 2, aborting

ERROR: command terminated with a non-zero exit status 1, aborting
8 
No rule to make target '/home/zabolotniy/swift-project/icu/icu4c/source/common/charstrmap.h', needed by 'locid.o'.

This seems to be causing the failure, maybe you didn't start a clean build and some stale artifacts from previous artifacts got in the way? Otherwise this could be an icu-specific issue and one would have to debug the icu build process in your specific environment to say for sure.

9

The 5.9 branch has been abandoned now, you're better off building the current release, 5.10.

Why not use a prebuilt compiler? I don't think Swift features like macros are available otherwise.

If there is no prebuilt Swift compiler in your PATH, these build flags do nothing and can be dropped.

When you checked out all the source branches again, as you mentioned above, did you start over with a clean build, ie rm -rf build/buildbot_linux/? If not, it may be a configuration issue from building on top of the previous ICU with a newly checked-out source tag for ICU.

10

No, didn't clean. Try to restart build after rm -rf build/.

11

Success! Thanks a lot!

12

Hi there, again!

Now I'm trying to build swift toolchain 5.9.2 on x86_64 mac.

./utils/build-script --preset=buildbot_osx_package,no_assertions,no_test

And it fails

FAILED: bin/LLDB.framework/Versions/A/LLDB
...
ld: warning: ignoring file '/usr/local/Cellar/python@3.11/3.11.6/Frameworks/Python.framework/Versions/3.11/Python': found architecture 'x86_64', required architecture 'arm64'
ld: warning: Could not find or use auto-linked library 'SwiftBasicFormat': library 'SwiftBasicFormat' not found
ld: warning: Could not find or use auto-linked library 'SwiftCompilerPluginMessageHandling': library 'SwiftCompilerPluginMessageHandling' not found
ld: warning: Could not find or use auto-linked library 'SwiftDiagnostics': library 'SwiftDiagnostics' not found
ld: warning: Could not find or use auto-linked library 'SwiftOperators': library 'SwiftOperators' not found
ld: warning: Could not find or use auto-linked library 'SwiftParser': library 'SwiftParser' not found
ld: warning: Could not find or use auto-linked library 'SwiftParserDiagnostics': library 'SwiftParserDiagnostics' not found
ld: warning: Could not find or use auto-linked library 'SwiftSyntax': library 'SwiftSyntax' not found
ld: warning: Could not find or use auto-linked library 'SwiftSyntaxBuilder': library 'SwiftSyntaxBuilder' not found
ld: warning: Could not find or use auto-linked library 'SwiftSyntaxMacroExpansion': library 'SwiftSyntaxMacroExpansion' not found
ld: warning: Could not find or use auto-linked library 'SwiftSyntaxMacros': library 'SwiftSyntaxMacros' not found
ld: Undefined symbols:
  _PyArg_Parse, referenced from:
      lldb_private::ScriptInterpreterPythonImpl::ExecuteOneLineWithReturn(llvm::StringRef, lldb_private::ScriptInterpreter::ScriptReturnType, void*, lldb_private::ExecuteScriptOptions const&) in liblldbPluginScriptInterpreterPython.a[4](ScriptInterpreterPython.cpp.o)
  _PyArg_UnpackTuple, referenced from:
      SwigPyObject_own(_object*, _object*) in LLDBWrapPython.cpp.o
  _PyBool_FromLong, referenced from:
      _PyInit__lldb in LLDBWrapPython.cpp.o
      _PyInit__lldb in LLDBWrapPython.cpp.o
      _wrap_SBAddress_IsValid(_object*, _object*) in LLDBWrapPython.cpp.o
      _wrap_SBAddress___nonzero__(_object*, _object*) in LLDBWrapPython.cpp.o
      _wrap_SBAddress___ne__(_object*, _object*) in LLDBWrapPython.cpp.o
      _wrap_SBAddress_OffsetAddress(_object*, _object*) in LLDBWrapPython.cpp.o
      _wrap_SBAddress_GetDescription(_object*, _object*) in LLDBWrapPython.cpp.o
      ...

Looks like it's trying to build for 'arm64' architecture. Full log is here. Should I somehow specify the architecture in a preset?

13

Removing infer-cross-compile-hosts-on-darwin from [preset: mixin_osx_package_base] worked.