Required dependencies for Ubuntu 22.04; error when compiling with -Xswiftc -static-executable

sspringer · October 11, 2022, 3:20pm

On the Getting Started page the requirements are only listed for Ubuntu 18.04 and Ubuntu 20.04, but not for Ubuntu 22.04. What are the requirements (dependencies) for Ubuntu 22.04?

Under Ubuntu 22.04 I get the following error when compiling with swift build -c release -Xswiftc -static-executable, maybe this is a consequence of having installed the wrong library versions (I installed the ones listed for Ubuntu 20.04)?

/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/lib/swift_static/linux/libswift_Concurrency.a(TaskGroup.cpp.o):TaskGroup.cpp:function swift::TaskGroup::offer(swift::AsyncTask*, swift::AsyncContext*): error: undefined reference to 'std::__throw_bad_array_new_length()'

sspringer · October 11, 2022, 4:31pm

From the docker file:

apt-get install \
    binutils \
    git \
    unzip \
    gnupg2 \
    libc6-dev \
    libcurl4-openssl-dev \
    libedit2 \
    libgcc-9-dev \
    libpython3.8 \
    libsqlite3-0 \
    libstdc++-9-dev \
    libxml2-dev \
    libz3-dev \
    pkg-config \
    tzdata \
    zlib1g-dev

..but of course, this should be documented on the Getting Started page.

But the undefined reference to 'std::__throw_bad_array_new_length()' error is still not resolved.

Update:

I found some answers concerning the error there, but do not know how to proceed.

compnerd · October 11, 2022, 6:27pm

You could work around this by using -Xswiftc -enable-experimental-cxx-interop. The problem is that you need to link the C++ runtime. That would require that you either:

use clang++ to link - which is what enabling the C++ interop will accomplish
correctly explicitly inject the linkage at the right location for the linker driver (clang). This is far more complicated and requires that you are also able to resolve the circularity between the runtime ABI support (the default being libsupc++) and the runtime (the default being libstdc++), as well as the transitive closure for its dependency (e.g. the unwinder - defaulting to libgcc, although if there are any shared libraries involved anywhere, be aware that you may need to switch to libgcc_s). The language runtimes should be linked at a specific position in the invocation as the traditional Unix style linking is order dependent (unlike Windows linking semantics).

This is not an issue of installed dependencies, though that may become an issue once you enable the c++ interop to ensure that the necessary static variants of the needed libraries are available for linking.

sspringer · October 11, 2022, 7:01pm

Thank you for your answer. For reference: The project that I am trying to compile is my "SwiftHelloPython" demo.

I tried using swift build -c release -Xswiftc -static-executable -Xswiftc -enable-experimental-cxx-interop, but it failed with a stack dump (with libraries as listed above):

Stack dump

swift build -c release -Xswiftc -static-executable -Xswiftc -enable-experimental-cxx-interop

Building for production...

remark: Incremental compilation has been disabled: it is not compatible with whoremark: Incremental compilation has been disabled: it is not compatible with whoPlease submit a bug report (https://swift.org/contributing/#reporting-bugs) and include the project and the crash backtrace.

Stack dump:

0. Program arguments: /usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend -frontend -c /home/stefan/projects/SwiftHelloPython/Sources/HelloPython/Main.swift -emit-module-path /home/stefan/projects/SwiftHelloPython/.build/x86_64-unknown-linux-gnu/release/HelloPython.swiftmodule -emit-module-doc-path /home/stefan/projects/SwiftHelloPython/.build/x86_64-unknown-linux-gnu/release/HelloPython.swiftdoc -emit-module-source-info-path /home/stefan/projects/SwiftHelloPython/.build/x86_64-unknown-linux-gnu/release/HelloPython.swiftsourceinfo -emit-dependencies-path /home/stefan/projects/SwiftHelloPython/.build/x86_64-unknown-linux-gnu/release/HelloPython.build/HelloPython.d -target x86_64-unknown-linux-gnu -disable-objc-interop -enable-cxx-interop -I /home/stefan/projects/SwiftHelloPython/.build/x86_64-unknown-linux-gnu/release -color-diagnostics -g -module-cache-path /home/stefan/projects/SwiftHelloPython/.build/x86_64-unknown-linux-gnu/release/ModuleCache -swift-version 5 -O -D SWIFT_PACKAGE -new-driver-path /usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-driver -entry-point-function-name HelloPython_main -empty-abi-descriptor -resource-dir /usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/lib/swift_static -use-static-resource-dir -module-name HelloPython -parse-as-library -num-threads 4 -o /home/stefan/projects/SwiftHelloPython/.build/x86_64-unknown-linux-gnu/release/HelloPython.build/Main.swift.o
1. Swift version 5.7 (swift-5.7-RELEASE)
2. Compiling with the current language version
3. While evaluating request ASTLoweringRequest(Lowering AST to SIL for module HelloPython)
4. While silgen emitArtificialTopLevel SIL function "@async_Main".
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0x5513c03)[0x5583a9789c03]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0x5511b2e)[0x5583a9787b2e]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0x5513f8a)[0x5583a9789f8a]
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7fc8f90b6520]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0xae4c12)[0x5583a4d5ac12]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0x179b282)[0x5583a5a11282]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0xf72f8e)[0x5583a51e8f8e]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0xfc3600)[0x5583a5239600]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0xf73dae)[0x5583a51e9dae]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0xf78409)[0x5583a51ee409]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0xff512f)[0x5583a526b12f]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0xff5049)[0x5583a526b049]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0xf7b0c8)[0x5583a51f10c8]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0xf78f24)[0x5583a51eef24]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0xa406c0)[0x5583a4cb66c0]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0xa42a6e)[0x5583a4cb8a6e]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0xa41d8d)[0x5583a4cb7d8d]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0x9140f9)[0x5583a4b8a0f9]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7fc8f909dd90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7fc8f909de40]
/usr/local/swift-5.7-RELEASE-ubuntu22.04/usr/bin/swift-frontend(+0x913955)[0x5583a4b89955]

My take-aways:

It seems to be complicated. (Or at least for now? Will that become easier?)
I really would have liked to produce a "stand-alone" binary, but I think I am able to work with an installed Swift + dependencies in this case. Not nice, but OK.

One further question:

I also tried to compile with swift build -c release -Xswiftc -static-stdlib, this compiles without an error but when I am starting my program, the program gets stuck. (The program compiled with just swift build -c release is running fine.) Could this be "the same problem"?

tomerd · October 11, 2022, 7:17pm

hi @sspringer some of this was discussed in https://forums.swift.org/t/pitch-package-manager-statically-link-swift-runtime-libraries-by-default-on-supported-platforms/

a few tl;dr take aways:

it is difficult to produce a fully statically linked binary in swift today due to system dependencies - especially glibc. there is a long term plan to make this easier.
for now, you would probably get better milage using -static-stdlib and avoiding (or manually bringing over) system dependencies that are not handled by -static-stdlib
need to investigate how -enable-experimental-cxx-interop interacts with -static-stdlib, if we are actually using C++ interoperability here. that is an experimental feature so I am not sure the implications of it on static linking are fully taken care of
we should update the docs accordingly, and also the dependencies list.

cc @Max_Desiatov @drexin @mishal_shah

sspringer · October 11, 2022, 7:22pm

For me, that sounds very good.

(And thank you for the link, I thought that both options -static-executable and -static-stdlib are already an "easy" given on linux.)

Note that I also have a problem using -static-stdlib, but this might be because of using PythonKit (it gets stuck when trying to execute the Python function)?

Max_Desiatov · October 11, 2022, 7:23pm

To me this does look like an issue related to C++ interop, especially as std::__throw_bad_array_new_length() was mentioned, which is a C++ symbol. C++ interop is still at an early stage, therefore the requirement for experimental flag. I'm not sure anyone tested it yet with static linking.

cc @zoecarver

sspringer · October 11, 2022, 7:26pm

Yes, those things take time... Thanks to all people involved.

compnerd · October 11, 2022, 9:24pm

@Max_Desiatov no, this is not a C++ interop issue. The issue is the standard library itself. That is written using C++, and thus needs to ensure that the C++ runtime is linked into the final executable when linked statically. However, we will drive the link using clang as normally you do not want to use clang++ to avoid over-linking in the case that the standard library is dynamically linked. Perhaps we should consider adding some complexity to the driver for the static linking case. I can look into that.