Trouble building lib/liblldb.so.17.0.0 on 6.0.1 using manually built 5.10.1

Development LLDB
1

I hope this is the correct category to post.
I have successfully (I think?) built the 5.10.1 based on these instructions here:
Building the Swift Project on Linux with lld instead of gold - #24 by itaiferber (thanks!!)

I've verified that I can compile swift programs with that binary, use swift repl so I think the swift 5.10.1 is all good. However, when I build the 6.0.1 using my version of 5.10.1, I can't build the lldb-server, nor the lib/liblldb.so.17.0.0 with missing symbols, like:

/usr/bin/ld: /home/build/swift-project/build/buildbot_linux/swift-linux-aarch64/lib/libswiftASTGen.a(SourceFile.swift.o): undefined reference to symbol '$ss10SetAlgebraP9formUnionyyxnFTj'
/usr/bin/ld: /home/build/swift-project/build/buildbot_linux/swift-linux-aarch64/lib/swift/linux/aarch64/libswiftCore.so: error adding symbols: DSO missing from command line
clang-15: error: linker command failed with exit code 1 (use -v to see invocation)

I'm using the utils/build-script like so (leaving out the install location for clarity):

swift/utils/build-script --preset=buildbot_linux,no_test

My biggest question is, would incorrectly built 5.10.1 cause the linker error for lldb-server? Just to narrow my search down.

If I leave out the lldb-server from the compilation, it produces a swift 6.0.1 but running swift repl reports version of 5.10.1.

Thanks in advance! Happy to provide more details if relevant, but wanted to validate my assumptions.

2

Just to verify: are you trying to build specifically with lld as your linker, or is your /usr/bin/ld an alias for gold?

If your /usr/bin/ld is lld, you need to set the LLVM_USE_LINKER=lld CMake variable in the build script to have LLVM, lldb, and the rest of the components know they're building with lld (which alters some of the linking behavior).

Is this your full build invocation (minus the install location), or are there any additional flags?

If you want to use the preset approach, you can add the CMake flag by editing swift/utils/build-presets.ini to add another preset at the bottom like so:

[preset: buildbot_linux,no_test,lld]
mixin-preset=buildbot_linux,no_test
extra-cmake-options=-DLLVM_USE_LINKER=lld

The name of the preset you create doesn't matter, just that you use it when building.

For reference, this is the preset I put together for building Swift 6.0.1 on Gentoo:

(Here you can see I'm passing multiple additional CMake options to the script; most of the other flags are likely irrelevant to your situation, though I would consider basing your build off the buildbot_linux,no_assertions,no_test preset to also disable assertions.)

Happy to try to help get you set up with a build!

3

Thanks for the reply, and I did see your gentoo related posts/links, thanks for those also. Yeah, I will try your suggestions, but I mainly wanted to rule out that I hadn't screwed something up with the building of 5.10.1. As in, is there a way that I may have built it somehow wrong, or missing something in 5.10.1, or can I rule that out as a potential problem for this link problem. Does that make sense? I'm not quite familiar enough to rule it out :)

4

Sorry, I meant to address this more clearly! If you managed to build the reset of Swift 6.0.1 and are running into a linker error with one component specifically, it looks like your Swift 5.10.1 is perfectly fine, and this is mostly a problem with the linker settings. (I've seen this before myself when the linker was incorrectly set.)

5

Awesomesauce, thank you! Just really wanted to rule that out so I know what problem I think I'm solving LOL. I'll play with those, much appreciated!

1 Like
6

I tried with both ld.gold and lld and they are now both getting this error:

CMake Error at /home/build/swift-project/build/buildbot_linux/llvm-linux-aarch64/lib/cmake/llvm/HandleLLVMOptions.cmake:330 (message):
  Host compiler does not support '-fuse-ld=lld'

And:

CMake Error at cmake/modules/HandleLLVMOptions.cmake:330 (message):
  Host compiler does not support '-fuse-ld=ld.gold'
Call Stack (most recent call first):
  CMakeLists.txt:918 (include)

I tried with the short modification you sent, as well as the full gentoo section one you had also provided.

7

Just to confirm, are you explicitly using clang to build, or is there a chance your invocation is falling back to gcc?

If you set

export CC="$(type -P clang)"
export CXX="$(type -P clang++)"
export LD="$(type -P ld.lld)"

in your environment prior to building, do you see a difference? (And what are the values of those variables?)

Edit: To be clear, the equivalent of the build invocation I'm running on Gentoo is

export CC="$(type -P clang)"
export CXX="$(type -P clang++)"
export LD="$(type -P ld.lld)"
export PATH="<path to Swift 5.10.1/usr/bin>:${PATH}"
build-script --preset=gentoo

This ensures that the build script can find all the Swift 5.10.1 binaries, and uses the system clang and lld for compilation of LLVM + lldb.

8

Thanks much for continued help :slight_smile:

~/swift-project # echo $CC
/usr/bin/clang
~/swift-project # echo $CXX
/usr/bin/clang++
~/swift-project # echo $LD
/usr/bin/ld.lld

And it still results with this:

CMake Deprecation Warning at /home/build/swift-project/llvm-project/cmake/Modules/CMakePolicy.cmake:11 (cmake_policy):
  The OLD behavior for policy CMP0116 will be removed from a future version
  of CMake.

  The cmake-policies(7) manual explains that the OLD behaviors of all
  policies are deprecated and that a policy should be set to OLD only under
  specific short-term circumstances.  Projects should be ported to the NEW
  behavior and not rely on setting a policy to OLD.
Call Stack (most recent call first):
  CMakeLists.txt:6 (include)


CMake Error at /home/build/swift-project/build/buildbot_linux/llvm-linux-aarch64/lib/cmake/llvm/HandleLLVMOptions.cmake:330 (message):
  Host compiler does not support '-fuse-ld=lld'
Call Stack (most recent call first):
  cmake/modules/LLDBStandalone.cmake:96 (include)
  CMakeLists.txt:42 (include)
9

Just to sanity-check: is this after also deleting the build directory? CMake caches values like CC and LD from the environment when you run it, so it's possible it's still using the old values otherwise.

10

Just deleted to get a fresh build :slight_smile: I'll be back in an hour :rofl:

1 Like
11

Yeah, so after rm -rf ./build followed by build-script with these settings:

mixin-preset=buildbot_linux,no_assertions,no_test
build-ninja=0
extra-cmake-options=
       -DLLVM_USE_LINKER=lld
       -DBUILD_TESTING:BOOL=NO
       -DSWIFT_INCLUDE_TESTS:BOOL=NO
       -DSWIFT_INCLUDE_TEST_BINARIES:BOOL=NO
       -DCOMPILER_RT_BUILD_ORC:BOOL=NO
       -DPython3_FIND_UNVERSIONED_NAMES=FIRST
libicu=0
llvm-targets-to-build=host
skip-build-curl
skip-build-zlib

Still getting that error:

-- Performing Test CXX_SUPPORTS_CUSTOM_LINKER
-- Performing Test CXX_SUPPORTS_CUSTOM_LINKER - Failed
CMake Error at /home/build/swift-project/build/buildbot_linux/llvm-linux-aarch64/lib/cmake/llvm/HandleLLVMOptions.cmake:330 (message):
  Host compiler does not support '-fuse-ld=lld'
Call Stack (most recent call first):
  cmake/modules/LLDBStandalone.cmake:96 (include)
  CMakeLists.txt:42 (include)
12

Hmm. Are you able to share the contents of your build/buildbot_linux/lldb-linux-x86_64/CMakeCache.txt and build/buildbot_linux/llvm-linux-x86_64/CMakeCache.txt? (These are going to be quite long, so I'd recommend uploading them to pastebin or similar)

And if you run

$ echo "int main() { return 0; }" >test.cpp
$ /usr/bin/clang++ -fuse-ld=lld test.cpp

what do you see? (This is approximately what CMake is trying to do here, and it claims this is failing.)

13 
~/swift-project # echo "int main() { return 0; }" >/tmp/test.cpp
~/swift-project # /usr/bin/clang++ -fuse-ld=lld /tmp/test.cpp
~/swift-project # ls -l ./a.out
-rwxr-xr-x    1 root     root          6968 Oct 26 11:45 ./a.out
~/swift-project # ./a.out

So that seems to work aok. Let me upload the stuff.

14

lldb: build/buildbot_linux/lldb-linux-aarch64/CMakeCache.txt · GitHub
llvm: CMakeCache.txt · GitHub

Thanks again!!!

15

One additional sanity check: it looks like lldb has CMAKE_EXE_LINKER_FLAGS:STRING=-Wl,--copy-dt-needed-entries set, which would turn the compilation invocation into

clang++ -fuse-ld=lld -Wl,--copy-dt-needed-entries Test.cpp

For me, this yields

$ clang++ -fuse-ld=lld -Wl,--copy-dt-needed-entries Test.cpp
ld.lld: error: unknown argument '--copy-dt-needed-entries'
clang++: error: linker command failed with exit code 1 (use -v to see invocation)

which would indeed prevent compilation. I assume you see the same?

I'm not seeing the copy-dt-needed-entries flag referenced anywhere in the CMake files (and many linkers don't support it), so I'm wondering if something is setting this in your environment, and CMake is picking it up. In the same shell you're building from, what do you see for

echo "$LDFLAGS"

?

16

Woot Woot!

./swift --version
Swift version 6.0.1 (swift-6.0.1-RELEASE)
Target: aarch64-unknown-linux-gnu

And:

./swift repl
Welcome to Swift version 6.0.1 (swift-6.0.1-RELEASE).
Type :help for assistance.

I think indeed using the lld was the key here. But before I switched to it, I had gone down a wrong path of using different flags and that copy-dt-needed-entries indeed was the one that allowed the lldb-server to get linked, but now the lib/liblldb.so.17.0.0 at which point I decided to check my assumptions and see if the bootstrap 5.10.1 might be at the root of the problem, at which point no flaggery would solve this. I am cleaning out everything up and doing everything from scratch with only this change and will report. But at least I got it working once, with your superb help, thank you so much!!!

17

Woohoo! That's great to see :smile:

If you get the chance and don't mind confirming something for me, please: could you try to execute something in the repl for this version that was just built (e.g., even just 1 + 1)? Based on my exploration in Swift 6.0.1 for Gentoo on GURU - #9 by itaiferber, I'd expect the repl to crash immediately. I plan on filing a bug report today with detail from a few OSes, but it'll be helpful to get even more confirmation than my own testing.

18

OFC! I'll do this once the build completes.

19

Was this with stripped or non-stripped? I did stumble on something somewhere in my adventures about repl / stripping, but I can't recall details right now.

20

I'm seeing this with symbols non-stripped (or, at least, I'm not enabling symbol stripping and I'm not seeing any symbols actually getting stripped); AFAICT, only Darwin platforms support stripping in the CMake setup