Strange error when compiling Swift on GNU/Linux

Hi all,
I'm compiling on GNU/Linux, the CPU is x86_64, the RAM is 6GB with swap space of 10GB.
Despite the specs, I am getting a "memory exhausted" error during linking.
I watched the compilation and the free RAM never went to zero. Here is a screenshot.
Does anyone know what the cause could be?
Thanks.

Addendum. I'm using 32-bit Linux, so the virtual memory space is 4GB, however I didn't see the free RAM go to zero. So I deduce that actually the amount of RAM available to any process is less than 4GB. The googles say that the limit is more like 3GB.

Time to install 64-bit Linux.

1 Like

Unfortunately, building really requires a 64-bit host, particularly with debug info. You might be able to get away with building release mode which should not have debug info.

1 Like

Depending on how many parallel jobs you have, 6GB might still be on the low side during parallel linking. Another thing that can help is to pass --extra-cmake-options="-DLLVM_PARALLEL_LINK_JOBS=1" to build-script.

Thanks, I'll give that a try. Unfortunately 64-bit Linux isn't working for me (Qemu freezes) so I'll have to start over again with 32-bit.
Eventually I'll give this a try on the Rpi 3 as well which currently runs 32-bit Linux.

OK I tried the cmake option but it is still not building on 32-bit Linux,
as you can see here:

It still is insufficient memory. What linker are you using? You should build without debug info to ensure that it can link, or ensure that you are using fission.

1 Like

Note that you're not even building any code from the Swift project at this point; this is just failing to build parts of LLVM. Saleem's advice seems spot-on to me.

It is just GNU ld, standard for Linux.
I'll try to build again with -R.

Also for some reason it is trying to build backends for many CPUs like ARM but my CPU is Qemu's i386 simulator. However there doesn't seem to be a way to prevent that. The --host-target option causes an error no matter what CPU I specify.

You might have better luck linking LLVM with gold or especially lld. You'll ultimately need either gold or lld for Swift to link with, because GNU ld does not support Swift's code model.

1 Like

Doesn't one of these come with the Swift build system?

I don't believe we build either of them as part of the build system, though I believe the setup scripts will install the Ubuntu package for gold for you.

Also for some reason it is trying to build backends for many CPUs like ARM but my CPU is Qemu's i386 simulator. However there doesn't seem to be a way to prevent that.

Try running build-script with --llvm-targets-to-build X86.

1 Like

Can you elaborate on what you mean by GNU ld not supporting Swift's code model? What exactly are you referring 'code model' in this context?

Specifically, Swift expects all the locally-defined symbols in an executable or dynamically linked image to be canonically defined in that image, so for ELF platforms, all our exported symbols have protected visibility. GNU ld still chooses to treat references to protected symbols as non-local on the off chance someone emits a COPY relocation to them, which is fundamentally incompatible with protected visibility. I reported this issue to the project, but it seems that, given the fundamental incompatibility here, they intentionally chose to err in the favor of copy relocations. See Joe Groff - Preventing preemption of 'protected' symbols in GNU ld 2.26 for more details.

So if my distro has ld 2.26 I need to upgrade, it sounds like.

I'm not sure newer versions of binutils address the issue, since when I talked to the team it sounded like the incompatible behavior change was intentional and they weren't interested in changing it back. Nonetheless, other linkers like gold and lld generally perform better than GNU ld and use less memory in my experience. Switching linkers might help you with your memory exhaustion problem building LLVM here too.

Another strange build error on GNU/Linux:

As you can see the compiler doesn't know what __atomic_load_8 is.

You need an explicit link against libatomic. You will find that in the build system, there are a few targets that explicitly add a -latomic to resolve that.

My current build target is X86. When I grep I don't see these other targets you are referring to. Maybe they were removed?