Strange error when compiling Swift on GNU/Linux


(Eben Up) #1

Hi all,
I'm compiling on GNU/Linux, the CPU is x86_64, the RAM is 6GB with swap space of 10GB.
Despite the specs, I am getting a "memory exhausted" error during linking.
I watched the compilation and the free RAM never went to zero. Here is a screenshot.
Does anyone know what the cause could be?

(Eben Up) #2

Addendum. I'm using 32-bit Linux, so the virtual memory space is 4GB, however I didn't see the free RAM go to zero. So I deduce that actually the amount of RAM available to any process is less than 4GB. The googles say that the limit is more like 3GB.

Time to install 64-bit Linux.

(Saleem Abdulrasool) #3

Unfortunately, building really requires a 64-bit host, particularly with debug info. You might be able to get away with building release mode which should not have debug info.

(Ben Langmuir) #4

Depending on how many parallel jobs you have, 6GB might still be on the low side during parallel linking. Another thing that can help is to pass --extra-cmake-options="-DLLVM_PARALLEL_LINK_JOBS=1" to build-script.

(Eben Up) #5

Thanks, I'll give that a try. Unfortunately 64-bit Linux isn't working for me (Qemu freezes) so I'll have to start over again with 32-bit.
Eventually I'll give this a try on the Rpi 3 as well which currently runs 32-bit Linux.

(Eben Up) #6

OK I tried the cmake option but it is still not building on 32-bit Linux,
as you can see here:

(Saleem Abdulrasool) #7

It still is insufficient memory. What linker are you using? You should build without debug info to ensure that it can link, or ensure that you are using fission.

(John McCall) #8

Note that you're not even building any code from the Swift project at this point; this is just failing to build parts of LLVM. Saleem's advice seems spot-on to me.

(Eben Up) #9

It is just GNU ld, standard for Linux.
I'll try to build again with -R.

Also for some reason it is trying to build backends for many CPUs like ARM but my CPU is Qemu's i386 simulator. However there doesn't seem to be a way to prevent that. The --host-target option causes an error no matter what CPU I specify.

(Joe Groff) #10

You might have better luck linking LLVM with gold or especially lld. You'll ultimately need either gold or lld for Swift to link with, because GNU ld does not support Swift's code model.

(Eben Up) #11

Doesn't one of these come with the Swift build system?

(Joe Groff) #12

I don't believe we build either of them as part of the build system, though I believe the setup scripts will install the Ubuntu package for gold for you.

(John McCall) #13

Also for some reason it is trying to build backends for many CPUs like ARM but my CPU is Qemu's i386 simulator. However there doesn't seem to be a way to prevent that.

Try running build-script with --llvm-targets-to-build X86.

(Simon Evans) #14

Can you elaborate on what you mean by GNU ld not supporting Swift's code model? What exactly are you referring 'code model' in this context?

(Joe Groff) #15

Specifically, Swift expects all the locally-defined symbols in an executable or dynamically linked image to be canonically defined in that image, so for ELF platforms, all our exported symbols have protected visibility. GNU ld still chooses to treat references to protected symbols as non-local on the off chance someone emits a COPY relocation to them, which is fundamentally incompatible with protected visibility. I reported this issue to the project, but it seems that, given the fundamental incompatibility here, they intentionally chose to err in the favor of copy relocations. See for more details.

(Eben Up) #16

So if my distro has ld 2.26 I need to upgrade, it sounds like.

(Joe Groff) #17

I'm not sure newer versions of binutils address the issue, since when I talked to the team it sounded like the incompatible behavior change was intentional and they weren't interested in changing it back. Nonetheless, other linkers like gold and lld generally perform better than GNU ld and use less memory in my experience. Switching linkers might help you with your memory exhaustion problem building LLVM here too.

(Eben Up) #18

Another strange build error on GNU/Linux:

As you can see the compiler doesn't know what __atomic_load_8 is.

(Saleem Abdulrasool) #19

You need an explicit link against libatomic. You will find that in the build system, there are a few targets that explicitly add a -latomic to resolve that.

(Eben Up) #20

My current build target is X86. When I grep I don't see these other targets you are referring to. Maybe they were removed?