Building recent snapshot fails on Arch Linux

I'm trying to build snapshot swift-DEVELOPMENT-SNAPSHOT-2019-06-02-a on my Arch Linux desktop but it fails compiling CoreFoundation stuff.
Here are the last few lines of build output:

[14/103][ 13%][0.222s] Building C object CoreFoundation/CMakeFiles/CoreFoundation.dir/Collections.subproj/CFBasicHash.c.o
FAILED: CoreFoundation/CMakeFiles/CoreFoundation.dir/Collections.subproj/CFBasicHash.c.o 
/home/segabor/Workspace/swift-dev/build/buildbot_linux/llvm-linux-x86_64/bin/clang -DCF_BUILDING_CF -DDEPLOYMENT_RUNTIME_SWIFT -DHAVE_SCHED_GETAFFINITY -DU_SHOW_DRAFT_API -D_GNU_SOURCE -I/home/segabor/Workspace/swift-dev/swift-corelibs-foundation/CoreFoundation -I/usr/include/libxml2 -I/home/segabor/Workspace/swift-dev/build/buildbot_linux/libicu-linux-x86_64/tmp_install/include -I/home/segabor/Workspace/swift-dev/swift-corelibs-libdispatch -I/home/segabor/Workspace/swift-dev/build/buildbot_linux/libdispatch-linux-x86_64/tests -isystem /home/segabor/Workspace/swift-dev/swift-corelibs-libdispatch/src/BlocksRuntime -O3 -DNDEBUG -fPIC   -F /home/segabor/Workspace/swift-dev/build/buildbot_linux/foundation-linux-x86_64 -I /home/segabor/Workspace/swift-dev/build/buildbot_linux/foundation-linux-x86_64/CoreFoundation.framework/PrivateHeaders -include /home/segabor/Workspace/swift-dev/swift-corelibs-foundation/CoreFoundation/Base.subproj/CoreFoundation_Prefix.h -fblocks -fconstant-cfstrings -fdollars-in-identifiers -fexceptions -fno-common -fcf-runtime-abi=swift -Wno-shorten-64-to-32 -Wno-deprecated-declarations -Wno-unreachable-code -Wno-conditional-uninitialized -Wno-unused-variable -Wno-int-conversion -Wno-unused-function -Wno-microsoft-enum-forward-reference -pthread -std=gnu99 -MD -MT CoreFoundation/CMakeFiles/CoreFoundation.dir/Collections.subproj/CFBasicHash.c.o -MF CoreFoundation/CMakeFiles/CoreFoundation.dir/Collections.subproj/CFBasicHash.c.o.d -o CoreFoundation/CMakeFiles/CoreFoundation.dir/Collections.subproj/CFBasicHash.c.o   -c /home/segabor/Workspace/swift-dev/swift-corelibs-foundation/CoreFoundation/Collections.subproj/CFBasicHash.c
In file included from /home/segabor/Workspace/swift-dev/swift-corelibs-foundation/CoreFoundation/Collections.subproj/CFBasicHash.c:11:
In file included from /home/segabor/Workspace/swift-dev/swift-corelibs-foundation/CoreFoundation/Collections.subproj/CFBasicHash.h:12:
In file included from /home/segabor/Workspace/swift-dev/build/buildbot_linux/foundation-linux-x86_64/CoreFoundation.framework/PrivateHeaders/CFInternal.h:156:
/home/segabor/Workspace/swift-dev/build/buildbot_linux/foundation-linux-x86_64/CoreFoundation.framework/Headers/ForSwiftFoundationOnly.h:592:12: warning: implicit declaration of function 'lstat' is invalid in C99 [-Wimplicit-function-declaration]
    return lstat(filename, buffer);
           ^
/home/segabor/Workspace/swift-dev/build/buildbot_linux/foundation-linux-x86_64/CoreFoundation.framework/Headers/ForSwiftFoundationOnly.h:599:20: error: use of undeclared identifier 'SYS_renameat2'; did you mean '_CF_renameat2'?
    return syscall(SYS_renameat2, olddirfd, oldpath, newdirfd, newpath, flags);
                   ^~~~~~~~~~~~~
                   _CF_renameat2
/home/segabor/Workspace/swift-dev/build/buildbot_linux/foundation-linux-x86_64/CoreFoundation.framework/Headers/ForSwiftFoundationOnly.h:597:12: note: '_CF_renameat2' declared here
static int _CF_renameat2(int olddirfd, const char *_Nonnull oldpath,
           ^
1 warning and 1 error generated.
ninja: build stopped: subcommand failed.
./swift/utils/build-script: fatal error: command terminated with a non-zero exit status 1, aborting
./swift/utils/build-script: fatal error: command terminated with a non-zero exit status 1, aborting

Arch Linux is a tough ground for Swift but so far I was able to build and use recent snapshots until this point. But now I don't have a clue what went wrong since then.

Some system calls, eg renameat2 do not actually have wrappers in glibc. See Glibc wrappers for (nearly all) Linux system calls [LWN.net] for details.

You probably don't have a header file defining SYS_renameat2, and in fact the renameat2 system call was not added to Linux until version 3.15. So it looks like a check will need to be added into corelibs-foundation to check the kernel version at runtime and provide an alternative if it is not there - the same thing was done for the statx system call as well.

3 Likes

I don't know if it has to do with SWIG binding but I recently noticed an extra flag in Swift AUR build script that uses swig3 for compatibility reason. I'm going to check if that's the case.
Anyway thanks for the explanation.

Ok, it has nothing to do with SWIG compatibility. Now I'm going to take a closer look into Foundation package.

I found the change set that possibly causes the build failure Parity: FileManager.replaceItem(…) · apple/swift-corelibs-foundation@c512c91 · GitHub

Dear @millenomi , your help is appreciated.
Thanks.

If you’d like to contribute a patch to check at compile time, I’d be happy to take it, I believe. It is not an ongoing goal for mainline development to explicitly support kernel versions older than the ones that are the default for the distros we target, though.

Ive created a patch for this, I will put a PR up in a bit although I will need it tested as its not required on Ubuntu hosts.

Can you please share with me? I will try it out and let you know the result.

I'm using kernel version 5.1.7 which is I think the most recent one. That's why I'm surprised this issue occurred on my Linux. I hope @spevans found the solution.

Here is a PR to try Check renameat2() is available on Linux by spevans · Pull Request #2337 · apple/swift-corelibs-foundation · GitHub
It should only enable the call if __SYS_renameat2 is defined AND the kernel version is >= 3.15

1 Like

Thanks! Just kicked off a build of the patched codebase waiting for result ...

Happy to report that snapshot swift-DEVELOPMENT-SNAPSHOT-2019-06-06-a is successfully built with your patch.
Thank you!

Note that this is a workaround that weakens the guarantees around that method, since this way it does not replace filesystem objects atomically anymore. I would open a bug with upstream to figure out why old headers are used even with newer kernels. (If you end up opening it, can you share a link here?)

You should be able to find the SYS_renameat2 in your header files:

ubuntu1804$ grep -r SYS_renameat2 /usr/include/
/usr/include/x86_64-linux-gnu/bits/syscall.h:# define SYS_renameat2 __NR_renameat2

Did the tests run successfully?

Well, it's there, I checked it. But I could not figure out why it did not include eventually. The only way I was able to make it work when I manually defined SYS_renameat2 in ForSwiftFoundationOnly.h.
When I moved #include <sys/syscalls.h> out of #ifdef block I got other errors like it cannot be included at that stage or something like that. I'm clueless.

I think all tests passed (or I missed out the log lines complaining on failed tests).

Could you actually get the preprocessed output for your build please? (pass -E to the clang invocation). Are you sure that you don't have a stray copy of kernel-headers installed to /usr/local? That would take precedence over the kernel headers in /usr/include.

I have only two headers under /usr/local/include but they don't interfere with Swift (dpi.h and xdo.h).
I started over a fresh build as the current working copy got messed up. Once build process aborts I will try to pass -E flag and get back with the outcome.

Ok, I managed to get swift-corelibs-foundation/CoreFoundation/Base.subproj/CFSystemDirectories.c pre-compiled. Result is captured on GitHub

Sorry, can you do -E -dD instead? That should make it much more useful to debug where the definitions are originating.

Sure! Just a sec ...