Linker error on Ubuntu 18.04 (Swift 5.1.2)

Hi there,

On a freshly-minted Ubuntu 18.04 server I get the error below when trying to build my Swift project. I have installed Swift 5.1.2. Any ideas?

BR,
Mark

swift build
/usr/bin/ld.gold: fatal error: out of file descriptors and couldn't close any
clang-7: error: linker command failed with exit code 1 (use -v to see invocation)
:0: error: link command failed with exit code 1 (use -v to see invocation)
[0/1] Linking ProjectName

You ran out of file descriptors which is a bit odd. Can you post the output of ulimit -a?

Thanks Johannes. The answer was as below.

$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 7728
max locked memory (kbytes, -l) 16384
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 7728
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

Okay, setting

ulimit -n 65536

solved it. So it was a real issue. Thanks!

2 Likes

I'm glad you resolved the issue but I'm slightly surprised that we need more than 1024 open files to compile a Swift project. What kind of project is that?

Hi Johannes,

This one: GitHub - finlabsuk/open-banking-connector: Open Banking Connector simplifies connections to UK Open Banking APIs by absorbing bank differences, handling bank registrations and tokens, and more.

BR,
Mark

We've had to increase ulimit for our project, too. It seems a common issue with big projects on Linux.

@mmfl / @yxckjhasdkjh Right, I think we should file a bug here, would any of you mind filing one?

Here some stats: These are the number of files per module in the open-banking-connector project:

$ cd Sources && for f in *; do find "$f" -name '*.swift' | wc -l | tr -d '\n' && echo "  $f"; done | sort -n
       1  AccountTransactionTypes
       1  PaymentInitiationTypes
       3  AccountTransactionLocalTypes
       3  BaseServices
       3  PaymentInitiationLocalTypes
       5  AccountTransactionTypeRequirements
       7  PaymentInitiationTypeRequirements
      40  OpenBankingConnector
     116  PaymentInitiationApiV3p1p1Types
     122  PaymentInitiationApiV3p1p2Types
     143  AccountTransactionApiV3p0Types
     199  AccountTransactionApiV3p1p1Types
     277  AccountTransactionApiV3p1p2Types

So the largest module has 277 files. The total amount of files the Swift compiler as a whole seems to produce for this module is

$ find .build/x86_64-apple-macosx/debug/AccountTransactionApiV3p1p2Types.build/ -type f | wc -l
    1388

which means it seems to produce roughly 5 output files per input file. So if it wants to open the input file as well as all the output files all in one go, it will need more than 1024 files... I would argue that is a bit excessive and we should look into options to reduce this, if the compiler team determine that this is impossible or too much work, then the compiler should still output at least an actionable error message telling you to raise ulimit -n to something high, it might even be able to give you an estimate given the number of source files.

Thanks Johannes for the analysis. I can have a go at a bug report but will be Fri or Sat as Finnovation Labs is exhibiting and demo-ing Open Banking Connector at a conference this week: https://fdata.global/summit/sponsors-2019/ :smile:

1 Like

Bug report created: [SR-11929] Linker exhausts file descriptors on Linux easily and without suggested remedy · Issue #54348 · apple/swift · GitHub.

1 Like