We have moderately complex Swift project with around 2,000 Swift files in a
single module.
Most of the Swift source files are deeply nested inside directories and 90%
of these files have absolute paths with more than 150 characters (assuming
project is in "/Volumes/Macintosh HD2/Project" folder)
Because Swift compiler (Xcode 8.3.2) takes the absolute paths of all source
files while compiling, the ARG_MAX limit is reached and build fails with:
Build operation failed without specifying any errors. Individual build
tasks may have failed for unknown reasons.
One possible cause is if there are too many (possibly zombie) processes; in
this case, rebooting may fix the problem.
Some individual build task failures (up to 12) may be listed below.
Have anybody experienced this issue before?
We temporarily fixed this issue by renaming longer folder names to shorter
ones.
I'm assuming that breaking this monolithic module into smaller ones might
fix the issue, but are there any other better solutions?
You're not the only person to run into this. rdar://problem/33635183 <rdar://problem/33635183> tracks the issue within Apple, and since it would need Xcode-side changes as well there's no point in having a report at bugs.swift.org <Issues · apple/swift · GitHub. I'm afraid we don't have a workaround right now other than what you've mentioned.
Jordan
···
On Aug 16, 2017, at 03:17, bhargav gurlanka via swift-users <swift-users@swift.org> wrote:
We have moderately complex Swift project with around 2,000 Swift files in a single module.
Most of the Swift source files are deeply nested inside directories and 90% of these files have absolute paths with more than 150 characters (assuming project is in "/Volumes/Macintosh HD2/Project" folder)
Because Swift compiler (Xcode 8.3.2) takes the absolute paths of all source files while compiling, the ARG_MAX limit is reached and build fails with:
Build operation failed without specifying any errors. Individual build tasks may have failed for unknown reasons.
One possible cause is if there are too many (possibly zombie) processes; in this case, rebooting may fix the problem.
Some individual build task failures (up to 12) may be listed below.
Have anybody experienced this issue before?
We temporarily fixed this issue by renaming longer folder names to shorter ones.
I'm assuming that breaking this monolithic module into smaller ones might fix the issue, but are there any other better solutions?
I'm working on a project that's bumping up against this issue and trying to figure out if there are any workarounds.
This project builds when it is cloned to a very short path off of the root of the hard drive. I have created a symlink at /x to the actual path and tried setting SRCROOT to /x in my build configuration. This works, but Xcode is still passing the fully resolved paths to the swiftc. Is there any way to get Xcode to use the shorter paths that go through the symlink?
We're hitting this on a project too, somewhere around 1600 Swift files. Deleting about 30 of these gets rid of the error. Symlinking to get shorter paths we couldn't get to work, and even unnesting some dependencies didn't seem to help too much, so not sure what our best recourse is. Splitting this up into frameworks won't work on this project either, as it's already heavily modularized; this is just the heaviest module.
Some great work has been done on this recently by @dabelknap. Soon swiftc should support passing parameters via response files (i.e., swiftc @foo.txt where foo.txt contains the arguments) just as clang and many other compilers do. Then it'll be up to Xcode to adopt that support to write arguments out to a response file when necessary.
Is there any update on this? I see the PR was merged but it's still not available on Xcode 10.1 and I don't see anything related on Xcode 10.2 beta changelog.
Xcode 10.1 fixed long command lines passed to swiftc, but missed an additional patch to make sure that frontend subprocesses spawned by swiftc also used args files as needed. This ends up hitting the merge-modules action that's used when you compile without whole-module-optimization, and this was patched in apple/swift#19449.
Xcode 10.2 should have the change; I don't know what branch the beta's toolchain was cut from but the patch is present from swift-5.0-branch-11-16-2018 on.
In the meantime, with Xcode 10.1, you can force whole-module-optimization on to get around the issue (assuming it's the merge-modules action causing the issue).
Sorry, I wasn't thinking about the Xcode part! The compiler feature is present in Swift 5.0, but Xcode appears to still need to be updated to write its command line arguments to a @params file in order for it to work.
To make sure that the issue is addressed properly, you may want to update your radar to emphasize that it's not a Swift compiler issue anymore, but that Xcode specifically needs to add support for the new behavior.
For what it's worth, we were able to get the command length down a ton by removing a lot of things that were stored as environment variables on our CI machines. All of the environment variables seem to be prepended to any command, and those were by far the majority of the length for us.
Xcode 11 beta 3 added a flag to use an unlimited number of Swift files. You have to add USE_SWIFT_RESPONSE_FILE to YES in Build Settings - User Defined section. But this only works if you use the New Build System.
Tested on a dummy project with 2000 files inside an extra long path and it's working.