Problems with `build-script` building compiler with `--xcode`

Hi,
Trying to build the compiler with --xcode and seeing an error

CMake Error in libswift/CMakeLists.txt:
  The custom command generating

    /build/Xcode-RelWithDebInfoAssert+swift-DebugAssert/swift-macosx-arm64/bootstrapping1/libswift/Optimizer.o

  is attached to multiple targets:

    bootstrapping1-libswift-Optimizer.o
    libswift-bootstrapping1

  but none of these is a common dependency of the other(s).  This is not
  allowed by the Xcode "new build system".


CMake Error in CMakeLists.txt:
  The custom command generating

    build/Xcode-RelWithDebInfoAssert+swift-DebugAssert/swift-macosx-arm64/bootstrapping1/libswift/Optimizer.o

  is attached to multiple targets:

    bootstrapping1-libswift-Optimizer.o
    libswift-bootstrapping1

  but none of these is a common dependency of the other(s).  This is not
  allowed by the Xcode "new build system".


CMake Error in stdlib/CMakeLists.txt:
  The custom command generating

   /build/Xcode-RelWithDebInfoAssert+swift-DebugAssert/swift-macosx-arm64/bootstrapping1/libswift/Optimizer.o

  is attached to multiple targets:

    bootstrapping1-libswift-Optimizer.o
    libswift-bootstrapping1

  but none of these is a common dependency of the other(s).  This is not
  allowed by the Xcode "new build system".

Here is the full command SKIP_XCODE_VERSION_CHECK=1 ./swift/utils/build-script --xcode --release-debuginfo --debug-swift --skip-build-benchmarks --swift-darwin-supported-archs "$(uname -m)" --sccache --skip-ios --skip-tvos --skip-watchos

I've seen some changes related to bootstrapping libswift like

There are any new flag that had to be added now e.g. --libswift=off? There are any new setup the have to be made?

Additionally, there are some issues with the Xcode build for a while (at least for me it's been a long time that I can't build the compiler with --xcode) so I wonder what most people use as setup instead of Xcode? I have seen some sharing that VSCode with a couple of plugins for c++ can be good enough for development, but I couldn't find it straightforward to setup.

Thanks in advance,
Best Regards,
Luciano

2 Likes

UPDATE: Using --libswift=off seems to make this error go away, but actually build still fails with a new one

xcodebuild: error: 'Swift.xcodeproj' does not exist.
ERROR: command terminated with a non-zero exit status 66, aborting

Still have to investigate that one

I just remember that we have to use --reconfigure as well when trying to use --libswift=off, but issue is that we go back to another issue that I've seen couple weeks ago

clang: error: no such file or directory: '${REPO_DIR}/build/Xcode-RelWithDebInfoAssert+swift-DebugAssert/llvm-macosx-arm64/Debug/lib/libLLVMSupport.a'
clang: error: no such file or directory: '${REPO_DIR}/build/Xcode-RelWithDebInfoAssert+swift-DebugAssert/llvm-macosx-arm64/Debug/lib/libLLVMDemangle.a'
Command Ld failed with a nonzero exit code

** BUILD FAILED **


The following build commands failed:
	Ld /${REPO_DIR}/build/Xcode-RelWithDebInfoAssert+swift-DebugAssert/swift-macosx-arm64/Debug/bin/swift-serialize-diagnostics normal (in target 'swift-serialize-diagnostics' from project 'Swift')
(1 failure)
ERROR: command terminated with a non-zero exit status 65, aborting

I'm not sure, but that one start happening after an llvm rebranch maybe related? Or coincidence? I'm not familiar enough with that. Have anyone else seen this?

What branch / commit are you trying to build?

Unfortunately, in my experience, trying to work on the compiler on an M1 is a world of pain [SR-14035] Compiler toolchain build error on M1 mac · Issue #56426 · apple/swift · GitHub though untill recently a --xcode build worked but I cloned main this afternoon and even that is failing at the moment. Seems like there really needs to be an M1 host added to the CI setup.

2 Likes

I on main branch, always update checkout before trying to build for the a couple of weeks or so :)

What is the error you been seeing?
It's been like that for a couple of weeks, I found that the issue with LLVM libraries is related to the fact that I'm building with --release-debuginfo --debug-swift and because of that LLVM libraries are built in llvm-macosx-arm64/RelWithDebugInfo/lib/libLLVMSupport.a but as we can see on the error it tries to find it in the Swift configuration folder for some reason. For what I could understand https://github.com/apple/swift/blob/7123d2614b5f222d03b3762cb110d27a9dd98e24/cmake/modules/SwiftXcodeSupport.cmake#L52 is designed to fix this problem but apparently is not being applied for those libraries ... so I'm still not sure what is the issue here.

The purpose of the question was more to know if people are using some workaround, or they are not using Xcode at all and using ninja build with some editor like VSCode which I tried, but was not so trivial to configure :(

I'm seeing a problem very early on with testing cmark dependencies.

    /bin/sh -c /Volumes/Data2/swift-m1/build/Xcode-RelWithDebInfoAssert/cmark-macosx-arm64/cmark.build/RelWithDebInfo/RUN_TESTS.build/Script-003C8D08B0008F35F218E126.sh
Test project /Volumes/Data2/swift-m1/build/Xcode-RelWithDebInfoAssert/cmark-macosx-arm64
    Start 1: api_test
1/9 Test #1: api_test .........................   Passed    0.41 sec
    Start 2: html_normalization
2/9 Test #2: html_normalization ...............   Passed    0.11 sec
    Start 3: spectest_library
3/9 Test #3: spectest_library .................***Failed    0.04 sec
    Start 4: pathological_tests_library
4/9 Test #4: pathological_tests_library .......***Failed    0.03 sec
    Start 5: roundtriptest_library
5/9 Test #5: roundtriptest_library ............***Failed    0.04 sec
    Start 6: entity_library
6/9 Test #6: entity_library ...................***Failed    0.03 sec
    Start 7: spectest_executable
7/9 Test #7: spectest_executable ..............***Failed    0.04 sec
    Start 8: smartpuncttest_executable
8/9 Test #8: smartpuncttest_executable ........***Failed    0.03 sec
    Start 9: regressiontest_executable
9/9 Test #9: regressiontest_executable ........***Failed    0.03 sec

I remember I fixed this in the past by removing some lines in cmark/test/CMakeLists.txt but it was enough to put me off trying to go any further today.

Update: I've reapplied that fix now and it's getting further with --xcode. I'll let you know how it turns out.

1 Like

OK, it got as far as the error you mentioned in the OP which is probably something to do with the "swift-driver" changes I'd guess. --xcode used to work so I'd probably bisect back through the daily tags doing a utils/update-checkout --tag TAG until you find one that works from a month or so ago or perhaps work off a release branch. Once you have an Xcode project that builds you can download the toolchain for that date and swap in the binary Xcode builds for testing.

I was just able to build a Xcode build successfully as suspected with --debug (the whole command SKIP_XCODE_VERSION_CHECK=1 ./swift/utils/build-script --xcode --debug --skip-build-benchmarks --swift-darwin-supported-archs "$(uname -m)" --sccache --skip-ios --skip-tvos --skip-watchos --libswift=off)
Although I think it is an issue not be able to build with --release-debuginfo --debug-swift --debug works at a cost of some disk space and slower build time.

@mishal_shah Can I open an SR for that issue? Just asking before open because maybe this is already reported somewhere

Note that to build successfully, we need also to apply another workaround for unsupported flag: No Xcode builds on CI - #2 by LucianoPAlmeida

I've been rolling the source code repo back and forth in time today and have found that for me, using an M1 mini, tag swift-DEVELOPMENT-SNAPSHOT-2021-09-09-a builds but tag swift-DEVELOPMENT-SNAPSHOT-2021-09-18-a does not if that helps. This is for --release-debuginfo and without --test.
Xcode 13.1.

2 Likes

According to this Clang: error: unknown argument: '-fswift-async-fp=always' - #18 by Buttaface, the tag swift-DEVELOPMENT-SNAPSHOT-2021-09-14-a is OK.

I can verify it is true.

Btw, I'm building on Apple silicon, using Xcode 13.1.
The full command is SKIP_XCODE_VERSION_CHECK=1 ./utils/build-script --xcode --debug --skip-build-benchmarks --swift-darwin-supported-archs "$(uname -m)" --sccache --skip-ios --skip-tvos --skip-watchos --libswift=off.

2 Likes

I'm trying to build on M1 Pro MBP and using the above on main I get pretty far in the compilation before ultimately failing on

xcodebuild: error: 'Swift.xcodeproj' does not exist.
ERROR: command terminated with a non-zero exit status 66, aborting

Any ideas about that?

I've seen this as well, normally is caused by another error earlier in the build, that causes Swift.xcodeproj to not be generated. You can try search the logs to see if that is the case =]

1 Like

Did you also need to add --libswift=off for this work? I had been working with a build I got a few weeks ago with

git checkout swift-DEVELOPMENT-SNAPSHOT-2021-09-14-a && utils/update-checkout --tag swift-DEVELOPMENT-SNAPSHOT-2021-09-14-a

followed by

env SKIP_XCODE_VERSION_CHECK=1 utils/build-script --skip-build-benchmarks \ --skip-ios --skip-watchos --skip-tvos --swift-darwin-supported-archs "$(uname -m)" \ --sccache --release-debuginfo --swift-disable-dead-stripping --debug-swift \ --xcode --reconfigure but needed to rebuild this due to an issue last week and now its no longer working.

The build now succeeds in terminal, then I manually create the All_Build and swift-frontend as schemes on Xcode, run All_Build (builds successfully), then build swift-frontend and there I run into an error that tells me nothing other than " :0: error: fatal error encountered during compilation; please submit a bug report and include the project" followed by a note that says " note: Compiler-internal integrated REPL has been removed; use the LLDB-enhanced REPL instead."

Any ideas? I'm on an Intel mac and 13.2.1 Xcode. Wondering if maybe I need to go to an older Xcode...

The REPL error doesn't mean there is anything wrong with your build. It just means the inbuilt interpreter was removed from the codebase (which was a shame). If you provide arguments for an actual compile it should work.

Could you explain what this means? I'm unsure if I've done this before.

Copy and paste the full command line arguments out of the build log of a project into the scheme. You can no longer use swift-frontend file.swift which used to interpret the file. It needs to be something like swift-frontend -c file.swift -o file.o.

1 Like

Hi Luciano. You're actually helping me with Xcode builds in another thread, but regarding the specific issue in this thread, where llvm files are being built with different configurations than the swift-frontend target in Xcode:

clang: error: no such file or directory: 'swift-project/build/Xcode-RelWithDebInfoAssert/llvm-macosx-arm64/Debug/lib/libLLVMSupport.a'
clang: error: no such file or directory: 'swift-project/build/Xcode-RelWithDebInfoAssert/llvm-macosx-arm64/Debug/lib/libLLVMDemangle.a'

Is the workaround still to rebuild everything with debug optimizations from the command line first? Then build the Swift project swift-frontend from Xcode?

Yes, exactly. You are probably using --release-debuginfo --debug-swift so try just --debug. You would be able to build just fine. Note that this means that build artifacts would require more disk space, but right now this is the only workaround I've found, hadn't be able to figure out the root cause of the problem yet.

Thanks, it works in Xcode using --debug. On an M1 took ~40 min and about 80 gb for build artifacts.

1 Like