Use xcodebuild swift package with binaryTarget logic flaw

Using Swift
1

Introduction binaryTarget feature in swift package manager 5.3. provides a convenient way to add libraries from the C++ universe using Objective C or pure C wrapper as external dependencies. This is an elegant and simple way. Before that, I had to use git submodules and Xcode subprojects with bash scripts.

I’m trying to refactoring FastRTPSBridge project - wrapper over the FastRTPS library, which is actively used in the TridentCockpit project.

Ok, let’s get started.

git clone -b binaryTarget https://github.com/DimaRU/FastRTPSBridge.git
cd FastRTPSBridge
swift build

Ok!

Tech is working for macOS.

Now try building for iOS Simulator. To do this, you will need to use xcodebuild.

xcodebuild -scheme FastRTPSBridge -destination ‘generic/platform=iOS Simulator’

Error, headers now found.

Okay, I guess maybe iOS isn’t fully supported yet.

Let’s try to build for macOS:

xcodebuild -scheme FastRTPSBridge -destination ‘generic/platform=macOS’

The same error, headers now found.

What is the difference between swift build and xcodebuild? Xcodebuild uses the ‘xcode new build system’, which makes extensive use of parallel builds.

Let’s try to investigate the error.

xcodebuild -scheme FastRTPSBridge -destination ‘generic/platform=macOS’ - IDEBuildingContinueBuildingAftererrors=YES

Error again, but compilation went further. Second run:

xcodebuild -scheme FastRTPSBridge -destination ‘generic/platform=macOS’ -IDEBuildingContinueBuildingAftererrors=YES

** BUILD SUCCEEDED **

Bingo!

The problem is that binaryTargets are processed in parallel with other tasks and simply do not have time to execute and create an environment for processing other targets.

If we force the build system to work after errors occur, then binaryTarget finishes processing and beginning from the second pass, the file structure is prepared for compiling the remaining targets.

This must be considered as logic flaw in xcodebuild system. All binaryTarget’s should be processed before all other targets!

The same problem is here: https://forums.swift.org/t/headers-not-found-for-an-xcframework-packaging-a-static-lib-for-ios-critical/41464 and may be here: https://forums.swift.org/t/packaging-static-library-in-spm-package-for-ios-executable/41245

3 Likes
2

@DimaRU this is awesome. I believe you found the issue, and that gives an answer I've been waiting for a while. I can't quite make it work with the IDEBuilding... option for my iOS example, but I can see how the issue is some missing logic flaw about processing binaryTargets first in xcodebuild.

3

@quentinfasquel

Unfortunately, the quirk with "- IDEBuildingContinueBuildingAftererrors " doesn't work with nested packages.
Please try:

cd MyWrapperLibrary
xcodebuild -scheme MyWrapperLibrary -destination 'generic/platform=iOS Simulator' -IDEBuildingContinueBuildingAfterErrors=YES
xcodebuild -scheme MyWrapperLibrary -destination 'generic/platform=iOS Simulator' -IDEBuildingContinueBuildingAfterErrors=YES
4

Yes @DimaRU I can perform this command a bunch of times in a row, it still keeps failing on my hand.

5

Just pack your binary target in separate package, as described in WWDC 10147/2020. This works in my case.

6

Can you elaborate? I have no idea what "WWDC 10147/2020" is a reference to.

I think this could possibly be the cause of the issues I'm seeing, where sometimes the compiler will choose to compile the code that depends on the binary target before it has bothered to copy the actual binaryTarget! (see: Bug with .binaryTarget in Swift Packages with Xcode?)

I don't understand why they made it "binaryTarget" instead of "binaryDependency", since really that's what we need, a "binaryDependency"... why did they make it a target? Clearly a "target" should be what is produced by your framework, not what is depended on by it.

7
8

That video is called "Distribute binary frameworks as Swift packages".

However I'm not trying to distribute anything. I have an existing app, with 40+ local framework targets and 10+ local Swift packages. The app uses Carthage. Carthage can now build XCFrameworks. Some of our Swift packages want to now use the local, dynamic XCFrameworks from the Carthage/Build directory, just like our frameworks do.

This doesn't work though. If we try to link to an XCFramework using ".binaryTarget" then it requires the XCFramework to be inside the Swift Package's own directory structure, so we have to make a symlink to the Carthage/Build directory inside the package. Even then, we're running into issues where Xcode says it cannot find the linked XCFramework.

What are we supposed to do?

This does NOT work:

Screen Shot 2021-03-02 at 9.08.43 AM
Screen Shot 2021-03-02 at 9.08.43 AM2532×1106 278 KB

And this does not work:

Screen Shot 2021-03-02 at 9.10.12 AM
Screen Shot 2021-03-02 at 9.10.12 AM2520×884 247 KB

And if we make it non-dynamic, it works but it results in an extra copy of SwiftProtobuf getting embedded into the app, then it fails validation. Maybe Xcode 12.5 fixes that issue, not sure though.

9

Also, the video you linked specifically says:

don't bundle other people's binary frameworks.

So how am I supposed to link our locally-declared Swift packages to our Carthage dependencies, which have been built as XCFrameworks?

10

After trying all the suggestions I could find, I came to the conclusion that .binaryTarget is not suitable for use with local frameworks in Xcode 12.3 due to the bug described above (link), unless there's something I'm missing here.

We've tried many different approaches, yet every time, we get intermittent failures where the build system can't find the XCFramework from the binaryTarget. Definitely feels like dependency-resolution bug in Xcode, but I can't rule out some bizarre project-file corruption in our app or similar.

On another approach, I came up with a half-working solution here using .unsafeFlags and .linkedFramework arguments: Binary Frameworks with SwiftPM - #19 by 1oo7 (in case it's helpful to anyone.)

11

This issue still persists in xcode 13 and doesn't bring fixed, does your workaround work for any xcframeworks that can be distributed inside a package instead of a Carthage build directory?

12

This issue still appears with Xcode 14.3, but at Xcode 14.2 project builds well.