Bug with .binaryTarget in Swift Packages with Xcode?

Using Swift
1

I'm building an App's test target with xcodebuild. The test target includes tests from a Swift Package, which has an XCFramework .binaryTarget—SwiftProtobuf.xcframework. We have used a symlink to link from a directory inside the Swift Package's directory, to the XCFramework, since you can't include a .binaryTarget from outside the Package's own directory structure (which isn't a bug, but nonetheless, is really dang annoying).

The build itself will encounter the error:

/REDACTED/TensorManifold/Sources/TensorManifold/base.pb.swift:13:8: 
error: no such module 'SwiftProtobuf'
import SwiftProtobuf

This error happens BEFORE we get to this build step:

ProcessXCFramework /REDACTED/TensorManifold/lib/SwiftProtobuf.xcframework 
(in target 'TensorManifold' from project 'TensorManifold')

The build command that was run:

set -o pipefail && env NSUnbufferedIO=YES xcodebuild \
 -workspace REDACTED.xcworkspace \
 -scheme AllUnitTests -configuration Debug_Testing \ 
 -derivedDataPath /private/tmp/build/REDACTED/ios/fastlane/output/derivedData \
 -destination 'platform=iOS Simulator,name=iPhone 7,OS=14.3' \
  clean build-for-testing

Here is what our Package.swift looks like:

import PackageDescription

let package = Package(
    name: "TensorManifold",
    platforms: [
        .iOS(.v13),
    ],
    products: [
        .library(
            name: "TensorManifold",
            targets: ["TensorManifold", "SwiftProtobuf"]
        ),
    ],
    dependencies: [
    ],
    targets: [
        .target(
            name: "TensorManifold",
            dependencies: ["SwiftProtobuf"]
        ),
        .binaryTarget(
            name: "SwiftProtobuf",
            path: "lib/SwiftProtobuf.xcframework" // this is a symlink
                                            // it builds just fine in Xcode
        ),
    ]
)
3 Likes
2

Another bizarre thing about this bug is that it seems to be quite intermittent. Our CI system has several fastlane "lanes" that each get built on a separate Mac. For reasons unknown, sometimes they will fail with the above-mentioned error, where SPM acts like like SwiftProtobuf doesn't exist:

/REDACTED/TensorManifold/Sources/TensorManifold/base.pb.swift:13:8: 
error: no such module 'SwiftProtobuf'
import SwiftProtobuf

I can assure you in all these cases, SwiftProtobuf.xcframework does in fact exist, and the path "lib/SwiftProtobuf.xcframework" is a valid path (it's a symlink to where the actual XCFramework is kept).

Can anyone think of a reason why this should fail intermittently like that?

3

Reading further, it seems there have been a lot of problems with using .binaryTarget in SPM, such as Xcode not signing them, or Xcode not copying them into the build folder before trying to compile the code in the module, etc.

It has started to occur to me—what we need is a way to list an XCFramework in the "dependencies" section of a Swift package. "Target" should only be used for when the binary is published as part of your package.

It seems .binaryTarget was created as a way to let someone wrap an XCFramework inside of a Swift Package so they can publish a closed-source package, like how Google now publishes Firebase.

But what we users need, is a way to link an XCFramework (that doesn't have a Swift package) into a locally-declared Swift package (that's not published anywhere and which we're basically just using as a way to avoid using .xcproj files to organize code in our app).

3 Likes
4

I'm running into this exact problem now. Wondering if you found any resolution. SPM and binaryTarget works great locally, but not on CI.

5

For some more context, I've found that my build locally with the same command line works for 12.5 but not 12.4. Our CI is 12.4 (Github Actions). However, using Xcode visually works with 12.4. I haven't figured out a reasonable workaround yet, but hoping we can either get 12.5 preview for Github, or something else. Otherwise kind of stuck currently.

6

I found that it works for 12.5.1 but not for 13.0... this is a real problem looks like

7

Did you manage to get this working again in xcode 13.2.1? I have a problem with a custom objective-c target that it does not work.

8

OK so I can confirm it works on xcode 13.2.1. For objective-c just had to add the bridging header

9

What works, exactly? How did you add the bridging header? (No matter what I do, I can't seem to get my local Swift packages' build products (e.g.: MyPackage_SA567A_PackageProduct.framework) to have a Headers directory like normal frameworks do, so that Obj. C can import the Swift code from them.

10

In fact scratch my last posts. In order for it to work in a swift package the binary target if it includes objective-c should have been build enabling modules. This you can do by adding in build settings DEFINES_MODULE=YES

Screenshot 2022-02-09 at 10.09.51

11

How do you set build settings like DEFINES_MODULE=YES on a Swift package? Your screenshot is very low resolution. Thanks.

12

You do not set it in the swift package. But the binary target you target in the swift package should be a framework build with DEFINES_MODULE=YES set to YES.

To verify that the binary target framework that contains objective-c code you should go the the framework and look into it (show package content). There you should find a folder Modules/modules.modulemap. This is what SPM needs to find the public headers so it can expose the objective-c code to swift.

Hope this helps or let me know more about your setup so I can help.

13

@doozMen Thanks a lot for the input and the explanation. Just a quick question - I understand that in the package.swift file below the binary target targeted by the swift package is the FraudForce target which includes the framework in the path /Sources/FraudForce.xcframework. The DEFINES_MODULE=YES build setting is in the project file that generates the framework. Is my understanding correct?

If the framework file is just distributed by the vendor and I don't have access to the project file. Is there another way to make this work?

// swift-tools-version:5.5
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "FraudForce",
    platforms: [
        .iOS(.v14)
    ],
    products: [
        .library(
            name: "FraudForce",
            targets: ["FraudForce"]),
    ],
    dependencies: [],
    targets: [
        .binaryTarget(name: "FraudForce", path: "./Sources/FraudForce.xcframework"),
    ]
)
14

No the vendor of the framework has to build it like that. But you can add the module file yourself in the framework directory to fix that. Just every time you update you will have to do that step.

In your case the framework shoud have a folder Modules that contains a file module.modulemap with the following content

framework module FraudForce {
  umbrella header "FraudForce.h"

  export *
  module * { export * }
}

That is assuming FraudForce.h is the umbrella header.

15

Bumping this thread, I appear to have the same issue:

.binaryTarget appears not to work with swift build, whereas it does with xcodebuild

Recreate:

  • Create a wrapper package for binary .xcframeworks
  • Create a project package for the parent
  • Add the wrapper package as a dependency

Now test build scenarious:

  • :white_check_mark: Works: xcodebuild -scheme <target> -destination "generic/platform=iOS Simulator" build
  • :x: Fails: swift build -Xswiftc "-sdk" -Xswiftc "xcrun --sdk iphonesimulator --show-sdk-path" -Xswiftc "-target" -Xswiftc "arm64-apple-ios13.1-simulator"
    • error: no such module '<The binary framework>'

Furthermore:

  • If the binary framework dependency is switched for a source-code version (e.g. from Github), then both can checkout and compile perfectly well

It appears like swift build doesn't implement the binary support...? In either case, module.modulemap is present

Hope someone can help

16

iOS is not a supported platform in swift build.

17

Hi @NeoNacho. Yes, indeed, that's fine. I'm not really trying to seriously build a target here; we're investigating migrating to SPM for our (large) project, and hitting significant performance issues, specifically at the build description stage. Currently it takes ~4 minutes using an M1 Macbook Pro when using Xcode{build}

I was hoping to do some profiling using "raw" swift build; since it's open source, I would have more of a chance of gathering learnings about what could be done about this. I was quite surprised to see that binary targets seemingly don't work here. Is it that they are somehow an extension that's built "on top" for xcodebuild, and not implemented at all for the upstream swift build?

It's not impossible that I could substitute our binary dependencies for source code in order to run profiling, although that would be a decent amount of preparation work.

Hope you can help! Thanks

18

It's implemented, but since iOS isn't supported, it might not work for iOS binary dependencies.

19

@itsthejb Are these local or remote binaries. I've had the same issue with Xcode just a week ago. I haven't manage to figure out what was causing it, most likely some internal Xcode remote fetching thing, but the performance was terrible. Once I moved to local binary dependencies, everything worked as expected.

20

They are local binaries. Using a SPM Package.swift wrapper to simply deliver the binaries. Works fine with XCode, but no with the CLI. This is pre-build, it should not matter that the resulting resolved tree can't be built