SwiftPM - Binary target with sub-dependencies

bielikb · September 11, 2020, 11:27am

Hi,

Xcode 12.0/Swift 5.3 toolchain introduced support for binary targets in Swift Package Manager.

While the feature works perfectly for single binary targets, is there a possibility to declare binary subdependencies for the binary targets?

In a scenario:
where binary package frameworkA is subdependency of binary package frameworkB and frameworkC
and furtherly all 3 binaries live in different repos

Im not able to find a way how to correctly declare the frameworkA as subdependency for frameworkB and frameworkC.

What I've tried sofar:

use the dependencies parameter in Package declaration. e.g. for frameworkB to pass in subdependency of frameworkA, e.g.

dependencies: [
        .package(url: "url-to-repo-with-frameworkA", .exact("1.0.0"))
    ]

At the integration point, the Xcode 12.0 beta 6 won't pick the specified subdependency frameworkA, only the added binary target, in this case frameworkB, will be added, which results in broken integration.

declare frameworkA as another binary target in package manifest for frameworkB and frameworkC.
This works fine when importing either frameworkB or frameworkC to the project.
Integrating both binary packages in Xcode 12.0 beta 6 results in "multiple targets named frameworkA in frameworkB, frameworkC" error.

Thanks for any help/guidance in advance!

Best,
Boris

SDGGiesbrecht · September 11, 2020, 5:08pm

It looks like none of the .binaryTarget factory methods have a dependencies parameter, which is how you would theoretically hook up such a dependency graph.

I don’t know if that is intentional, or just an oversight. I can imagine either being the case. Two binary packages depending on the same source package could be problematic, so maybe it is disallowed to prevent that scenario. But if so, I would expect it to be mentioned in the evolution proposal and it isn’t.

@hartbit, you were the primary implementer. Do you know?

bielikb · September 11, 2020, 7:52pm

Hi, thanks for reply.

Does it mean, that the current solution can be used only in monorepo alike setups, where the integrator need to fetch all the binaries regardless if he actually only need 2 out of 3?

Just to clarify the details in the above scenario:
frameworkA, frameworkB and frameworkC are all binaries (xcframeworks).

I dont want the module stable binaries depend on non-module stable source packages .

SDGGiesbrecht · September 11, 2020, 9:46pm

It means the parameter by which you would declare the dependency does not currently exist. So there is currently no way to make a binary target depend on anything.

I spent about half an hour hunting for a workaround to bypass access control and get at the private initializers that have both the binary target details and a dependencies parameter. I even thought of decoding a Target from a JSON literal, but alas discovered it was only Encodable, not Decodable. I came up empty.

However, thinking about it again after your second message, I suspect you could still trick SwiftPM into fetching all the needed pieces in a reliable manner:

Specify the binary target A like normal.
Specify a source target that is just an empty stub.
Make the empty stub dependent on B and C.
Specify the product A such that it includes both target A and the stub.

Then anyone who depends on product A will get B and C resolved as a side affect of the empty stub.

Let me know if it works.

paulb777 · September 11, 2020, 9:50pm

We used a wrapper target to solve dependencies for a binary framework. See firebase-ios-sdk/Package.swift at master · firebase/firebase-ios-sdk · GitHub

SDGGiesbrecht · September 11, 2020, 9:55pm

Yes, that is almost what I described, except it puts the stub over A as well. I’d say that is sufficient evidence that it works. Thanks, @paulb777.

bielikb · September 12, 2020, 7:59pm

First of all, @paulb777 @SDGGiesbrecht thanks for the help folks. Without your guidance & real world example, I wouldn't be able to find the right solution!

My manifest file for frameworkC that's dependent on frameworkA now looks like this:

It lists the binary target both for frameworkC and frameworkA
It declares additional (stub) target FrameworkCTargets, that has 2 target dependencies -> frameworkC and frameworkA
Additionally, the stub target FrameworkCTargets declares the custom path towards dummy(empty) source file.
NOTE: I had to commit an empty source file (.m or .swift) to FrameworkCTargets subfolder on my repo, for the target FrameworkCTargets to be properly processed when integrated to project X. Without commiting the dummy source file for the stub target to the repo with the manifest, the libSwiftPM just thrown an error about missing source files.
Product frameworkC's only target is the FrameworkCTargets

// swift-tools-version:5.3
import PackageDescription
let package = Package(
    name: "FrameworkC",
    platforms: [
        .iOS(.v13)
    ],
    products: [
        .library(
            name: "FrameworkC",
            targets: ["FrameworkCTargets"]
        )
    ],
    targets: [
        .binaryTarget(
            name: "FrameworkC",
            url: "url-to-framework-c",
            checksum: "checksum"
        ),
        .binaryTarget(
            name: "FrameworkA",
            url: "url-to-framework-a",
            checksum: "checksum"
        ),
        .target(name: "FrameworkCTargets",
                dependencies: [
                    .target(name: "FrameworkA", condition: .when(platforms: .some([.iOS]))),
                    .target(name: "FrameworkC", condition: .when(platforms: .some([.iOS])))
                ],
                path: "FrameworkCTargets"
        )
    ],
    swiftLanguageVersions: [.v5]
)

I hope this helps somebody else, too.

Furtherly, I believe, declaring binary subdepedencies in Package.swift could be done in a way more declarative/straightforward way by following the similar rules that are outlined for non-binary targets.
Should I use feedbackassistant to report this back to Apple team?

SDGGiesbrecht · September 12, 2020, 8:13pm

Absolutely.

No. SwiftPM is part of the open‐source Swift project, so bug reports for it belong at bugs.swift.org.

richardgroves · October 26, 2020, 4:38pm

I've tried making a binary target outside the Package call, and modifying linkerSetttings before putting the target into the package. But XCode (Version 12.1 (12A7403)) crashes when it tries to process it.

// swift-tools-version:5.3
import PackageDescription

// Edit these for a new version
let version = "3.3.0-alpha.6"
let frameworkChecksum = "58189551dc306034b4face3779e15598854c7ac797ec250ccf8e0ee2678e84a4"

// Trying to modify the binary target linker settings to avoid the need for a wrapper target
var fwBinaryTarget = Target.binaryTarget(
	name: "UXCam",
	url: "https://raw.githubusercontent.com/uxcam/ios-sdk/\(version)/UXCam.xcframework.zip",
	checksum: frameworkChecksum
)

fwBinaryTarget.linkerSettings =
	[
		.linkedFramework("AVFoundation"),
		.linkedFramework("CoreGraphics"),
		.linkedFramework("CoreMedia"),
		.linkedFramework("CoreVideo"),
		.linkedFramework("CoreTelephony"),
		.linkedFramework("MobileCoreServices"),
		.linkedFramework("QuartzCore"),
		.linkedFramework("Security"),
		.linkedFramework("SystemConfiguration"),
		.linkedFramework("WebKit"),
		.linkedLibrary("z"),
		.linkedLibrary("iconv")
	]

let package = Package(
    
    name: "UXCam",
    
    platforms: [ .iOS(.v9) ],
    
    products: [ .library( name: "UXCam", targets: ["UXCam"]) ],
    
    targets:
    [
		fwBinaryTarget
    ]
)

As soon as there is anything in the fwBinaryTarget.linkerSettings array XCode will crash when the file is saved.

Edit: Here is a minimal package file that triggers XCode to crash:

// swift-tools-version:5.3
import PackageDescription

// Trying to modify the binary target linker settings to avoid the need for a wrapper target
var fwBinaryTarget = Target.binaryTarget(
	name: "FWName",
	url: "some url",
	checksum: "random checksum"
)

fwBinaryTarget.linkerSettings =
	[
		.linkedFramework("AVFoundation")
	]

let package = Package(
	
	name: "FWName",
	
	platforms: [ .iOS(.v14) ],
	
	products: [ .library( name: "FWName", targets: ["FWName"]) ],
	
	targets:
	[
		fwBinaryTarget
	]
)

Comment out line 26 to stop it crashing XCode.

fgarciaDev · February 1, 2021, 8:42pm

@bielikb and all, is there a supported solution yet, or is the workaround to create a dummy target to import a binary framework and it's dependencies the only way right now?

kean · February 9, 2021, 12:11am

I can't seem to get the workaround to work because of the static linkage.

My setup is:

A (XCFramework)
A depends on B (Swift package I have control over)
B depends on C (Apple package, no linkage specify, Xcode defaults to static)

XCFramework seems to look for dependencies using dyld (dynamically). I can change my package B to be a dynamic library, then iOS is able to find it. But I can't enforce dynamic linkage for C. Any suggestions?

Update: Apparently Apple says in WWDC 2019 that binary frameworks cannot depend on Swift packages. I'm not sure that's fully true because my experiment with dynamic library proved to work. Anyway, I managed to get this to partially work by rewriting the frameworks so that B doesn't depend on C. I now have A -> BCore and B -> BCore, C. Both A and BCore are now binary frameworks as Apple intended.

RonanMcH · February 17, 2021, 10:39am

Hello,
although this workaround of using a dummy target to declare a binary sub-dependency is great and a huge help, we have encountered an issue with it. In a project, we have imported two dependencies that share the same dependency, the frameworks Foo and Bar are the two frameworks we are importing and FooBar is the shared framework. The respective package.swift files look as so:
Framework Foo:

// swift-tools-version:5.3
import PackageDescription
let package = Package(
    name: "Foo",
    platforms: [
        .iOS(.v10)
    ],
    products: [
        .library(
            name: "Foo",
            targets: [
                "FooTargets"
            ]
        )
    ],
    dependencies: [
        .package(name: "FooBar",
                 url: "https://github.com/foobar",
                 from: "1.0.0")
    ],
    targets: [
        .binaryTarget(
            name: "Foo",
            path: "Foo.xcframework"
        ),
        .target(name: "FooTargets",
                dependencies: [
                    .target(name: "Foo"),
                    .product(name: "FooBar", package: "FooBar")
                ],
                path: "Sources")
    ]
)

Framework Bar:

// swift-tools-version:5.3
import PackageDescription
let package = Package(
    name: "Bar",
    platforms: [
        .iOS(.v10)
    ],
    products: [
        .library(
            name: "Bar",
            targets: [
                "BarTargets"
            ]
        )
    ],
    dependencies: [
        .package(name: "FooBar",
                 url: "https://github.com/foobar",
                 from: "1.0.0")
    ],
    targets: [
        .binaryTarget(
            name: "Bar",
            path: "Bar.xcframework"
        ),
        .target(name: "BarTargets",
                dependencies: [
                    .target(name: "Bar"),
                    .product(name: "FooBar", package: "FooBar")
                ],
                path: "Sources")
    ]
)

Framework FooBar:

// swift-tools-version:5.3
import PackageDescription
let package = Package(
    name: "FooBar",
    platforms: [
        .iOS(.v10)
    ],
    products: [
        .library(
            name: "FooBar",
            targets: [
                "FooBar"
            ]
        )
    ],
    targets: [
        .binaryTarget(
            name: "FooBar",
            path: "FooBar.xcframework"
        )
    ]
)

The issue we are encountering is that there is a warning generated in the project stating:
Skipping duplicate build file in Copy Files build phase: /Users/rmchugh/TestIntergrations/TestSPMBug/DerivedData/TestSPMBug/SourcePackages/checkouts/foobar/FooBar.xcframework/ios-i386_x86_64-simulator/FooBar.framework

Is anyone else here experiencing any similar issues with this workaround or have a different implementation that doesn't generate this warning?

Thanks,
Ronan.

bielikb · February 17, 2021, 2:32pm

I guess I encountered this warning in past.

imho, this bug is ready to be reported through the feedback assistant and maybe also on bugs.swift.org

RonanMcH · February 17, 2021, 2:37pm

Thanks @bielikb , I opened a report, see here: [SR-14245] Allow `binaryTarget`s to declare `dependencies` in the Swift packages · Issue #4449 · apple/swift-package-manager · GitHub (you will need an account to view this).

tomasim · August 25, 2022, 2:25pm

Hi I am still having this issue when creating a swift package with both my binary framework and a swift package dependency, its not allowing me to run the package saying there is no target, any idea?

This is my setup:

let package = Package(
    name: "FirstFramework",
    platforms: [
        .iOS(.v15)
    ],
    products: [
        .library(
            name: "FirstFramework",
            targets: ["FirstFrameworkTargets"]),
    ],
    dependencies: [
        .package(url: "https://github.com/maxkonovalov/MKRingProgressView.git", .upToNextMajor(from: "2.3.0"))
    ],
    targets: [
        .binaryTarget(
            name: "FirstFramework",
            path: "./Sources/FirstFramework.xcframework"
        ),
        //.target(name: "FirstFrameworkDependency", dependencies: ["MKRingProgressView"], path: "FirstFrameworkDependency"),
        .target(name: "FirstFrameworkTargets",
                dependencies: [
                    .target(name: "FirstFramework"),
                    .target(name: "FirstFrameworkDependency")
                ],
                path: "FirstFrameworkTargets"
        )
    ]
)

CassiusPacheco · October 13, 2022, 5:28am

Thanks for pointing out an example @paulb777! Just to confirm, does the FirebaseAnalytics binary target directly depend on any of the dependencies listed on the wrapper? I would image so, right?

I'm asking because, I've followed your example and I get crashes in the runtime app dyld[79642]: Library not loaded: @rpath/Valet.framework/Valet, which is one of the dependencies of my SDK.

Here's what the Package file that I'm referencing in my app looks like:

import PackageDescription

let package = Package(
    name: "MySDK",
    platforms: [
        .iOS(.v13),
        .macOS(.v10_15),
    ],
    products: [
        .library(
            name: "MySDK",
            targets: ["MySDKTarget"]
        ),
    ],
    dependencies: [
        .package(url: "https://github.com/square/Valet.git", from: "4.1.2"),
        .package(url: "https://github.com/somethingelse", from: "4.1.2"),
    ],
    targets: [
        .target(
            name: "MySDKTarget",
            dependencies: [
                .target(name: "BinaryWrapper"),
            ],
            path: "MySDK" // contains an empty file
        ),
        .target(
            name: "BinaryFrameworkWrapper",
            dependencies: [
                .target(name: "BinaryFramework"),
                .product(name: "Valet", package: "Valet"),
                .product(name: "somethingelse", package: "somethingelse"),
            ],
            path: "BinaryFrameworkWrapper" // contains an empty file
        ),
        .binaryTarget(
            name: "BinaryFramework",
            url: "https://link/BinaryFramework.xcframework.zip",
            checksum: "sha256"
        ),
    ]
)

My BinaryFramework.xcframework specifies on its own Package file the dependencies I've added to the BinaryFrameworkWrapper. I imagined that the wrapper would somehow solve this?

Would you, or anyone else, have any idea why this would happen? Is it something I need to change in the framework path or something else?

Update:
I've created a sample project that shows the current problem

paulb777 · October 13, 2022, 2:37pm

Yes.

CassiusPacheco · October 14, 2022, 7:43am

Okay, thanks again @paulb777. Would be able to tell me whether there's a trick for exporting the XCFramework or just the standard way described by Apple in the docs is enough? As you can see in my sample project above, I've followed the same steps as Analytics' but no luck.

aibo-cora · November 7, 2022, 9:37pm

How can it be dependent on any of them if they are all declared on the same level?

aibo-cora · November 8, 2022, 11:06pm

Insightful read: XCFrameworks | kean.blog