Binary targets wrapping static libraries produce fragile dynamic linking arrangement

Development Package Manager
,
1

Hey. I want to highlight some behavior around binary targets and dynamic linkage that I find surprising; I don’t have any suggestions for what a better model of linkage could look like, but I would be curious to know if anyone else has ideas or can point out my misunderstanding of the problem.

When Xcode builds SwiftPM dependencies, in cases such as a diamond arrangement where 2 dynamic products depend on a product with an unspecified linkage type, such a product can be promoted to be a dynamic library. From my understanding, this promotion is intended to avoid machine code duplication, even though it can be permitted by two-level namespace linkage. If I understand correctly, the same promotion behavior might come to swift build in the future.

One case where I find this promotion produces results I don’t expect is with binaryTargets, which wrap static frameworks” especially when a package has multiple such binary targets, and they have a linkage dependency between them.

I will do my best to explain my understanding of why linkage works and doesn’t work in the case of such binary targets, but please correct me if I am wrong:

When a product with source-based targets is promoted to a dynamic library, the linker receives the underlying compilation artifacts in the form of .o in the filelist, which exposes all “external” symbols to the downstream consumers of the dynamic library. The same doesn’t happen with static libraries in binary targets; the static library is not -force_loaded into the dynamic library, and so in the following arrangement:

products: [
    .library(
        name: "ToBePromotedProduct", targets: ["XCFrameworkWithStaticBinaryWrapper"]
    ),
],
targets: [
    .target(
        name: "XCFrameworkWithStaticBinaryWrapper",
        dependencies: ["XCFrameworkWithStaticBinary"],
        // some other configuration like resources distributed in the package
    ),
    .binaryTarget(name: "XCFrameworkWithStaticBinary", path: "XCFrameworkWithStaticBinary.xcframework"),
],

The ToBePromotedProduct dynamic library ends up exposing only the extern symbols of the static binary that are consumed by XCFrameworkWithStaticBinaryWrapper, which often means nothing is exposed.

In my understanding this, for example, breaks the build of FirebaseFirestore library:

dependencies: [
    .package(url: "https://github.com/firebase/firebase-ios-sdk.git", exact: "12.11.0"),
],
targets: [
    .target(
        name: "XCFrameworkWithStaticBinaryWrapper",
        dependencies: [
            .product(name: "FirebaseFirestore", package: "firebase-ios-sdk")
        ]
    ),
],

Because there are 2 static binaries involved: the absl binary is necessary for linking FirebaseFirestore, but if promoted, it is passed as a wrapper that doesn’t actually contain any of the absl implementation.

What is further surprising is that sometimes the linkage can work, but for reasons that seem accidental. For example, even though nothing will be exported from ToBePromotedProduct, downstream linkages can still succeed because of autolinking and a global PackageFrameworks search path where the build system unpacks all XCFrameworks.

For example, if the ToBePromotedProduct’s XCFramework was created with the following sources:

// XCFrameworkWithStaticBinary/XCFrameworkWithStaticBinary.m
#import "XCFrameworkTest.h"
@implementation ObjCType
@end

void bar(void) {};

// XCFrameworkWithStaticBinary/foo.m
void foo(void) {};

then consumed by the following package:

products: [
    .library(name: "Right", type: .dynamic, targets: ["Right"]),
],
dependencies: [
    .package(path: "../ToBePromotedPackage")
],
targets: [
    .target(
        name: "Right",
        dependencies: [.product(name: "ToBePromotedProduct", package: "ToBePromotedPackage")]
    ),
],

with a source such as

import XCFrameworkWithStaticBinary

public func right() {
    ObjCType()
}

and an equivalent product on the “Left”, then finally, all of these are consumed in the executable:

import XCFrameworkTest
import Right
import Left

@main
struct Foo {
    static func main() {
        ObjCType()
        right()
        left()
        foo()
        bar()
    }
}

then the ObjCType is duplicated between Left and Right dynamic libraries, the _bar and ObjCType in the executable are bound to either Left or Right dynamic libraries and the _foo is placed in the executable itself.

I hope these examples make sense. Please let me know if more detailed e2e reproductions of these issues would help explain the problem.