Weak linking of frameworks with greater deployment targets

haikuty · December 4, 2019, 9:11pm

:) Yep! Good catch! Going too fast… :)
Well, in the mean time, I now have a simple test program to demonstrate it…

haikuty · December 4, 2019, 9:22pm

Hmmm. Still seeing the issue.

Confirmed from build log that it's using the Dec 3 master trunk snapshot:

/Library/Developer/Toolchains/swift-DEVELOPMENT-SNAPSHOT-2019-12-03-a.xctoolchain/usr/bin/swiftc

repo is here if you want to try:

I just build it (inside Xcode so as to make selecting the toolchain easy) and then execute the built object in Terminal from within the DerivedData Build folder.

You have to build and run it (or at least run it…) on macOS 10.14.6 (or macOS < 10.15.x) for the CryptoKit dylib to be missing and thus see the problem which is:

dyld: Library not loaded: /System/Library/Frameworks/CryptoKit.framework/Versions/A/CryptoKit
Referenced from: /Users/haikuty/Desktop/testwlbug 2019-12-04 13-18-42/Products/usr/local/bin/testwlbug
Reason: image not found
Abort trap: 6

I'm using Xcode Version 11.2.1 (11B500) running in macOS 10.14.6 (18G103).

haikuty · December 5, 2019, 3:08am

Quite possible I'm using if #available() incorrectly. The "*" at the end isn't something I wanted to include but Xcode wouldn't compile without it and suggested it as the "fix-it". ¯_(ツ)_/¯

Think I may have found the issue?

I disassembled the generated binary with Hopper and there appears to be a lazy witness table accessor for a CryptoKit stuff that the compiler seems to have generated that is outside the test for #available os; that seems like it could make the linker think weak linking was not possible... ???
(I haven't looked at what the Swift compiler is doing before so it's likely I'm out in left field here, but it's the only thing I can see in the binary that looks suspicious…).

Here's the routine as Hopper disassembled it into code:

int _$s9CryptoKit8InsecureO10SHA1DigestVAeA0E0AAWl() {
    rax = *lazy protocol witness table cache variable for type CryptoKit.Insecure.SHA1Digest and conformance CryptoKit.Insecure.SHA1Digest : CryptoKit.Digest in CryptoKit;
    var_8 = rax;
    if (rax == 0x0) {
            rax = type metadata accessor for CryptoKit.Insecure.SHA1Digest(0xff);
            rax = swift_getWitnessTable(*protocol conformance descriptor for CryptoKit.Insecure.SHA1Digest : CryptoKit.Digest in CryptoKit, rax, rax);
            *lazy protocol witness table cache variable for type CryptoKit.Insecure.SHA1Digest and conformance CryptoKit.Insecure.SHA1Digest : CryptoKit.Digest in CryptoKit = rax;
            var_8 = rax;
    }
    rax = var_8;
    return rax;
}

UPDATE: (since this forum doesn't allow more than 3 replies in a row by the same person (o.O)):

Filed a bug at bugs . swift . org since it seems like that might be a better place:

github.com/apple/swift

[SR-11907] Inferred Weak Linking when building an SPM package does not seem to be working

opened 10:01PM - 05 Dec 19 UTC

haikusw

bug

| | | |------------------|-----------------|… |Previous ID | SR-11907 | |Radar | https://feedbackassistant.apple.com/feedback/7471728 | |Original Reporter | @haikusw | |Type | Bug | <details> <summary>Environment</summary> Running on macOS 10.14.6 (so CryptoKit isn't present) Xcode 11.2.1 with default toolchain as well as the Dec 3 snapshot of master trunk of the swift toolchain from here: <https://swift.org/builds/development/xcode/swift-DEVELOPMENT-SNAPSHOT-2019-12-03-a/swift-DEVELOPMENT-SNAPSHOT-2019-12-03-a-osx.pkg> </details> <details> <summary>Additional Detail from JIRA</summary> | | | |------------------|-----------------| |Votes | 0 | |Component/s | | |Labels | Bug | |Assignee | None | |Priority | Medium | md5: 036f233fd430111751b42d893446ce55 </details> **Issue Description:** Swift compiler is not inferring that an imported system framework should be weak linked based on availability guards around code that accesses symbols in that framework. Or the compiler is erroneously marking the protected symbols as strongly referenced. I've filed this as FB7471728 but realized it should probably have been filed here instead. I found this related swift forum post ( <https://forums.swift.org/t/weak-linking-of-frameworks-with-greater-deployment-targets/26017> ) which lists several feedback's filed on this from back in June (SwiftUI in their case, but it's a general problem as can be seen by this non-SwiftUI case: FB6170959 and FB6187287). In my case, I create a simple executable Swift package manage project and import CryptoKit (which is macOS 15+ only) and then put the code that references it inside \#if available( iOS 13, macOS 15, \*) { // code referencing CryptoKit here } On macOS 10.14.6 the project will fail to launch with the error: $ swift run dyld: Library not loaded: /System/Library/Frameworks/CryptoKit.framework/Versions/A/CryptoKit Referenced from: /Users/tyler/Desktop/testpkg/.build/x86_64-apple-macosx/debug/testpkg Reason: image not found Abort trap: 6 if I comment out the code that accesses CryptoKit then the code will run fine. Sample project is on github [here](https://github.com/haikusw/testwlbug): Steps: 1\. clone the repo 2. cd into the repo directory 3. swift build 4. once built, execute the resulting binary with: ./.build/debug/testwlbug -\> If you're on 10.14.x it should give you the same error. -\> On 10.15.x it should work fine. On 10.14.x if you comment out the four lines inside the { } block protected by the if \#available(...) and rebuild and run it it will run just fine. (if using Xcode then you'll have to go into Library/Derived Data/... to find the binary to execute it on the command line, but otherwise the same steps of get repo, build it, execute the resulting binary in terminal work to show the result). ------------------------------------------------------------------------ Update: Looking at the built binary with Hopper, I wonder if this is related? There is a function outside of the \#available macro code that the compiler generated that accesses CryptoKit: ``` java int _$s9CryptoKit8InsecureO10SHA1DigestVAeA0E0AAWl() { rax = *lazy protocol witness table cache variable for type CryptoKit.Insecure.SHA1Digest and conformance CryptoKit.Insecure.SHA1Digest : CryptoKit.Digest in CryptoKit; var_8 = rax; if (rax == 0x0) { rax = type metadata accessor for CryptoKit.Insecure.SHA1Digest(0xff); rax = swift_getWitnessTable(*protocol conformance descriptor for CryptoKit.Insecure.SHA1Digest : CryptoKit.Digest in CryptoKit, rax, rax); *lazy protocol witness table cache variable for type CryptoKit.Insecure.SHA1Digest and conformance CryptoKit.Insecure.SHA1Digest : CryptoKit.Digest in CryptoKit = rax; var_8 = rax; } rax = var_8; return rax; } ```

jigar007 · August 20, 2020, 2:33pm

Seems like this not fixed yet.

Compiling for iOS 11.0, but module 'Framework' has a minimum deployment target of iOS 13.0.

Can not link minimum deployment target of iOS 13.0 Framework in iOS 11 project.

pjcau · August 27, 2020, 7:41am

I have already the same issue on framework on ios13 minimun version on project ios10.3, on xCode 11.6 (11E708)

xavier.lowmiller · February 4, 2021, 3:39pm

Sorry to open this after a long time, but what if I don't have control over the third party framework?

My problem is the following:

A framework that requires iOS 13 but doesn't have @available statements
An app that runs on iOS 11+

Even if I put all my calls to the framework in if #available(iOS 13, *) statements, the app will crash on launch on iOS 11 and 12.

Is there anything I can do besides ask the framework provider to add @available statements?

EmDee · July 29, 2022, 2:50pm

Has there been any update here for SPM here or any workaround?

We‘re still experiencing this issue with the latest Xcode 14, beta 3.

tshortli · July 29, 2022, 4:43pm

It's been a number of years since this was first reported and it's hard to say whether everyone in the thread is actually reporting the same bug or different bugs with similar symptoms. @EmDee could you be specific about what framework your code is linking, what deployment target the target has, and the symbol that dyld is reporting as missing? Putting those details in a GitHub issue or a Feedback would be helpful. I can take a look at it once those details are available.

@xavier.lowmiller You should be able to use the approach Jordan detailed above to force the entire framework to be weakly linked:

If you're using an Xcode project you can also achieve this by marking the framework as "Optional" in the linked framework settings for the target.

PatrickN · August 1, 2022, 7:40am

@tshortli Will reply on behalf of @EmDee here.

We're trying to integrate Braintree via SPM for some specific functionality which needs a minimum deployment target of iOS 12. Our target app still has a deployment target of iOS 11. When importing the library, import Braintree generates the compiler error Compiling for iOS 11.0, but module 'Braintree' has a minimum deployment target of iOS 12.0. This error of course makes sense but neither marking the library as "Optional" in the linked framework settings nor using the -weak_library or -weak_framework linker flag resolves the issue.

jrose · August 1, 2022, 3:58pm

I believe in order for -weak_library / -weak_framework to work, the library has to be absent on disk at run time, not just invalid to load. There’s not an inherent technical reason for this that I know of, but it does mean that unless you’re building different versions of your app you won’t get the “weak” effect.

Now, that said, you could build different versions of your app. So it seems like there ought to be some way to tell the compiler that this is okay, as long as you can tie yourself to the OS availability model (i.e. you will provide the library on iOS 12 and later and it will be absent on iOS 11), or you are very limited in what you do with the library (completely undocumented and subject to change).

So there’s a compiler issue here for “I know what I’m doing, please treat the version as minimum availability within this module”, and a possible dyld feature request to treat version load failures as valid for a weakly-linked library.

Meanwhile, your best bet is probably to lower the deployment target and then pepper every top-level declaration with @available. :-(

jrose · August 1, 2022, 4:02pm

To the old questioners: this means if someone sends you a framework compiled for iOS 13 and you want to use it conditionally, you can’t do that save for using Bundle/dlopen to get at it dynamically. Part of the reason for that is the metadata format for ObjC and Swift types changes from OS to OS, and when you compile with a minimum deployment target of iOS 13, the compiler will use the best metadata format for iOS 13, which iOS 12 might not be able to read. And some of this metadata might need to be consulted at load time, rather than on first use.