Distribute XCFramework with mixed implementation only and exported dependencies

We have an SDK that we would to share as XCFramework.
This SDK has some proprietary transitive dependencies, MVP on GitHub here

SDK
—>ImpOnlyA
—>ImpOnlyB
—>PublicSDK

ImpOnly* are imported as @_implementationOnly while PublicSDK must be exposed by SDK and is imported as @_exported

Each SDK has its own Xcode project file with Enable Libraries For Distribution at true.

We build our XCFramework from the SDK root using Carthage script carthage build --platform iOS --no-skip-current --use-xcframeworks

Once built we add it to a consumer app. The xcframework is added as usual inside the app project and we are able to build, launch and archive, basically “It works on my machine”.
The issue comes when we distribute the xcframework, right after trying to build it an error message is displayed
AppDelegate.swift:9:8: Missing required modules: 'ImpOnlyA', 'ImpOnlyB', 'PublicSDK'

I’ve also discovered that if I delete the DerivedData I can reproduce the same error, and that if I use again the script to build the SDK the error disappear.
So it seems that the consumer is looking for something in the derived data that comes form the SDK xcframework creation.
Dependencies are added to the SDK with SPM, but I tried also to add them as Xcode project and add linking manually, but the problem is till the same.

Following a lot of discussions on Swift forums, SO and Apple developers forum I’ve found this conclusion

Mostly I think that this discussion from @NeoNacho sump pretty well the issue

“The general issue here is that the Swift compiler needs to be able to see all the modules being used at build-time, either via .swiftmodule/.swiftinterface or module maps. A single XCFramework can only contain one such module, though.”

Then he suggests some workarounds:

  • You could use @_implementationOnly import for the modules from the third-party SDK if you aren't using anything from that SDK as part of your module's public interface.
  • You provide the third-party SDK separately, e.g. as its own XCFramework. The important part here is that you need to make sure that it doesn't get linked statically into your other XCFramework, otherwise you will run into duplicated symbols. For packages that means making any library products dynamic. This might require some significant effort since there could be more transitive dependencies.
  • You provide the additional module definitions out-of-band and make sure they get found at build-time by the compiler using search paths.

The first one is not violable for us, since we want to expose “PublicSDK”
The second neither because we would like to avoid our clients to access the dependencies
The third could be interesting because it seems that along with the xcframework we just need to add module files but I’ve tried a lot without solving the issues, so far I’ve done:

  • Create an xcframework for each dependencies to leverage Xcode create the .swiftmodule
  • Tried to reference them in the consumer app using Swift Search Path -> Import path
  • Tried reference them from “header search path”
  • Tried reference them from “”framework search path”
    Basically I don’t know which Xcode flag should I use and where to point it.

The MVP project is available here

References:

Another interesting issues that I've found that could confirm why wiping DerivedData doesn't make the project compile anymore, is that in .swiftModule it seems that there are absolute paths to Intermediate directory :thinking:

/Users/XXX/Library/Developer/Xcode/DerivedData/SDK-epfocesjjjtmcwbvklsuzrhppzip/Build/Intermediates.noindex/ArchiveIntermediates/SDK/BuildProductsPath/Release-iphoneos/PackageFrameworksŸh/Users/XXX/Library/Developer/Xcode/DerivedData/SDK-epfocesjjjtmcwbvklsuzrhppzip/Build/Intermediates.noindex/ArchiveIntermediates/SDK/BuildProductsPath/Release-iphoneos/PackageFrameworksŸh/Users/XXX/Library/Developer/Xcode/DerivedData/SDK-epfocesjjjtmcwbvklsuzrhppzip/Build/Intermediates.noindex/ArchiveIntermediates/SDK/BuildProductsPath/Release-iphoneos/PackageFrameworksY\/Users/XXX/Library/Developer/Xcode/DerivedData/SDK-epfocesjjjtmcwbvklsuzrhppzip/Build/Intermediates.noindex/ArchiveIntermediates/SDK/BuildProductsPath/Release-iphoneos:/Applications/Xcode_13.4.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/Library/FrameworksJ/Applications/Xcode_13.4.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.5.sdk/Developer/Library/FrameworksI\/Users/XXX/Library/Developer/Xcode/DerivedData/SDK-epfocesjjjtmcwbvklsuzrhppzip/Build/Intermediates.noindex/ArchiveIntermediates/SDK/BuildProductsPath/Release-iphoneosI//Applications/Xcode_13.4.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/usr/lib$ImpOnlyA$ImpOnlyB

No response from the community for such a complete and well-structured question with an example project and all?

Needless to say, we've also a similar expected setup and would be wonderful to make it actual.

2 Likes

I agree, I have gone rounds with Apple on this. It would be nice if the Swift Compiler team and Xcode Build team worked together to move toward using Swift Packages in Frameworks to a slightly more advanced degree. It is a complicated subject, but basically we need Swift Packages to be Dynamic Libraries when we link them into a framework from which we generate an XCFramework. You can use @_implementationOnly to hide the symbol from the modular interface, but that cares for only part of the problem. What if you want to use that Swift Package in other parts of the SDK? Or in other frameworks that the SDK uses, with out a doubt we get lead down a rabbit hole Modules not being found if they are not marked as @_implementationOnly or a load ton of warnings in Xcode from importing that needed library in the said framework. This all seems to be due to how Swift Package Manager statically links its targets/libraries rather than providing them as dylibs inside of a framework. I have had more than one DTS ticket on this subject and spent time on WWDC Labs only coming to one potential solution with a little help (that is not fun at all fun to implement) which is take a huge Library like NIO and create a Framework version of it from a forked repo, which none the less is hardly maintainable in a realistic scenario; simply to remove certain symbols(Which honestly is not all that simple) from being found in multiple libraries which if we don't potentially could lead to unexpected behavior. The issue comes down to the compiler, linker, and nature of the programming language(The warning seems to indicate a Objective-C runtime thing which seems to be generated from the swift interface for interoperability) which has complicated it to a further degree. Any thoughts/suggestions compiler teams?

I tried to reach Apple about that issue, first asking for a lab session, but they told me to ask for DTS.
I used the DTS and they told me to open a bug.
I've opened a bug and the bug had no answer.

I think that this is definitely not a bug. I spoke with DTS over it for like 8 months, they did an in depth investigation of the situation, they know about the problems and limitations at least. It is a super complicated scenario that really needs functionality added to the XcodeBuild/Swift Package Manager/Swift Compiler/ObjectiveC runtime technologies. I really hope they work on this. Maybe with enough noise they will consider it. I think the largest problem is that it spans over so many technologies, everyone on the low level tools teams are going to need to work together on it.

2 Likes