Migrating to SPM from mix of embedded frameworks and static libraries

At work we have a fairly large project with many, many projects (97 targets in the scheme including test targets and frameworks and whatnot). There is a mix of embedded frameworks, static libraries, a few 3rd party frameworks and a lot of mix and match Swift and Objective-C code with bridging headers in nearly all the static libraries so we could continue adding Swift files to them.

We're wanting to restructure this stuff in pieces and migrate to using SPM with a bunch of local targets that all produce static libraries. The first thought was to start at the root with an embedded framework that we have that doesn't have any other dependencies, convert that to a package and move onto the next. The problem is that has a mix of ObjC and Swift code. I'm able to break that framework out into two targets under a package and get that to compile separately but when I attempt to @import FormerEFNowPackageTargetObjC into an Objective-C file in one of the other embedded frameworks it says it can't find the module. I have it linked to the root project and added to our scheme (we have find implicit dependencies turned off) but it can't find it.

Here's where it gets interesting. I also tried to import it into one of the static libraries we have further down the line and it can't find it unless I explicitly link against the library generated by the package, but when I do that it runs into an issue with duplicate symbols.

tldr;

How (or can you) import a Swift package (either ObjC with include directory or Swift) into Objective-C code in both embedded frameworks and static libraries. If so are there any good approaches to doing this?

Edit: This is on Xcode 11.4 beta 2

1 Like

You cannot import the same static library into multiple dynamic libraries without getting duplicate symbols when you try to link those dynamic libraries into the same application.

As a temporary measure while you work your way up the dependency chain, any time you convert a dynamic library or framework, I would make the converted package products explicitly .dynamic to start with. Then, once you are certain that it ultimately has only one dynamic clientβ€”either a single dynamic library or the top‐level applicationβ€”then you can go back and switch that whole section of the dependency tree back to automatic (nil instead of .dynamic).

1 Like

I don’t have any input on the big picture issue here, but there’s one nit I need to pick (sorry!)…

You cannot import the same static library into multiple dynamic
libraries without getting duplicate symbols once those dynamic
libraries are linked into the same process at runtime.

The two-level namespace on Apple’s platforms means that this won’t be a problem at runtime. Two dynamic libraries can export the same symbol just fine. Clients always reference symbols in the context of a specific library [1].

Share and Enjoy

Quinn β€œThe Eskimo!” @ DTS @ Apple

[1] There are exceptions here, most notably dlsym with RTLD_DEFAULT.

1 Like

That’s fine. I can tell when someone is smarter than me.

To clarify for my own sake then, you’re only saying there will be no problems at runtime. You still get errors at some point during the build, right? (When trying to import? It’s been a while since I actually got the error myself, so I don’t remember for sure how far through the build and launch it occurs.)

I ask because dealing with the duplicate symbol error is a recurring question on the forums. I have usually recommended importing the whole package tree into a single framework that @exported imports everything. From there you can safely depend on the framework from multiple places. Is that advice out of date?

You still get errors at some point during the build, right?

Yeah, that story is much less clear. The static linker has traditionally assumed a C language model, where the input files form a single flat namespace, and that has problems with duplicate symbols. I don’t know if that’s changed with the advent of Swift.

Regardless, it’s not something I’d want to rely on. Your goal of having a single copy of each static library is clearly correct [1], and if you have to use a dynamic library to achieve that goal then so be it.

Share and Enjoy

Quinn β€œThe Eskimo!” @ DTS @ Apple

[1] Indeed, if there’s Objective-C, is a hard requirement.

What I did recently was this. I migrated all our orgβ€˜s frameworks to SPM and resolved their dependencies. Then to avoid duplicates in the main host app I came up with a good trick that works great for us.

Instead of embedding all different libs from packages through Xcodeβ€˜s UI, I created a project local package with a single library target/product. This local package would then have all the dependencies for our frameworks, which are now completely migrated to SPM. This moves completely all the static vs. dynamic linking resolution to SPM and avoids any duplicate symbols. The last step is only to embed the lib from the local package into the host app.

5 Likes

@DevAndArtist @SDGGiesbrecht @eskimo Thank you for your suggestions so far they have helped me move along. I've been going through the arduous task of resolving dependencies (and thousands of missing imports) but I've hit sort of a roadblock and wondering if you have any ideas around it.

One of the packages converted from a framework is Swift-only. It is being used by embedded framework Foo (dynamically linking to this package through another framework called "Shimmy" which is just a shim used to work my way up the list). The Foo framework is Swift-only as well but has some things exposed to ObjC. Those things that are exposed to ObjC have a dependency on that Swift-only package and when it generates the compatibility header the build system can't seem to find the module for the Swift-only package. Is this an issue with the new SPM compatibility integration (new in 5.2) or something with my build structure?

Any help you or anyone else can give is greatly appreciated.

I'm not sure I can provide you further help with that particular issue, but here is a visual representation of my projects dependencies that I could fully resolve by not using Xcode's UI to add all different dependencies to the host application, but instead moving everything one level up into a local package and let SPM do the resolution and linking job.

You can scroll around in this markdown code block:

                                                        β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                    
           β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”‚   RxSwift    β”‚                    
           β”‚                      β”‚                     β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                    
           β”‚                      β”‚                             β”‚                           
           β”‚                      β”‚         β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”       
           β”‚                      β”‚         β”‚                   β”‚                   β”‚       
           β”‚                      β”‚         β–Ό                   β–Ό                   β–Ό       
           β”‚                      β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
           β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”‚   RxRelay    β”‚    β”‚      A       β”‚    β”‚RxBluetoothKitβ”‚
           β”‚                      β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
           β”‚                      β”‚         β”‚                   β”‚                   β”‚       
           β”‚                      β”‚         β”‚          β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”         β”‚       
           β”‚                      β”‚         β”‚          β–Ό        β”‚         β–Ό         β”‚       
           β”‚                      β”‚         β”‚  β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”  β”‚       
           β”‚                      β”‚         β”‚  β”‚      B       β”‚ β”‚ β”‚      C       β”‚  β”‚       
           β”‚                      β”‚         β”‚  β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜  β”‚       
           β”‚                      β”‚         β”‚          β”‚        β”‚         β”‚         β”‚       
           β”‚                      β”‚         β”‚          β””β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜         β”‚       
           β”‚                      β”‚         β”‚                   β”‚                   β”‚       
           β”‚                      β”‚         β”‚                   β–Ό                   β”‚       
           β”‚                      β”‚         β”‚           β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”            β”‚       
           β”‚                      β”‚         β”‚           β”‚      D       β”‚            β”‚       
           β”‚                      β”‚         β”‚           β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜            β”‚       
           β”‚                      β”‚         β”‚                   β”‚                   β”‚       
           β”‚                      β”‚         β”‚                   β”‚                   β”‚       
           β”‚                      β”‚         β”‚                   β–Ό                   β”‚       
           β”‚                      β”‚         β”‚         β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”          β”‚       
           β”‚                      β”‚         β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”‚        E         β”‚          β”‚       
           β”‚                      β”‚         β”‚         β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜          β”‚       
           β”‚                      β”‚         β”‚                                       β”‚       
           β”‚                      β”‚         β”‚                                       β”‚       
           β”‚                      β”‚         β–Ό                                       β”‚       
           β”‚                      β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”            β”‚       
           β”‚                      β”‚ β”‚      F       │───▢│      G       β”‚β—€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜       
           β”‚                      β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                    
           β”‚                      β”‚                             β”‚                           
           β”‚                      β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”            β”‚                           
           β”‚                      β”‚ β”‚      J       β”‚            β”‚                           
           β”‚                      β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜            β”‚                           
           β”‚                      β”‚         β”‚                   β”‚                           
           β”‚                      β”‚         β”‚                   β”‚                           
           β”‚                      β”‚         β–Ό                   β–Ό                           
           β”‚                      β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                    
           β”‚                      β”‚ β”‚      I       │───▢│      H       β”‚                    
           β”‚                      β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                    
           β”‚                      β”‚                             β”‚                           
           β”‚                      β”‚                             β–Ό                           
           β”‚                      β”‚                   ╔═══════════════════╗                 
           β”‚                      β”‚                   β•‘    MyRemoteLib    β•‘                 
           β”‚                      β”‚                   β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•                 
           β”‚                      β”‚                             β”‚                           
           β”‚                      β”‚                             β”‚                           
           β”‚                      β”‚                             β”‚                           
           β”‚                      β”‚                             β”‚                           
           β–Ό                      β–Ό                             β”‚                           
╔════════════════════╗ ╔════════════════════╗                   β”‚                           
β•‘   UserInterface    β•‘ β•‘ SomeOtherRemoteLib β•‘                   β”‚                           
β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β• β•šβ•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•β•                   β”‚                           
           β”‚                      β”‚                             β”‚                           
           └──────────────────────┴──────────────────────────────                           
                                                                β”‚                           
                                                                β–Ό                           
                                                     β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                 
                                                     β”‚     Assembler      β”‚                 
                                                     β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜                 
                                                                β”‚                           
                                                                β”‚                           
                                                                β–Ό                           
                                                    β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”                
                                                    β”‚   Host Application   β”‚                
                                                    β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜       

I’m not sure which versions of SwiftPM can successfully have an Objective C target import a Swift target. It wasn’t possible with the initial Objective C support, but then support was added. After that some bugs popped up, but they were then fixed. However, that is all with respect to the master branch and I don’t know how any of that timeline corresponds to actual releases. It could even be that nothing released so far can do it.

Here is one related thread, but if you search, you’ll find more:

Yup I was involved in that conversation haha. The current version does have the capability of doing so (Swift package imported into an ObjC package and the other way around). The error seems to only rear its head when an embedded framework is producing its compatibility header while being dependent on a Swift only swift package. My solution right now is to press through ALL embedded frameworks before working about getting anything else downstream to build.

I'm having this problem with a 11.3 project. I had a Swift Package used by "A" and "B", and "B" uses "A". I don't want to embed the package dynamically, it needs to be statically linked. Why did this stop working?

I changed the Swift Package to be dynamic because of the issues from Xcode 11.4. Now I'm embedding and signing on the main app target, but it failed to upload to the App Store when embedding on thee extensions like Today widget for example. When I make it "Do Not Embed" on the extension, the Swift Package disappears but I can't compile anymore because the code depends on it. How do I handle app target and extensions scenario now?

I found a workaround by using an internal Framework that deals with SPM libraries.
You can found a step by step here: GitHub - renaudjenny/Swift-Package-Manager-Static-Dynamic-Xcode-Bug: Workaround about SPM (Swift package manager) deal with Xcode 11.4 and Swift 5.2 with external static libraries. Adding an internal dynamic library to resolve static code duplication error

This looks promising, do you mind me asking what your build scheme looks like? I'm at a stage now where I've already done the dynamic linking (through a .dynamic type swift package instead of a framework, same concept though) but I'm building for tests and running into the duplicate linking issue again for the test targets.

Hi happy if that helped you. Actually, you can directly download the project to see the schemes and stuff directly into Xcode. I made different commits to help you figure it out what happened step by step :blush:.

1 Like