Swift Packages in multiple targets results in “This will result in duplication of library code.” errors

I think it would help to write feedback assistant and/or bug reports for SwiftPM and sharing them here and also if there is no thread about it making an appropriately tagged thread on Apple Dev Forums (always tagging the bug report reference number). If you have already reported this bug, please post it and others duping it will help raise its priority.

I was more than happy to report the following SPM issues:

  • App with a local Swift Packages sometimes can't be built if Thread Sanitizer or Undefined Behavior Sanitizer are on FB9217731
  • Bitcode bundle cannot be generated for app in release config with locally-declared dynamic library Swift Package targets if Deployment Postprocessing is on FB8956614
  • Swift Package Manager dependency on XCFramework fails due to Swift compiler version FB9028470
  • Remote swift package dependencies sometimes fail to resolve and take too long to resolve FB8944736 (they said potential fix in Xcode 12.5 but Google Firebase still takes 15 minutes to go from pasting the URL into the "Add Swift Package" dialog to the screen where you get to select which package products to depend on—and then another 15 minutes after that to deep clone the whole repo—and then another 15 minutes anytime a new employee clones our repo and tries to open it in Xcode or a new CI node spins up that doesn't have the ~800MB deep git clone of all of Firebase's repos and all of its dependencies' repos)
  • "Unexpected Duplicate Tasks" bug in Xcode FB9005461 (they said potential fix in Xcode 12.5 however it wasn't fixed, it was made much worse as described in my post above, because rather than giving you an error it just goes ahead and builds tons of frameworks into your app and it doesn't fail until you try to submit to the app store!)
  • Xcode Previews fail when using a dynamic Swift Package library that's not embedded in a framework FB8955594
  • @testable import doesn't work in Swift packages when using an Xcode configuration with a name other than "Debug" (e.g. a configuration named "Testing" maps to SPM's "Release" configuration which has @testable import disabled) FB8914293
  • Xcode fails to install xcappdata to simulator when running tests FB9070373 (OK it's not an SPM issue but this bug has been in Xcode ever since version 6.0... just so you realize what kind of operation we're dealing with)

Considering that the framework duplication bug is really just a failure caused by 12.5's "fix" for the "Unexpected Duplicate Tasks" bug, I went ahead and just updated FB9005461 with the details of how the removal of ".dynamic" from the package descriptions causes Xcode to now auto-embed every package as a framework into every other framework or package-framework that links to it, leading to app store submission failures.

Apple should open-source Xcode. It would be less work for people like you and I to actually just fix all these bugs ourselves and PR them to Xcode, rather than to constantly be creating sample projects to reproduce bugs and submitting them to a Feedback Assistant queue that advances at the rate of continental drift and half the time when a fix finally trickles down, it not only doesn't solve the problem, but actually makes it worse...

8 Likes

@ferologics are you able to run on a physical device using that script?

Yes! we use this workaround to submit the app for our TestFlight testers. Without it we get a lot of errors about duplicate symbols during the upload of the binary...

fyi we're using this improved version of the script in production now:

movedFrameworks=()
        cd "${CODESIGNING_FOLDER_PATH}/Frameworks/"
        for framework in *; do
            if [ -d "$framework" ]; then
                if [ -d "${framework}/Frameworks" ]; then
                    echo "Moving nested frameworks from ${framework}/Frameworks/ to ${PRODUCT_NAME}.app/Frameworks/"
        
                    cd "${framework}/Frameworks/"
                    for nestedFramework in *; do
                        echo "- nested: ${nestedFramework}"
                        movedFrameworks+=("${nestedFramework}")
                    done
                    cd ..
                    cd ..
        
                    cp -R "${framework}/Frameworks/" .
                    rm -rf "${framework}/Frameworks"
                fi
            fi
        done
        
        if [ "${CONFIGURATION}" == "Debug" ] & [ "${PLATFORM_NAME}" != "iphonesimulator" ] ; then
            for movedFramework in "${movedFrameworks[@]}"
            do
                codesign --force --deep --sign "${EXPANDED_CODE_SIGN_IDENTITY}" --preserve-metadata=identifier,entitlements --timestamp=none "${movedFramework}"
            done
        else
            echo "Info: CODESIGNING is only needed for Debug on device (will be re-signed anyway when archiving) "
        fi
13 Likes

thanks a ton for this, the script fixed my issue as well

Thanks everyone for sharing scripts to remove nested frameworks.

That issue really feels bad because it doesn't allow to share some code between host app and an app extension :expressionless: Any plans to fix it? Xcode 13.0 and 13.1 still has this problem

3 Likes

This was the one script that worked for me. In case anyone is going through the same problem:
I was having this issue trying to upload it to App Store

Besides solving the issue with the script, the failing framework "Moya" had to be taken out of the main target, and now it's only on my secondary targets. I hope this helps someone!

Thanks everyone for scripts to remove duplicated embeddings in swift package libraries!

@NeoNacho Could you please provide us with more information about this issue? I'm pretty sure that removing excess embedded frameworks script is not quite a good solution, so it would be great to understand is it planned to be fixed or maybe this is not an issue from Apple point of view and it's OK to stay with such a script.

As for my case is kind of an urgent issue, because our team develop modular white-label project, where SPM is a perfect manager to divide project into modules as local Swift Packages. We have abandoned Cocoapods and very satisfied with SPM, except of this issue.
The thing is that any modular project structure may have Shared Modules with some common functionality, tools, etc. Shared Modules Swift Packages are declared as dynamic libraries to make it possible to import them into independent Feature Modules without duplicating a source code.
It means that for modular development such dependencies graph is kind of required:

  • Shared Module S (dynamic lib) is used in Module A
  • Shared Module S (dynamic lib) is used in Module B
  • Shared Module S (dynamic lib) is used in main app target
  • Module A is used in main app target
  • Module B is used in main app target

I hope that our case and motivation is sufficiently clear and could be in demand by all swift community.

Thanks in advance :slight_smile:

6 Likes

I did exactly that, however, now, months and months after our original feedback was filed, still this bug remains in Xcode. I don't understand what kind of process Apple internally has? Do they not have tests? How can this kind of thing ever make it to production, let alone, get left that way for multiple releases?

3 Likes

Just to add an additional data point. I have a collection of AUv3 app extensions, all with their own host app. I spent considerable effort to move the common code for the hosts and app extensions into Swift packages, finally hitting on a solution that seems to work when running locally:

  • AUv3Support -- package used by both host and app extension by way of being linked to a common framework shared between the two
  • AUv3Support_macOS -- package used by macOS host for functionality common across host apps. Linked to the app but depends also on AUv3Support in the Package.swift file.
  • AUv3Support_iOS -- same as above but for iOS host apps.

So, there is a common framework that depends on AUv3Support package. The host and app extension both depend on this common framework. The host additionally depends on AUv3Support_ package, which also depends on AUv3Support by way of the Package.swift file where it is defined.

This works fine until I validate the archive that results. Looking at the archive, I see the dreaded embedded framework for AUv3Support:

Host app:
-- AUv3Support.framework (from package)
-- Common.framework:
---- Frameworks:
------ AUv3Support.framework (duplicate)

My AUv3Support Package.swift file has no type attributes set in the product specifications.

What works for me is to just delete the 'Frameworks' folder since the AUv3Support.framework already exists. Doing so, my app validates and AppStore accepts it without error.

The only remaining issue is the pesky warning that now persistently appears about linking to an unsafe dylib. This is definitely an Xcode issue since the error appears only after adding the platform specific package library to the host app (the app extension remains unchanged).

2 Likes

This stopped working in Xcode 13.3 (was working perfectly fine in Xcode 13.2.1). It seems that binary frameworks, which are added as SPM dependencies to other frameworks are no longer embedded in the .app.

3 Likes

YES! This was the post I was looking for. Using the default linking (not specifying .static or .dynamic) no longer embeds PackageFrameworks multiple times in Xcode 13.3. For me, this meant SwiftUI previews which were dynamically linking an external framework now work perfectly! But nothing in the Xcode 13.3 release notes about it :confused:

But you can't specify .static or .dynamic type for products referencing binary targets. Xcode shows the following error message:

invalid type for binary product 'XXXXXXX'; products referencing only binary targets must have a type of 'library'

The error message mentioned here also happens when a product P depends on multiple libraries in a 3rd party dependency D. And the Package.swift file of dependency D sets the multiple libraries with type ".dynamic". Those multiple libraries correspond to internal targets that internally have a dependency on each other. This is because internal target linking is always static and thus causes the conflict error message.

What would have been a great addition is if internal targets could depend on each other dynamically

Thanks @pewe. We're using this script also and works fine for typical simulator building. But when archiving, the script doesn't appear to work - resulting in app store rejection.

we actually removed this script when upgrading to Xcode 13.3 or so. not sure what changed there, but something else broke so we found we removed it and did some other hack to make it work again (I don't remember the details, sorry. it was all very confusing with lots of annoying trial and error)

1 Like