Swift Packages in multiple targets results in “This will result in duplication of library code.” errors

So dynamic linking seems to finally work since Xcode 12.5RC....until you're trying to submit the app, which will give you the infamous ITMS-90205/ITMS-90206 error messages.

I already submitted an example project (FB9028703) to Apple, but no response.

It has the following components:

  • iOS App Target (BundleFormat)
  • Framework 1 (Framework)
  • Framework 2 (OtherFramework)
  • Local SPM Package (Package)
  • Remote SPM Package Dependency (DeviceKit)

With this dependencies:

App > OtherFramework
OtherFramework > Framework
OtherFramework > LocalPackage
LocalPackage > RemotePackage
Framework > RemotePackage

And produces the an app package that looks like this:

As you can see it contains a lot of nested frameworks, which the app store doesn't accept.

I have no idea how to work around this.

The 12.5 RC still has deal-breaker problems for our team.. we worked around the 12.4 linker issues by forcing all our SPM products to be .dynamic and using DISABLE_DIAMOND_PROBLEM_DIAGNOSTIC.

We had hoped that Xcode 12.5 would be our saviour. When the RC came out we went through all our packages and removed the harcoded .dynamic from each Package.swift - however when using 12.5 to build we get this error:

The Xcode build system has crashed. Please close and reopen your workspace.

(sometimes the error is prefixed with unexpected service error:)

The funny thing is.... 12.5's xcodebuild command works just fine!

DEVELOPER_DIR=/Applications/Xcode12.5.app/Contents/Developer xcodebuild -workspace ./Redacted.xcworkspace -scheme Redacted_iOS\ Prod  build

Our package structure is a little complex but we would definitely expect Xcode to handle it. The issue stops happening if we remove the package with the blue background (and therefore packages that it depends on).

The diagram above shows our SPM dependency structure with our app target at the top. Some of these products exist in the same package, others have their own package (there are 8 packages all together). Something else to note that we think may be related is that the package shown in blue has some products that are only used in our unit test targets.

Has anyone else has experienced this in 12.5? For now we are sticking to 12.4 and (again) hoping that the next RC solves all our problems...

this embedded framework issue only seemed to happen when a package was embedded in an xcode framework project (that hasnt been migrated to package manager)

if that is the case, i think just changing the "embed" to "dont embed" in the list of dependencies should fix the issue

(it did for me/us anyways)

hope that helps :sweat_drops:

So we finally seem to have solved our problem with a post-build script that moves all embedded frameworks to "APP.app/Frameworks" (and re-signs everything):

cd "${CODESIGNING_FOLDER_PATH}/Frameworks/"
    for framework in *; do
        if [ -d "$framework" ]; then
            if [ -d "${framework}/Frameworks" ]; then
                echo "Moving embedded frameworks from ${framework} to ${PRODUCT_NAME}.app/Frameworks"
                cp -R "${framework}/Frameworks/" .
                rm -rf "${framework}/Frameworks"
            fi
        fi
    done
    
    if [ [ "${CONFIGURATION}" == "Debug" ] & [ "${SDKROOT}" != *Simulator* ] ] ; then
        find "${CODESIGNING_FOLDER_PATH}" -name "*.framework" -print0 | while read -d $'\\0' framework
        do
            if [ "${framework}" != "NULL" ] ; then
                codesign --force --deep --sign "${EXPANDED_CODE_SIGN_IDENTITY}" --preserve-metadata=identifier,entitlements --timestamp=none "${framework}"
            fi
        done
        
        echo "Resigning ${PRODUCT_NAME}.app after moving frameworks"
        codesign --force --deep --sign "${EXPANDED_CODE_SIGN_IDENTITY}" --preserve-metadata=identifier,entitlements --timestamp=none "${CODESIGNING_FOLDER_PATH}"
    else
        echo "Info: CODESIGNING is only needed for Debug on device (will be re-signed anyway when archiving) "
    fi

This has also solved this problem for us: SwiftPM binaryTarget dependency and code signing

2 Likes

@pewe Thanks for providing the script! however, when I run it I get [: too many arguments.
The fix is to remove the extra [ ] wrapping from:

if [ [ "${CONFIGURATION}" == "Debug" ] & [ "${SDKROOT}" != *Simulator* ] ] ; then

to be:

if [ "${CONFIGURATION}" == "Debug" ] & [ "${SDKROOT}" != *Simulator* ] ; then

Anyway, after I tried it Xcode just says "Unable to install "App"" error and in the details it has "No code signature found".

Did you set a Team in Signing & Capabilities? (Automatically manage signing)

Yes, automatic code signing is enabled.

After diving into this we managed to work around the issue with a similar but modified version of the script posted by @pewe (many thanks!). In our case we had to remove some leftover nested SPM packages. Also, we only sign frameworks that are missing a signature. Hope it helps:

# Flatten and code sign nested frameworks
# adapted from https://forums.swift.org/t/swift-packages-in-multiple-targets-results-in-this-will-result-in-duplication-of-library-code-errors/34892/57

cd "${CODESIGNING_FOLDER_PATH}/Frameworks/"

# flatten nested frameworks by copying to APP.app/Frameworks
for framework in *; do
    if [ -d "$framework" ]; then
        if [ -d "${framework}/Frameworks" ]; then
            echo "Moving embedded frameworks from ${framework} to ${PRODUCT_NAME}.app/Frameworks"
            cp -R "${framework}/Frameworks/" .
            rm -rf "${framework}/Frameworks"
        fi
    fi
done

# remove any leftover nested frameworks (i.e. 'PackageName_359AFEED79E48935_PackageProduct.framework')
for framework in *; do
    if [ -d "$framework" ]; then
        if [ -d "${framework}/Frameworks" ]; then
            echo "Removing embedded frameworks from ${framework} to ${PRODUCT_NAME}.app/Frameworks"
            rm -rf "${framework}/Frameworks"
        fi
    fi
done

# codesign for Debugging on device
if [ "${CONFIGURATION}" == "Debug" ] & [ "${SDKROOT}" != *Simulator* ] ; then

    echo "Code signing frameworks..."
    find "${CODESIGNING_FOLDER_PATH}/Frameworks" -maxdepth 1 -name '*.framework' -print0 | while read -d $'\0' framework
    do
        # only sign frameworks without a signature
        if ! codesign -v "${framework}"; then
            codesign --force --sign "${EXPANDED_CODE_SIGN_IDENTITY}" --preserve-metadata=identifier,entitlements --timestamp=none "${framework}"
            echo "Added missing signature to '${framework}'"
        fi
    done
fi
5 Likes

I just made the same mistake as you, by removing ".dynamic" from all our local packages and updating their Swift Tools versions to 5.4. Except rather than crashing, instead, now when Xcode builds our app, for every Swift package that it decides to build as a dynamic framework, it embeds a copy of it into a Frameworks folder inside every. single. other. package. product. or. Xcode-project-defined. framework. that. depends. on it.

Of course, this "automatic embedding" behavior causes the app to fail app store validation due to multiple copies of the same bundles being deposited throughout the app like easter eggs throughout a backyard on Easter Sunday. WTF.

Like with all the other unwanted, automatic behaviors that plague the SPM/Xcode integration, this also has zero documentation around it, and no way to turn it on or off.

See my post here for details:

I'm 100% certain this problem (described in my post linked above) is the exact same problem that @ferologics is describing, which they have "solved" by writing a shell script to go through their entire app after it's built and remove all the extra copies of frameworks that SPM has sprayed everywhere, and then resign the whole thing.

Our app is 800 MB with 100+ modules... so I find as unacceptable any "solution" that extends our already long CI build times by repeating steps in order to work around an issue that could be avoided by simply not using Swift Packages for anything related to enterprise or business-critical software.

Maybe in Swift 7 or 8, Swift Packages will be able to fully replace Xcode project files without everything caving in on itself. But I've seen enough to now believe that it would be less work to redo our 50+ local packages as Xcode projects than it would be to continue supporting them as Swift Packages.

1 Like

I think it would help to write feedback assistant and/or bug reports for SwiftPM and sharing them here and also if there is no thread about it making an appropriately tagged thread on Apple Dev Forums (always tagging the bug report reference number). If you have already reported this bug, please post it and others duping it will help raise its priority.

I was more than happy to report the following SPM issues:

  • App with a local Swift Packages sometimes can't be built if Thread Sanitizer or Undefined Behavior Sanitizer are on FB9217731
  • Bitcode bundle cannot be generated for app in release config with locally-declared dynamic library Swift Package targets if Deployment Postprocessing is on FB8956614
  • Swift Package Manager dependency on XCFramework fails due to Swift compiler version FB9028470
  • Remote swift package dependencies sometimes fail to resolve and take too long to resolve FB8944736 (they said potential fix in Xcode 12.5 but Google Firebase still takes 15 minutes to go from pasting the URL into the "Add Swift Package" dialog to the screen where you get to select which package products to depend on—and then another 15 minutes after that to deep clone the whole repo—and then another 15 minutes anytime a new employee clones our repo and tries to open it in Xcode or a new CI node spins up that doesn't have the ~800MB deep git clone of all of Firebase's repos and all of its dependencies' repos)
  • "Unexpected Duplicate Tasks" bug in Xcode FB9005461 (they said potential fix in Xcode 12.5 however it wasn't fixed, it was made much worse as described in my post above, because rather than giving you an error it just goes ahead and builds tons of frameworks into your app and it doesn't fail until you try to submit to the app store!)
  • Xcode Previews fail when using a dynamic Swift Package library that's not embedded in a framework FB8955594
  • @testable import doesn't work in Swift packages when using an Xcode configuration with a name other than "Debug" (e.g. a configuration named "Testing" maps to SPM's "Release" configuration which has @testable import disabled) FB8914293
  • Xcode fails to install xcappdata to simulator when running tests FB9070373 (OK it's not an SPM issue but this bug has been in Xcode ever since version 6.0... just so you realize what kind of operation we're dealing with)

Considering that the framework duplication bug is really just a failure caused by 12.5's "fix" for the "Unexpected Duplicate Tasks" bug, I went ahead and just updated FB9005461 with the details of how the removal of ".dynamic" from the package descriptions causes Xcode to now auto-embed every package as a framework into every other framework or package-framework that links to it, leading to app store submission failures.

Apple should open-source Xcode. It would be less work for people like you and I to actually just fix all these bugs ourselves and PR them to Xcode, rather than to constantly be creating sample projects to reproduce bugs and submitting them to a Feedback Assistant queue that advances at the rate of continental drift and half the time when a fix finally trickles down, it not only doesn't solve the problem, but actually makes it worse...

7 Likes

@ferologics are you able to run on a physical device using that script?

Yes! we use this workaround to submit the app for our TestFlight testers. Without it we get a lot of errors about duplicate symbols during the upload of the binary...

fyi we're using this improved version of the script in production now:

movedFrameworks=()
        cd "${CODESIGNING_FOLDER_PATH}/Frameworks/"
        for framework in *; do
            if [ -d "$framework" ]; then
                if [ -d "${framework}/Frameworks" ]; then
                    echo "Moving nested frameworks from ${framework}/Frameworks/ to ${PRODUCT_NAME}.app/Frameworks/"
        
                    cd "${framework}/Frameworks/"
                    for nestedFramework in *; do
                        echo "- nested: ${nestedFramework}"
                        movedFrameworks+=("${nestedFramework}")
                    done
                    cd ..
                    cd ..
        
                    cp -R "${framework}/Frameworks/" .
                    rm -rf "${framework}/Frameworks"
                fi
            fi
        done
        
        if [ "${CONFIGURATION}" == "Debug" ] & [ "${PLATFORM_NAME}" != "iphonesimulator" ] ; then
            for movedFramework in "${movedFrameworks[@]}"
            do
                codesign --force --deep --sign "${EXPANDED_CODE_SIGN_IDENTITY}" --preserve-metadata=identifier,entitlements --timestamp=none "${movedFramework}"
            done
        else
            echo "Info: CODESIGNING is only needed for Debug on device (will be re-signed anyway when archiving) "
        fi
10 Likes

thanks a ton for this, the script fixed my issue as well

Thanks everyone for sharing scripts to remove nested frameworks.

That issue really feels bad because it doesn't allow to share some code between host app and an app extension :expressionless: Any plans to fix it? Xcode 13.0 and 13.1 still has this problem

3 Likes

This was the one script that worked for me. In case anyone is going through the same problem:
I was having this issue trying to upload it to App Store

Besides solving the issue with the script, the failing framework "Moya" had to be taken out of the main target, and now it's only on my secondary targets. I hope this helps someone!

Thanks everyone for scripts to remove duplicated embeddings in swift package libraries!

@NeoNacho Could you please provide us with more information about this issue? I'm pretty sure that removing excess embedded frameworks script is not quite a good solution, so it would be great to understand is it planned to be fixed or maybe this is not an issue from Apple point of view and it's OK to stay with such a script.

As for my case is kind of an urgent issue, because our team develop modular white-label project, where SPM is a perfect manager to divide project into modules as local Swift Packages. We have abandoned Cocoapods and very satisfied with SPM, except of this issue.
The thing is that any modular project structure may have Shared Modules with some common functionality, tools, etc. Shared Modules Swift Packages are declared as dynamic libraries to make it possible to import them into independent Feature Modules without duplicating a source code.
It means that for modular development such dependencies graph is kind of required:

  • Shared Module S (dynamic lib) is used in Module A
  • Shared Module S (dynamic lib) is used in Module B
  • Shared Module S (dynamic lib) is used in main app target
  • Module A is used in main app target
  • Module B is used in main app target

I hope that our case and motivation is sufficiently clear and could be in demand by all swift community.

Thanks in advance :slight_smile:

5 Likes

I did exactly that, however, now, months and months after our original feedback was filed, still this bug remains in Xcode. I don't understand what kind of process Apple internally has? Do they not have tests? How can this kind of thing ever make it to production, let alone, get left that way for multiple releases?

1 Like

Just to add an additional data point. I have a collection of AUv3 app extensions, all with their own host app. I spent considerable effort to move the common code for the hosts and app extensions into Swift packages, finally hitting on a solution that seems to work when running locally:

  • AUv3Support -- package used by both host and app extension by way of being linked to a common framework shared between the two
  • AUv3Support_macOS -- package used by macOS host for functionality common across host apps. Linked to the app but depends also on AUv3Support in the Package.swift file.
  • AUv3Support_iOS -- same as above but for iOS host apps.

So, there is a common framework that depends on AUv3Support package. The host and app extension both depend on this common framework. The host additionally depends on AUv3Support_ package, which also depends on AUv3Support by way of the Package.swift file where it is defined.

This works fine until I validate the archive that results. Looking at the archive, I see the dreaded embedded framework for AUv3Support:

Host app:
-- AUv3Support.framework (from package)
-- Common.framework:
---- Frameworks:
------ AUv3Support.framework (duplicate)

My AUv3Support Package.swift file has no type attributes set in the product specifications.

What works for me is to just delete the 'Frameworks' folder since the AUv3Support.framework already exists. Doing so, my app validates and AppStore accepts it without error.

The only remaining issue is the pesky warning that now persistently appears about linking to an unsafe dylib. This is definitely an Xcode issue since the error appears only after adding the platform specific package library to the host app (the app extension remains unchanged).

1 Like
Terms of Service

Privacy Policy

Cookie Policy