-ObjC flag causes duplicate symbols with Swift Packages

I think we'll need more information on which symbols are duplicated and what the contents of the various pieces here are. I wasn't able to reproduce this by replicating your set up in a simple example.

1 Like

Thanks for looking into this! I've pushed a repo of the project that I'm able to repro this in here: GitHub - ciauri/Halp: Sample Project to reproduce Xcode 11 Bug FB7040693 when dealing with Swift Packages and the -Objc flag

Just init the Package dir as a git repo, make a dummy commit, then add it as a swift package dependency to the StaticLib project via the xcode tool and then attempt to build the App target. The static lib target will build fine, it's just the App that seems to have this issue.

Here is the resulting error message:

Showing Recent Messages
Ld /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator/App.app/App normal x86_64 (in target 'App' from project 'App')
    cd /Users/stephen.ciauri/Downloads/Halp-master/App
    /Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -target x86_64-apple-ios13.0-simulator -isysroot /Applications/Xcode-beta.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator13.0.sdk -L/Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator -F/Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator -filelist /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Intermediates.noindex/App.build/Debug-iphonesimulator/App.build/Objects-normal/x86_64/App.LinkFileList -Xlinker -rpath -Xlinker @executable_path/Frameworks -dead_strip -Xlinker -object_path_lto -Xlinker /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Intermediates.noindex/App.build/Debug-iphonesimulator/App.build/Objects-normal/x86_64/App_lto.o -Xlinker -export_dynamic -Xlinker -no_deduplicate -Xlinker -objc_abi_version -Xlinker 2 -fobjc-link-runtime -L/Applications/Xcode-beta.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator -L/usr/lib/swift -Xlinker -add_ast_path -Xlinker /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Intermediates.noindex/App.build/Debug-iphonesimulator/App.build/Objects-normal/x86_64/App.swiftmodule -ObjC -Xlinker -sectcreate -Xlinker __TEXT -Xlinker __entitlements -Xlinker /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Intermediates.noindex/App.build/Debug-iphonesimulator/App.build/App.app-Simulated.xcent /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator/libStaticLib.a -Xlinker -dependency_info -Xlinker /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Intermediates.noindex/App.build/Debug-iphonesimulator/App.build/Objects-normal/x86_64/App_dependency_info.dat -o /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator/App.app/App -Xlinker -add_ast_path -Xlinker /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Intermediates.noindex/StaticLib.build/Debug-iphonesimulator/StaticLib.build/Objects-normal/x86_64/StaticLib.swiftmodule

duplicate symbol 'Package.Package.init(text: Swift.String) -> Package.Package' in:
    /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator/libStaticLib.a(Package.o)
duplicate symbol 'default argument 0 of Package.Package.init(text: Swift.String) -> Package.Package' in:
    /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator/libStaticLib.a(Package.o)
duplicate symbol 'Package.Package.text.modify : Swift.String' in:
    /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator/libStaticLib.a(Package.o)
duplicate symbol 'Package.Package.text.getter : Swift.String' in:
    /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator/libStaticLib.a(Package.o)
duplicate symbol 'property descriptor for Package.Package.text : Swift.String' in:
    /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator/libStaticLib.a(Package.o)
duplicate symbol 'variable initialization expression of Package.Package.text : Swift.String' in:
    /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator/libStaticLib.a(Package.o)
duplicate symbol 'Package.Package.text.setter : Swift.String' in:
    /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator/libStaticLib.a(Package.o)
duplicate symbol 'Package.Package.init() -> Package.Package' in:
    /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator/libStaticLib.a(Package.o)
duplicate symbol 'type metadata accessor for Package.Package' in:
    /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator/libStaticLib.a(Package.o)
duplicate symbol 'nominal type descriptor for Package.Package' in:
    /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator/libStaticLib.a(Package.o)
duplicate symbol 'type metadata for Package.Package' in:
    /Users/stephen.ciauri/Library/Developer/Xcode/DerivedData/App-czvrzrbbjmoyjdcjoekskyustvxd/Build/Products/Debug-iphonesimulator/libStaticLib.a(Package.o)
ld: 11 duplicate symbols for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Thanks, I was able to reproduce this issue. This issue isn't related to OSS SwiftPM, would you mind filing a bug report at https://feedbackassistant.apple.com?

1 Like

Wonderful news!

Donezo: FB7040693

Thanks again for your help, Aciid.

Thank you!

Hello ciauri, did you found any solution? I am facing the same issue with using of SPM and -ObjC flag...
Thank you

Sadly, no. I did not investigate this any further. Let me know if you learn anything!

This behavior is still reproducible in the Xcode 11 GM Seed. :sob:

@Aciid is there something that we're doing wrong here? Are you aware of any workarounds to this? I suppose it's likely a compiler issue.

No, this is an Xcode issue. I am not aware of any workarounds.

3 Likes

@ciauri I'm having the same issue… did you find anything? This post is the closest to my problem, and you all say there's no workaround :sob:
In addition, I have 1122 duplicate symbols, so I have no idea where it comes from :confused:

Sadly I haven't found anything! This is a pretty big blocker for my project starting to use SPM. In lieu of using SPM, I just added the source files directly to the parent project's compilation phase and went the git submodule route. :\ Definitely not ideal for large packages. All I can suggest is to file a radar so that It can be marked as a dupe and increased in severity.

1 Like

Two weeks ago, I decided to reorganise my app files' structure, and I came up with this example workspace. It worked fine, but I realised I could use a Static Library instead of another project with linked code in each other project.

I changed everything in my app, and I had no problem since this morning. But for some reason XCode once compiled and said I had 1622 duplicate symbols :flushed:

By removing a Linked Library in Build Phases, I managed to reduce it to 1122… but I never managed to completely remove it. I even tried removing every Compile Source in each project and adding them back few by few to see which file caused the problem. XCode could not compile due to missing symbols, but as soon as it found every symbol, it showed 500+ duplicate symbols once again :sob:

I'm not sure this is an SPM-only issue. I have a framework project which produces duplicate symbol errors when running the test target, and it's integrated through CocoaPods. CP does add the -ObjC flag though, so the root cause may be the same.

1 Like

I removed the -ObjC flag, and now it compiles but Firebase Firestore does not work anymore :confused:

I can log in and call the functions, but I get no answer after fetching documents :thinking: Does the -ObjC flag prevents the compiler from compiling some of the code (it looks like the functions are here, but empty)?

The -ObjC flag tells the linker that it should pull in all the object files from static libraries, instead of just the ones containing symbols that are as-of-yet unresolved. But if you're dealing with Objective-C code, its dynamic behavior makes things more complicated, and unresolved symbols are only inserted for classes, not methods. In particular, if you have an object file that only contains implementations of methods defined in an Objective-C category, you need to pass -ObjC or the linker will never know to include them.

Looking at @ciauri's project above, the problem seems to be that Xcode is adding some object files to the static library twice—if you look at the build log when it runs libtool, the object file is in the file passed to -filelist, and then it's also passed again on the command line separately. So the resulting static library has Package.o in it twice, and when the linker later on sees the -ObjC flag, it dutifully does what you're asking it to do and pulls in every object file—including the duplicate.

Unfortunately, if this is an issue with the build plan that Xcode 11 generates from SPM dependencies, then I don't know if there's a clean way to work around it in the meantime.

6 Likes

I have a similar problem that described in my post here: Direct & indirect static libs dependencies with SPM + Xcode

I don't think this is related to the -ObjC flag. Looking at the github sample project, the problem is that Xcode will link the Package static lib into the static lib target, and then it will do it again on the app, leading to duplicate symbols. We need finer control in Xcode: being able to declare a dependency to another static lib with or without linking it. (without linking means that some other target will have to do it manually)

In your case, I see 2 possible workarounds. The best one, since your Package is inside the static lib Xcode project, you can declare a dependency through Xcode Build Phases pane, and remove the linked libraries.

For the other case when the package is not part of the Xcode project, but just resides in the same Xcode workspace (my case), I do have a workaround too: have the package create that same lib as a dynamic one, and make the static lib target link against this one (hint: it won't actually link against it, since it's dynamic). The main app target can then just link against the static one, as it should be.

    .library(
        name: "Package",
        type: .static,
        targets: ["Package"]),
    .library(
        name: "Package-stub",
        type: .dynamic,
        targets: ["Package"]),

I sure hope this gets addressed, as this is a common use case. The scenario to focus on is a number of dependent static libs, either defined through Xcode or SPM, and an app that link those both directly (some) and indirectly (some other ones). FYI, I also went through a related problem: the module map of inner libs wouldn't be correctly exposed to (outer) targets that would link indirectly.

1 Like

If you're referring to @ciauri's project above, there's no evidence of that in the build log. What's happening is:

  1. The Package target (Swift package target) is built, producing a relocatable object file (instead of a static library) (clang -r ... -o Package.o).

  2. The StaticLib target (Xcode static library target) is built, producing a static library. The file list passed to libtool contains the .o file from its own .swift source and the Package.o from the dependency. But, Package.o is also passed separately on the command line outside of the file list. libtool doesn't deduplicate these inputs, and even emits a warning about the conflict:

    warning same member name (Package.o) in output file used for input files: /path/to/Build/Products/Debug-iphonesimulator/Package.o and: /path/to/Build/Products/Debug-iphonesimulator/Package.o (due to use of basename, truncation, blank padding or duplicate input files)

  3. The App target (Xcode app target) is built. The linker takes the object files for the target's own .swift files as inputs in the file list, and then the command line also includes libStaticLib.a. It does not separately contain Package.o from step 1.

There's another way that we can verify that -ObjC is causing the problem here—just remove it from the App target. If we do that, the App links fine, because there are no references to symbols from Package in App. Even if we have App reference a symbol in Package, it still works without -ObjC because the linker stops looking once it finds the first object file in libStaticLib.a that defines the symbols it's looking for. Since the two copies of Package.o are the same, it will never find symbols in the second one that it hasn't already found in the first one, so the duplicate never gets pulled in unless you force it.

4 Likes

you're right: there is indeed an interaction between the -ObjC flag on the app (3) and the (erroneous) duplicate linking in (2). The workarounds I suggested can still provide a solution, but they do require to move the link to the outer binary (which may not always be feasible).

That being said, I still think we should be able to state in Xcode that a static library depends on another without linking against it. That can be done in embedded subprojects (Build Phase / Dependencies), but not generally in a workspace. Cross-projects dependencies like this are not possible, they require a (full) 'link' to establish a dependency as an implicit one (Build Phase / Link).

SwiftPM is fuzzy about that as well: when building a static library that depends on another one, should the resulting library include both library symbols, or is it left to the final (outer) binary to assemble the libs? Stating the intent somehow would make sense (I have used both approaches for different purposes). I could be wrong as I'm discovering the tools, but it looks like that not all use cases are covered.

1 Like

Xcode 14.0 beta (14A5228q) – still having this issue

4 Likes

Still a problem for me in Xcode 14.3.1