[Xcode 12.5 RC] Duplicated artifacts of binary frameworks in DerivedData

I've just rechecked a proposed use of binary packages in our project and found a weird behaviour of such artifacts checking in the DerivedData folder.

Imagine the following project structure:

App
- Places (local Swift package)
- CountryCodes (local Swift package)
- LocalizedStrings (local Swift package)
- CarrierProviding (local Swift package)
- ImageDownloader (local Swift package)
…
- ExternalDependencies (local Swift package)
  - Facebook SDK (binary xcframework dependency via HTTPS)
  - Firebase SDK (binary xcframework dependency via HTTPS)

Where:

  • The App target depends on all of the mentioned local Swift packages
  • ExternalDependencies is a wrapping package which contains external dependencies referenced via HTTPS URL link to a ZIP file

Binary xcframeworks are downloaded and extracted to a DerivedData/APP/SourcePackages/artifacts folder, yet not directly, but to a subfolder named by a package. :+1:

I'd expect to either fetch all artifacts to be either downloaded side-by-side (workspace-wide, but that could be problematic with different packages requesting different versions, so that's not the case) or per-package (depending on which package actually calls for such binary artifacts – which I presume should be the case then). With this in mind, I'd expect to see a ExternalDependencies subfolder here which would contain all those referenced binary dependencies, downloaded and extracted, linked in the Workspace JSON file.

What actually happens is, all those binary packages are stored in an artifacts subfolder named by a random one of the local Swift packages mentioned above, where none of those depends on either no or only another local Swift package (but never the ExternalDependencies one), but not on any binary dependency nor the ExternalDependencies package alone – that one is only linked by the App target.

What's worse, every time I close the project and reopen it, ANOTHER folder in artifacts is created and the binaries are fetched again and again into multiple folders (named by unrelated SPM packages) every time on project opening.

The App target alone compiles fine, with every new project opening, the workspace-state.json file is updated with this newly created folder so the linking goes fine, but imagine we have ~27 local Swift packages to nicely separate code, having to download ~150–200MB of binary dependencies on every project open – that's over 5GBs of redundant files instead of just a single fetch. :see_no_evil:

Here's a conflicting, cleaned up workspace-state.json file, you can easily spot the duplicated references to same artifacts being linked to unrelated Swift packages instead of the right one.

workspace-state.json

Here's how it looks in Finder:

Just tested and applies freshly to Xcode 12.5 RC (12E262), first discovered in previous Beta 3.

Reported as FB9084662.

2 Likes

Moving this thread into #swift-users as this isn't strictly about SPM development.

Bugs can be discussed in the forums, but unlikely to be tracked unless they were reported using the Apple's feedback assistant for things that are not open source or at bugs.swift.org for the open source related parts such as SPM.

Thanks for alignment. :+1: I'm not really sure whether it's somehow touching the open-source part, but I've added a note about Apple Bug Tracker report.

Possibly Cc @NeoNacho with a kind request on helping :thinking: I'll be available for assistance in case of troubles with reproducing.

2 Likes

Thanks for reporting this, I'll take a look. The intent was that the subfolder was named after the package containing the binary target, but seems like this isn't working as intended anymore.

1 Like

Thank you! :+1: My teammate just found out that the binaries are downloaded to a proper artifacts/ExternalDependencies subfolder in Xcode 12.4, so this issue started misbehaving with the 12.5 cycle. :information_source:

1 Like

I've got a feedback reply that it was not possible to reproduce the issue, so I've updated it with a demo workspace and a screen capture video demonstrating the issue. :+1:

The demo workspace project is also available here. :bulb:

1 Like

Thanks for the demo workspace, much appreciated!

1 Like