Dealing with projects with a lot of Swift Packages

Using Swift
1

I'm dealing with a heavily modularised codebase where features are split into their own packages, each within a respective GitHub repo, with some dependencies between them, forming a dependency graph that generally has 50+ Swift Packages.

The codebase and the modularisation is amazing and providing us with a huge amount of benefits, but the whole team's productivity suffers because of the tooling.

Opening an Xcode project with a lot of Swift package dependencies is a game of roulette. More often than not, the dependency resolution will fail with various errors, like caching issues, SwiftPM.SPMRepositoryError error 3, error 5, etc...

Resetting package caches often solves the problem, but sometimes you have to manually delete all caches, restart Xcode, or even restart a computer. The problem is that dependency resolution is extremely slow even with a gigabit connection. It can easily take up to five minutes on the M1 Pro and then fail.

When this happens often, and to the whole iOS team, the amount of wasted time becomes very worrying.

It's worth saying that we haven't noticed any issues when dealing with pure Swift packages through the command line, but working with them from Xcode is a nightmare. Unfortunately, most of our products are iOS apps, so we can't avoid Xcode.

I'm of course waiting for Xcode 14 to see if that will help in any way. If that doesn't happen, I was thinking of spinning up a package registry in a hope of at least making the resolution faster, but that will only be possible if Xcode 14 supports the registry, or at least using packages that use the registry.

I'm starting this thread to see if there are other people experiencing similar problems in hope that somebody has found potential remedies to the problems.

6 Likes
2

Your remedy is to use a monorepo so Xcode doesn't have to load the packages remotely. You can see an example of hyper modularization in a monorepo in isowords.

Xcode 14 seems slightly better than 13 in handling packages but still isn't reliable. But a monrepo will help a lot.

6 Likes
3

I swear I've lost months of coding time to this. Something that is working for me is to maintain two different project directories for each heavily-modularized project. (Do this for the end-node executable packages, not the library packages.)

One directory contains my working project, where Package.swift has all of the child dependencies being developed alongside as a local dependency with .package(path:) The other directory has all child dependencies pointing to their remote homes (on GitHub), and is used only to reconcile updates in Package.resolved when a dependency is updated. Both are pointing to the same repository and can check in, but the working project always has lines in its Package.swift that can't be checked in.

I don't work on a team so I don't know if there are thorns to this approach if you adopt this, but it beats the roulette game by a mile.

1 Like
4

Thanks @Jon_Shier.

Moving all packages to a single repo is not really an option in my case, as each of the packages has to be versioned and maintained separately.

It does give me an idea of an option, though, that might be worth considering: some kind of a tool that could flatten the whole package graph into a single package, use the flattened package from Xcode, so that we don't have to rely on the package resolution going through Xcode. The tool could leverage the Swift CLI or the SPM library itself. The dependency management would then be controlled manually through CLI, removing Xcode from the equation.

Seems like a lot of work though for something of a questionable ergonomics.

5

It does give me an idea of an option, though, that might be worth considering: some kind of a tool that could flatten the whole package graph into a single package, use the flattened package from Xcode, so that we don't have to rely on the package resolution going through Xcode.

FYI at first glance this seems a lot like what Tuist tries to achieve with it's packaging scheme. Except it's flattened into a single Xcode project rather than a package.

6

You could check out all your libraries locally alongside your main project.
Then create an Xcode Workspace besides all the checked out directories.
Drag your main project file into this workspace and all the repos for your packages also parallel to your project into the workspace.

Now when you open your workspace, you will have your project use all the local checkouts instead of the remote dependencies.

To update all your checkouts, you could use a shell command like

for dir in ./*; do (cd "$dir" && git pull) &; done; wait

It's just a suggestion, but then at least you don't have to resolve any remote dependencies ever when using the project.

2 Likes
7

This is quite clearly an XCode bug.

It seems to be tested only on public repositories, and has numerous issues as soon as it has to deal with private repos which usually require SSH access (aka the standard case for people developing software for their clients).

The Readme is never displayed in this case, updates are mostly random and as you wrote: The whole thing costs a LOT of time. This is especially infuriating as the built in git client updates the very same repos without any issues.

I wouldn't have less of an issue if it takes 5 minutes to update for a complete reload, if it would do that reliably. Still, simply updating a single package should not need longer than a few seconds if there are no changes to the dependencies.