SwiftPM manual cache management?

Development Package Manager
1

i have a CI/CD system that consumes a lot of network bandwidth downloading git repositories from GitHub. this happens because the build system clears the SwiftPM cache (.swiftpm/cache/repositories) after every build due to disk space limitations.

i’m interested in setting up some sort of manual SwiftPM cache management system to reduce the amount of data we need to download over the internet. the idea in my head right now is to archive the cloned dependencies after every build and persist them in some less-expensive proximate storage (e.g. S3 in the same region). then on the next build, the system would use its internal dependency metadata to repopulate the SwiftPM cache with the unarchived repositories to improve the hit ratio.

has anyone tried this before? if so, how did it go, and what worked/didn’t work for you?

2

You can use SwiftPM mirrors with a prepopulated cache stored in there. If any of the packages contain submodules that are important to you, you'll have to create your own git config with insteadOf values for each submodule URL.

1 Like
3

are you referring to SE-0219? i’m not sure if i understand it correctly, but i thought that it requires setting up a git server, which sounds like a much bigger task than backing up some .git folders in an S3 bucket.

4

No, you don't need a git server, it accepts any valid URL. Here's how a record in mirrors.json could look like for a local file system mirror:

{
  "original" : "https://github.com/apple/swift-docc-plugin",
  "mirror" : "file:///tmp/MIRRORS/swift-docc-plugin-a84d795b/swift-docc-plugin"
},
2 Likes
5

ah, that’s great to know, thanks!