SE-0292: Package Registry Service

[mattt]

@0xTim

Are there any implementation details for how SwiftPM's dependency resolution is going to change? My concern is that whilst this sounds great for pinned versions or small projects, very large projects may suffer. For instance, currently to resolve the dependencies, SwiftPM downloads the single Git repository (which admittedly can be large) and then can quickly change between tags to check transitive dependencies. With the new proposed implementation, it seems that either a new manifest needs to be downloaded for each version of every target (which is a lot of network connections) or the entire package needs to be downloaded for every version.

You can see the latest working implementation in this PR to apple/swift-package-manager.

You're correct in pointing out that the fixed cost of downloading the entire history of a project by cloning its source repository may, under some circumstances, may be less than the sum of downloading multiple releases through a registry. However, in practice, I believe a registry will be as fast if not faster than a repository for a few reasons:

• We can see all of the available releases for a package from a single endpoint
• We define a separate endpoint for downloading Package manifests at individual versions, without downloading the full archive
• In general, globally-distributed CDNs are faster at serving Zip files over HTTP than repository hosts are at serving code via Git (even if that's Git over HTTP)

Ultimately, these are empirical claims that will have to be verified by real-world usage. I have a benchmark harness that you can use to try it out for yourself. At the moment, the results are promising:

$ bundle exec rake clobber benchmark
time ./spm run
       57.70 real        87.12 user        10.88 sys
time ./spm run --enable-package-registry
       16.24 real        35.85 user         4.75 sys

@0xTim

I also haven't seen anything that addresses private dependencies. Currently you can integrate private packages using git@github.com:0xTim/MyPrivatePacakge.git - this will continue to work but will it mean that the entire dependency resolution will revert to Git based dependency resolution or just for that package?

If a repository-based external dependency itself has external dependencies that can be resolved through a registry, then SPM will attempt to do so.

@0xTim

And has there been any thoughts into how private dependencies can be managed with the new service?

This came up in a previous discussion thread, but you're right — this isn't mentioned in the proposal or specification.

@mattt

Information disclosure

An attacker could scrape public code repositories for Package.swift files that use hardcoded credentials in dependency URLs, and attempt to reuse those credentials to impersonate the user.

dependencies: [
  .archive(name: "TopSecret",
           url: "https://<token>:x-oauth-basic@github.com/Mona/TopSecret/archive/1.0.0.zip",
           checksum: "2c4a4ce92225fb766447c1757abb916e13f68eba0459f1287ee62e4941d89bbf")
]

This kind of attack can be mitigated on an individual basis by using an unauthenticated URL and setting a mirror.

$ swift package config set-mirror \
    --original-url https://github.com/Mona/TopSecret/archive/1.0.0.zip \
    --mirror-url https://<token>:x-oauth-basic@github.com/Mona/TopSecret/archive/1.0.0.zip

The risk could be mitigated for all users if Swift Package Manager forbids the use of hardcoded credentials in Package.swift files.

Alternatively, you can provide credentials through a .netrc file.

@0xTim

Finally, how does this fit in with the new system wide cache of dependencies for SwiftPM - I would guess that by default SwiftPM will default to the cache, but there's no mention of it in the proposal.

These features have been developing in parallel, and the current draft implementation of registry support doesn't support the system-wide cache. That said, I think it'd make a lot of sense to add support eventually.