SE-0292: Package Registry Service

[mattt]

Thanks for taking the time to review and share your thoughts, @Karl. Responding to your individual points:

@Karl

1. Ban credentials in URLs

[...]

Given all of this, I would propose that we outright ban credentials in registry URLs, and I'd actually go one step further and ban them from all package URLs.

I agree that this is a topic for further consideration, but I don't think that this is particularly relevant to this proposal. Both package registries and conventional, repository-based dependencies will share a common authentication mechanism through Swift Package Manager. The only reason I brought up hardcoded credentials in my response to @0xTim was to describe an existing strategy for accessing non-public external packages.

For context, Swift Package Manager only recently added support for .netrc files [1]. Any decision to ban credentials in URLs should take a considered approach, where deprecation warnings and documentation give SPM users an opportunity to migrate to a better solution.

@Karl

2. Underspecified components

The proposal mentions certain conditions for registry URLs, but not all components are specified. Can registry URLs contain query strings?

Packages are uniquely identified by a canonicalized form of a URL that locates their source code. The exact behavior of this is described by the CanonicalPackageIdentity, which was introduced in this PR to apple/swift-package-manager on November 18, 2020.

@Karl

3. My suggestion

[...]

So out of everything URLs can do, and all the quirks they have to support, we basically only need 2 things: the hostname of the registry, and a package name (and a port if we want to support nonstandard ports).

That's not entirely correct. As I said before, and describe in the proposal, packages are identified by URLs. We use URLs as identifiers not just because a URLs components map to the parameters we need, but because they locate a server resource. As I discussed in my response to @StanTwinB:

@mattt

Our approach uses URLs, which uniquely identifies packages and confers a host of additional benefits, including global addressability (you can copy-paste a URL into your browser), security (you can authenticate via existing TLS certificate infrastructure), and naming authority (we can delegate any ownership claims to ICANN and domain registrars).

From the beginning, Swift Package Manager has taken a federated approach to identity. I think that's a good thing. In contrast to other systems, there's no centralized naming authority for packages. You aren't requesting a package named mona/LinkedList that happens to be hosted on GitHub, you're requesting the package github.com/mona/LinkedList.

By using canonicalized URLs to identify packages, we're saying that https://github.com/mona/LinkedList and ssh://git@github.com:mona/LinkedList.git are the same, and we use HTTP content negotiation to upgrade requests to use a faster, more secure registry interface, when available.

Getting back to the concerns you raised about edge cases in parsing URLs: I'm having trouble imagining a scenario where this ambiguity could be exploited. If an invalid or ambiguous URL is provided, it will either fail to resolve in an HTTP request or it will cause dependency resolution to fail. Do you have a specific concern about how these URLs could cause problems for the package ecosystem?