URLs as Swift Package Identifiers

[mattt]

@xwu

This is a big caveat, and I'm actually somewhat concerned about this. I cannot find the proper references to cite in a pinch, but URI handling is notoriously difficult, and it differs among even major browsers.

This was discussed at some length by @Karl in the review thread:

@Karl

There's a lot to say about the deficiencies of URLs, but luckily I can recommend a couple of videos instead (one short, one long).

1. HOW FRCKN' HARD IS IT TO UNDERSTAND A URL?! - uXSS CVE-2018-6128 (15 mins). Explains a universal cross-site scripting bug that affected WebKit on iOS. I haven't looked in to the bug in detail, but either there were different URL parsers in the OS which saw different hosts for the same URL, or there was an idempotency bug (parsing -> serialising -> parsing the URL changed its meaning).
2. A new era of SSRF (47 mins). A now famous talk by Orange Tsai about how to exploit quirks in different URL parsers. It includes this slide, demonstrating how each of the common parsers used by Python sees a different host from the same URL. It's so beautiful I'm thinking about getting a poster made of it:

@xwu

This all isn't merely for the sake of pursuing perfection for its own sake. Rather, I would expect any discrepancy or ambiguity in mapping URIs unambiguously to a code package could be a potential security exploit allowing for unintended code to be downloaded, compiled, and/or executed.

I don't want to diminish the potential for ambiguity or complexity in parsing URIs. However, I do think it's important to focus on how package identifiers would be used.

The list of URI transformations in the proposal are specific to Swift Package Manager, and are a way to reduce the incidence of duplicate nodes in the dependency graph. When the user does swift build --enable-package-registry, SPM takes the canonicalized URIs, prepends the https:// scheme and sends a HEAD request to the resulting URL to see if they support Swift package registry. If the server responds accordingly, SPM attempts to resolve that dependency through the registry interface; otherwise, it falls back to Git (unless the user opts out by setting --disable-repository-fallback).

Any dependency URLs declared in Package.swift were put there by the developers authoring that package. Any transitive dependencies were put there by the maintainers of those direct dependencies. Would you say that the potential for security exploits due to URI parsing ambiguity is distinct from those inherent from importing 3rd-party code?