As discussed with @ktoso, yes, they do. For example Rust, NPM, Haskell.
I am arguing that this is the best option proposed so far, and also good enough. If we all came to the conclusion that this is indeed the case and we proceeded with opaque ID, then there's a separate discussion to have: How are we assigning the opaque IDs. Can anybody register any string? Do they need to start with a reverse DNS that you need to prove you have when registering? Something else? But that's entirely a second step in my mind. The first discussion to have is whether the location of the code and the package ID are linked. In my mind they should not be linked.
AFAIU, the code for all the dependencies can get downloaded from the registry (so we don't need to git clone
anymore). And you can mirror that registry into your corp network.
No, at build time, SwiftPM knows (without internet) that opaque ID 1 and opaque ID 2 are the same package when their byte representation is equal. The registry would just make sure that there is of course only one package registered with a given name (sure, two registries could in theory have the same package ID). Again, must like Rust's crates.io (package name is tokio
and Crates uses https://crates.io/crates/tokio
if you want more information about it), Haskell's stackage (package name for example is bytestring
and Hackage uses https://hackage.haskell.org/package/bytestring
if you want more info).
You would have your internal mirrors and internal registry that always gives you all the code from inside of your corp network.
Isn't the main claimed advantage of the URL the "trust model" it gives you (I don't buy there's actually any more trust but we discussed that before). I'll quote @mattt :
And the URI-based proposal is not supposed to make "moving packages around" easy:
What you @Jon_Shier seem to be pitching is essentially opaque IDs where the opaque ID happens to be in the form of a URL. With one addition: That registries can also hold aliases. Is that correct?
Personally, at this stage I don't think it's helpful to discuss the shape of the opaque package ID (I think we should first decide that we want opaque package IDs). What I mean with that is that I personally don't mind if the SwiftNIO is called:
swift-nio
, as in Rust (Crates), Haskell (Hackage/Stackage), JavaScript NPM, ...com.apple.swift-nio
, as I think in the JVM ecosystem (with initial proof that you ownapple.com
)apple/swift-nio
like Swift Package Index seems to useafb757d66ed2e63824e8aff4984d6f97c43d85ea
(just a hash)https://github.com/apple/swift-nio
What I do care about is that the package name is comparable by a simple string comparison. If we agreed on that, then I'd be very happy. Of course, I also don't think that URIs make good opaque IDs because location may change (and the ID now can't) and people may think it's a URI and not an opaque ID (so they may assume that https://github.com/apple/swift-nio
== https://github.com/apple/swift-nio/
or so).
That is, if Registry1 and Registry2 both claim to have a package from
https://github.com/apple/swift-nio
, we know that must be the same package since it's not possible for two people to claimgithub.com
and have code at that path.
Isn't this a counter example? Assume your package graph contains both https://gitHub.com/apple/swift-nio
and https://gitLab.com/apple/swift-nio
. I think we agree that SwiftPM needs to de-dupe and find the "correct" SwiftNIO. However,
- Registry 1 claims: I have a package, canonically hosted at
https://gitHub.com/apple/swift-nio
with an old aliashttps://gitLab.com/apple/swift-nio
. - Registry 2 claims: I have a package, canonically hosted at
https://gitLab.com/apple/swift-nio
with an old aliashttps://gitHub.com/apple/swift-nio
. - Registry 3 claims: I have a package, canonically hosted at
https://gitHub.com/apple/swift-nio
, no aliases - Registry 4 claims: I have a package, canonically hosted at
https://gitLab.com/apple/swift-nio
So two registries agree that SwiftNIO has 2 package IDs but they disagree which one is the canonical one. The other two registries see SwiftNIO at only one place. Similar situations can even happen with non-malicious registries if for example one registry can't reach (or doesn't properly implement) the "forwarding". It's a distributed system of mutually distrusting parties, we can't easily reach one definitely correct view of the world (there's a blockchain joke somewhere in here).
What I'm saying is that the trust model is more or less "I trust the registry" (if I have multiple, I probably need an order how much I trust them) in both the URI-based and the opaque ID system. The opaque ID system is however much simpler and solves all the common cases.
So far, URLs seem to be the only option which give the community any way to do that.
I don't think they do. The one thing that URLs can give you is that when you first register, you prove that your code is hosted at a certain URL. Opaque IDs can do a very similar thing if we (at a later stage) decide that to register an opaque ID, you need to prove ownership of a certain domain/website/... . That's I believe what they do in the JVM ecosystem, I think you can only register com.corp.package
if you provably own package.corp.com
at the time of registration of the package. Conveniently, Github would then allow everybody (through Github pages) to claim io.github.user-name
without having to pay money for a domain etc. The crucial thing is that there is only ever one ID that is immutable and doesn't change.
Our proposal associates SHA256 checksums to versioned source archives in
Package.resolved
. If you do a fresh build of your project (such as in a CI) and a host tries to send you something else, Swift Package Manager would raise an error.
That's great. But it's not related to URI vs. opaque IDs though, right? And once you swift pacakge update
your packages, there's again no more trust than what "opaque IDs+trusted registries" gives you too.
As I said in a previous reply, tooling — whether that's a new subcommand or making the output of
build
andresolve
more verbose — could go a long way to helping us understand what's being downloaded from where.
The same would apply if we use opaque IDs, right? In both, the URI and the opaque ID case, we usually download the code from the registry. In both cases can also get information about the source host and download it manually directly from there.
We've pitched an enhancement that allows users to configure an intermediate registry proxy. Whereas a mirror lets you redirect a single dependency, an intermediate proxy would route all packages directly.
Right, which adds extra complexity and I fail to see the benefit. We also need to trust these intermediate registry proxies.