SE-0292 (2nd review): Package Registry Service

Evolution Proposal Reviews
42

Sorry if I missed this or didn’t grasp how it could be done, but with this proposal, is it possible to have more than one registry in use in a project at one time? I understand that the existing url resolution could be used alongside a designated registry to achieve something similar, but is there a way to resolve packages from multiple registries?

43

Yes. In addition to a default (i.e. unscoped) registry, a user may configure one or more registries, each for a particular scope. A full example of this behavior is described under the heading "Global registry configuration" in the proposal.

3 Likes
44

Associating a registry with a scope

The user can associate a package scope with a custom registry by passing the --scope option.

For example, a user might resolve all packages with the package scope example (such as example.PriorityQueue ) to a private registry.

$ swift package-registry set https://internal.example.com/ --scope example
$ cat .swiftpm/config/registries.json

{
  "registries": { 
    "example": {
      "url": "https://internal.example.com"
    }
  },
  "version": 1
}

In addition to associating a scope with a specific registry (as indicated in the proposal), would it make sense to allow for "re-scoping" a package in the event of scope clashes across registries (which I imagine are rare, but could happen, especially for internal registries)? For example the following...

{
  "registries": { 
    "rescoped": {
      "url": "https://internal.example.com",
      "scope": "example"
    }
  },
  "version": 1
}

would "re-scope" the example scope in an internal registry, giving the ability to refer to its packages now with rescoped.* and different packages from the default registry with example.*. This simple remapping would make it possible to benefit from packages that are distributed under the same scope on different registries.

Thoughts?

45

Maybe. I'd want to see a real-world example of the problem you're describing before jumping to any particular solution.

It seems to me that, in a situation in which you'd want to override the resolution of packages across multiple scopes, you could accomplish this different way — either by:

  1. Setting the same custom registry for the affected scopes, or
  2. Setting that custom registry as the default and setting custom registries for all everything else
46

First of all, +1 on the proposal overall. I look forward to the day where adding dependencies is as simple as swift package add-dependency scope.package-name...

With regards to the "issue" I mentioned above, while I don't have a "real" real-world example, my brain can't help but imagine that someone would be stuck with this problem. Admittedly, the examples in my head are rather contrived, but I've added one below to give a look into my thoughts...

Contrived Example

If I am understanding this correctly (from the proposal):

A custom registry may be associated with one or more scopes, but a scope may be associated with only a single registry at a time. Scoped custom registries override any unscoped custom registry.

This means that for any given scope that exists on two different registries, someone would have to resort to using URL based resolution to fetch packages from one of registries. For example, assume two (different) packages exist on separate registries:

  • the Vapor web framework (vapor/vapor hosted on github)
  • A helper library used internally at a given organization (vended on an internal registry from vapor/helpers)

These two packages use the vapor scope and therefore using a single registry for this scope wouldn't solve the problem. You would have to resort to using URL based resolution for one of the packages and would not be able to use a scoped package identifier (scope.package-name) for both of them.

Again, the example is contrived and is only really an issue in the event that (1) there was a project requirement to use scoped package identifiers and (2) the scope in the local/internal registry could not be renamed.

If anything let the example serve as an exercise to figure out if this has potential to be a problem the proposal could solve.

47

In this hypothetical, the identifier shouldn't be vapor.helpers (unless your organization is, in fact, the Vapor open source project). Instead, it'd be something like example.VaporHelpers. If you insisted on using the vapor scope internally, you'd either have to host the vapor package in your custom registry or declare the vapor dependency by URL.

1 Like
48

Overall I'm pretty happy with how this effort is shaping up. I would like to suggest though, to leave out the global registry configuration.

My reasoning for this is threefold: Firstly, I share the reservations others have expressed that it would become an expectation to have that global registry configured to github, or packages might fail to build.

Secondly, it's an additive change. If we leave out the global configuration now, and find such a mechanism is needed in real-world usage, we can introduce it in the future without breaking anything.

Finally, it violates an ideal that has been expressed as a goal for spm multiple times: As far as feasible, a package should describe the information needed to build it in its manifest/other package files, with no external configuration or setup. I think asking package maintainers to set up a default registry per package is not that big of an ask.

For these reasons, I think leaving out the global registry concept for now would be a good move.

Anecdotally, CocoaPods also started support for custom sources with the pod repo-add (if memory serves) command, but switched to declaring sources in the Podfile later.

And, as a last footnote, and then I really must be going: If global configuration is included after all, it really should not go into ~/.swiftpm, but in the platform-appropriate directories ($XDG_CONFIG_DIR on linux, probably somewhere in ~/Library on mac, and I'm not sure about win :D).

4 Likes
49

I agree that the proposal doesn't preclude multiple registries but it does so in way that would likely result in multiple probably-mostly-compatible package eco-systems around each public registry rather than one package eco-system with multiple co-operating public registries.

I don't think its enough to say multiple registers == a de-centralized ecosystem.

A number of people previously have highlighted the desire to retain the de-centralized approach that the git-based model gave us. As far as I can see, the current approach doesn't actually give us the benefits of such a de-centralized model while still giving up some benefits of a central public repository (simpler eco-system/ease of configuration/bootstrapping).

50

This second review for SE-0292 has concluded and the proposal was returned for revision to refine some of the configuration and API details.

Thank you to everyone for the feedback and contributions to this proposal.

1 Like