URLs as Swift Package Identifiers

Key part of the core team's SE-0292 review feedback was to explore the question of identity - and more specifically, resolve the question of identities that carry location information such as URLs, compared to identities that are opaque with regards to location, such as simple strings or reverse-DNS.

The reason our choice of identity is so important is that SwiftPM today is not able to reliably deduplicate packages, and this issue is becoming urgent as the packages ecosystem grows. SR-11338 is a real-world example of such issue,
where a repository that was moved ended up in the dependencies graph twice (under different URLs), causing SwiftPM to fail.

As the ecosystem grows, and given that moving and renaming repositories is fairly common, this kind of issues will also become common. As such, the solution we choose must embrace and design for this reality.

The package identifier scheme we choose will also become the foundation for resolving module name conflict across packages, which is another problem becoming urgent as the packages ecosystem grows. The canonical example for this issue is two separate packages that both vend a "Utilities" module.

Today, Swift cannot deal with such module name duplication and the user must choose between the packages. If we add compiler support, we could support having two modules named “Utilities” from different packages by prefixing the module name with the package identifier.

The core team has expressed a desire to solve this long lasting issue by adding such compiler support, making it possible for SwiftPM to prefix the modules it generates. Since the module names need to be unique, the package identifier is a good candidate for such prefix, if we can make sure it is unique.

Here too we must embrace the reality that moving and renaming repositories is fairly common, and choose an identity scheme that would make for good module names over extended period of time.

Some package systems have no need to deduplicate packages because they only support languages in which code duplication is not a problem. However, that is not true of SwiftPM. SwiftPM cannot change the ABI of C, C++, or Objective-C to allow multiple copies of a package containing such code to be loaded into a program at once. Even in pure Swift packages, where such ABI changes are possible, package duplication can cause spurious build or runtime failures in programs that rely on the uniqueness of shared types and state. Therefore, reliable deduplication of packages is a basic requirement for SwiftPM.

As such, the core team asked in its SE-0292 review feedback that "This topic needs to be further explored in a dedicated forum thread in preparation to the next revision.". This is that thread.

This thread gathered a lot of attention, and has provided insight into the practical advantages and disadvantages of the two approaches. At this point, it seems like the technical arguments have all been laid out. While there are still different opinions, there is little new information coming from the ongoing discussion.

Since we have seen that both solutions could be made to work, and both carry a set of unique challenges, the remaining question is which one makes better tradeoffs?

The implication of location-based identifiers is that we would be asking SwiftPM users to manage URL mapping files, or set up proxies and other infrastructure to resolve dependencies deduplication issues. In our opinion, this is not practical at a significant scale and time horizon, and will cause continuous pain down the road.

In other words, location-based identifiers land the complexity on the end user, while opaque identifiers land the complexity on the package registries. From where we stand, putting the complexity on the registries would be setting the stage better for a future where we can reliably deduplicate packages and solve the module name clashes in the vast majority of cases.

SE-0292 is critical to the success of the Swift ecosystem. The core team has discussed this topic on our last couple of meetings, and concluded that opaque identifiers are a better fit to the Swift ecosystem for these reasons. While this thread is not a formal review, the core team wanted to share its position that SE-0292 needs to be amended to adopt opaque identifiers before it can be sent back to a second review.

11 Likes