What exactly is the real name of this package?

taylorswift · May 7, 2023, 10:24pm

once upon a time, we had a pretty solid system for naming packages:

package names must be valid URL characters (no spaces, punctuation, etc.)
package names must be lowercased, and use hyphen-case (“kebab case”)
package names must start with “swift-”

of course, this was always something of a fiction. pretty much nobody outside of Apple ever really observed rule #3, and the only enforcement mechanism we had for rules #1 and #2 was the fact that packages once used their repository URLs as a notion of identity, and those two things were (rightly) decoupled a few toolchain releases ago.

and yet, by giving up on “URL path components as package identities”, we must acknowledge we have given up on a few important properties:

we lost the notion of a “machine-readable” package name, e.g. a string with format restrictions such that we could use the package identity for things like generating directory names or assigning URLs.
we lost a visual indicator for quickly determining if a name refers to a package or a module. (e.g. “swift-syntax” is the package, while “SwiftSyntax” is the module.)

i did a quick survey of a handful of “official” Apple packages, and it seems even among them, chaos now reigns supreme:

repository	package identity	package identity (mangled)
GitHub - apple/swift-package-manager: The Package Manager for the Swift Programming Language	`SwiftPM`	`swiftpm`
GitHub - apple/swift-system: Low-level system calls and types for Swift	`swift-system`	`swift-system`
GitHub - apple/swift-syntax: A set of Swift libraries for parsing, inspecting, generating, and transforming Swift source code.	`SwiftSyntax`	`swiftsyntax`
GitHub - apple/swift-nio: Event-driven network application framework for high performance protocol servers & clients, non-blocking.	`swift-nio`	`swift-nio`
GitHub - apple/swift-markdown: A Swift package for parsing, building, editing, and analyzing Markdown documents.	`swift-markdown`	`swift-markdown`
GitHub - apple/swift-argument-parser: Straightforward, type-safe argument parsing for Swift	`swift-argument-parser`	`swift-argument-parser`

as we can see, “mangling” the package identities into something suitable as a machine identifier doesn’t give consistent results, and moreover it’s not clear to me how to mangle package identifiers that contain spaces or other special characters.

what are the names of these packages?

xwu · May 8, 2023, 4:07am

Mmm...where are you getting this rule from? My understanding was that this convention is intended only for packages that fall under the umbrella of the Swift project itself.

taylorswift · May 8, 2023, 5:42am

it is a common naming convention not at all exclusive to Apple-backed projects, and i mentioned it only to serve as a contrast with the more irregular strings found in the package “name” field, which is apparently different from the package “identity” SPM uses elsewhere.

SDGGiesbrecht · May 8, 2023, 8:13am

The name is the thing listed in the manifest under the name: parameter; the identity is something completely different.

The naming inconsistencies across the ecosystem arose because changes over time in the SwiftPM team lead to changes in its goals and philosophies.

Originally the name’s only purpose was as a user facing label. Style recommendations from that era corresponded to a standard English book title or a UI label, i.e. “My Package”. Hence “SwiftPM”. However, packages revolving around a single module or executable often lacked any name distinct from that of the target, which then inherited the target’s naming conventions, such as “SwiftSyntax” from import SwiftSyntax.

The name’s first machine use was as an optional disambiguator between two identically named targets from different packages, where it went by Swift’s standard string == semantics. This saw little actual use, because clashing module names impeded builds anyway in most cases.

With the proposal of target‐based resolution, the name was to became the link between the target dependency and the package dependency so the resolver could know whether it needed to fetch the dependency in the first place. The manifest API changes were merged, erroneously connecting things by last path component instead of the name under the hood. The resolver improvements were never merged, but over the course of several versions the API was fixed to mach the proposals, then broken again.

During that era of chaos, SwiftPM was able to both infer more of the necessary connections, and support more toolchain versions at once (without #if) when the name matched the last path component. Wanting to save users from typing as much, many packages that sprouted in that time period adopted repository names as package names, such as “swift-argument-parser” (while its read‐me simultaneously calls it “Swift Argument Parser”, which is what the name field was intended to convey).

The swift- prefix in some of the repository names you cite was supposed to mark them as belonging to the Swift project, not as being written in Swift. No one else was supposed to use it. But here too the intent was lost on users in the wild and now it is meaningless.

Later on, changes in the development team lead to a desire to eliminate the name from all its machine uses and attach everything to the last path component (whereas some of the original team had considered URLs ugly, internationalization unfriendly, and something to be contained and not spread throughout the source). Since the change, the display name has even ended up marked with deprecation attributes in places I don’t believe it should be.

Finally, with the introduction of registries, the identity was invented. The last path component that was by then pervasive in the implementation did not exist for registries, so they needed something else. What you read in the registry proposals describes the constraints and conventions of registry identities. They do not apply to URL or path dependencies, which continue to support any URL or path, no matter how gnarly, and likely always will. (Removing support would be severely breaking to extant packages.) A registry identity is the identifier by which a user interacts with a registry dependency (and is thus designed to be machine friendly). But a general identity is the union of registry identities, URLs, etc. that SwiftPM internally uses to track whether two things are the same package or not. It was theoretically never supposed to be surfaced to users.

But then Xcode failed to respect SwiftPM’s API boundaries and integrated it in ways that spilled implementation details out its user interface. Users have been able to see several variations of the supposedly internal identity across a handful of Xcode releases.

So you are right that in the end it is a tangled mess. The best advice I can give is...

If you are referring to the package in something intended for a user, such as in a read‐me heading, UI list or error message prose, use the string verbatim from the name field.
If you are describing to users how to depend on a package, or providing code to copy‐and‐paste, use the URL or registry identity as defined externally by the hosting platform.
If you are implementing something mechanical that needs to tell packages apart internally, or namespace their contents into some index (such as for a DocC feature) then I guess make use of SwiftPM’s general identity, knowing you have to sanitize it yourself and can not expect it to remain stable as SwiftPM evolves.

That is the state as far as I know, but my participation has waned late and I may have missed still further changes if they were recent enough.

taylorswift · May 8, 2023, 11:04pm

thanks Jeremy! this is pretty close to what i’m using package names/identities for:

but what exactly is a “general package identity”?

if i go to, say swift-syntax - Swift Package Registry , it says that that package’s identity is simply “package”, probably because the site’s backend always clones packages into a directory named package.

{
  "url": "/usr/src/package",
  "name": "SwiftSyntax",
  "path": "/usr/src/package",
  "version": "unspecified",
  "identity": "package",
  "dependencies": [
    {
      "url": "https://github.com/apple/swift-argument-parser.git",
      "name": "swift-argument-parser",
      "path": "/usr/src/package/.build/checkouts/swift-argument-parser",
      "version": "1.1.4",
      "identity": "swift-argument-parser",
      "dependencies": []
    }
  ]
}

SDGGiesbrecht · May 8, 2023, 11:53pm

Yes. SwiftPM’s general identity is solely concerned with relationships in the present resolution graph, not the external ecosystem. The root package plays by different rules in all sorts of ways, and in regards to this discussion, it is a lot like a local path‐based package. If you want to know the general identity that will be assigned when used as a dependency in a particular way, you have to actually depend on it that way from something else and then dump what SwiftPM decided in that situation (where it will be derived in some way from the URL, registry identity, local path, etc. by which it was fetched). Maybe SwiftPM happens to expose enough API to determine what that it would be from a hypothetical dependency declaration without doing any actual resolution work, but such a service has never been on the developers’ minds.

If you need identifiers that are stable regardless of the dependency mechanism or that are unique across the ecosystem, I do not think such a thing exists at present. Registry identities have both traits within the repository niche, but not in their interactions with the outside world of clones, local paths, remote tags and so on. If you limit yourself to a registry query, what you get is reliable in the ways you seek. But by the action of cloning to inspect the package in a development state, you end up in the wider realm full of placeholders, overrides and overlaps. (These quirks generally exist to facilitate the development itself and are vital to it. No one wants to have to publish a release first in order to run unit tests!)

finestructure · May 9, 2023, 10:19am

Over in the Swift Package Index we take the package name from the Package.swift manifest. This results in SwiftSyntax for https://github.com/apple/swift-syntax:

I wish packages would drop the swift- prefixes, at least from their package names if not their URLs. I always thought it's to namespace them on Github where all manner of languages live and you want to provide guidance at a glance what a package is about.

However, it ends up spilling into all sorts of places, most prominently the package dependencies sidebar in Xcode and VSCode:

I really hope that package registries give us another shot at having nice package names, decoupled from any URL or identifier.

jrose · May 9, 2023, 4:31pm

For comparison, https://rust-lang.github.io/api-guidelines/naming.html:

Crate names should not use -rs or -rust as a suffix or prefix. Every crate is Rust! It serves no purpose to remind users of this constantly.

Max_Desiatov · May 9, 2023, 7:14pm

FWIW, Swift Async Algorithms package name was made consistent recently, while I've just created corresponding PRs for SwiftSyntax and SwiftPM.

SDGGiesbrecht · May 9, 2023, 7:28pm

It should be fine for those three, but be careful changing names when the tools version is older. If the package purports to support circa 5.5* or older, then the name is still used in ways that are a breaking change for clients.

*a rough estimate from memory without having looking it up

taylorswift · May 9, 2023, 9:05pm

in light of the recent discussion at [Second review] SE-0386: `package` access modifier i wonder if the proliferation of swift- prefixes is because of a particular handicap of SPM: right now we cannot have more than one package per git repository.

this has caused us to treat repositories and packages as one and the same, and so SPM uses the repository name as the package identity. and repositories must have swift- prefixes (since they coexist in the same namespace as projects written in other languages), which implies that package identities must too have swift- prefixes.

i hope if SPM gets support for multi-package repositories, that we can dispense with the swift- prefixes once and for all.

as an aside: i don’t think there is any value in a package name: field in the manifest that deviates from the identity SPM uses everywhere else - it is just confusing. perhaps it should be renamed to displayName:, or removed entirely.