# Swift Package Registry Service **URL:** \ **Category:** Pitches\ **Tags:** packagemanager\ **Created:** [June 4, 2020, 6:20pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219 "2020-06-04T18:20:33Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![mattt](https://avatars.discourse-cdn.com/v4/letter/m/46a35a/32.png) [@mattt](https://forums.swift.org/u/mattt)\ **Post date:** [June 4, 2020, 6:20pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/1 "2020-06-04T18:20:33Z")
**Hello!** Last June, Apple and GitHub announced that the [GitHub Package Registry will support Swift packages](https://github.blog/2019-06-03-github-package-registry-will-support-swift-packages/). A few months ago, [Bryan Clark](https://github.com/clarkbw) started a [thread](https://forums.swift.org/t/github-swift-package-management-service/30406) to gather ideas about a standard package registry API that could be implemented by anyone, not only GitHub. Over the past few weeks, I’ve had the privilege to work with Bryan, as well as [Whitney Imura](https://github.com/whitneyimura) and other great folks at GitHub, to define a draft specification for a Swift package registry service. I’m thrilled to share it with you all today. Before diving into the proposal itself, I wanted to make two points at the outset, to help frame our discussion: - This proposal defines an open standard for Swift package registry services. GitHub will implement these APIs as part of its [GitHub Packages](https://github.com/features/packages) offering, and anyone else will be able to create their own registry as well by implementing these endpoints. - This proposal defines the package registry API _only_. Swift Package Manager will need to add support for package registries as a client of this specification. We intend to determine that integration in a separate proposal. * * * ## Introduction Swift Package Manager downloads dependencies using Git. Our proposal defines a standard web service interface that it can use to also download dependencies from package registries. ## Motivation Today, a package dependency is specified by a URL for a Git repository. When a project is built for the first time, Swift Package Manager clones the repository for each dependency and attempts to resolve the version requirements with the available tags. Although Git is a capable version-control system, it's not well-suited to this kind of workflow for the following reasons: - **Speed** : Cloning a Git repository is significantly slower than downloading and expanding a Zip archive with the same content. - **Efficiency** : Cloning a Git repository downloads the entire history of a project, when only a particular version is used. - **Security** : Git tags can be reassigned to another commit at any time. - **Durability** : Git repositories can be moved or deleted. ## Proposed solution Define a standard interface for package registry services that Swift Package Manager can access to download dependencies as an alternative to direct Git access. Many language ecosystems have a package registry, including [RubyGems](https://rubygems.org "RubyGems: The Ruby community’s gem hosting service") for Ruby, [PyPI](https://pypi.org "PyPI: The Python Package Index") for Python, [npm](https://www.npmjs.com "The npm Registry") for JavaScript, and [crates.io](https://crates.io "crates.io: The Rust community’s crate registry") for Rust. In fact, many Swift developers develop apps today using [CocoaPods](https://cocoapods.org "A dependency manager for Swift and Objective-C Cocoa projects") and its index of libraries. Each of these systems follows what we describe as a "push" model. When a package owner releases a new version of their software, a client runs a command locally and pushes the results to a server. However, the "push" model reflects a tradition of software deployment that predates modern source code management and build automation. As package maintainers can attest, this approach often involves a lot of manual effort and trial-and-error guesswork. It also lacks strong guarantees about reproducibility and software traceability. We describe our approach as a "pull" model, which takes inspiration from current best-practices like continuous integration (CI) and continuous delivery (CD). When a package owner releases a new version of their software, their sole responsibility is to notify the package registry. The server does all the work of downloading the source code and packaging it up for use. A "pull" model for package registries not only means less work for maintainers but better outcomes for consumers. > **Note** : This proposal covers only the package registry interface. Adding support for package registries in Swift Package Manager will be discussed in a separate proposal. * * * _(Continued in next post)_ ---
**Author:** ![mattt](https://avatars.discourse-cdn.com/v4/letter/m/46a35a/32.png) [@mattt](https://forums.swift.org/u/mattt)\ **Post date:** [June 4, 2020, 6:21pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/2 "2020-06-04T18:21:18Z")
## Detailed design ### Notations The following terminology and conventions are used in this document. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC 2119](https://tools.ietf.org/html/rfc2119 "Key words for use in RFCs to Indicate Requirement Levels"). API endpoints that accept parameters in their path are expressed by Uniform Resource Identifier (URI) templates, as described in [RFC 6570](https://tools.ietf.org/html/rfc6570 "URI Template"). ### Definitions The following terms, as used in this document, have the meanings indicated. - _Package_: A named collection of Swift source code that is organized into one or more modules according to a `Package.swift` manifest file. - _Release_: The state of a package after applying a particular set of changes that is uniquely identified by an assigned version number. - _Version Number_: An identifier for a package release in accordance with the [Semantic Versioning Specification (SemVer)](https://semver.org/ "Semantic Versioning"). ### Conventions This document uses the following conventions in its description of client-server interactions. #### Application layer protocols A client and server MUST communicate over a secured connection using Transport Layer Security (TLS) with the `https` URI scheme. The use of HTTP 1.1 in examples is non-normative. A client and server MAY communicate according to this specification using any version of the HTTP protocol. #### Authentication A server SHOULD require authentication for client requests to publish or unpublish a package release. A server MAY require authentication for client requests to access information about packages and package releases. A server SHOULD respond with a status code of `401` (Unauthorized) if a client sends a request to an endpoint that requires authentication without providing credentials. A server MAY respond with a status code of `404` (Not Found) or `403` (Forbidden) when a client provides valid credentials but isn't authorized to access the requested resource. A server MAY use any authentication model of its choosing. However, use of a scoped, revocable authorization framework like [OAuth 2.0](https://tools.ietf.org/html/rfc6749 "The OAuth 2.0 Authorization Framework") is RECOMMENDED. #### Error handling A server SHOULD communicate any errors to the client using "problem details" objects, as described by [RFC 7807](https://tools.ietf.org/html/rfc7807 "Problem Details for HTTP APIs"). For example, a client sends a request to create a package release with an invalid `tag` parameter and receives the following response: ``` HTTP/1.1 404 Content-Version: 1 Content-Type: application/problem+json Content-Language: en { "detail": "tag '2.0.0' not found" } ``` #### Rate limiting A server MAY limit the number of requests made by a client by responding with a status code of `429` (Too Many Requests). ``` HTTP/1.1 429 Content-Version: 1 Content-Type: application/problem+json Content-Language: en Retry-After: 60 { "detail": "try again in 60 seconds" } ``` A client SHOULD follow the guidance of any `Retry-After` header values provided in responses to prevent overwhelming a server with retry requests. It is RECOMMENDED for clients to introduce randomness in their retry logic to avoid a [thundering herd effect](https://en.wikipedia.org/wiki/Thundering_herd_problem "Thundering herd problem"). #### API versioning Package registry APIs are versioned. API version numbers are designated by decimal integers. The accepted version of this proposal constitutes the initial version, `1`. Subsequent revisions SHOULD be numbered sequentially (`2`, `3`, and so on). API version numbers SHOULD follow Semantic Versioning conventions for major releases. Non-breaking changes, such as adding new endpoints, adding new optional parameters to existing endpoints, or adding new information to existing endpoints in a backward-compatible way, SHOULD NOT require a new version. Breaking changes, such as removing or changing an existing endpoint in a backward-incompatible way, MUST correspond to a new version. A client SHOULD set the `Accept-Version` header field to specify the API version of a request. ``` GET /mona/LinkedList/list HTTP/1.1 Accept: application/json Accept-Version: 1 ``` A server SHALL set the `Content-Version` header field with the API version number of the response. ``` HTTP/1.1 200 OK Content-Type: application/json Content-Version: 1 ``` If a client sends a request without an `Accept-Version` header, a server MAY either return `400 Bad Request` or process the request using an API version that it chooses, making sure to set the `Content-Version` header field accordingly. If a client sends a request with an `Accept-Version` header that specifies an unknown or invalid API version, a server SHOULD respond with a status code of `400` (Bad Request). ``` HTTP/1.1 400 Bad Request Content-Version: 1 Content-Type: application/problem+json Content-Language: en { "detail": "invalid API version" } ``` If a client sends a request with an `Accept-Version` header that specifies a valid but unsupported API version, a server SHOULD respond with a status code of `415` (Unsupported Media Type). ``` HTTP/1.1 415 Unsupported Media Type Content-Version: 1 Content-Type: application/problem+json Content-Language: en { "detail": "unsupported API version" } ``` #### Namespace and package name resolution A namespace consists of zero or more path components preceding a package name and is used to distinguish between packages with the same name. For instance: - A registry that qualifies package names with the username of the person or organization responsible for the package (for example, `mona/LinkedList`) - A registry that qualifies package names with a username and domain of the source host (for example, `github.com/mona/LinkedList`) - A registry that doesn't qualify package names, creating an implicit, global namespace for the domain (for example, `LinkedList`) Namespaces and package names are both case-insensitive (for example, `mona` ≍ `MONA`) and normalization-insensitive (for example, `n` + `◌̃` ≍ `ñ`). A server MAY normalize or disallow characters in a namespace or package name according to the security considerations of Internationalized Resource Identifiers (IRIs) discussed in [RFC 3987](https://tools.ietf.org/html/rfc3987 "Internationalized Resource Identifiers (IRIs)") and [Unicode Technical Report #36](http://www.unicode.org/reports/tr36/ "Unicode Technical Report #36: Unicode Security Considerations"). ### Endpoints A server MUST respond to the following endpoints: | Link | Method | Path | Description | | --- | --- | --- | --- | | [[1]](#endpoint-1) | `GET` | `{/namespace*}/{package}` | List package releases | | [[2]](#endpoint-2) | `GET` | `{/namespace*}/{package}/{version}` | Fetch a package release | | [[3]](#endpoint-3) | `PUT` | `{/namespace*}/{package}/{version}` | Publish a package release | | [[4]](#endpoint-4) | `DELETE` | `{/namespace*}/{package}/{version}` | Unpublish a package release | * * * #### List package releases A client MAY send a `GET` request for a URI matching the expression `{/namespace*}/{package}` to retrieve a list of the available releases for a particular package. A client SHOULD set the `Accept` header with the `application/json` content type and MAY append the `.json` extension to the requested URI. ``` GET /mona/LinkedList HTTP/1.1 Accept-Version: 1 Accept: application/json ``` A server SHOULD respond with a JSON document containing all of the releases for the requested package. ``` HTTP/1.1 200 OK Content-Type: application/json Content-Version: 1 Link: ; rel="latest-version", ; rel="payment" { "releases": { "1.1.1": { "url": "https://swift.pkg.github.com/mona/LinkedList/1.1.1" }, "1.1.0": { "url": "https://swift.pkg.github.com/mona/LinkedList/1.1.0", "problem": { "status": 410, "title": "Gone", "detail": "this release was removed from the registry" } }, "1.0.0": { "url": "https://swift.pkg.github.com/mona/LinkedList/1.0.0" } } } ``` The response body SHALL contain a JSON object nested at a top-level `releases` key, whose keys are version numbers for releases and values are objects containing the following fields: | Key | Type | Description | Requirement Level | | --- | --- | --- | --- | | `url` | String | A URI for the release | REQUIRED | | `problem` | Object | A problem details object. | OPTIONAL | A server SHOULD communicate the unavailability of a package release using a ["problem details"](https://tools.ietf.org/html/rfc7807 "Problem Details for HTTP APIs") object. A client SHOULD consider any releases with an associated `problem` to be unavailable for the purposes of package resolution. A server SHOULD list releases in order of precedence, starting with the latest version. However, a client SHOULD NOT assume any specific ordering of version numbers in the response. A server SHOULD respond with a link to the latest published release of the package if one exists, using a `Link` header field with a `latest-version` relation. A server MAY respond with additional `Link` entries, such as one with a `payment` relation for sponsoring a package maintainer. #### Fetch a package release A client MAY send a `GET` request for a URI matching the expression `{/namespace*}/{package}/{version}` to fetch a package release. ##### Source archive A client MAY send a `GET` request for a URI matching the expression `{/namespace*}/{package}/{version}` to retrieve a release's source archive. A client SHOULD set the `Accept` header and SHOULD append the corresponding extension to the requested URI to indicate the desired archive format. The following table lists the supported archive formats along with the requirement level for a server. | Media Type | Extension | Kind of Document | Requirement Level | | --- | --- | --- | --- | | `application/zip` | `.zip` | ZIP archive | REQUIRED | | `application/gzip` | `.tar.gz` | GZip Compressed Tape Archive | RECOMMENDED | ``` GET /mona/LinkedList/1.1.1.zip HTTP/1.1 Accept: application/zip Accept-Version: 1 Want-Digest: sha-256 ``` If a release is found for the requested URI, a server SHOULD respond with a status code of `200` (OK) and a `Content-Type` header corresponding to the archive format used (for example, `application/zip` for a Zip archive). Otherwise, a server SHOULD respond with a status code of `404` (NOT FOUND). ``` HTTP/1.1 200 OK Accept-Ranges: bytes Cache-Control: public, immutable Content-Type: application/zip Content-Disposition: attachment; filename="LinkedList-1.1.1.zip" Content-Length: 2048 Content-Version: 1 Digest: sha-256=a2ac54cf25fbc1ad0028f03f0aa4b96833b83bb05a14e510892bb27dea4dc812 ETag: e61befdd5056d4b8bafa71c5bbb41d71 Link: ; rel=describedby; type="application/pgp-signature", ; rel=duplicate; geo=jp; pri=10; type="application/zip" ``` A server SHOULD respond with a `Content-Length` header set to the size of the archive in bytes. A server SHOULD respond with a `Content-Disposition` header set to `attachment` with a `filename` parameter equal to the name of the package followed by a hyphen (`-`), the version number, and file extension (for example, "LinkedList-1.1.1.zip"). A server SHOULD respond with a link to a digital signature for the archive using a `Link` header field entry with a `describedby` relation and an `application/pgp-signature` type. It is RECOMMENDED for clients and servers to support range requests as described by [RFC 7233](https://tools.ietf.org/html/rfc7233 "Hypertext Transfer Protocol (HTTP/1.1): Range Requests"), caching as described by [RFC 7234](https://tools.ietf.org/html/rfc7234 "Hypertext Transfer Protocol (HTTP/1.1): Caching"), and instance digests as described by [RFC 3230](https://tools.ietf.org/html/rfc5843 "Instance Digests in HTTP") and [RFC 5843](https://tools.ietf.org/html/rfc5843 "Additional Hash Algorithms for HTTP Instance Digests"). A server MAY specify mirrors or multiple download locations using `Link` header fields with a `duplicate` relation, as described by [RFC 6249](https://tools.ietf.org/html/rfc6249 "Metalink/HTTP: Mirrors and Hashes"). A client MAY use this information to determine its preferred strategy for downloading. A server that indexes but doesn't host packages SHOULD respond with a status code of `303` (See Other) and redirect to a hosted package archive if one is available. Otherwise, a server MAY respond with a status code of `501` (Not Implemented). ``` HTTP/1.1 303 See Other Content-Version: 1 Location: https://swift.pkg.github.com/mona/LinkedList/1.1.1.zip ``` ##### Digital signature of source archive A client MAY send a `GET` request for a URI matching the expression `{/namespace*}/{package}/{version}` to retrieve a detached digital signature that can be used to verify the release's source archive according to [RFC 4880](https://tools.ietf.org/html/rfc4880 "OpenPGP Message Format"). A client SHOULD set the `Accept` header to `application/pgp-signature` and SHOULD append `.asc` to the originally requested URI (for example `.zip.asc` for the signature of the Zip archive). ``` GET /mona/LinkedList/1.1.1.zip.asc HTTP/1.1 Accept: application/pgp-signature Accept-Version: 1 ``` If a release is found for the requested URI, a server SHOULD respond with a status code of `200` (OK) and a `Content-Type` header set to `application/pgp-signature`. Otherwise, a server SHOULD respond with a status code of `404` (NOT FOUND). ``` HTTP/1.1 200 OK Cache-Control: public Content-Type: application/pgp-signature Content-Disposition: attachment; filename="LinkedList-1.1.1.zip.asc" Content-Length: 833 Content-Version: 1 ``` A client SHOULD verify the contents of a downloaded source archive using the provided signature (for example, using the command `gpg --verify LinkedList-1.1.1.zip.asc LinkedList-1.1.1.zip`). A server that indexes but doesn't host packages SHOULD respond with a status code of `303` (See Other) and redirect to the digital signature for the archive if one is available. Otherwise, a server MAY respond with a status code of `501` (Not Implemented). ``` HTTP/1.1 303 See Other Content-Version: 1 Location: https://swift.pkg.github.com/mona/LinkedList/1.1.1.zip.asc ``` ##### Metadata A client MAY send a `GET` request for a URI matching the expression `{/namespace*}/{package}/{version}` to retrieve metadata about a release. A client SHOULD set the `Accept` header with an `application/json` content type, and MAY append the `.json` extension to the requested URI. ``` GET /mona/LinkedList/1.1.1 HTTP/1.1 Accept: application/json Accept-Version: 1 ``` If a release is found for the requested URI, a server SHOULD respond with a status code of `200` (OK). Otherwise, a server SHOULD respond with a status code of `404` (NOT FOUND). ``` HTTP/1.1 200 OK Content-Type: application/json Content-Version: 1 Link: ; rel="latest-version", ; rel="predecessor-version" ``` A server SHOULD respond with a `Link` header containing the following entries: | Relation | Description | | --- | --- | | `latest-version` | The latest published release of the package | | `successor-version` | The next published release of the package, if one exists | | `predecessor-version` | The previous published release of the package, if one exists | A link with the `latest-version` relation MAY correspond to the requested release. A server MAY include additional fields in its response. It is RECOMMENDED that package metadata be represented in [JSON-LD](https://w3c.github.io/json-ld-syntax/ "JSON-LD 1.1: A JSON-based Serialization for Linked Data") according to a structured data standard. For example, this response using the [Schema.org](https://schema.org/) [SoftwareSourceCode](https://schema.org/SoftwareSourceCode) vocabulary: ``` { "@context": ["http://schema.org"], "@type": "SoftwareSourceCode", "identifier": "@mona/LinkedList", "name": "LinkedList", "description": "One thing links to another.", "keywords": ["data-structure", "collection"], "version": "1.1.1", "codeRepository": "https://github.com/mona/LinkedList", "license": "https://www.apache.org/licenses/LICENSE-2.0", "readme": "https://github.com/mona/LinkedList/blob/master/README.md", "issueTracker": "https://github.com/mona/LinkedList/issues", "programmingLanguage": { "@type": "ComputerLanguage", "name": "Swift", "url": "https://swift.org" }, "author": { "@type": "Person", "@id": "https://github.com/mona", "givenName": "Mona", "middleName": "Lisa", "familyName": "Octocat", } } ``` #### Publish a new package release A client MAY send a `PUT` request for a URI matching the expression `{/namespace*}/{package}/{version}{?commit,branch,tag,path,url}` to publish a release of a package. ``` PUT /mona/LinkedList/1.1.1 HTTP/1.1 Accept-Version: 1 ``` A client MAY publish releases in any order. For example, if a package has existing `1.0.0` and `2.0.0` releases, a client MAY publish a new `1.0.1` or `1.1.0` release. A client MAY include additional information in its request body encoded as a JSON document with the `Content-Type` header set to `application/json`. It is RECOMMENDED that package metadata be represented in [JSON-LD](https://w3c.github.io/json-ld-syntax/ "JSON-LD 1.1: A JSON-based Serialization for Linked Data") according to a structured data standard, as discussed in ["Fetch a package release"](#endpoint-2). A client MAY send subsequent requests to update metadata for existing releases. A server MAY refuse to update the metadata for a package release by responding with a status code of `403` (Forbidden). If no release currently exists for the requested version, a server SHOULD respond quickly with a status code of `202` (Accepted) to acknowledge that the request is being processed. This response SHOULD also contain a `Location` header with a URL that the client can poll for progress updates and a `Retry-After` header with an estimate of when processing is expected to finish. A server MAY locate the status resource endpoint at a URI of its choosing. However, the use of a non-sequential, randomly-generated identifier is RECOMMENDED. ``` HTTP/1.1 202 Accepted Content-Version: 1 Location: https://swift.pkg.github.com/submissions/90D8CC77-A576-47AE-A531-D6402C4E33BC Retry-After: 120 ``` A server SHOULD attempt to access the source of the package release before sending a response, so that it may communicate any errors to the client immediately. A server MAY set a timeout to guarantee a timely response to each request. If a request to publish a new package release were to fail, a server MUST communicate that failure in the same way if sending an immediate response as it would if responding to a client polling for status. ##### Source repository location A server SHOULD determine the location of a package's source repository from the requested URI, if possible. A client MAY specify an explicit location with a `url` query item parameter. ``` PUT /mona/LinkedList/1.1.1?url=https://github.com/mona/LinkedList.git HTTP/1.1 Accept-Version: 1 ``` A server MAY refuse the specified source repository location by responding with a status code of `403` (Forbidden). ``` HTTP/1.1 403 Forbidden Content-Version: 1 Content-Type: application/problem+json Content-Language: en { "detail": "invalid parameter, 'url'" } ``` If a server is unable to access the repository, it SHOULD respond with `502` (Bad Gateway) with additional details about the failure in the response body. ``` HTTP/1.1 502 Bad Gateway Content-Version: 1 Content-Type: application/problem+json Content-Language: en { "detail": "unable to connect to repository" } ``` ##### Source revision The `version` path parameter in a client's request MUST be a valid version number for the release and SHOULD correspond to a tag in the package's source repository. A client MAY specify a different commit reference with any combination of `commit`, `tag`, and `branch` query item parameters. ``` PUT /mona/LinkedList/1.1.1?tag=v1.1.1 HTTP/1.1 Accept-Version: 1 ``` A server SHOULD respond with a status code of `404` (Not Found) if it's unable to resolve the requested version in the repository with the provided `commit`, `tag`, or `branch` parameters. ``` HTTP/1.1 404 Not Found Content-Version: 1 Content-Type: application/problem+json Content-Language: en { "detail": "tag '2.0.0' not found" } ``` A server SHOULD respond with a status code of `409` (Conflict) if any provided `commit`, `tag`, or `branch` parameters cannot be resolved to a single revision. ``` HTTP/1.1 409 Conflict Content-Version: 1 Content-Type: application/problem+json Content-Language: en { "detail": "tag '2.0.0' doesn't refer to commit '1F34E'" } ``` A server MAY refuse to publish a package if the specified tag or commit isn't signed by responding with a status code of `403` (Forbidden). ``` HTTP/1.1 403 Forbidden Content-Version: 1 Content-Type: application/problem+json Content-Language: en { "detail": "tag '2.0.0' must be signed" } ``` A server SHOULD support Git repositories. A server MAY support other source control systems, such as Mercurial or Subversion. ##### Package location A client MAY use `path` and `url` parameters together to publish a single package located in a subdirectory of a repository or to publish multiple packages from different paths in a single repository. If a client specifies a `path` parameter, a server SHOULD look for a `Package.swift` file at that location. Otherwise, it SHOULD look at the root directory. If a `Package.swift` file doesn't exist at that location, a server SHOULD respond with a status code of `404` (Not Found). #### Unpublish a package release A client MAY send a `DELETE` request for a URI matching the expression `{/namespace*}/{package}/{version}` to remove an existing release from a package registry. ``` DELETE /mona/LinkedList/1.1.1 HTTP/1.1 Content-Version: 1 ``` Support for this endpoint is OPTIONAL. A server that chooses not to process requests for unpublishing a package release SHALL respond with a status code of `405` (Method Not Allowed). ``` HTTP/1.1 405 Method Not Allowed Content-Version: 1 Content-Type: application/problem+json Content-Language: en { "detail": "packages cannot be unpublished through the API" } ``` If a server accepts a request for removal, it SHOULD respond quickly with a status code of `202` (Accepted) to acknowledge that the request is being processed. This response SHOULD also contain a `Location` header with a URL that the client can poll for progress updates and a `Retry-After` header with an estimate of when processing is expected to finish. A server MAY locate the status resource endpoint at a URI of its choosing. However, the use of a non-sequential, randomly-generated identifier is RECOMMENDED. ``` HTTP/1.1 202 Accepted Content-Version: 1 Location: https://swift.pkg.github.com/submissions/0CF1FC65-D928-41E5-A825-3BEB9D69C004 Retry-After: 120 ``` Once a package release is removed, a server SHOULD NOT publish any new releases with the same combination of namespace, package name, and version number. If a client attempts to unpublish a package that has already been removed from the registry, a server SHOULD return `204` (No Content). ``` HTTP/1.1 204 No Content Content-Version: 1 ``` If a client fetches a package that has been unpublished from the registry, a server SHOULD respond with a status code of `410` (Gone), or MAY instead redirect to an alternate release by responding with a status code of `301` (Moved Permanently). ``` HTTP/1.1 410 Gone Content-Version: 1 Content-Type: application/problem+json Content-Language: en { "detail": "this release was removed from the registry" } ``` * * * _(Continued in next post)_ ---
**Author:** ![mattt](https://avatars.discourse-cdn.com/v4/letter/m/46a35a/32.png) [@mattt](https://forums.swift.org/u/mattt)\ **Post date:** [June 4, 2020, 6:21pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/3 "2020-06-04T18:21:31Z")
### Normative References - [RFC 2119](https://tools.ietf.org/html/rfc2119 "Key words for use in RFCs to Indicate Requirement Levels"): Key words for use in RFCs to Indicate Requirement Levels - [RFC 3230](https://tools.ietf.org/html/rfc5843 "Instance Digests in HTTP"): Instance Digests in HTTP - [RFC 3986](https://tools.ietf.org/html/rfc3986 "Uniform Resource Identifier (URI): Generic Syntax"): Uniform Resource Identifier (URI): Generic Syntax - [RFC 4880](https://tools.ietf.org/html/rfc4880 "OpenPGP Message Format"): OpenPGP Message Format - [RFC 5843](https://tools.ietf.org/html/rfc5843 "Additional Hash Algorithms for HTTP Instance Digests"): Additional Hash Algorithms for HTTP Instance Digests - [RFC 6249](https://tools.ietf.org/html/rfc6249 "Metalink/HTTP: Mirrors and Hashes"): Metalink/HTTP: Mirrors and Hashes - [RFC 6570](https://tools.ietf.org/html/rfc6570 "URI Template"): URI Template - [RFC 7230](https://tools.ietf.org/html/rfc7230 "Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing"): Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing - [RFC 7231](https://tools.ietf.org/html/rfc7231 "Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content"): Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content - [RFC 7233](https://tools.ietf.org/html/rfc7233 "Hypertext Transfer Protocol (HTTP/1.1): Range Requests"): Hypertext Transfer Protocol (HTTP/1.1): Range Requests - [RFC 7234](https://tools.ietf.org/html/rfc7234 "Hypertext Transfer Protocol (HTTP/1.1): Caching"): Hypertext Transfer Protocol (HTTP/1.1): Caching - [RFC 7807](https://tools.ietf.org/html/rfc7807 "Problem Details for HTTP APIs"): Problem Details for HTTP APIs - [RFC 8288](https://tools.ietf.org/html/rfc8288 "Web Linking"): Web Linking - [SemVer](https://semver.org/ "Semantic Versioning"): Semantic Versioning ### Informative References - [BCP 13](https://tools.ietf.org/html/rfc6838 "Media Type Specifications and Registration Procedures") Media Type Specifications and Registration Procedures - [RFC 6749](https://tools.ietf.org/html/rfc6749 "The OAuth 2.0 Authorization Framework"): The OAuth 2.0 Authorization Framework - [RFC 8446](https://tools.ietf.org/html/rfc8446 "The Transport Layer Security (TLS) Protocol Version 1.3"): The Transport Layer Security (TLS) Protocol Version 1.3 - [TR36](http://www.unicode.org/reports/tr36/ "Unicode Technical Report #36: Unicode Security Considerations"): Unicode Technical Report #36: Unicode Security Considerations - [JSON-LD](https://w3c.github.io/json-ld-syntax/ "JSON-LD 1.1: A JSON-based Serialization for Linked Data"): A JSON-based Serialization for Linked Data - [Schema.org](https://schema.org/) A shared vocabulary for structured data. ## Security A package registry can offer stronger guarantees for safety and security compared to downloading dependencies using Git. Providing a checksum for the source archive of a package release allows a client to validate the integrity of the downloaded content. Providing a digital signature allows a server to certify that content to a client in a verifiable way. We acknowledge an inherent privacy concern with repository hosts and package registries logging the IP addresses for each package requested. The use of proxies and other techniques can help mitigate this risk, but we believe the best solution is for services to operate according to a stated privacy policy. ## Impact on existing packages Current packages won't be affected by this change, as they'll continue to be able to download dependencies directly through Git. Swift developers can opt-in to package registries on a per-dependency basis by changing its URL from one that points to a repository to one that resolves to a package endpoint. ``` let package = Package( name: "MyPackage", dependencies: [ - .package(url: "https://github.com/apple/swift-argument-parser.git", from: "0.0.1"), + .package(url: "https://swift.pkg.github.com/apple/swift-argument-parser", from: "0.0.1"), ] ``` ## Alternatives considered We first considered the more traditional "push"-based package registry design. However, this model creates a separate release process that runs parallel to version control, which not only duplicates effort but has the potential to introduce discrepancies between tags and releases. ## Future directions Defining a standard interface for package registries lays the groundwork for several useful features. ### Swift Package Manager integration A package registry can be useful in its own right, but enjoying its full benefit requires additional functionality in Swift Package Manager. We intend to submit a separate proposal for doing this. ### Offline cache Swift Package Manager could implement an [offline cache](https://yarnpkg.com/features/offline-cache "Offline Cache | Yarn - Package Manager") that allows it to work without network access. While this is technically possible today, a package registry allows for a simpler and more secure implementation than would otherwise be possible with Git repositories alone. ### Security auditing The response for listing package releases could be updated to include information about security advisories. ``` /* straw-man proposal */ { "releases": { /* ... */ }, "advisories": [{ "cve": "CVE-20XX-12345", "cwe": "CWE-400", "package_name": "mona/LinkedList", "vulnerable_versions": "<=1.0.0", "patched_versions": ">1.0.0", "severity": "moderate", "recommendation": "Update to version 1.0.1 or later.", /* additional fields */ }] } ``` Swift Package Manager could communicate this information to users when installing or updating dependencies or as part of a new `swift package audit` subcommand. ``` $ swift package audit ┌───────────────┬──────────────────────────────────────────────────────────────┐ │ High │ Regular Expression Denial of Service │ ├───────────────┼──────────────────────────────────────────────────────────────┤ │ Package │ RegEx │ ├───────────────┼──────────────────────────────────────────────────────────────┤ │ Dependency of │ PatternMatcher │ ├───────────────┼──────────────────────────────────────────────────────────────┤ │ Path │ SomePackage > PatternMatcher > RegEx │ ├───────────────┼──────────────────────────────────────────────────────────────┤ │ More info │ https://example.com/advisories/526 │ └───────────────┴──────────────────────────────────────────────────────────────┘ Found 3 vulnerability (1 low, 1 moderate, 1 high) in 12 scanned packages. Run `swift package audit fix` to fix 3 of them. ``` ### Package removal reason The request for unpublishing a package could be amended to accept additional parameters in the request body that explain why a package is being removed. For example, a maintainer could unpublish a package release in response to a security vulnerability or because they didn't intend to publish it in the first place. This endpoint could also be amended to distinguish between removal and deprecation of package releases. ### Binary framework distribution The package registry specification could be amended to add support for distributing packages as [XCFramework](https://developer.apple.com/videos/play/wwdc2019/416/ "WWDC 2019 Session 416: Binary Frameworks in Swift") bundles. ``` GET /mona/LinkedList/1.1.1.xcframework HTTP/1.1 Accept: application/vnd.apple.xcframework Accept-Version: 1 ``` Swift Package Manager could then use XCFramework archives as [binary dependencies](https://github.com/apple/swift-evolution/blob/master/proposals/0272-swiftpm-binary-dependencies.md "Package Manager Binary Dependencies") or as part of a future binary artifact distribution mechanism. ``` let package = Package( name: "SomePackage", /* ... */ targets: [ .binaryTarget( name: "LinkedList", url: "https://swift.pkg.github.com/mona/LinkedList/1.1.1.xcframework", checksum: "ed04a550c2c7537f2a02ab44dd329f9e74f9f4d3e773eb883132e0aa51438b37" ), ] ) ``` ---
**Author:** ![SDGGiesbrecht](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/sdggiesbrecht/32/4768_2.png) [@SDGGiesbrecht](https://forums.swift.org/u/SDGGiesbrecht)\ **Post date:** [June 4, 2020, 6:49pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/4 "2020-06-04T18:49:32Z")
> [@mattt](#): > > Swift developers can opt-in to package registries on a per-dependency basis by changing its URL from one that points to a repository to one that resolves to a package endpoint. > > ```swift > let package = Package( > name: "MyPackage", > dependencies: [ > - .package(url: "https://github.com/apple/swift-argument-parser.git", from: "0.0.1"), > + .package(url: "https://swift.pkg.github.com/apple/swift-argument-parser", from: "0.0.1"), > ] > > ``` This will wreak havoc on graph resolution. How is SwiftPM supposed to know both URLs point at the same thing, so that you don’t end up with both in your graph at once? Is there no way to stick with the existing URL and have an API that basically asks “Do you support the registry API?” Then SwiftPM could opt to interact with the registry if it got an intelligible response, and fall back to the old Git method if not? ---
**Author:** ![armcknight](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/armcknight/32/3486_2.png) [@armcknight](https://forums.swift.org/u/armcknight)\ **Post date:** [June 4, 2020, 7:03pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/5 "2020-06-04T19:03:02Z")
I'm not sure I'm understanding the problem here, did you notice that it is a diff-style example where the github url was removed and replaced with the swift.pkg.github url? I'm just asking because you said "both URLs" but there would only be one. ---
**Author:** ![itaiferber](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/itaiferber/32/33840_2.png) [@itaiferber](https://forums.swift.org/u/itaiferber)\ **Post date:** [June 4, 2020, 7:08pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/7 "2020-06-04T19:08:36Z")
I believe what @SDGGiesbrecht is concerned about is that you could end up with two dependencies each depending on the same package, one specifying “[github.com](http://github.com)” and the other specifying “[swift.pkg.github.com](http://swift.pkg.github.com)”. ---
**Author:** ![mattt](https://avatars.discourse-cdn.com/v4/letter/m/46a35a/32.png) [@mattt](https://forums.swift.org/u/mattt)\ **Post date:** [June 4, 2020, 7:37pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/8 "2020-06-04T19:37:58Z")
> [@SDGGiesbrecht](#): > > How is SwiftPM supposed to know both URLs point at the same thing, so that you don’t end up with both in your graph at once? To be clear, that code listing under "Impact on existing packages" is presented as an example of how Swift Package Manager integration could be implemented, not a specific proposal for how it should be done. If this approach were to be adopted, I can think of a few different ways to resolve transient dependencies with different URLs: - Add a `Link` header entry to the package registry response for a package release that points to the canonical source repository url - Update `.package` in the next `swift-tools-version` to include a `name` or some other canonical mapping - Define a mechanism for git URLs to automatically redirect to package registry API endpoints when accessed by Swift Package Manager, such as a User-Agent or other header I'm not sure how Swift Package Manager currently deals with case differences in shared dependencies (e.g. `github.com/alamofire/alamofire.git` vs. `github.com/Alamofire/Alamofire.git`), but I think it'd be instructive to consider these problems together. Anyway, this is absolutely something that we should figure out in the follow-up proposal for Swift Package Manager integration. Thanks for calling this out! ---
**Author:** ![tanner0101](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/tanner0101/32/675_2.png) [@tanner0101](https://forums.swift.org/u/tanner0101)\ **Post date:** [June 4, 2020, 7:50pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/9 "2020-06-04T19:50:26Z")
This looks great! Thanks @mattt + 💯 > [@SDGGiesbrecht](#): > > How is SwiftPM supposed to know both URLs point at the same thing, so that you don’t end up with both in your graph at once? Couldn't you see that both have the same `Package.name`? ---
**Author:** ![SDGGiesbrecht](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/sdggiesbrecht/32/4768_2.png) [@SDGGiesbrecht](https://forums.swift.org/u/SDGGiesbrecht)\ **Post date:** [June 4, 2020, 8:00pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/10 "2020-06-04T20:00:50Z")
> [@mattt](#): > > I'm not sure how Swift Package Manager currently deals with case differences in shared dependencies (e.g. `github.com/alamofire/alamofire.git` vs. `github.com/Alamofire/Alamofire.git` ), There are plenty of bugs in this area. SwiftPM invalidates all pins, retries resolution, and refuses to pin the offending package. Super slow, but it does succeed in the end. Xcode just crashes. But I agree it doesn’t necessarily have to be solved here, only that it be on our minds so we don’t exacerbate it. > [@tanner0101](#): > > Couldn't you see that both have the same `Package.name` ? As I mentioned above, the whole thing is currently a mess. The declared name would be suboptimal as a unique ID, because if it clashes between two legitimately different packages, SwiftPM would think they were the same and never fetch the second. The diagnostics would then be completely tangled, sending you on a wild goose chase. However, maybe SwiftPM could fetch _both_ and then treat them the same if their entire contents match, thereafter discarding the extraneous copy? But I’ll leave it at that, because we’re drifting off topic. ---
**Author:** ![Aciid](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/aciid/32/429_2.png) [@Aciid](https://forums.swift.org/u/Aciid)\ **Post date:** [June 4, 2020, 8:25pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/11 "2020-06-04T20:25:38Z")
> [@SDGGiesbrecht](#): > > As I mentioned above, the whole thing is currently a mess. +1 I think we really need to take a look at package identity/name clashes in SwiftPM but that deserves its own separate discussion. ---
**Author:** ![hartbit](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/hartbit/32/738_2.png) [@hartbit](https://forums.swift.org/u/hartbit)\ **Post date:** [June 4, 2020, 8:30pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/12 "2020-06-04T20:30:23Z")
I'm very excited to see some movement in this space. A small comment below: > [@mattt](#): > > A server SHOULD communicate any errors to the client using "problem details" objects, as described by [RFC 7807](https://tools.ietf.org/html/rfc7807). For example, a client sends a request to create a package release with an invalid `tag` parameter and receives the following response: I just read through RFC 7807, and I'm not entirely sure I got this right, but shouldn't the `type` member be mandatory? If that's the case, shouldn't the current specification define some standard type URIs? ---
**Author:** ![mattt](https://avatars.discourse-cdn.com/v4/letter/m/46a35a/32.png) [@mattt](https://forums.swift.org/u/mattt)\ **Post date:** [June 4, 2020, 8:49pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/13 "2020-06-04T20:49:10Z")
> [@hartbit](#): > > I just read through RFC 7807, and I'm not entirely sure I got this right, but shouldn't the `type` member be mandatory? If that's the case, shouldn't the current specification define some standard type URIs? To be honest, I'm not entirely confident in my own reading of RFC 7807. I would have liked for there to be a few more SHALL / SHOULD / MAY for each field. Looking at [section 3.1](https://tools.ietf.org/html/rfc7807#section-3.1) (emphasis added): > A problem details object can have the following members: > > - "type" (string) - A URI reference [[RFC3986](https://tools.ietf.org/html/rfc3986)] that identifies the problem type. This specification encourages that, when dereferenced, it provide human-readable documentation for the problem type (e.g., using HTML [[W3C.REC-html5-20141028](https://tools.ietf.org/html/rfc7807#ref-W3C.REC-html5-20141028)]). **When this member is not present, its value is assumed to be "about:blank".** As far as I can tell, an omitted `type` value shouldn't be a problem. I think it's acceptable to delegate this decision to each server implementation, as they may have their own preferred semantics / existing support ontology. ---
**Author:** ![tachyonics](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/tachyonics/32/37286_2.png) [@tachyonics](https://forums.swift.org/u/tachyonics)\ **Post date:** [June 4, 2020, 9:28pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/14 "2020-06-04T21:28:47Z")
Perhaps having this example in the proposal is more of a distraction than it is worth considering there are many issues regarding the final SwiftPM integration that would have to be worked out in the future proposal. Maybe just stating that dependencies from a registry will be specified in a manner that is distinct from and complementary to the current git-based dependencies is sufficient for this proposal. ---
**Author:** ![tachyonics](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/tachyonics/32/37286_2.png) [@tachyonics](https://forums.swift.org/u/tachyonics)\ **Post date:** [June 4, 2020, 10:04pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/15 "2020-06-04T22:04:16Z")
One use case made possible by a registry is having a mirror that tracks and curates (for example based on approved licenses) releases as they occur. Is there mechanisms for such mirrors to discover activity from its root repository to achieve such use cases? At the very least seems like this would require a GET endpoint to list packages/sub-namespaces at each level. Feels like this endpoint would also be useful for discoverability/visibility. Two questions for the current release list API- - will it be paginated or return the complete list of all releases for the package - is there a defined order that the releases are required be returned in ---
**Author:** ![mattt](https://avatars.discourse-cdn.com/v4/letter/m/46a35a/32.png) [@mattt](https://forums.swift.org/u/mattt)\ **Post date:** [June 4, 2020, 10:45pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/16 "2020-06-04T22:45:15Z")
> [@tachyonics](#): > > Maybe just stating that dependencies from a registry will be specified in a manner that is distinct from and complementary to the current git-based dependencies is sufficient for this proposal. I agree. I'll update that for the final proposal. > [@tachyonics](#): > > One use case made possible by a registry is having a mirror that tracks and curates (for example based on approved licenses) releases as they occur. Is there mechanisms for such mirrors to discover activity from its root repository to achieve such use cases? There are various strategies for coordinating new releases between a mirror and its source, but those are outside the scope of this specification. For example, a package registry could provide users a web hook that can be pinged to initiate a refresh. Or it could simply poll at a fixed interval. I think a likely option will be for registries to provide a CI integration, such as a GitHub Action, to automate the `PUT` request to publish a new version whenever a tag is pushed. The thing to keep in mind is that this specification defines a minimal set of common APIs. A registry can implement any additional functionality on top of this internally or though other API endpoints. > [@tachyonics](#): > > Two questions for the current release list API- > > - will it be paginated or return the complete list of all releases for the package The current design doesn't paginate. My rationale is that the overwhelming majority of packages have fewer than 100 releases. Yet, as I type this, I now recall Apple's practice of tags for nightly releases of their open source libraries, and how it overwhelm's GitHub's web UI. So I take your point. I'll add an affordance for pagination to the specification (probably something lightweight using `Link` relations). Thanks! > [@tachyonics](#): > > - is there a defined order that the releases are required be returned in According to the spec, the precedence ordering is SHOULD but not SHALL. That might have to change if we add pagination. ---
**Author:** ![davbeck](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/davbeck/32/32237_2.png) [@davbeck](https://forums.swift.org/u/davbeck)\ **Post date:** [June 4, 2020, 11:03pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/17 "2020-06-04T23:03:25Z")
I wonder how SPM handles differences between https urls and ssh urls. ---
**Author:** ![davbeck](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/davbeck/32/32237_2.png) [@davbeck](https://forums.swift.org/u/davbeck)\ **Post date:** [June 4, 2020, 11:12pm UTC](https://forums.swift.org/t/swift-package-registry-service/37219/18 "2020-06-04T23:12:18Z")
I really like this proposal. Whenever a Swift package registry is brought up, I get nervous that we will end up with a centralized approach, or something that encourages name squatting. I do wonder how SPM would differentiate http git urls from package urls. Github has long supported dropping `.git` from urls so that wouldn't be a reliable way to distinguish the 2. Perhaps a custom url scheme (that is actually just https) would work? Something like `swift://swift.pkg.github.com/apple/swift-argument-parser`? Or we could break the urls up and have a sources search list and a package from a registry would just be the name like `.package(name: "apple/swift-argument-parser", from: "0.0.1")`. ---
**Author:** ![bok1](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/bok1/32/12014_2.png) [@bok1](https://forums.swift.org/u/bok1)\ **Post date:** [June 5, 2020, 1:31am UTC](https://forums.swift.org/t/swift-package-registry-service/37219/19 "2020-06-05T01:31:07Z")
I think its a great proposal in general and am happy to see traction in this area. On the scope, I feel like package discovery is something that could definitely be improved upon but doesn't appear to be addressed here, would it be up to package registries to add search as part of their platforms if appropriate? Would we not want a standard way to search for packages to support a search command (eg. `swift package search`)? ---
**Author:** ![mattt](https://avatars.discourse-cdn.com/v4/letter/m/46a35a/32.png) [@mattt](https://forums.swift.org/u/mattt)\ **Post date:** [June 5, 2020, 3:05am UTC](https://forums.swift.org/t/swift-package-registry-service/37219/20 "2020-06-05T03:05:49Z")
> [@bok1](#): > > On the scope, I feel like package discovery is something that could definitely be improved upon but doesn't appear to be addressed here, would it be up to package registries to add search as part of their platforms if appropriate? Package discovery is something that I'm excited for registries to help solve. That said, I think search is a complex problem that could be better-served by existing standards like [OpenSearch](https://developer.mozilla.org/en-US/docs/Web/OpenSearch), or left as an implementation detail for each individual registry. > [@bok1](#): > > Would we not want a standard way to search for packages to support a search command (eg. `swift package search` )? Without a centralized search index, I'm not sure how this would work. Certainly any individual registry could provide their own search tool, but it may not make as much sense in a subcommand of `swift package`. ---
**Author:** ![mattt](https://avatars.discourse-cdn.com/v4/letter/m/46a35a/32.png) [@mattt](https://forums.swift.org/u/mattt)\ **Post date:** [June 5, 2020, 3:12am UTC](https://forums.swift.org/t/swift-package-registry-service/37219/21 "2020-06-05T03:12:26Z")
> [@davbeck](#): > > Github has long supported dropping `.git` from urls so that wouldn't be a reliable way to distinguish the 2. Not saying that this is the correct solution, but GitHub could respond differently based on a different media type query (`Accept: vnd.org.swift+json`) or user agent header (`User-Agent: swift package manager 5.2.4`). > [@davbeck](#): > > Perhaps a custom url scheme (that is actually just https) would work? Something like `swift://swift.pkg.github.com/apple/swift-argument-parser` ? I would discourage the use of custom URI schemes. Apple used this in early versions of iOS to support app linking, but has since — for good reason — moved towards universal links instead. > [@davbeck](#): > > Or we could break the urls up and have a sources search list and a package from a registry would just be the name like `.package(name: "apple/swift-argument-parser", from: "0.0.1")` . This is the approach I'm most familiar with by way of RubyGems. I'd have to ask around to see how Ruby developers feel about these in practice — especially with the advent of GitHub Packages. [Next page](https://forums.swift.org/t/swift-package-registry-service/37219.md?page=2)