[Vision] Redesign of the SSWG Package Incubation Process

Server Workgroup Updates
1

Hello everyone,
I'd like to share an update about the evolving thinking around the SSWG's package incubation process. This is an initial vision and we're interested in hearing feedback from the community about it.

Swift Server Incubation Workflow (2.0)

This proposal outlines a new and updated SSWG Incubation process.

Introduction

History

The incubation process was designed to give developers and companies more confidence in the Swift on Server ecosystem. Its goal was to spread good practices, and also ensure longevity of server focused packages. It also served as a way to discover packages.

The first incubated packages were put through the process in 2018/2019, and the process has continued ever since, with bi-annual status reviews. In the years since, the Swift server changed in many ways and resulted in a vibrant and coherent ecosystem that's steadily growing.

The process is focused on accepting and promoting packages through the Maturity Phases, with specific requirements associated with specific levels.

933×239 8.36 KB

You can review the incubation process and maturity level requirements here:

Analysis of Incubated packages

Before we propose the new process, we'd like to explain the state of the incubation process and incubated packages.

The incubation process has been steadily adding packages over the years, and the current list of packages is:

  • Graduated (14): SwiftNIO, SwiftLog, SwiftMetrics, PostgresNIO, AsyncHTTPClient, APNSwift, SwiftStatsdClient, gRPC Swift, Swift Crypto, Soto for AWS, MultipartKit, JWTKit, Vapor

  • Incubating (11): SwiftPrometheus, OpenAPIKit, Service Lifecycle, GraphQL, Graphiti, CassandraClient, SQLite NIO, Swift Service Context, Swift Distributed Tracing, Swift OpenAPI Generator, MongoKitten, Hummingbird

  • Sandbox (7): RediStack, Swift AWS Lambda Runtime, MQTT NIO, Swift Distributed Actors, DiscordBM, Oracle NIO, SwiftASN1

Projects are sometimes accepted at higher levels immediately, because they meet the specified criteria for a given level. However, at the same time, projects do not necessarily go through the whole process until Graduated unless they were well established projects to begin with.

The levels were historically meant to inspire more confidence in developers choosing packages, that a package would not randomly disappear or become un-maintained. Packages do not necessarily have to progress through until Graduation, because they may be facing e.g. shortage of additional core committers which historically has been preventing single-person but well maintained projects from entering the Graduated level.

The SSWG members have historically stepped in to "take over" maintenance of some projects, however we have found this process not sustainable, and some packages will just have to become replaced with others.

The challenges

  • Discovery: Improve discovery of well-maintained packages
  • Clarity: It is not very clear what the incubation process is giving developers, and why they should go through the process. Perfectly well maintained packages exist outside of the incubated list.
  • Simplicity: Streamline the incubation process; the current process is heavyweight, including a formal public review period.

Proposed solution

Goals

We propose to improve the incubation process in three aspects:

  • Make packages easier to discover
  • Make the benefits clearer to both package authors and developers discovering packages
  • Make the process lighter and easier to participate

By making it easier to participate and clarify the quality parts of packages, we hope the process can benefit both package authors as well as developers seeking new packages to use in their projects.

As additional benefit the process may also be adopted by other workgroups (Wasm and Android come to mind), however it is not a goal to force the adoption of this process onto any workgroups. We do believe its new form could scale up throughout the ecosystem though.

Package Discovery with Swift Package Index

  • Make packages easier to discover

In recent years, the community-run Swift Package Index has increasingly been listing thousands of packages (currently more than 10,000 packages), and has become the best place to list and discover Swift packages.

The incubated package list, on the other hand, is buried deep within the Swift.org website and is rarely how newcomers actually discover packages. In practice, general search engines and specialized tools like the Swift Package Index are far more useful for package discovery than a static list.

Key point:

  • Swift Package Index subsumes the utility of the incubated package listing for purposes of discovery.
  • The Swift.org website also has a Packages page which features popular packages, and directs to the Swift Package Index for more.

Checklists and Badges

  • Make the benefits clearer to both package authors and developers discovering packages

We propose to turn the incubation levels into "checklists" which can be verified (partially automatically, in the future), and associated "badges" which will be listed next to a project on the Swift Package Index.

For example, Minimal Requirements to become an SSWG incubated project in the Sandbox level include:

  • Be able to use the library on Server systems (e.g. Linux)
  • Use SwiftPM
  • Use some form of CI, unit tests, and semantic versioning
  • Licensing should be open-source compatible, e.g. Apache 2, MIT or other licenses are recommended
  • Follow the general Swift and Swift-on-Server best practices and API design guidelines

These are valuable, but basic, requirements. In addition to those, a Graduated project must:

  • Have stable API (no pending/planned breaking API changes), with at least one published major release (e.g. 1.0.0)
  • Support new GA versions of Swift within 30d
  • CI setup for two latest Swift.org recommended versions of Swift
  • CI setup for at least one of Swift.org recommended Linux distributions
  • CI setup for each platform supported by the library or tool
  • Unit tests for both macOS and Linux
  • Use Swift.org docker images, when appropriate
  • Documented release methodology
  • Documented support strategy for at least one previous major version.
  • Explicitly define a project governance and committer process, ideally laid out in a GOVERNANCE.md file and OWNERS.md files respectively
  • Include list of adopters for at least the primary repo ideally laid out in an ADOPTERS.md files
  • Optionally, have a Developer Certificate of Origin or a Contributor License Agreement

We propose to reform this list of requirements into independent "checks" or "badges" which can be attributed to a project and listed on the Swift Package Index. And some of those checks even can become automatic in time:

Automatic checks

  • Package index identified platform compatibility (already exists)
  • Use SwiftPM for build and distribution
  • Automatic CI and pull request validation
  • Open Source compatible license
  • Tagged 1.0 release / stable API
  • Check for DocC documentation
  • Check for RELEASING.md and other well-known files

There still will be some manual checks, which we'll discuss in the next section. Just to list a few immediately though, we envision:

Manual checks (require human review before badge is applied)

  • Consistency with swift-server ecosystem libraries (e.g. uses swift-log, metrics and tracing)
  • Consistency with Structured Concurrency practices (e.g. prefers structured concurrency and offers un-structured alternatives)
  • ...

The SSWG-Recommended badge

The SSWG will periodically review repositories which have been tagged as "recommended".

This is similar to the existing mechanism located at: PackageList/custom-package-collections.json at main · SwiftPackageIndex/PackageList · GitHub

Such tag may be granted with a specific date, so even slow reviews still provide indication because a tag would not be refreshed in 2026 for example. This tag is only additional to the other useful signals.

New light-weight review process

  • Make the process lighter and easier to participate

We propose to remove the current "public review" that packages are undergoing right now. In practice those have not been very helpful in the past. It is not the same as a Swift Evolution proposal where a "feature" is being discussed. It is difficult to put a whole package under review and then attempt to ambiguously judge its value. The purpose of the public reviews in spirit is the same as with Swift Evolution proposals, but since it is about an already existing piece of work, and not a new feature, the discussions have historically not really been very helpful -- it is a difficult task to jump and read through the entire package.

Packages can however themselves employ a light-weight evolution style process, for discussing new features. And we've seen foundational packages like log, metrics, distributed-tracing and others already adopt this independently. The public review makes sense for features, however it has not proven very useful for a general package review.

Given the introduction of the badges and checks, most of the process is already subsumed by those and the Package Index. Some checks cannot be automated however, and this is where the role of the SSWG (or other workgroups) remains.

We propose the creation of a sswg-packages which would be able to "tag" repositories with specific badges which require a manual review. For example, a package might want to request a review about Swift Structured Concurrency use and style to get the badge. We propose the flow would be:

  • Package wants to get a number of manual "checks" done by the SSWG
  • Package author opens issue on new repository called "sswg-packages" ( name TBD)
    • The new issue template includes the list of checks one can request to be checked for
  • An SSWG member is assigned and performs the review -- potentially helping out polish up the package, and help educate the maintainer about patterns or techniques they could apply to make their package better.
  • The SSWG member marks the package listing in the "sswg-packages*" listing (a JSON file) with any of the tags it shall now receive.
  • The Package Index periodically ingests this JSON file and applies additional "tags" to the package in the Package Index.

This way any workgroup may run a "review and tag" process for any specific tag that the package index shall list. Since these tags are ONLY granted by a specific workgroup working in partnership with the Package Index, they provide additional trust and verification of quality of such packages, that developers can rely on.

Proposed Migration Path

We propose to "drop" the current incubation levels and embrace the notion of the badges. These may be the one specific sswg-recommended (or some other name) badge, or there may be multiple ones which would be assigned the same way.

As migration strategy, all incubated packages would currently gain the new bage that we decide on.

Potential use by other workgroups

We also acknowledge that this mechanism is actually fairly general and could be used by other workgroups. The package index work just needs to be done once to allow the badge mechanism, and next other "blessed" badges could be created and managed by workgroups.

For example, perhaps wasm (@Max_Desiatov?) or @android-workgroup may potentially interested in badging with some form "verified to work great on ..." badges? This isn't a requirement for our process update, however we're thinking that in the Swift project's scale of things, this could be a generally useful mechanism.

We look forward to your feedback and ideas on the topic!

-- Konrad & @server-workgroup

19 Likes
2

Just finished reading this, I think this is a great direction for the incubation process to go in, and I agree that other workgroups should follow suit. I have raised this vision with the Android workgroup in some discussion we were having about similar package recommendations.

This is a great list of attributes to have manual and automated checks for, I'd like to add two more categories that I think are important:

Security- Supply chain security is a huge issue in software these days, we need to do something to enable it in Swift also. This could be everything from automated checks to flag issues, to manual reviews for core components. All badges must be tied to specific source tags, so devs know exactly which version of the software was badged.

Architecture- It would be good practice to require an Architecture.md, where the developer lays out their package's architecture, calling out core algorithms and modules to look at. This could be undergirded by Swift tooling that helps automate this description as much as possible.

Then, there is the thousand-pound gorilla at the base of open source, from which every other problem flows: funding. Much better mechanisms to fund open source packages are needed, as source forges like GitHub have manifestly failed at enabling this. I don't know if the Swift project and its corporate contingent wants to tackle this, as it largely hasn't so far, but the anemic funding model for open source is the root cause of most of these problems.

8 Likes
3

I really like this new direction. :raising_hands:

In particular:

It seems that the old process mixed two - somewhat related, but still different - concerns: quality of the product with risks to future sustainability ("are there enough contributors?"). I support pivoting to a process that emphasises quality more.

And there is even an opportunity: what if you provide a badge that indicates that a project is looking for or could benefit from more contributors?

Would you consider copy-left licenses (like GPL) compatible? And do you want to define a list of accepted licenses? On one hand that might be restrictive, but it does provide clarity.

Yes! :plus:1

3 Likes
4

As a former maintainer of a library within the v1 process, I’m very much in favor of this new direction.

It always felt a little unfair punishment that the project was never able to progress to a higher status simply because no one else wanted to step forward to also help maintain the project. Nor was the definition of what a “maintainer” is clearly defined.

This should hopefully reduce the burden on the working group, while opening up the list of packages within to overall improve the ecosystem.

3 Likes
5

Maintainer was/is "person with write/admin rights to the repo and is active in the project (responsive)" and this was used consistently in the process. We did add co-owners and moved repos to shared orgs to achieve this in the past. This also doesn't scale as it ended up putting more and more responsibility on SSWG members to "co maintain" random repositories just to move them along the process... This wasn't a good practice long term wise, even though we did "save" a few repos, like the statsd and prometheus libraries... It's not something we'll want to continue doing.

The requirement stemmed from worrying about the longevity of single-person projects. Realistically though it indeed was hard to resolve in practice so I think we're agreeing this will be something to likely relax on. Worst thing, one can fork a "dead" project after all.

1 Like
6

Overall, I like this.

None of the six evaluation criteria mention security at all, which is a significant concern for server software. Should we also have some minimum expectation around having a SECURITY.md, a security disclosure process, and a reasonable response time for CVEs?

Some of this was covered in the original SSWG criteria, and maybe it’s worth rolling it in here?

4 Likes
7

Thank you! That's a good callout, SSWG did previously require having a security policy for packages so yes we should roll this into here as well. Thank you :slight_smile: