I would like to publish some private modules and submodules under the swift-server organization on GitHub. what’s the process for getting added to that organization?
Hi there @taylorswift!
Yes, that org is managed by SSWG members - maybe you can shoot me a DM on the forums or email what those packages are and we can take it from there?
Why would you prefer to put them under that org rather than your personal github account / org? In general repositories there are also endorsed by SSWG so the process to follow that is outlined here: sswg/incubation.md at main · swift-server/sswg · GitHub (though we're more than happy, it's perhaps better even, to "endorse" repos outside this org, in order to suggest that this isn't some magical special place, it's just a good place for some of the packages we worked on as a group).
Also happy to discuss it right away here in the open, whichever works better for you.
1 Like
the thing about the incubation process document is, reading through, i just felt like the whole thing was over my pay-grade.
Incubation is made of the following stages: Pitch , Proposal , Development , and Recommendation . The Development stage is where the majority of incubation take place. The SSWG will maintain a public "Swift Server Ecosystem" index page that will list all recommended tools and libraries as well as projects that are part of the incubation process and their respective incubation level.
Pitches are an introduction to an idea for a new library or tool. They can also introduce ideas for new features or changes to existing tools. Pitches are used to collect feedback from the community and help define the exact scope of a project prior to writing code. They should demonstrate how they align with the SSWG's goals to improve Swift on the server. Pitches are submitted by creating a new thread in the Swift Server forum area.
this just doesn’t seem to line up with my understanding of how swift packages occur in the wild. it is not very often a dev gets an idea for a module, spends a week lobbying key voters, figuring out who all the eventual API users are going to be, and drafting a proposal document to anticipate and balance all their separate needs, without even starting to write the code.
and that’s just the pitching process. to even be called a proposal it needs two sponsors on the server committee to vouch for it, it needs an insider to care enough to put it in front of the whole committee, it needs the committee to get around to assigning it a review manager, and this is before it has even been scheduled for a review.
this whole time the committee won’t even dare refer to it as a “sandbox project”.
the proposal section is ten paragraphs long. the sections for the subsequent stages are even more detailed. but the initial “pitch” section is one paragraph long, and there isn’t anything on what to do before the pitch, other than a vague:
The incubation process is designed to help nurture and mature projects ensuring standardization, quality, and longevity. It also seeks to increase the visibility of ideas, experiments, or other early work that can add value to the SSWG mission.
this is just the opposite of how contributions really come about. in general, you will constantly see tons and tons of half-started, incomplete, and experimental packages random authors keep in personal github repos. some of these packages might become good enough to be used with care in production, perhaps in a dev’s own project, or by a few users that consider it reliable enough. as packages mature, there are fewer and fewer of them and they naturally evolve into the community’s idea of a “quality library” and the detailed project guidance and acceptance criteria are not really needed because everyone knows a mature framework when they see one.
the process just doesn’t seem to be geared towards someone who wants to spin-off part of a project that they think will be useful to a wider audience, or someone who authored a framework for an employer that went under and the dev retained the rights to the framework and are thinking about open-sourcing it. situations that represent how a majority of packages are born. especially in a community like Swift on Linux where there is not a lot of preexisting library ecosystem and people are used to rolling their own.
they already live in my personal github in one form or another. github private repos are great for version control. but even if i make it public, no one will ever find it, and the next server-side dev who needs, say, a JSON parser without floating point encoding issues (i feel like every week someone comes here asking that) will just write another in their own personal repo, and so on and so on. and everyone will keep feeling like server-side swift is a desert and all the nice things are over in iOS/SwiftUI land.
github doesn’t really have a nice “package collections” feature (i wish it did). so github organizations end up becoming the place for people to look for packages, especially for server-side swift devs, since we’re the only group of swift devs who deploy to linux as the exclusive target and not a second platform.
i know the committee “endorses” things occasionally. but the endorsement is hidden away so carefully that even people who know that it exists have to navigate the labyrinth to find it. no one even told me the committee endorsed atomic blonde (edit just realized you did, oops)
for example, the incubation page references a
public "Swift Server Ecosystem" index page that will list all recommended tools and libraries as well as projects that are part of the incubation process and their respective incubation level.
but it’s not hyperlinked. you have to scroll all the way past the diagram (why is there a diagram?) before you can click on the Swift Server Ecosystem Index, which takes you to swift.org.
the url fragment is called #projects, which doesn’t exist and which i assume is referring to the id="development-guides" heading. and the development guides come after the “Swift Server Workgroup” section, because that’s the first thing we need to teach someone who’s just looking to get started with server-side swift.
the (search) keywords 'package', 'ecosystem', 'index' don’t appear anywhere on the page. you have to breadth-first search through
- Setup and code editing
- Building
- Testing
- Debugging Memory leaks
- Performance troubleshooting and analysis
- Debugging multithreading issues and memory checks
- Deployment
none of which could reasonably be called an index, except for the IDE editors page which is only considered “server-side related” because all the macOS/iOS people use XCode.
would it really be so bad if the committee allowed more “pre-pitch” repositories to live under the organization directory? it’s not like it’s overflowing with repositories. including the VSCode extension, i only count six.
Right now i have packages for (in various stages)
- CRC-32
- SHA-256
- Base64
try?-based constructive parsing and diagnostics- JSON
- timestamps
- decimals
- a high-level
async/awaitwebsocket client (and a much less mature websocket server) - FIX
- an
async/awaitPublisher
2 Likes
You may be attributing a much larger gravitas to this repository than is actually is, or is intended to be I feel somehow.
I absolutely agree that "packages in the wild" occur naturally, and as you say "the community realizes they're high quality" etc. The SSWG has nothing to do with this natural process at all, nor does this organization.
It should not be a goal to throw repositories into the swift-server organization for just a reason of "people will find it more". It's an anti goal even I'd say. As you also seem to agree with:
This misses the point of the group though. The "recommendations" and maturity levels are modeled after organizations like the https://www.cncf.io etc. The purpose is for people who have no idea about projects, to be able to quickly identify "okey, those core important libraries I really care about are vetted by the SSWG and will not randomly die out" which should lead larger organizations to feel comfortable to "bet on using Swift (on the server)" after all.
This isn't going to matter for small organizations or even single people projects, but is hugely important the larger an organization grows. Think about "Large Corporation X CTO" having to be convinced this Swift thing isn't just a joke, the group's "rubber stamping" is there to help teams make a strong, argument that certain packages are expected to be solid, supported, and taken care of from a security aspect so the "bet" on Swift becomes less risky.
This has me very confused. We both seem to be cheering for and advertising for a "bazaar"-style ecosystem where everyone puts out best libraries they can and the people naturally find the good libraries. The SSWG's purpose in life is to help such bazaar come to fruition and give larger organizations more confidence in this model, and "random repositories" which they might not have otherwise.
Yet, in order to see this model you advertise blossom, one has to get packages in the open, right? And keeping them in private repositories, claiming that no-one would find them if they were open is a bit confusing, that's how people are even able to find them in the first place, they must be "out there"!
We should not double down on the "cathedral" model where "all repositories which matter" end up in the swift-server organization -- this would honestly be a failure of the SSWG. We want to help others grow and not centralize packages.
Since, we both (you, me, the SSWG), want to encourage the bazaar model, and you also seem to cheer for that model - so... let's do it, and not try to add bricks to the cathedral which is an anti goal of the group. Let's get those repos open 
We also have "package collections" as well as the https://swiftpackageindex.com which both are designed to solve the discoverability issue which seems to be your primary concern in not just open sourcing what you have under your own repositories. And there are efforts to make packages more and more discoverable via such approaches
If you want, we'd be happy to also "rubber stamp" them using the incubation process, but that's a different topic somewhat, right?
// edit: For what it's worth the Apple SDK's are developed in the cathedral model; but this isn't what we want to foster for the open source and/or server ecosystem; The bazaar can move quicker and is not tied to release schedules and "privileged" groups only being able to release etc.
9 Likes
Thanks for the mention, Konrad :) I'm not sure if it helps in this instance but I just wanted to point out that we're automatically exposing all packages within an org (Github user) as package collections. For example:
It would therefore be quite simply to group certain packages in a collection by moving them into their own organisation.
4 Likes
i never liked the bazaar model because it assumes your customer is infinitely motivated to hunt down your specific product, that you have an infinite supply of user attention that’s allocated just for you, that you can bury gold in the desert and people will be swarming the area with metal detectors the next day.
in 2022, attention is one of the scarcest things in the world. people in the commercial world are bidding top dollar just for the right to deliver the equivalent of a 30 second elevator pitch.
CNCF claims to have 147K contributors. when server-side swift is getting that much attention then it would make sense to be this selective. but i just don't think that's where we're at right now.
larger organizations are more likely to participate in a project if it already has a lot of small participants. but you're not going to attract a lot of small participants if you frame the group as mainly geared towards big participants and small participants are not valued or welcome.
actually, i used the bazaar model for years. i’ve been publishing packages for Swift on Linux since before these forums existed and everyone was still on the mailing list and complaining about the exact same "ecosystem" issues. i gave up on it because it simply isn’t working in 2022.
there’s a big valley of death between when a package is functional for what it was built for, and when a package is "fit for public consumption". a library author probably wrote the library for themselves or for an employer. to take it public, this person needs to:
- fully document its API, write references, and set up a documentation website. (before DocC, they also had to bootstrap their own docgen tool.)
- set up a CI pipeline so other people can contribute
- write advertising copy for the README so visitors don’t bounce back to google
- write tutorials, manuals, and other guides so people actually know how to use the package
- travel the talk show circuit (these forums, Reddit, RW forums, etc.) to raise awareness of the package, on platforms the dev may not have a lot of reputation on because they spend most of their time coding or on GitHub.
in my experience, there is not a lot of incentive for someone to complete all these steps, nearly all of this work will go to waste. nobody will visit the API website, or contribute to the repository. the dev will not get paid for the tutorials, and cannot sell them to RW or some other vendor. the dev will get accused of spam and chased off by a Reddit moderator. etc etc.
i have to reiterate, in the for-profit world, people pay top dollar for the attention of others. a non-profit open source project with no marketing budget has no chance of success, unless we help each other.
5 Likes
It is worth noting that the SSWG actually did start out as a community project. That was hijacked around the SwiftNIO release timeframe and became that funny thing we have today.
On the one hand, I agree - it is definitely a lot of work and you need to be really motivated to bring the project up to that level of quality. And yeah, for an individual, open-source development is a bit of a thankless task. Don't expect to get paid for any of it. Ever.
On the other hand, I think all of those things are really important. Would you want to use a package with no documentation? Or which isn't continuously tested? Just "works for me"? Or whose README does not even explain what the package does?
IMO, it's good that the SSWG enforces these sorts of quality standards. It's good for users to know that somebody is keeping an eye on things and will say something if the developer gets lazy or forgetful and fails to maintain a certain level of quality, and hopefully that gives them the confidence to try packages which they otherwise might not have.
4 Likes
of course i think all those things are important. i did them in the past. but that's not what we’re talking about, it’s about how a package crosses that fold and why the process seems to be built to actively disincentivize people from taking that gamble and spending a month of their valuable time to bring it up to spec just for the committee to say it is not even worthy of being called a "sandbox project".
im not asking for a lot. i'm literally just asking for access to a GitHub organization. something that is free in a country where just about everything costs money. so why does it feel like i am begging people to let me donate my time and my work to their organization?
2 Likes
I mean, looking at the proposals, I think that is perhaps a bit of a mischaracterisation of the committee. Everything seems to get accepted at some level. Sometimes they have feedback, but they work with the package authors to help get things up to snuff. It's not a cold, distant panel giving a plain 
for no reason.
The levels are sometimes up for debate. For example, I know @Mordil has expressed concerns about the need for a certain number of contributors, and I agree that it can be difficult for some packages to achieve.
It's still unclear to me why you want this, although I confess I don't really understand this whole GitHub organisations business. The closest I can find to an explanation is this:
Which appears to me to be saying that you want your projects to live under the SSWG banner in some form, to give them more exposure or imply endorsement without going through the existing process. Apologies if I'm misunderstanding, but I'm not sure I agree with that.
1 Like
+1 to these concerns. I considered submitting some of my libraries to SSWG for review, was even asked to submit XMLCoder by other people. In my opinion XMLCoder has enough documentation, has a pretty decent CI pipeline set up, quite comprehensive README, etc.
Then I read about the incubation process and this was overall a no-go in terms of the amount of time and effort required to go through all of the review stages. Especially a requirement to have 2+ maintainers. It's a chicken and egg problem for sure. I think an average library is unlikely to get a second maintainer and enough motivation to comply with all of the requirements if it doesn't receive an endorsement through something like SSWG. But you can't get into SSWG unless you have a second maintainer and comply with all of the requirements.
3 Likes
why is the assumption always that there are a million unscrupulous devs trying to sneak their low-quality projects, lest they steal some of the committee’s pristine reputation (for, what exactly? if not to give back to the community?)
i just don't think any organization can treat people like this and expect people to keep coming to them.
exactly. and this is just for "incubation". i just don't think that's what is meant by the word "incubation".
1 Like
To be clear - you absolutely can get a package into the SSWG incubation process with just a single maintainer. It's the 'graduated' packages that we require multiple maintainers to ensure that anyone relying on them isn't going to be out of luck if the maintainer moves on/loses interest/gets another job etc. The entry requirements for sandbox packages are not too major and it sounds like XMLCoder would easily make it. Additionally the pitch process doesn't require any code to be written if you just want to see what the appetite is for a library
5 Likes
Firstly, I'm not a member of the SSWG, so please don't read my opinion as being any sort of official opinion.
Secondly, that's not what I'm saying. I'm definitely not implying any sort of malice on your part. I'm saying that the SSWG has quality bars, I think that's good, and from what I can see, they are appreciative of everybody's efforts and give thoughtful feedback to try to help everybody reach them.
We're not assuming anyone is trying to sneak any packages into the org. Any packages in the swift-server org are effectively packages the SSWG are committing to maintaining (unless archived). As Konrad stated the GH org's aim is not to become the 'official' collection of packages. We want packages from all over to be submitted and a large section of the packages in incubation do not live in the swift-server (or Apple) org. So generally the only packages in the swift-server org are packages the SSWG are building themselves and I don't see this changing
3 Likes
sure, that’s one approach they could take. but there just doesn’t seem to be any space for that middle ring of projects that are explicitly Swift on Linux, server-side, etc but aren’t authored by someone who already sits on the committee.
i understand the fear:
- someone adds a project to the SSWG organization on GitHub
- they move on and stop maintaining the package and things gradually start breaking
- dead projects accumulate and the organization gets cluttered and is no longer useful as a package collection
but it just seems like it's assuming the worst out of anyone brave enough to walk in the door. after all,
-
the dev might be more motivated to maintain the project if it has users and attention
-
other users might find the project, learn how to use it, and write guides and tutorials for it
-
the dev may eventually wish to join the SSWG, once they have some positive experiences contributing to the organization
if you assume people are going to abandon the projects, that's what they're going to end up doing.
1 Like
maybe it’s just me, but i feel like new projects tend to start with code and no english, more often than an author has a proposal written but no code to go along with it. for me at least, i usually build the thing first, and then write about it…
This is something that the entire ecosystem of Swift packages are dealing with and not just server deployed focused ones.
We should be trying to solve this issue as a Swift ecosystem rather than just using a tool that conveniently happens to exist for the server-side community in the swift-server organization.
... and that solution is in progress. We just got package collections with SwiftPM and the Swift Package Index team has been doing awesome work for the last 2 years.
The contention seems to be that all we have is this hammer, and everything is looking like a nail. The purpose of the organization is not to be a source of all projects "blessed" or indexed by the SSWG - it's just a central place to collaborate on clearly shared pieces such as async-http-client. If SwiftNIO wasn't an Apple IP, it would definitely live there.
But RediStack isn't and probably won't be unless I actually step away, because it was a library I started on my own initiative, and not every server-side project will rely on it.
I very much agree with @ktoso that the cathedral model is actually worse for the ecosystem given that it would further give the implication that if it's not in the swift-server organization, then it's not worth it.
Much like how some projects I worked on refused to use anything other than Foundation or writing their own that they could trust.
I think it's time to revisit the outline of the incubation process of expectations and process - because some of the proposals have happened in retrospect of code being written. I think only really SwiftLog and SwiftMetrics started on paper before any code was written.
If I recall correctly, there's even been a few proposals that skipped the pitch phase.
Yeah it is fine to pitch things after there is already code written and I believe several libraries currently in the incubation process already existed at the time of being pitched. At the time we pitched the MongoDB driver (which was before I joined SSWG) I had been working on it for quite a while and was in the process of adding the EventLoopFuture version of the API. I agree the wording there is something we should revisit