Namespacing of packages/modules, especially regarding SwiftNIO

Server Development
1

First of all, I apologise for bringing this up only now but I believe this is very important for the long-term health ecosystem and it's not too late yet.

What's the issue?

In a given Swift(PM) application, 3 things about a package need to be unique across the whole application:

  1. it's package basename (for example swift-nio-http2)
  2. it's package name (let package = Package(name: "swift-nio", ...) in Package.swift)
  3. all module names

Arguably, that can lead to tricky issues when combining multiple packages. We realise that SwiftNIO is becoming a dependency of more and more libraries and therefore we want to make your lives easier by defining rules for ourselves on how to name things.

So what is NIO doing to help?

Because all those identifiers (basename, package name, module, and type names) can not be changed without releasing a new public API version we try to make the ones we add predicatable. For SwiftNIO, we have tried to be very careful and we have specified what exactly we will and won't do, and how we namespace new symbols in SwiftNIO's public API declaration.

Basically: Unless we release a new major version, we will only add symbols (in minor version bumps) that have NIO* / nio* / CNIO* / cnio* / c_nio* prefixes. All of our modules are and will always remain NIO*/CNIO* too. And our packages are swift-nio*.

How does this help?

Unless you name your package swift-nio-* or create modules that start with NIO/CNIO and follow the recommendations NIO's public API declaration there should be very little room that your package is clashing with SwiftNIO today or in the future, so it should remain compatible. That is very important for the ecosystem to work long-term when we have an order of magnitude more packages that depend on each other.

Why am I bringing this up?

The reason I bring this up now is that mostly as part of the SSWG, there are a lot of great new packages being created that all use SwiftNIO and we couldn't be more excited about it. Examples are:

The problem here is that there is a high chance of name collisions in the future because a lot of those packages are either swift-nio-* or have modules/types called NIO*. This will basically break everything we try to help you with by setting out rules for ourselves in the public API guidelines. In other words: If everybody names they modules/types NIO* and their packages swift-nio-*, then breakages are similarly likely to if we just didn't use our namespaces at all.

I believe that we're solving a real issue here and therefore my proposal would be to sort of claim packages prefixed swift-nio as well as module/type names starting with NIO/CNIO as SwiftNIO's namespace. I realise that it's very late asking people to rename their packages/modules to something else and it some cases it won't be possible anymore. By all means, if you have released a public version that people are using, please don't worry to change things until at least you release a new major version. For upcoming SSWG packages however I think it would be great to not use swift-nio*/NIO*/CNIO*.

Does that make sense? Is this too much to ask? And what do you think about this?

CC @tanner0101 / @IanPartridge / @kylebrowning / @Mordil / @artemredkin / @lukasa / @Joannis_Orlandos

5 Likes
2

Good point. I haven't considered this impact yet, but I'm completely behind this.

1 Like
3

Im cool with this. I think it's important to protect namespace.

1 Like
4

I agree protecting the namespace seems important. But for NIOPostgres at least, I really can't think of a better name. Does anyone have ideas? A pattern that we could use consistently would be nice. Maybe a special prefix for NIO-community things?

NIOPostgres is very similar in design to NIOHTTP1, NIOWebSocket, etc. That's why I like the name quite a bit. From my perspective, a user importing something with the NIO* prefix should expect things to be lower level. It's a nice pattern.

Another idea is that maybe we don't need to have such a broad rule. For something like NIOPostgres or NIORedis, where it's very unlikely that Apple will be creating a driver, we can let it slide. Then we only have to worry about things that are more likely to clash, like NIODNS.

3 Likes
5

Tanner voices my thoughts exactly. Agree with the goal, but not sure what the alternative is without being cute with the naming, overly broad, or redundant.

1 Like
6

What about PostgresNIO, RedisNIO for the module names?

7

Same here, what name do you think would be acceptable for NIOHTTPClient? Could we maybe make a registry of accepted libraries and their namespaces (at least for NIO*)?

8

/cc @Aciid to see if he ideas from swift-pm point of view

9

Honestly, for some of these lower level libraries I'm a little willing to claim some very basic module names: HTTP, Redis, etc. while the package might just be as simple as swift-*

I'd very like to see import HTTP similar to other language ecosystems

Higher level libraries will be responsible for naming, such as *Kit or other scheme.

1 Like
10

I thought the idea is that once that becomes the official working group implementation, the name would be allowed to enter that namespace. Is it not so?

So you anticipate that Apple will "secretly" produce NIO packages directly competing with official "SSWG packages" people are working on?
From an OpenSource community project standpoint I'd say in this case Apple should prefix such packages until they replace the community ones with a new API version (if that community decides so).

Or in other words: I'd prefer the NIO prefix to be community owned :slight_smile:

1 Like
11

"Official working group implementation" sounds very formal to me. The SSWG is really just there to help coordinate and mature the ecosystem by running and incubation process. Depending on SSWG packages might also be a slightly easier dependency because you can rest assured that there are people looking after it, at the very least 'archiving' an old obsolete project. Also you can rely on the licensing and just a few more pairs of eyes but that's all, see minimum requirements.

The SSWG also doesn't say 'our implementation for XYZ' is the only one and the best. There could be two packages wanting that implement the same thing. (see ecosystem index).

That's not what I said at all. I'll give you better examples: Let's assume someone creates a module called NIOBasics, or NIOHelpers, or maybe NIOSomethingElse. Or someone adds a type called NIOEventLoopProvider or similar. There's no way we can then guarantee that this package will be compatible with future NIO versions because as we announced, our strategy to prevent clashes is to always prefix everything new (apart from in new major versions) with NIO. That just doesn't help if everybody is creating NIO modules/types.

But then we would need to have some registry of stuff using NIO* to be able to safely add modules/types to SwiftNIO, right? Or should SwiftNIO switch to a different prefix in the next major version? What is your proposed fix?

12

NIOHTTPClient is a good example of something I believe we should better rename. NIOHTTPClient is a very near miss to the executable target called NIOHTTP1Client in SwiftNIO.

A registry would of course work but I was hoping we can avoid that complexity by just name-spacing things :slight_smile:.

1 Like
13

My suggestion is that the NIO prefix is reserved for SSWG incubated projects, not for Apple. If a NIOBasics package/module/type got accepted as part of the process, that is it. The WG acts as the registry. The "to help coordinate and mature the ecosystem" sounds very much in line with that.

If someone else (like myself) also does a NIOxyz outside of the WG, that would be entirely my problem (though I would probably just blame Swift/SPM :slight_smile:)

There could be two packages wanting that implement the same thing

While I'm not entirely sure why this is desirable at all, I'd say it's first come first serve in this case.
In any case there is no good solution for this in any case due to the lack of proper namespacing in Swift (e.g. via DNS names). What would happen? One imp would call itself RedisNIO, well, and the other too!

Note that I do think that implementations/APIs can be exchanged. Lets say we first have a NIOHTTPClient community project everyone agreed to incubate, and now Apple also produces a NIOHTTPClient project. That could perfectly go in as a new major version if everyone agrees that this is the better implementation. In the transition phase the Apple thing could be called NIOAppleHTTPClient.

Something which I think would be a step backwards and counter the idea if people put vendor names into the released modules. NIOVaporPostgreSQL, NIONozeRedis, NIOKituraXYZ.
I'm in strong favour of using NIOPostgreSQL for a PG NIO driver w/o additional dependencies, or NIORedis, etc.

14

For Redis, that may be considered trademark violation. We had that issue with SQLite at least: Trademark Violation · Issue #47 · vapor/sqlite-kit · GitHub

That seems like a reasonable compromise to me at least.

I agree 100% with this. I definitely want to avoid making it seem like these packages are anything but "pure NIO". In other words, NIOPostgres is, at its core, NIO + Postgres. Adding any other words there, especially the name of a framework, could make it seem like the package is more specialized.

15

But this is a counter example, he seems happy w/ NIOSQLite.

(which BTW sounds very weird to me, because SQLite has nothing to do w/ non-blocking I/O)

16

I'm aware of this consideration, and my understanding is the primary concern being the name of the project in GitHub.

How will people be able to easily disambiguate xyz/redis from a fork of the actual Redis project and a Redis client library - which ours are?

Naming it nio-redis (or swift-redis) as a package/repo is fine, and naming the module Redis is fine as it matches ecosystem convention - or something like that.

1 Like
17

After some more thought, I've realized that we already have an established process for naming these packages and modules if we follow what we came out with Logging & Metrics.

We're working in some aspects a bit too hard to get simple names to do double or triple duty in providing information and context.

  • NIORedis as a package could become swift-redis and the module just Redis
  • NIOPostgres -> swift-postgres / Postgres
  • NIOAPNS -> swift-apns / APNS
  • NIOHTTPClient -> swift-http / HTTP
    ...
18

The SSWG isn't about picking friends or electing one library as 'the only one' for a given task. If the SSWG were a forum to elect the 'best and only' library for a given task, that would be a huge issue. What if someone developed a better library for a task that the SSWG has a library for? Should we ignore that, or kick the existing one or, or force them to merge projects? I think all of those options are bad.
I believe that without knowing the future, it's impossible to figure out what will become the best version of a library and therefore the SSWG explicitly states that there might be more than one project that solves a given project (ordered by popularity).

The guarantees given by an SSWG-approved project are at least the ones listed under minimum requirements. Therefore I don't think we should give SSWG-approved packages any magic special powers. I think it's rather the opposite, for SSWG-approved packages we can require a certain set of rules (the minimum requirements), for example regarding licensing. Nobody can stop you from creating a GPL3 licensed driver for a database but the SSWG requires 'Apache 2 by default' (into which I personally read that it should be Apache 2 unless there are good reasons and even then it should be a permissive license comparable and compatible to Apache 2).

Of course you can do that and maybe later at some point you might even want to pitch NIOxyz to the SSWG? The only issue with that would be if everybody who starts writing a 'xyz' client or server names their package and module NIOxyz. Then we have what nobody wants: A set of mutually exclusive 'xyz' implementations.

Please don't get me wrong, this isn't about giving any project (including NIO) special powers, this is about making sure that as many packages as possible can be mixed with others without clashes. For example, let's assume you create a package nio-foo and somebody else creates swift-nio-foo, and both would have a NIOFoo module. At some point, one of you two pitches say nio-foo to be added to the SSWG, it gets approved, and then some of the very popular big web frameworks starts using nio-foo somewhere in their dependency tree. Now, the community has a problem because it's now impossible for anybody to use that popular web framework together with swift-nio-foo which is a real bummer.
Worse, the problem won't be hit by the authors of those libraries, it will be hit by some developer who (possibly long ago) chose swift-nio-foo as their favourite Foo library and a certain poplular web framework as their favourite web framework and at some point that just stops working (because the web framework pulls in the other Foo library).
We'll run into similar issues with packages that provide higher-level APIs for stuff that NIO covers or will cover (HTTP/1&2, SSL/TLS, WebSocket, QUIC, DNS, UDP, TCP, event loops, ...) or very general names. For example we recently added the NIOTestHelpers module to make it easier for developers to verify their NIO code in their own unit tests. If anybody else created such a module in say their database driver, we're in trouble I think.

I don't think this is desirable either but it might happen. Joining efforts to make one great library is certainly better than creating lots of half-baked alternative versions.
And you're right naming a Foo package just FooNIO (instead of NIOFoo) isn't magically solving all problems because two packages might do the same.

I think I opened this discussion way too NIO-centric (because that was the concrete problem I was worried about and well because I work on NIO :slight_smile:). Maybe a better of phrasing the question is:

Should we come up with a certain set of best practises around package/module/type naming and name-spacing? I believe that yes and I reckon the only way to do this is that when a new project is created, github/gitlab/... are searched for potential clashes and we make an active effort not to reuse anybody's package (base)name or their module prefixes.

I agree with that but we need to be careful to not create other clashes. For example I bet there are already lots of packages called HTTP out there. And also: Should this be for an HTTP client or server? Same for Postgres, APNS, and Redis just not quite as bad as HTTP.

As above: I think we should aim to make package/module names unique globally (and not only in the SSWG) if possible and also foresee extensions (after a HTTP client, there might be a HTTP server, ...). This all is not bullet proof because someone could also later start a new project with the same name as an existing one. But as a set of best practises, what about trying to

  • come up with a package (base)name that's unique globally if possible
  • not hogging anyone else's module prefix if possible

These are not meant as absolutes but as something we could aim for. Of course, I know this is not easy, I just wanted to talk about the issue of clashes which will become more and more real the more the ecosystem matures as we'll likely share much more code than we used to (which is great!).

1 Like
19

This is definitely in the charter, and I think was a community misinterpretation in some regards. I also think it's inevitable due to human nature to see a committee that has recommendations and processes behind them to be seen as an Authority Body and the recommendations being "official" in nature, even not by name.

I'm not sure of the value of this? If I see two libraries that implement a Redis client - I would assume they're mutually exclusive, even if they're not to the build tools. I probably wouldn't want to use both in my library / application for making clients, and at some point I would have to write adapter code to even use parts that I might want from RedisLib-1 with RedisLib-2.

I thought the purpose of a framework was to rely on them to pull on the disparate libraries so that you don't have this distinction? Otherwise, they're in no different of a situation than web developers when say, choosing between React, Vue, or Angular

I think having a standard for naming among current or upcoming SSWG libraries is a good idea - but to have it completely unique is extreme overkill. We'll end up with cute or overly long/verbose names rather than very helpful and descriptive names.

I think what we're really getting at in this discussion is that there are problems with Swift tooling in regards to package composability (which, personally, that aspect of the Swift project has been neglected) and it's an uphill battle.

20

As a minor note - my hope is that I would be able to have a Client, and server(s) available from the module eventually.

HTTPClient, HTTP1Server, etc all vended by the HTTP module