This was originally mentioned on the swift-nio issue tracker here, but the issue tracker doesn't necessarily get a load of traffic so I wanted to bring this up on the forums.
Specifically, it seems like it would be useful to try to come to a community understanding of how semantic versioning should map to Swift. Swift is in a uniquely difficult place becuase of the way it allows users to extend arbitrary types.
For the rest of this post I'll use a hypothetical package A, that depends on packages B and C. In particular, this package can run into the following cases:
Package A may conform a type from Package B to a protocol from Package C. If Package B later adds that conformance, this will cause a compile error. (This is very common in the Swift community in the case where Package C == the Swift standard library).
Package A may extend a type from Package B with a method or computed property. At some later time, Package C publicly extends the same type from Package B with the same method or computed property. This will cause a compile error in package A.
These possibilities require us to revise what is commonly accepted as a "breaking" change in SemVer for the Swift community. Given that the Swift language entirely allows both 1) and 2) above, in principle it becomes almost impossible to make a SemVer-minor change to a Swift library, as it could always conflict with a hypothetical public extension in a third-party library that depends on it. This interpretation makes SemVer almost entirely useless for Swift, as there will never be a SemVer-minor: only SemVer-patch and SemVer-major.
Because we don't want SemVer to be useless, we should as a community come up with a set of guidelines as to what kinds of changes are safe for SemVer-minor. This will need to involve defining a set of best-practices regarding type extensions such that software that follows them will safely get the SemVer guarantees.
I'd like to use this thread to solicit feedback as to what those guidelines should be.
The NIO team have noted the following cases should probably be considered violations of best-practices.
Conforming Types You Don't Own To Protocols You Don't Own
In general, if you conform someone else's type to someone else's protocol you are at risk of being broken in a minor version bump. This is because the owner of the type (or the protocol) may choose to add their own conformance, which will conflict with yours.
For this reason, you should never conform a type you don't own to a protocol you don't own unless you're willing to encounter breakage in a SemVer minor release. Again, note that libraries do not own standard library types or standard library protocols.
If your codebase defines the type or the protocol, adding a conformance is fine. You only have to own one of the parts.
Public Extending Types With Common Method Signatures
This is an extension of the first case. If you provide a public extension of a type you do not own, and that extension provides methods or computed properties that both a) have a simple name and b) use only types/protocols your code does not own, you are at risk of breaking either your own or other people's code.
If you wish to provide a public extension to someone else's type, the following situations are fine:
Use a name that is unlikely to collide. For example, provide a prefix to all method names and computed property names that relates to your package in some way.
Have your method signature include a type/protocol that your code owns. That will make it impossible for someone else to add the same method signature unless they themselves break this rule.
I disagree with that. You "just" have to carefully design your API with proper extension points and avoid SemVer problematic constructs in the API.
In a way the reverse of what you expect consumers of your API to do
It's been my policy to completely ignore the fact people can add methods in extensions when considering version numbers. If we started to consider that, then yes, you would be bumping major versions every time you add a method or property. Same goes for adding a protocol conformance, although that's been rarer in my experiences.
hmm, I don't think I can reproduce this: If I add this
extension ByteBuffer {
mutating func set<S: Sequence>(bytes: S, at index: Int) -> Int where S.Element == UInt8 {
return 5
}
}
which clearly clashes with a ByteBuffer from NIO defined method and I call precondition(5 == self.b.set(bytes: DispatchData.empty, at: 0)) that works just fine. So Swift correctly picks up the one defined in this module.
It would be great is Swift had a disambiguation mechanism for this stuff built in so that if there was a conflict, users have a way of disambiguating without the framework makers having to use gross prefixes or containers. This seems like a feature that could come along with namespace or submodule support.
Indeed it would. Unfortunately, until such time as Swift ships such a feature, I think we still need to work out a set of guidelines about what the community can and cannot safely do with most Swift libraries.
I disagree that it is in the spirit of semantic versioning to restrict a downstream client for what an API author can foresee. In this case, new public conformances to protocols should really be major version bumps, IMHO.
@millenomi unfortunately in Swift that affects every new (public) method on any public type and every new top-level type and function.
Say you add a public function to one of our types and we happen to add one with the same name, your code wouldn’t compile anymore. Interpreting SemVer that way would mean that pretty much any API addition becomes a major release.
IMO this is sufficiently well covered in the SemVer spec. Yes, you need to plan ahead and lock down your API before releasing it. This is what SemVer is all about. You can't iterate on it on the back of your users and risk DLL hell.
Why do you even need to add a public method or a public type to 1.x? This can go into 2.x. If you really really need to have new stuff, there are other ways to accomplish it, for example using additional modules or proper extension points.
P.S.: I still think outlining the rules is useful, it just doesn't have anything to do with SemVer
In most cases I think users are interested in when the API actually breaks, ie. method renamed, parameter/function/type removed, ... That is what I think constitutes a backwards incompatible change.
If you declare even the tiniest addition as a backwards incompatible change, then there's no digit left in SemVer express that something is actually a breaking change. So in the end, following that model in Swift would lead to everybody having a different locked down major version which leads to lots of incompatibility.
But the API does actually break in Swift if you add methods. (Funny enough we seem to agree what backwards means if we are talking about sub minor versions.)
NIO is currently releasing a new minor version every two weeks or so. Is there a reason for this? Just marketing? Why can't you just fix bugs in 1.x and delay additions and other breaking changes to 2.x?
As Jon said there is no particular reason to be afraid of major releases. After all that is the model Swift itself is doing (every year a new breaking release).
Totally agreed @Jon_Shier. We're not afraid of releasing a new major at all, in fact we're already collecting PRs for NIO 2.0.0. But I believe it's also important to not release new major versions for not actually breaking changes. Because then we had no 'automated way' of communicating that something is actually a breaking change.
For the ecosystem I believe it's good if we try to restrict the number of breaking versions because then a large number of packages using from: "1.0.0" can work together in one large package graph. Had we released a new major version for every addition to the NIO API, then today we would have released NIO 8.0.0. And some of our clients would probably have a Package.swift stating from: "1.0.0", others from: "2.0.0", yet others from: "7.0.0" despite the fact that they're all totally compatible and we haven't removed a single type, function, parameter, or module nor changed the 'meaning' of them.
So basically we just collecting breaking API changes in bulk and as soon as we have a good reason, we'll release them all together as NIO 2.0.0.
@Helge_Hess1 we want to be a healthy open-source project and we're literally at the beginning. So we want to encourage people to contribute and give them access to the latest & greatest as soon as we have it. It's quite discouraging if you contribute something but have to wait half a year to actually use it in a released version.
From what I understand, the compilation will only fail iff any of the following is true
you extend on of NIO's types with a public method/property,
you conform one of our types to a protocol that you don't control
or we get an unfortunate clash of a public symbol
(2) is definitely bad style and asking for trouble; (1) kind of smells too (you can add methods, just not public ones) and (3) is preventable by using (the arguably not so great) import struct NIO.ByteBuffer syntax.
I wish there were less pitfalls in Swift and we had qualified imports etc but unfortunately we don't have that (yet).
So I think there is a workable model for Swift without bumping the major version for any API addition.
Out of interest, can you or @millenomi outline what kinds of changes would constitute semver minor in your reading of the spec? What kind of change can we make to NIO that would justify a x.y+1 release?
My concern is that in the model where extending our own types is semver major I can’t think of any, so I’d like to know where I have gotten lost.
Yes, I absolutely understand this and fully support that (also your desire to iterate fast - lets be honest and admit that NIO is not 1.x but 0.x, and SemVer actually has good rules for this).
That is why I'm saying that we need those best practices and a SwiftVer effort (and maybe even SPM support for it, with the relaxed rules). This could live alongside SemVer which is very necessary for package resolution and linking. You really help no one if complex package graphs fail to build or run (a "it is all your fault by extending xyz" is not going to help when it happens). And I do really hope that we eventually have a healthy Swift package market comparable to npm. (Without the issues and leftpad of course ;-) )
I have no immediate good answer for that. Maybe tag stuff as swiftver_1_x_y, and keep 1.x.y for SemVer?
For now I would do this: Just stop calling it SemVer, it simply is not SemVer and doesn't provide the guarantees it gives. Instead just do your versioning thing the way you already do it and explain it.
Then lets see how we can improve the situation.
Since NIO is such a low level component, it is indeed very difficult to envision changes that would warrant a minor release bump. What API would you want to add to a 1.x version of a base socket library? I really can't imagine any if there was a designed API in advance. A new Channel? No, we do SSL and HTTP2 in separate modules. Which is good.
E.g. the ByteBufferView may be nice. Do I need this in a 1.x? No. I'd rather have stability.
Actually an example for a minor would be adding HTTP2 support. The user facing API could be exactly the same.
In the C and Java worlds you often have explicit extension points for environments that require hard API compatibility, for example the Apache API, or the COM variants (OLE, XPCOM etc). Or OSGI in Java.
I only read the announcements, but the Vapor "services" stuff seems to go in that direction. You define a common protocol or base class and can extend based upon that.
But that doesn't really work for a base library like NIO.
