As pointed out in the pitch thread, a number of libraries are already making use of the _modify accessor:
This thread is to discuss how we should deal with that. The problem is that if you use unstable language features, your library and its releases are by definition not stable:
One idea might be to use the compiler's support for optional language features. For example, the following code compiles:
We could add a feature, like $ModifyAccessorV1, so that these libraries could conditionally support _modify accessors with compatible compilers, and issue a warning while falling back to regular getters and setters if _modify ever gets dropped or substantially revised.
This wouldn't solve the problem of existing releases being unstable, nor the problem of adoption by ABI-stable libraries, but it would allow future releases of source packages to degrade gracefully, while also allowing third-party packages to safely experiment with the feature.
... there is no ambiguity here. Underscored attributes are not source stable and that means you need to deal with the consequences of them not being source stable, whether you're a leaf target or a package.
I would assume that the assumption that _modify is a source-stable feature constitutes programmer error, regardless of how "canon" one's package is to the Swift ecosystem. I like swift-nio's shim: a function meant to emulate the behavior of the _modify accessor in a way that meets the library's needs without resorting to unstable underscored keywords.
If a library author must have the behavior of _modify as it was in a particular version of Swift, that's what #if is for, and should suffice as long as that version of Swift is available to compile against. Once _modify is removed in some newer version of Swift, library authors should transition to the new syntax that then-modern Swift will provide.
I may be missing something, but I don't see this issue as any different from the many syntax transitions that Swift has undergone at every major milestone in its development.
Also, async-http-client uses@inline(__always), but I'm not sure if that counts as first-party.
Now, I'm not blaming the authors of these packages - it is clear that this is a widespread problem, showing a general lack of discipline across the community when it comes to publishing releases making use of unstable language features. But I think developers do and should expect better, especially from first-party packages.
If a package, distributed in source form, makes use of underscored keywords or attributes, it is not guaranteed to compile with the next patch/minor/major release of the compiler. The entire package, and any projects which depend on them, become compromised and are subject to the same lack of stability guarantees.
This is a significant danger lurking in the package ecosystem; it undercuts the idea that Swift is source-stable if so many important packages are publishing releases using unstable language features.
Also, it's just poor form. Why are some packages allowed to use these features and claim to be stable, but my packages cannot? They are able to offer better performance, modularity, and nicer APIs than is possible using actual, stable Swift language features.
This sounds to me like an undeserved blame. People work with the tools that are available. I would not look at people when the tools are lacking.
EDIT: my own sentence may sound like a blame as well. Apologies. I just really wish we'd talk about problem people have, instead of things people do in an incorrect way.
The goal here is not to blame anybody, but to accept that:
This is a problem
We need to fix it before it gets worse
We need to prevent it happening again
The obvious solution to (2) is to work to get these features through evolution, so they can become official, supported parts of the language. Solving (3) is perhaps a bit trickier, but could involve making better use of the feature-test abilities present in more recent compilers, as mentioned in the OP. That way, packages could continue to compile, perhaps with degraded performance, if these unstable features ever disappeared or were significantly changed.
Regarding ③, for unstable/experimental features, Kotlin has the concept of @RequiresOptIn which force the user of annotated API to pass compiler flags to enable them.
In our case, forcing usage of compiler flag would at least force package maintainer to distribute their packages using the SwiftSetting.unsafeFlag directive, which will prevent accidental release of source using unsupported feature.
The _enclosingInstance subscript variant for a property wrapper comes to mind too.
I think the problem is Swift keeps adding interesting features without standardizing them. Then Apple (generally) is quick to adopt those non-standard features in their own frameworks, which creates pressure for others to do the same. People write articles to explain how they work (like this one where I found a mention of _enclosingInstance) and various people start using it.
I don't think an underscore means much to most people. A different compiler flag to enable each feature would be more effective, but in the end if there's demand people will adopt the feature despite the nuisances you throw at them.
Once the feature is present in the compiler, it reduces the pressure for standardizing it because those who needed it will now use it despite the underscore. And we end up in this limbo world of non-standard features accumulating and no one having a self-interest in doing the work to standardize them.
Currently, packages developed by Apple have access to special dialects which enable more expressive, more efficient APIs than are technically possible with the official Swift language, and IMO that is not acceptable. It does not seem honest to preach to third-party developers that they should limit their distributed source code to official language features, when first-party developers of open-source packages do not respect the same rules.
It doesn't seem fair to single out any particular people giving that advice, but you can look at Ben's reply quoted above, and a quick search should reveal many more - from @_cdecl to @_disfavoredOverload to @_implements to @_implementationOnly and @_exported imports, developers acknowledge that these things exist, and could solve their problems, but that they aren't allowed to use them.
Developers even police each-other to help ensure that a reliance on these features doesn't spread through the ecosystem.
Apple absolutely does this (e.g. SwiftUI using result builders while they were still being designed), but I don’t think swift-atomics is an example of it. Packages that are part of the Swift project are in a better position to align their releases with compiler changes.
Rust has a formalized notion of opting in to unstable features, while Swift does not. But having that notion doesn’t make the features any less unstable. You can argue that unstable features should not exist, or at least that they should not be indefinitely unstable (unless they’re limited to the stdlib or something), but that’s somewhat separate from which packages get the “benefit” of such features.
I may be completely wrong, but I personally don't see anything in Ben's reply quote above that could be interpreted as limiting my use of such features.
What do you mean by "aren't allowed to use them"? This is the first time I'm hearing I'm not allowed to use these features. I actually rely on @_exported in a few of libraries I co-maintain quite extensively.
I never witnessed policing that you're describing here, but maybe it's just me.
*raises hand* When I was at Apple I definitely warned people not to use unstable features, and I continue to do so. Some features are more unstable than others, and the compiler team isn’t unaware of their use in the wild, but ultimately they are not part of the main source or binary compatibility goals of the project. That includes @_exported.
Sure, but verbal warning is very far from "limiting" or "not allowing". I'm warned, but I continue to use them as I don't see any viable alternative. I would only agree with "not allowed" interpretation if use of such features triggered a compiler error.
I think this is a bit of a fallacy - it does not matter who you are, or who you work for, or how closely you watch the compiler. If you release source code which uses underscored language features, that source code is not stable.
And for packages distributed as source code, that's a problem. Somebody might be using a pinned or mirrored copy of a particular release, and that code can break with a minor compiler update. In turn, this makes the entire ecosystem more fragile and dilutes the promise that Swift is a source-stable language.
An atomics library is always going to be tied intimately to the compiler because of its interactions with the ordinary semantics of code. Inter-thread ordering and memory models are language-level concepts.
Sure, and the compiler team may be happy to support @_semantics("atomics.requires_constant_orderings") for this purpose indefinitely. There is an argument that, if it is indeed supported, it should be a regular attribute with documented semantics and no underscore, but I'm happy to leave that discussion to focus on the broader point. It's possible that compile-time evaluable function parameters (which have been pitched) will solve this particular case anyway.
But swift-collections, swift-system, and swift-numerics all use (at least) @inline(__always). Some also use _modify, some use @_exported imports, and there are might be more that I didn't think to search for.
If these packages, which are not tied intimately to the compiler, are unable to express their APIs or meet their performance targets using only official Swift, that is:
A damning indictment of the language. You shouldn't need to use unstable features to implement a great Deque or OrderedSet type, or to organise your FilePath type's API.
An indication for other developers that it's okay to shift the burden of source-stability on to their clients. To quote Ben's response in the other thread:
This is why I disagree so much with the "I work close to the compiler; I don't need to care" philosophy. It won't be the developers of these packages who suffer if their older code stops compiling - it'll be the developers of other packages, who (rightly) assumed those dependencies were source-stable. That will affect their clients, and so on, down the chain.
But I do mean what I said previously about not blaming individual library authors. It should be clear by now that this is a widespread problem. Nobody takes it seriously that these features are unstable.
I started this thread (and the previous discussion in the _modify thread) because I was apprehensive about launching a stable 1.0 release of WebURL while it used _modify. Now? I pretty much don't care; it's no longer a blocking issue. Nobody else is playing by the rules, either, and that's a sad situation. I wish the core team would have more of a response.
In concrete terms, what I would like to see from the core team is a statement acknowledging that we are not making good on our promises of source stability, and that scrubbing these underscored features has to be a top priority. For the next version of Swift, before adding new features, we need to focus on stabilising the features that apps and libraries are already depending on. For backwards compatibility, we need a way to support these features in conditional compilation blocks, and for the future, we need better tools in the compiler/package manager to flag the use of unstable language features.
Yes. These backdoor hooks are never used by any of the code I work with (except Swift components themselves), and the one package that cannot live without @_exported has been sitting in 0.x versions for years for no other reason than that.
That said, I am not opposed to backdoor hooks being used by the same people or organizations that expose them. The same packages I work with which refuse to use Swift’s underscored guts tent to expose similar sorts of hooks that are used secretly by other software belonging to the same organization. And so I do not think the swift- packages you mention are doing anything wrong.
Those situations are not remotely similar. A special API to be used by another module is not the same thing as using an unstable language feature. Your special API doesn't affect source stability - those are just regular Swift functions and types, which you discourage others from using. Unstable language features mean the compiler team must forever support those features using the exact semantics and spelling they have now, otherwise old code will fail to compile with newer versions of the compiler.
I don't see how the core team can say, on the one hand, that these features are not source stable, and then on the other hand remain silent while countless packages use these features while claiming to be source stable.
Imagine if Swift 6 no longer compiled code using @_specialize or _modify, and run through that scenario in your mind. Suddenly, NIO and swift-system will fail to compile, and anybody who didn't update would see their packages break, and all downstream packages would also break. It's basically untenable; the compiler team is forced to keep supporting it, even if they don't want to, and even if the official feature ends up looking or working differently -- which is exactly what they hoped to avoid by warning that the features are not stable.
Not to mention that this creates a special dialect of Swift (another thing they hoped to avoid), where Apple is able to create Swift libraries that nobody else can -- just by virtue of being Apple, and for no other reason. That's what you're saying. I, for one, certainly do not agree with that position.
I thought NIO was immune to this, as they tend to be very strict about underscored features. So imagine my surprise when I saw