yes, this was my thinking as well.
we can already do this in a crude fashion for sendable classes, what is needed is a way to specify nonisolated var for a stored actor property.
2 Likes
Back in 2020, we promised that noncopyable types would be The Right Way to get atomics, locks, etc in Swift.
All of these approaches have serious shortcomings, and they're likely to be quickly superseded by move-only atomics in the future. How far that future is from the present is an open question, and having some sort of low-level API for atomic primitives could still have a small niche for advanced users even with move-only atomics as the primary interface.
After three long years, non-copyable types are now becoming a real thing. However, modeling synchronization primitives isn't a flagship use case for them; in fact, properly modeling these things continues to be an elusive thing.
It has been three years; I do not think it is tenable that Swift still does not provide such constructs.
I think concerns about not putting known-suboptimal constructs in the stdlib should come with an expiration date. The only thing we achieve by not having synchronization primitives in the Swift Standard Library at this point is the proliferation of half-baked, ill-considered attempts at implementing the same, throughout the entire ecosystem.
I deeply regret not pushing harder for including ManagedAtomic/UnsafeAtomic in the stdlib.
10 Likes
I don't think this is true. Once the implementation of SE-0390 is mature, then we're really only missing one thing necessary to model atomics or locks with move-only types, namely the ability to put a block of raw memory inside such a type. Figuring that out, either in the general case or in a special case just for atomics, would be a great flagship use for move-only types.
10 Likes
Let's make that happen! Every single day I'm getting blocked by not being able to use atomics, and the questionable alternative designs that I use to avoid them are starting to pile up too high and are getting embedded too deeply for comfort.
8 Likes
thank you Karoy, for the honest assessment of the current state of affairs with multi-platform atomics. unfortunately atomics today only work comprehensively on Apple platforms, and that is inhibiting the development of multi-platform concurrent libraries.
there are two things that can be done right now to improve the multi-platform experience for swift atomics:
-
enabling doublewide atomics on non-Apple platforms (PR), which will allow us to use the official swift-atomics package on linux (instead of a constellation of everyone’s personal forks), and
-
merging this PR, which i am told will allow us to use swift-atomics on Windows.
i really hope to see some movement on these items soon.
5 Likes
Ah, thanks for reminding me about that PR.
2 Likes
As Saleem pointed out, this PR would regress the installer size, so we need to rework it in some way. However, this is the correct direction. For working around the problem, see here.
FYI, if you upgraded to Visual Studio 17.5, you can build Atomics without modifying the toolchain now thanks to native stdatomic.h support.
“Installer size” is far less important than “basic language operations work,” so if we have to regress installer size, I don’t see why we wouldn’t do that. If there’s another way to handle this, great, but as Swift pushes into more platforms we will run into more missing and broken C and C++ features, and need to be willing to bundle our own implementations when necessary.
10 Likes
OK, here are some concrete steps that I can start making on my part:
-
Ship a feature release for Swift Atomics, switching its focus from obsolete Swift compilers to modern ones, and delivering some sorely missing new functionality. This is fully in progress now: see the thread [Announcement] Planning for Swift Atomics v1.1
-
Replace C atomics with llvm intrinsics that are native to Swift, initially as an experimental package variant. Ideally wrappers of these intrinsics would then ship as underscored additions in the stdlib, enabling the package to wean itself off of its kludgy _AtomicsShims module. This would let people who are currently blocked on SPM's inconsistent support for C modules to start using the package. (Including, e.g., people who want to use it on platforms such as Swift Playgrounds, or people who struggle with weird build system problems elsewhere.)
Doing this will make it far less important to ship this package as part of the toolchain distribution, but, of course, it will not eliminate the urgent need for atomics directly shipping in the Swift Standard Library. (I don't really get the point of, e.g., shipping
@unchecked Sendablewhile we don't have any synchronization constructs beyondisKnownUniquelyReferencedin the stdlib.) -
Meanwhile, I'll keep loudly crying about the continued lack of language support for properly modeling these things. The idea of accessing blocks of memory with a stable address is a really promising one. It has been around since Nov 2019 at least.
I'm a library/frameworks engineer with way too many things on my plate already, so unfortunately it doesn't seem likely I would be able to contribute a compiler patch myself -- but I'd jump on any opportunity to try integrating things...
If anyone is currently twiddling their thumbs looking for a small/mid-scale compiler project, implementing Joe's idea would be a good way to make an enormous impact with relatively small investment!
13 Likes
Yes! this makes my life so much easier. i'll be trying out the new builds this week.
Oh, I may have given the wrong impression -- getting rid of the C module isn't going to be a weekend project. I can get started working on step 2, but the end result will definitely not ship in 1.1.0.
The crucial llvm intrinsics aren't accessible to packages, and the stdlib doesn't expose wrappers for them. Either/both of these are solvable problems technically, but they both require toolchain changes, so they won't happen overnight. (Exposing stdlib wrappers would be far preferable to exposing the raw intrinsics, because the stdlib is aware of the general concept of source stability, and by and large people try to avoid randomly breaking things there -- unlike in the Builtin module.)
If it is possible to set custom compiler flags in Swift Playgrounds, then perhaps there is a way to come up with a shorter-term workaround, but it is unlikely it would be possible to do that while also continuing to pick up swift-atomics as a normal package dependency.
1 Like
my two sentences above are actually disconnected. i don't expect the playground fix on monday. i'll still be trying the new builds :)
1 Like
i'm still a bit confused on what move-only types will do for me. My mental model for the situation I have is: i have two or more threads each racing to change a pointer to point at their version of some data. Once the pointer is set, many threads may read, but none may write. Writer threads which fail know they failed because they tried to update with a CAS.
Arc-style non-exclusive ownership seems the right model for this. The Law of Exclusion would have the compiler verify that only one thread could modify. This seems to behave as a pessimistic lock when what I want in this case is optimistic.
What am i missing that exclusive ownership via a move-only type would give me?
@taylorswift mentioned that there are/or have been? problems with Swift atomics on Windows — of course this is not good for such an important package, and even less good if distributed as part of the toolchain. (Sorry for bothering everyone with this Windows topic again…)
Update: Just seen the new topic Windows Testing Required on Pull Request so maybe my remark is kind of superfluous (or just a reminder).
Ok, I am officially thrilled @lorenty. swift-atomics 1.1 works in playgrounds and I don't have to continue to maintain a fork. Thanks for all the work here!
Now on to understanding how move-only types are going to help me write better atomic algorithms.
6 Likes
Huh! That's great to hear, but I wouldn't have expected 1.1 would noticeably change things for playgrounds -- the package is still heavily relying on a C module, and as such it is still subject to limitations/bugs in this area.
1 Like
Sigh... sorry, @lorenty my bad. I was using atomics in a workspace and even though I had changed my package to point at 1.1.0, there was an app in the workspace that was still pointing at my fork and that caused the playgrounds to use the wrong version. The problem with playgrounds still exists.
The Playground breakage is not an issue with Atomics being a C module per se. It is completely that the swift_retain_n and swift_release_n cannot be seen by the compiler bundled into playgrounds. My fork gets around that by looping over the retain and the release as suggested in your comment here. It's lousy from a performance standpoint, and when prepping my package for release I point back to apple/swift-atomics, but in playgrounds performance is not that big a deal.
As per the original topic of this thread, this could easily be fixed by either:
- moving Atomics into the std lib or
- having Unmanaged in the std lib expose [retain|release]
_n in the same way it currently exposes [retain|release]
in re: #2 above, as I think about it, that actually does not seem like an awful idea. Unmanaged exists to provide unsafe access to retain/release, so as long as we're being unsafe, might as well embrace the entire thing.
1 Like
Yes, adding Unmanaged._retain(by:)/_release(by:) to the stdlib is the right way to fix that issue.
These are extraordinarily niche entry points, so I wouldn’t bother proposing them as public API — unless someone invents more use cases for them, leaving them underscored is a-okay from where I’m looking.
Patches are, as always, welcome! The tricky part here is to expose these runtime entry points without confusing the compiler: it doesn’t currently expect to see them directly used by user code.
1 Like
The other thought I have had is to make this injectable. Atomics just needs some function that can bump retain counts up and down by values other than 1.
I'll probably modify my fork to require that the consuming application provide that function. Then I can track the atomics library easily and not have different package dependencies between the playgrounds I use as documentation for the underlying package and the package itself.
The interesting thing about this is issue is that it hits directly at the point of this thread. If Atomics were actually implemented as move-only types, then we wouldn't have retain/release to worry about to begin with 
I'm still not seeing how my use cases can avoid being ARC'd however. When the new borrowable/move-only/non-copyable raw memory feature @Joe_Groff discusses above is implemented I hope to no longer see through this glass darkly.
1 Like