“Installer size” is far less important than “basic language operations work,” so if we have to regress installer size, I don’t see why we wouldn’t do that. If there’s another way to handle this, great, but as Swift pushes into more platforms we will run into more missing and broken C and C++ features, and need to be willing to bundle our own implementations when necessary.
10 Likes
OK, here are some concrete steps that I can start making on my part:
-
Ship a feature release for Swift Atomics, switching its focus from obsolete Swift compilers to modern ones, and delivering some sorely missing new functionality. This is fully in progress now: see the thread [Announcement] Planning for Swift Atomics v1.1
-
Replace C atomics with llvm intrinsics that are native to Swift, initially as an experimental package variant. Ideally wrappers of these intrinsics would then ship as underscored additions in the stdlib, enabling the package to wean itself off of its kludgy _AtomicsShims module. This would let people who are currently blocked on SPM's inconsistent support for C modules to start using the package. (Including, e.g., people who want to use it on platforms such as Swift Playgrounds, or people who struggle with weird build system problems elsewhere.)
Doing this will make it far less important to ship this package as part of the toolchain distribution, but, of course, it will not eliminate the urgent need for atomics directly shipping in the Swift Standard Library. (I don't really get the point of, e.g., shipping
@unchecked Sendablewhile we don't have any synchronization constructs beyondisKnownUniquelyReferencedin the stdlib.) -
Meanwhile, I'll keep loudly crying about the continued lack of language support for properly modeling these things. The idea of accessing blocks of memory with a stable address is a really promising one. It has been around since Nov 2019 at least.
I'm a library/frameworks engineer with way too many things on my plate already, so unfortunately it doesn't seem likely I would be able to contribute a compiler patch myself -- but I'd jump on any opportunity to try integrating things...
If anyone is currently twiddling their thumbs looking for a small/mid-scale compiler project, implementing Joe's idea would be a good way to make an enormous impact with relatively small investment!
13 Likes
Yes! this makes my life so much easier. i'll be trying out the new builds this week.
Oh, I may have given the wrong impression -- getting rid of the C module isn't going to be a weekend project. I can get started working on step 2, but the end result will definitely not ship in 1.1.0.
The crucial llvm intrinsics aren't accessible to packages, and the stdlib doesn't expose wrappers for them. Either/both of these are solvable problems technically, but they both require toolchain changes, so they won't happen overnight. (Exposing stdlib wrappers would be far preferable to exposing the raw intrinsics, because the stdlib is aware of the general concept of source stability, and by and large people try to avoid randomly breaking things there -- unlike in the Builtin module.)
If it is possible to set custom compiler flags in Swift Playgrounds, then perhaps there is a way to come up with a shorter-term workaround, but it is unlikely it would be possible to do that while also continuing to pick up swift-atomics as a normal package dependency.
1 Like
my two sentences above are actually disconnected. i don't expect the playground fix on monday. i'll still be trying the new builds :)
1 Like
i'm still a bit confused on what move-only types will do for me. My mental model for the situation I have is: i have two or more threads each racing to change a pointer to point at their version of some data. Once the pointer is set, many threads may read, but none may write. Writer threads which fail know they failed because they tried to update with a CAS.
Arc-style non-exclusive ownership seems the right model for this. The Law of Exclusion would have the compiler verify that only one thread could modify. This seems to behave as a pessimistic lock when what I want in this case is optimistic.
What am i missing that exclusive ownership via a move-only type would give me?
@taylorswift mentioned that there are/or have been? problems with Swift atomics on Windows — of course this is not good for such an important package, and even less good if distributed as part of the toolchain. (Sorry for bothering everyone with this Windows topic again…)
Update: Just seen the new topic Windows Testing Required on Pull Request so maybe my remark is kind of superfluous (or just a reminder).
Ok, I am officially thrilled @lorenty. swift-atomics 1.1 works in playgrounds and I don't have to continue to maintain a fork. Thanks for all the work here!
Now on to understanding how move-only types are going to help me write better atomic algorithms.
6 Likes
Huh! That's great to hear, but I wouldn't have expected 1.1 would noticeably change things for playgrounds -- the package is still heavily relying on a C module, and as such it is still subject to limitations/bugs in this area.
1 Like
Sigh... sorry, @lorenty my bad. I was using atomics in a workspace and even though I had changed my package to point at 1.1.0, there was an app in the workspace that was still pointing at my fork and that caused the playgrounds to use the wrong version. The problem with playgrounds still exists.
The Playground breakage is not an issue with Atomics being a C module per se. It is completely that the swift_retain_n and swift_release_n cannot be seen by the compiler bundled into playgrounds. My fork gets around that by looping over the retain and the release as suggested in your comment here. It's lousy from a performance standpoint, and when prepping my package for release I point back to apple/swift-atomics, but in playgrounds performance is not that big a deal.
As per the original topic of this thread, this could easily be fixed by either:
- moving Atomics into the std lib or
- having Unmanaged in the std lib expose [retain|release]
_n in the same way it currently exposes [retain|release]
in re: #2 above, as I think about it, that actually does not seem like an awful idea. Unmanaged exists to provide unsafe access to retain/release, so as long as we're being unsafe, might as well embrace the entire thing.
1 Like
Yes, adding Unmanaged._retain(by:)/_release(by:) to the stdlib is the right way to fix that issue.
These are extraordinarily niche entry points, so I wouldn’t bother proposing them as public API — unless someone invents more use cases for them, leaving them underscored is a-okay from where I’m looking.
Patches are, as always, welcome! The tricky part here is to expose these runtime entry points without confusing the compiler: it doesn’t currently expect to see them directly used by user code.
1 Like
The other thought I have had is to make this injectable. Atomics just needs some function that can bump retain counts up and down by values other than 1.
I'll probably modify my fork to require that the consuming application provide that function. Then I can track the atomics library easily and not have different package dependencies between the playgrounds I use as documentation for the underlying package and the package itself.
The interesting thing about this is issue is that it hits directly at the point of this thread. If Atomics were actually implemented as move-only types, then we wouldn't have retain/release to worry about to begin with 
I'm still not seeing how my use cases can avoid being ARC'd however. When the new borrowable/move-only/non-copyable raw memory feature @Joe_Groff discusses above is implemented I hope to no longer see through this glass darkly.
1 Like