Shared mutable state, Sendable, and locks

I've found that as the language improves with regards to memory safety, I've still sometimes found myself reaching for a lock, due to the need to have shared mutable state that is accessible from synchronous contexts that may be used on multiple threads. This is especially relevant when working with legacy code in a large codebase that we're attempting to modernize. I've also encountered this with cross-platform code, where NSLock (which lives in Foundation) may not be the best option, but os_unfair_lock is also not possible to use across the board. I've noticed as well that in Apple's public Swift repos, locks are still frequently used behind the scenes, and it seems there's a fairly elegant cross-platform solution already available, but it's not available to consumers of the API; the internal CriticalManagedState (or a similar idea) must be copied into the local library.

I know that using actors is generally the more recommended way of dealing with shared mutable state, but it's not always practical to do so. I also know that discussing future plans can get into awkward territory around here. That said, I'd love to hear if there are plans to introduce native locks into the Swift standard library, as I think it can fill a vital role especially in cross-platform code. I'd also love to hear others' thoughts on this topic.

3 Likes

Not in the stdlib but for the sake of completeness also worth linking to OSAllocatedUnfairLock.

2 Likes

Is OSAllocatedUnfairLock also available on Swift for Linux and Windows ?
Where I can find a Swift APIs reference document that shows on what platforms a function is available, not only Apple specific platforms ?

To my knowledge, OSAllocatedUnfairLock is only available on Apple platforms, and is not available in any public repos. It's indeed worth bringing up, as it's a great API, but it's a non-starter for cross-platform work.

In my experience, if something isn't in the standard library and is in the documentation materials that come with Xcode, it's pretty safe to assume it's not available on non-Apple platforms.

3 Likes

In server-side applications we typically use NIOLock from SwiftNIO, which is almost dependency-free (except for some atomics stuff) and works on Darwin, Linux and Windows natively.

1 Like

NIOLock itself should be entirely dependency free, and is licensed under Apache 2.0, so users are free to copy it out and use it directly. Note also NIOLockedValueBox, which is broadly similar in API to OSAllocatedUnfairLock.

1 Like

Yes! Maybe I should make it clearer: the implementation of NIOLock is totally dependency-free, but using it from NIO requires dependency on NIOConcurrencyHelpers, which also contain some atomic stuff.

I guess the linker cannot swap out CNIOAtomics even if NIOAtomic is not used? Given that NIOAtomic is deprecated now, is it worth splitting the atomics and lock modules?

1 Like

Correct, this is not something the linker is capable of today.

In principle we could do that, but I suspect it is more trouble than it's worth. It might be better for the community to produce a micro-package for the "I just want a lock" situation.

surely there is no need for yet another micropackage, a dependency-less module in the existing repo with the lock implementation would be sufficient, no?

i have grown increasingly annoyed with the amount of version timelines that need to be reconciled to use the various server side libraries. (testable builds matter!) we need to stop hitting the "create new GitHub repository" button every time we build something new.

Yeah, I agree with you, having a micropackage isn't good. I just think having a whole ecosystem depend on NIO just to get a cross-platform lock isn't great either. Foundation also has a lock, but right now the options are those two big projects. I kinda feel like another project might be a better fit.

Alternatively, someone could propose this for the standard library. The implementation is sitting right there.

3 Likes

A lock that the compiler is aware of and integrates with the concurrency features would be fantastic.

1 Like

having a whole ecosystem depend on NIOCore is bad, but there is nothing wrong with the whole ecosystem depending on the repository NIOCore lives in. (assuming cloning the repo history doesn't become a bottleneck)

I think we do want to add some kind of locking primitives eventually. Probably to the standard library.

I seem to recall @lorentey mentioning it at some point, and @John_McCall has mentioned it as well:

3 Likes

I'm pretty sure that the general mood towards building an "official" lock type is generally positive, and it's only a matter of time. It'd abstract away the underlying differences between platforms as all those floating around implementations do.

To be honest lack of such lock is already felt even by the stdlib with more features having to deal with concurrency and a lock just being the right tool for the job sometimes -- like the recent Observation work had to roll its own lock again which was unfortunate.

12 Likes

The SE-0395 implementation is going to include _ManageCriticalState. It's going to be a part of the stdlib. But it's not public, as implemented. Would be so easy to turn that internal into public...

3 Likes