SE-0433: Synchronous Mutual Exclusion Lock

The following pattern is common but impossible with closure based API (at least for now):

let valueA: A
let valueB: B
let valueC: C
mutex.withLock {
  valueA = ...
  valueB = ...
  valueC = ...
}

Are there any workarounds except returning a large tuple or using variables of implicitlyUnwrapedOptional types instead of let? Or is it recommended to wait for Mutex.Guard API?

As @FranzBusch mentioned above – "One thing that surprised me in the proposed API is the requirement for the Result of both with methods to be Sendable."
I want to clarify some points:

  1. It may be feasible to allow returning non-Sendable value if its Type != State
let mutex: Mutex<NonSendableA> =  ...
let value = mutex.withLock { state in
  return NonSendableB(state.property)
}
let value2 = mutex.withLock { state in
  let foo = Int.random()
  state.updateWith(foo)
  return NonSendableB(foo)
}

Any plans?
2. What if we try to simply return the state?

mutex.withLock { state in
  return state
}

This case varies whether the Sendable constraint is lifted and / or Result become transferable. Nevertheless, it is possible the State to be a Sendable and move-only-Type.
3. If I understand it the right way, move-only ~Copyable types are not copied implicitly by compiler. Having a mutex with move-only type, what happens if we try to explicitly copy the state or the Mutex itself? In other words words, is there a way to make a copy explicitly or compiler will ultimately deny such attempts?

The last one is about mutability. Mutex will be decorated with the @_staticExclusiveOnly, so it is impossible to declare a var.
How then willSet / didSet observers can be done?

Will it trigger set / get of enclosing instance?

struct Foo {
  let state: Mutex<State>
}

let foo = Foo() // immutable
foo.state.withLock { ... } // is it allowed or a compiler error?

var foo2: Foo {
  get {}
  set {}
}
foo2.state.withLock { ... } // will it trigger `get / set`?
3 Likes

You can do:

let (valueA, valueB, valueC) = mutex.withLock {
  return (..., ..., ...)
}

You would have to implement such behavior yourself. It seems to me like there are too many possibilities of how the observation logic would interact with the boundaries of the lock guard for there to be a one-size-fits-all feature for this.

Because Mutex is ~Copyable, Foo also has to be in order to hold a Mutex. let in this situation doesn't really mean "immutable" but that "nobody can get exclusive access". You can still mutate the guarded state using foo.state.withLock { ... } because the lock guard dynamically ensures that you acquired exclusive access to the guarded state, even if you started from having only shared access to the state. A getter like on foo2 will return a new temporary foo, so foo2.state.withLock {} will create a new temporary Foo, access the state of it, and then throw the temporary Foo away.

2 Likes

How is it possible for a "new temporary Foo" be created? I didn't made Foo a ~Sendable explicitly but anyway Foo itself is also a move-only type.

A getter returns a new value, which the caller takes ownership of. For a noncopyable type, that means that it generally can only either construct a new value or move an existing value out of some place:

var foo2: Foo {
  get { return Foo(...) }
}
1 Like

How far will this back-deploy?

(We ended up reimplementing OSAllocatedUnfairLock already because it didn't support iOS 15)

2 Likes

Mutex will share a lot of the same restrictions other new types in the standard library have in that they are only available since their first release.

Can it be used inside of _modify without lock()/unlock() methods? or what's the best approach here?

1 Like

+1, think it's much needed in lot's of places, and design is good. :+1:

Would be interesting to see how community will adopt new stuff, having Atomic, Mutex and actors soon. :slightly_smiling_face:

1 Like

I think it would be useful, especially for those coming to Mutex with Swift as their first/only language, for the code example in the documentation comments to include the method that reads/returns the protected cached value.

class Manager {
  let cache = Mutex<[String: Resource]>([:])
  
  func saveResouce(_ resource: Resouce, as key: Key) {
    cache.withLock {
      $0[key] = resource
    }
  }

  // Add the method to read / return the resource
  // Is this correct?
  func readResource(for key: Key) -> Resource {
     cache.withLock {
       return $0[key]
    }
  }
}
7 Likes

Yeah, we can choose to make the result transferring instead of requiring it to be Sendable since all withLock does is return the result of the closure as the overall result of the function; it doesn't actually need to use the result value concurrently.

13 Likes

My first ultra basic try with Mutex, using the swift-PR-71383-1207-osx.tar.gz toolchain.

import Foundation
import Synchronization

let counter = Mutex<Int>(0)

func incrementCounter() {
    counter.withLock { counterValue in
        counterValue += 1
    }
}

func readCounter() -> Int {
    counter.withLock { counterValue in
        return counterValue
    }
}

incrementCounter()
print(readCounter())

I have the following error: 'counter' is borrowed and cannot be consumed.
What is wrong ?

2 Likes

This is the same issue described in this thread: Are atomic globals supposed to work?. It's a known bug, and a pretty severe one, but yeah globals variables as well as static variables do not currently work with Atomic, Mutex, or even something simple like:

struct Hello: ~Copyable {
  let x: Any

  borrowing func hello() {}
}

let x = Hello(x: 123)

func something() {
  x.hello() // error: 'x' is borrowed and cannot be consumed
}
4 Likes

Understood thanks !
Otherwise it seems to me that Mutex will really be welcome.

I read with interest the section "Differences between mutexes and actors". It's very technical and not sure to understand everything immediately and deduce in which cases Mutex will be preferable to actor. It would be nice to have some simple examples showing where Mutex is preferable to actor and vice versa.

2 Likes

IMHO think description is actually good for proposal as it should cover rather technical questions.

In my mind Atomic, Mutex and actors are better understood when thinking in the scale from low level to high level—atomics for low-level system programming; mutex for protecting some shared state, maybe with a bit of async (e.g. inside classes); and actors as an abstraction are good for designing an app overall.

I'm in two minds about this. It's good to formalize common ad hoc patterns, and there are no doubt countless implementations of locks in Swift. Having a single standard is good, just like when they introduced Result.

My doubts are that we have been down this path before, and it doesn't end well. As already mentioned by others, if you introduce Mutex, pretty soon you need RecursiveMutex, etc etc It's a rabbit hole of deadlocks and other concurrency issues, and was the motivation for other approaches like async/await.

This makes me reserved about it, because introducing it formally also endorses this approach in general, and I'm not sure that we should do that for 90% of app programming Swift developers. There are cases where you need this low-level high performance concurrency, but in my experience, in everyday app level development, they are uncommon. If I ever reach for a lock, I check myself and ask if I really need it, or am I actually just taking on technical debt.

Personally, I would love a Mutex like solution inside of actors. As mentioned in the proposal, async/await and Actors suffer from reentrance and interleaving, and this introduces a new category of race. It would be great if you could mark an Actor func as being @nonreentrant, whereby you could assume that only one instance of that func can be in flight at a time.

Of course, this also makes deadlocks possible. Given these funcs would only be on Actors, I would think it would be possible to impose restrictions at compile time such that you are guaranteed never to get a deadlock. Eg. No call inside a @nonreentrant func can be made outside the actor itself, and no call to any func in the @nonreentrant func can make its way back to that same func.

This would introduce a very simple queuing/transaction mechanism similar to Mutex but for the async-await universe, and be very beneficial to the 90% who should never really look at locks.

2 Likes

You're right, a proposal is a technical document about the goals and implementation.
I should have specified that my suggestion be in a blog or other place with examples when Mutex will be available in Swift 6.

Is it still the right mindset to have "sync" vs "async" here; can the Lock not support both simultaneously? i.e. have both sync and async versions of withLock etc.

Conceptually the only difference is whether you do the context-switching in kernel-space or user-space.

My current codebases tend to have a mix of sync and async code, but the need for mutual exclusion often spans both domains. While you can sometimes get away with "sync" locks even in async context, provided the lock isn't too highly contended and is never held for long, I'd prefer to just use an awaiting call anyway.

I know some folks want mutexes and their ilk to be considered archaic and unnecessary in the face of actors, but - whether that's valid in principle or not - I just don't see that being practical anytime soon. Maybe once actors have a mode where they don't permit re-entrancy, that might change the balance a bit - but even then, it might not be ideal to force use of such actors in all places, especially given they'll then have the same deadlock concerns as simple mutexes anyway.

Is there any benefit to Sendable? Would transferring strictly give us more options because Sendable stuff can be transferred AND non sendable stuff can be transferred?

Big +1 for me on this one. I really like how this fully fits in with concurrency and isolation so it will be easy to use for non sendable stuff. I still would like to remove the Sendable requirement on Result but I don't know if there is a strong reason that is required.