@Observable With Locks

Using Swift
, , ,
1

How is @Observable intended to interact with mutable state protected by a lock?

Consider a class like:

final class Counter: Sendable {
  private let _value = Mutex(0)
  var value: Int {
    _value.withLock { $0 }
  }
  func increment() {
    _value.withLock { $0 += 1 }
  }
}

I’d like to be able to observe Counter.value while keeping it Sendable. The approach I’ve been using is to just keep it isolated to the main actor:

@Observable
@MainActor
final class Counter {
  private(set) var value: Int
  func increment() {
    value += 1
  }
}

I would think invoking access(keyPath:) and withMutation(keyPath:_:) manually would solve the issue, but it leads to some ambiguities:

@Observable
final class Counter: Sendable {
  private let _value = Mutex(0)
  var value: Int {
    access(keyPath: \.value)
    return _value.withLock { $0 }
  }
  func increment() {
    // Should `withMutation` go inside or outside the lock?
    withMutation(keyPath: \.value) {
      _value.withLock { $0 += 1 }
    }
    // or
    _value.withLock {
      withMutation(keyPath: \.value) {
        $0 += 1
      }
    }
  }
}

The relative ordering of how willSet(keyPath:)/lock() and didSet(keyPath:)/unlock() execute is the problem. Upon further thinking, it seems like neither of these two options are particular safe.

withMutation(keyPath:) then withLock(_:)

Consider the case where withMutation(keyPath:) wraps withLock(_:). The relative ordering of operations would look something like:

func increment() {
  willSet(keyPath: \.value)
  _value.lock() // Doesn't actually exist, but for the sake of the example

  value += 1

  _value.unlock()
  didSet(keyPath: \.value)
}

willSet(keyPath:) synchronously invokes any observers, including any consumers using withObservationTracking(_:onChange:). Looking at the proposal and implementation for the new Observations async sequence, it’s implemented such that it schedules values to be yielded on the next suspension of the observing isolation, which may or may not be the isolation on which increment() is being called:

let counter = Counter()

@MainActor
func increment() {
  counter.increment()
}

final actor AnotherActor {
  func observe() {
    Observations { counter.value }
  }
}

In this example, Counter.increment() is being called on the main actor, but then counter.value is being observed on AnotherActor. That implies a counter.value will be yielded whenever AnotherActor yields. But then if the Counter._value lock happens to be highly contentious, there’s a chance that the order of operations is:

  1. Counter.increment() is called on the main actor
  2. willSet(keyPath: \.value) is called, invoking all observers using withObservationTracking
  3. Observations is one such observer and schedules counter.value to be read and yielded.
  4. The _value mutex is highly contentious and increment() does not acquire the lock yet, blocking the main actor from updating the value.
  5. AnotherActor yields, allowing the Observations block to observe counter.value at its pre-incremented state, causing Observations to yield the wrong value.
  6. The _value mutex is eventually acquired and the value is incremented.
  7. didSet(keyPath: \.value) is eventually called, but withObservationTracking is explicitly implemented to only respond to willSet, so the Observations stream never sees the new value and never yields it.

This isn’t an issue when @Observable is restricted to a single actor since reading from the value can only be done from that same actor and the order of await suspensions ensures everything works smoothly. But when it’s a lock that allows for synchronous access on another actor, the guarantee that the value is actually mutated on time before it gets observed is not upheld. This quote from SE-0475 implies Observations should be compatible with locks:

in the Sendable cases the mutation is guarded by some sort of mechanism like a lock

withLock(_:) then withMutation(keyPath:)

Reversing the order of operations (sort of) solves the issue above but introduces another. The relative order of operations would look like:

func increment() {
  _value.lock() // Doesn't actually exist, but for the sake of the example
  willSet(keyPath: \.value)

  value += 1

  didSet(keyPath: \.value)
  _value.unlock()
}

By ensuring the lock is held before calling willSet(keyPath:), it makes it much more likely that the value actually be incremented on time. willSet(keyPath:) will tell the Observations to schedule up another observation, but then value += 1 is incremented immediately after. Technically, since the two could be running on different threads, it’d be possible to run into the same sequence of events where AnotherActor yields and Observations observes value before it gets incremented, but it’s at least extremely unlikely.

A bigger problem arises from this code

withObservationTracking {
  counter.value
} onChange: {
  print("counter.value is about to change from \(counter.value)")
}

It doesn’t seem particularly conspicuous, but this results in a runtime crash. withObservationTracking‘s onChange runs synchronously on the same isolation as where willSet(keyPath:) is called. That means the print is going to be executed with this relative ordering:

func increment() {
  _value.lock()
  willSet(keyPath: \.value)
  // Synchronously calls:
  print("counter.value is about to change from \(counter.value)")
  
  value += 1

  didSet(keyPath: \.value)
  _value.unlock()
}

The onChange is accessing counter.value! Which is protected by the same lock that was just acquired, resulting in attempting to take the lock again on the same thread, which leads to a crash if you’re not particularly careful to use NSRecursiveLock instead of Mutex.

Mutex and most other locks don’t support recursively locking themselves, and it isn’t particularly clear that this would be an issue to begin with. Going back to the case from before where Observations attempts to read counter.value from another actor than where it’s being incremented, it means it’s still possible (albeit unlikely) to trigger this runtime crash even if you’re careful to never access the observed value from within onChange directly. If the lock hasn’t been yielded by the time someone else tries to read the value it’s protecting, it’s going to result in a crash.

Conclusion

Is there a way to use @Observable safely with locks??

It seems like one of the root issues is that the withObservationTracking is implemented such that onChange is called on the willSet, but there’s just no way to guarantee the accompanying didSet is ever actually called on time without resorting to putting everything on the same isolation. If that’s the case, it should probably be called out that @Observable is just incompatible with synchronization techniques other than using actors.

2 Likes
2

Make sure to call the access and mutation calls OUTSIDE the locks else bad things might happen (like deadlocks etc)

3

I agree that calling it inside the locks is more problematic since it’s calling outside the code that may attempt to require the lock, but what about the concern about the lock not being acquired on time? I.e., this sequence of events:

Since Observations introduces a suspension point between when it’s notified about the willSet and when it next observes the value, it feels like unlikely but not impossible that the lock is not acquired on time since it’s happening on different isolations, and the wrong value is observed (i.e., the sequence of events I listed above). Maybe I’m misunderstanding though?

4

For another case where calling willSet(keyPath:) outside the lock fails, consider this series of events:

  1. On one thread (Thread A), a consumer calls Counter.increment(), which calls willSet(keyPath: \.value) before attempting to acquire the lock and perform the mutation. At this point in time, there are no observers, so willSet does nothing.
  2. On another, concurrently-executing thread (Thread B), withObservationTracking is installing a tracker that reads counter.value. Since the observation closure is calling counter.value, it attempts to acquire the lock to read the current value.
  3. The two happen to coincide and the lock is yielded to Thread B first by chance. Thread B acquires the lock and reads the current counter.value (let’s assume it’s still 0). After reading the value and logging an access(keyPath: \.value), it relinquishes the lock.
  4. Thread A, which has been waiting on the lock, acquires it and performs its mutation, incrementing counter.value to 1.
  5. Thread B though, at this point in time, already read counter.value == 0 and is installing its willSet(keyPath:) tracker too late: Thread A has already called willSet, so Thread B will never see that counter.value has changed from 0 to 1. Even though Thread A is guaranteed to call didSet(keyPath:) after the mutation, withObservationTracking only tracks willSets and will never observe the mutation.

This goes back to the point I made at the end of my post: having no way to observe didSets (outside of @_spi(SwiftUI)) means there’s no just no way to fix this: with a lock, there will always be a chance for concurrent reads/writes to happen in an order that drops observation updates.

5

Note that there are currently bugs with observation and concurrent mutation, and you can't expect to receive updates made concurrently with calls to withObservationTracking (even if the call to withObservationTracking is made by the Observations sequence)