[Pitch] Transactional observation of values

I have taken a bit to look at this as an alternative and it really looks much better to me, I think this is worthwhile pursuing as a refinement for this pitch. There is another slight alteration this opens up; we could perhaps remove the @Sendable requirement... however I have yet to prove that is actually safe.

If the isolation is provided then the call site being inherited to the block means that the block MUST be as sendable as the exterior to the closure. That means that when the exterior is non-isolated then the interior must be safe already. If the exterior is isolated then the interior must be safe to the same isolation too. So that means that if the closure is always called on the matching isolation then (either isolated or not) then that means that we should be able to preserve the correct safety guarentees. Even if the type does not require @Sendable.

1 Like

As a follow-up; I have altered the pull request of the proposal to reflect using the @isolated(any) - it does require @Sendable still even though it could potentially drop that; the practical impact is that dropping that would make some of the more advanced uses impossible to implement instead of just commensurately difficult.

1 Like

I think Sendable can be removed only if closure is isolated to an actor, but IIRC all closures isolated to an actor are Sendable.

The only case that Sendable excludes are non-Sendable closures not isolated to an actor. Such closures can be used safely if entire Observed is not Sendable. So maybe it is worth to consider making conformance Observed: Sendable conditional (and somehow abstracting over the closure type).

Allowing non-Sendable Element types would be another plus.

But actually I don’t mind to not support nonisolated closures at all - both Sendable and non-Sendable. Since Observed does not group transactions for such closures, I think it’s best to be explicit about this fact and provide different API for those use cases.

Maybe even API with explicit transactions for grouping changes.

Thinking about transactions, I’ve realized that Observed does not support grouping changes that involve other observers.

E.g, there are observed free variables A and B, and there is derived C = f(A, B). But f is expensive to compute, so C is cached and uses Observed to listen to changes in A and B and recompute.

There is also D that uses A, B and C and listens to changes using Observed. After A and B have changed, C and D are notified in arbitrary order, and D may see old value of C.

2 Likes

Im not sure what you mean by other "observers"; if you mean other @Observable types then it works just fine to have multiple observed types in the closure - as long as the initial access is safe (which by construction it has to be) then those will work just fine for the same shared isolation; they don't work by individual property didSet but instead tracking to the access KeyPaths on the specific objects and then gathering the willSet of the first of any within that set of accesses. That part is exactly how SwiftUI does it. The only difference here is that this supports changes on other isolation domains other than the main actor; and if your isolation domain is none - then it will require the objects properly handle the case of being Sendable. Which is more than just atomicity of properties; it is consistent states upon concurrent access - if the state is somehow inconsistent in-between accesses then it is not adhering to Sendable correctly. @Observable or Observed can't make make unsafe things somehow safe.

One additional note of erratera - It is very strongly suggested to build in Swift 6 mode else some Sendable characteristics may be too pessimistic about Sendable warnings.

@Philippe_Hausler I'm still working on this issue with the initial implementation code.

    Task {
      person.firstName = "first"
      await Task.yield()
      person.lastName = "last"
    }

If you do this kind of thing β€” which you may well end up doing by mistake if you are calling a function to get a name that you need to await for example β€” then the changes can happen fast enough that the sequence breaks and never emits again. This is not robust enough for real use.

It's possible this is the issue brought up in the PR, or possibly something else. It might be that even with the lock, if two changes happen fast enough with the task yielded, then we do not catch the next one in time and call continuation.resume() on the wrong one, or something like that.

Is anyone else seeing this with the implementation?

There are a few issues with the code you have listed - im not sure the problem you are outlining is rather clear. Is it that you are claiming that you expect the first and last name assignments would be conjoined? if so then that code is unsafe and is breaking the data consistency expected by isolation; types MUST have consistent/stable values upon suspension - any time you suspend you are able to have re-entrancy on that actor. Or are you claiming the last name setter might be ignored since that is at the end of the task? That is presuming there is never a suspension later in that isolation the Task has inherited. Which seems like a hot-loop to me which would not be good for app performance at all and would definitely cause stalls.

Neither of those issues are really observation falling over but instead the application itself doing something that is not desirable in general. It might be good to outline exactly the case you expect to work and perhaps I can help elucidate where the mismatch is.

Thanks for getting back β€” and sure thing, I can outline my full example here. So I made an test view like this: β€”


@Observable
class Person {
  var firstName: String
  var lastName: String

  init(firstName: String, lastName: String) {
    self.firstName = firstName
    self.lastName  = lastName
  }
}

struct ContentView: View {
  @State private var person = Person(firstName: "John", lastName: "Appleseed")
  @State private var names: Observed<String, Never>? = nil

  let firstNames = ["John", "James", "Tim", "Bob", "Constantinople"]
  let surnames = ["Appleseed", "Smith", "Bobbins"]

  @State var count = 1

  var body: some View {
    VStack {
      Button("Change name") {
        Task {
          count += 1
          person.firstName = firstNames[count % firstNames.count]
          await Task.yield() // sequence breaks if this is included
          person.lastName = surnames[count % surnames.count]
        }
      }
    }
    .padding()
    .task {
      if names == nil {
        names = Observed { [weak person] in
          guard let person = person else { return nil }
          return person.firstName + " " + person.lastName
        }
      }
      if let names = names {
        for await name in names {
          print("Name changed to '\(name)'")
        }
        print("Sequence ended.")
      }
    }
  }
}

Now without the yield, everything works as expected, the names are printed without tearing and all is good. When you add the yield, I expected to get multiple outputs and see inconsistent names like you say, but for it to keep working never the less. What actually happens β€” most of the time β€” is that you see one torn value eg with just the first name changed, but after that the sequence stops emitting. The sequence hasn't ended, the guard let person has not returned nil, but the sequence never emits again.

I think in a real situation this needs to be considered. It would be easy to write code like β€”

Task {
   person.firstName = await namesGetter.firstName(count: count)
   person.lastName = await namesGetter.surname(count: count)
 }

β€” and not realise that because of the await boundaries, your sequence is invalidated. Sure this code is incorrect from a transaction point of view and produces an inconsistent view. But the app would suddenly not react ever again and you'd have no way of knowing it happened. There needs to be resilience so that even if this operation is not transactional because it is across an await boundary, the sequence still emits even if it is multiple times, and also continues to emit in the future.

Note that Observe might not even be being used in SwiftUI so there are many cases where this could accidentally happen.

I do think this is a bug β€” with print statements in the code to try to track it down, it happens less often and you have to click the button faster to see it. So I think it could be a race within the calls that set up the next continuation. It's quite possible I'm wrong of course!

2 Likes

huh, I will have to dive into that; there was a tiny race condition that my initial PR had in it so maybe that is something to re-check. I agree that this is just an implementation bug... but that doesn't seem to be something substantively impactful to the shape of the API.

Thanks! Yeah that is very true that it's not feedback on the actual API design.

I'm trying to backport this basically, because I think it has the potential to be very useful for us, but I can't tell unless I can actually try it as the devil will be in the details. It will be far too long to wait for this to actually make its way into an iOS that we can actually support, alas!

You have a couple of SwiftUI mistakes in your code I thought worth pointing out. For classes you must use @StateObject not @State otherwise you'll cause extra heap allocations of Person and potentially more allocations internally like its ObservationRegistrar. To use @StateObject with an @Observable class in the current version of SwiftUI simply also conform it to ObservableObject. In .task you don't need to check if names is nil or save names to optional @State, just init it in a local let and it survive for the lifetime of the async/await code.

I might be better if the Observed values sequence only worked with a single consumer to be consistent with all other sequences like CLLocationUpdates.liveUpdates and CLMonitor().events

I don't think it's right to use Observation and ObservableObject is it? That seems wrong to me, you would give it two mechanisms to update rather than one. That isn't what it says here: β€”

And I retained the sequence to be sure that wasn't the reason it suddenly stopped working. You are right that isn't required.

1 Like

I feel like that would make it significantly less useful no?

Conforming to ObservableObject is currently a requirement to use @StateObject. If it helps, just think of @Observable as a replacement for @Published.

Sadly that documentation has not been rolled back yet since after @State was changed back to be value types only.

State autoclosure added to 17b1 (to make it like StateObject):

State autoclosure removed from 17b5 (probably because it made regular value States inefficient or perhaps buggy):

Apologies, since you were using it in a for await in loop I assumed the Observer is an async stream, seems it isn't. Maybe it should be? It reminds me of the new lines async stream which prior to async/await was not possible for a line reading API because of buffer allocation issues which seems similar to the issues with the observation's storage.

import Foundation

let fileURL = URL(fileURLWithPath: "/path/to/your/file.txt")

do {
    for try await line in fileURL.lines {
        print(line)
    }
} catch {
    print("Error reading file: \(error)")
}

per my understanding of the current prototype implementation, only one iterator will ever see the 'initial' value of the Observed sequence. there's an internal flag shared across every iterator created for the sequence that governs whether the initial value is returned before awaiting the next 'will set'. is that behavior intentional?

i tested your sample code a bit, and i believe the issue is that the current prototype appears to suspend when attempting to register the next 'will change' continuation. this allows the observation tracking block to fire before the iterator's continuation intended to await it has been created. once missed, it can no longer 'see' subsequent changes to the Observable object, as the observation tracking blocks are one-shot (IIUC), so it ends up stuck waiting for a continuation that no longer resumes when the observed object changes. if you thread the iterator's isolation through to that method it seems to resolve that particular problem.

but this example does highlight a more general question – if a tracked property changes between when the observation tracking was installed and the point at which an iterator registers its 'will set' continuation... does the system fall apart?


overall, i think the proposal would benefit from some more detailed explanations of how the change tracking & update delivery model functions and is expected to work under various configurations (same isolation, different isolations, non-isolated, multiple consumers/producers, etc). i think there are currently some sharp edges that could make things seem like they work, but then lead to the system not functioning as expected (potentially due to nondeterminism), so it would be good to try and ensure developers have the appropriate mental model for how to use a tool like this, and what its limitations are.

5 Likes