SE-0395: Observability

I wanted to share some further thoughts that I originally wrote up in a separate thread to hopefully clarify my position:

In the context of SE-0395, it seems to me that, the intended primary purpose seems to be 'fulfilling the needs of GUI apps' as that was the context given by the proposal – but perhaps the aim was more general.

For me, I would like a cohesive solution, the parts of which can be used together seamlessly; using SE-0395's proposed ObservationTracking mechanism, should work well when also using the ObservedChanges/ObservedValues mechanism.

One example of what I mean by this is that: if I have some View directly observing a property on each of ObservableA and ObservableB using the ObservationTracking mechanism, then ObservableA also directly observes some property on ObservableB (a dependent property) using whatever the ObservedChanges/ObservedValues mechanism becomes, they all need to be able to synchronise in the current event loop cycle.

As SE-0395 is currently proposed, when ObservableB is updated, triggering both an observation event on the View and ObservableA, the View will receive its event in the current event loop cycle, while ObservableA will receive its update (via an async sequence) in some future event loop cycle. So ultimately the View will receive updates for this single update across multiple event loop cycles.

Cue 'Flickery UI's.

This is what I mean when I'm talking about invariants. (In this context, at least.) I realise that invariants can still be temporarily broken in the current frame/event loop cycle, but I think that's a lot easier to reason about than invariants broken across multiple frames/event loop cycles.

I do wonder if the attempt to combine both 'fulfilling the needs of GUI apps' and inter-actor observation in the same mechanism is perhaps too ambitious. Not quite satisfying either goal. Perhaps focusing the proposal on a more specific goal would lead to a better overall solution. Then, as part of a future direction, it can be considered whether or not the mechanism can be successfully expanded to achieve inter-actor observation without compromising the existing primary purpose.

9 Likes

I've been thinking some more about the problem above and I'm wondering if perhaps I'm missing something obvious here. Here's some code which more or less models the diagram above, but leans on withTracking instead of the async sequence bits:

I have a View (SomeView) observing a property on each of ObservableA and ObservableB.

The property being observed on ObservableB is called directlyObservedProperty and, as the name suggests is being observed directly by the view.

The property being observed on ObservableA is called dependentProperty and it itself is updated via an observation on ObservableB using the ObservationTracking mechanism. Specifically, when indirectlyObservedProperty changes.

In code:


struct SomeView: View {
  
  @StateObject private var observableA = ObservableA()
  
  var body: some View {
    VStack {
      Text("\(observableA.observableB.directlyObservedProperty)")
      Text("\(observableA.dependentProperty)")
    }
  }
}

@Observable final class ObservableA {
  var dependentProperty = "Some Value"
  let observableB = ObservableB()
  init() {
    resetObservationOnB()
  }
  
  func resetObservationOnB() {
    ObservationTracking.withTracking {
      self.dependentProperty = "\(observableB.indirectlyObservedProperty). Woohoo!"
    } onChange: { [weak self] in
      self?.resetObservationOnB()
    }
  }
}

@Observable final class ObservableB {
  var directlyObservedProperty = "Some Value"
  var indirectlyObservedProperty = "Some Value"
  func updateBothProps() {
    self.directlyObservedProperty = "I'm changing"
    self.indirectlyObservedProperty = "And so am I!"
  }
}

My questions is, if you'll be gracious enough to humour me @Philippe_Hausler:

Is this a supported use of the withTracking mechanism? Can I assume that when updateBothProps() is called on ObservableB, the update to the directlyObservedProperty and dependentProperty will occur in the same view update/transaction? Do you see any issues arising from using this kind of chained/indirect observation pattern?

In that example, would it be possible to write a unit test for observable A without a handful of arbitrary waits?

3 Likes

I have a feeling that is a bit more complicated than it needs to be:

struct SomeView: View {
  @State private var observableA = ObservableA() // @StateObject is for `ObservableObject`

  var body: some View {
    VStack {
      Text("\(observableA.observableB.directlyObservedProperty)")
      Text("\(observableA.dependentProperty)")
    }
  }
}

@Observable final class ObservableA {
  var dependentProperty: String { observableB.indirectlyObservedProperty }
  let observableB = ObservableB()
}

@Observable final class ObservableB {
  var directlyObservedProperty = "Some Value"
  var indirectlyObservedProperty = "Some Value"
  func updateBothProps() {
    self.directlyObservedProperty = "I'm changing"
    self.indirectlyObservedProperty = "And so am I!"
  }
}

So lets suss that apart a bit:

Consider when the body is executed for SomeView. There are a few property accesses that happen. First the \ObservableA.observableB happens then \ObservableB.directlyObservedProperty, and then \ObservableA.dependentProperty but executing that ALSO hits \ObservableB.indirectlyObservedProperty.

The tracking then is installed as observing those things. When the first one changes out of that set of key paths and instances the view is flagged as needing update and the next pass of the rendering engine comes along and re-evaluates the view accordingly.

That means that when updateBothProps is invoked two of the properties given in the list of observations change (the first one invalidates the view).

3 Likes

That's interesting.

I was overcomplicating for the sake of the example – this isn't the way i'd structure something this simple – but I must say I think my reservations regarding intra-actor communication are beginning to evaporate as the capabilities of the withTracking mechanism become clearer to me.

I'm still interested in the question of whether chained observations as originally described could work, though. I can imagine there could be situations where this would be useful. Perhaps there's some other action that needs to be taken when indirectlyObservedProperty is updated. Would a chained observation as described work?

To be clear; yes that could work if you needed it to.
Perhaps with some slight modifications - particularly it would probably need to re-evaluate things on the next run loop tick in there somewhere.

3 Likes

Well, in that case... consider me converted!

Personally, I think I'd still be cautious about using the ObservedChanges/ObservedValues mechanism for same actor communication. But as I can no longer think of any reason why you'd use ObservedChanges/ObservedValues over the withTracking mechanism for same actor communication – it's a non issue. And, as I say above, for inter-actor observation I think the ObservedChanges/ObservedValues mechanism will suit me well.

Can't wait to take it for a test drive.

2 Likes

Apologies, just noticed the edit on your post there.

I think that's what my concern would be – if an observation became spread across multiple run loop cycles – that's going to cause the Flickery UIs. Granted, the use-cases might be considered 'advanced'. But I can imagine withTracking being useful outside of just the UI render tree.

It would be really nice if we could get chained usage synchronised!

1 Like

I note almost all the examples are made with final class. What happens with a non-final class? Can the subclass properties be observed too? If not, can you add @Observable on subclasses?

3 Likes

The final is used because the protocol requirements having Self in the keypath require that the types be final. There are ways that could be relaxed, however doing so may have some pretty steep impact - namely creating ambiguity over what keypath it is referring to; either the subclass or the superclass.

As currently proposed the requirement is that classes must be final.

1 Like

So if you have a class hierarchy, you can only add @Observable on leaf classes (which must be final). That's a limitation I could not find in the proposal. Thank you for clarifying.

Now I wonder what this means for properties of a base class. Do they become observable when @Observable is applied on the derived class? I don't see how that'd work, so I suppose there are other ways to make them observable, perhaps by overriding them in the derived class?

I'm trying to see if @Observable can work (reasonably) with a class hierarchy, and it would appear the answer is mostly no. To me this is a very major downside of this proposal.

3 Likes

macros only apply to the properties that are members to the application, so it does not apply to superclasses (since macros don't mutate code, they only add to it)

1 Like

That's quite serious limitation IMHO. As far as I understand the previous version of the pitch was not using macros and it didn't have this limitation, right? Can you list all pros and cons the new macro based implementation has compared to the one that doesn't use macros.

1 Like

I guess my followup question is whether it's possible to manually implement observability in those cases. Perhaps like this:

class Base {
   var value: Int // non-observable value in base class
}

@Observable
final class Derived: Base {
   // making Base.value observable in derived class
   override var value: Int {
      get {
         _$observationRegistrar.access(self, keyPath: \.value)
         return super.value
      }
      set {
         _$observationRegistrar.withMutation(self, keyPath: \.value) {
            super.value = newValue
         }
      }
   }
}

Would this work? It's neither convenient nor pretty, but it's good to know there's an escape route when you've painted yourself in a corner.

That will work as you expect.

2 Likes

Cycling back to a few points of feedback:

There is a slightly inaccurate but founded concern about flickering UI. I think it might be better to phrase that as more so the issue underlying that; the concern is consistency. Which is a very reasonable worry. The point of consistency problems crops up (paraphrasing a few posts by folks) that if a property will be observed and the change in that property only happens once (and never happens again) and the confluence of events is such that the property observation via values(for:) is called asynchronously (e.g. in a Task) the first value could get lost.

After working through all of the ramifications and implications of this; there are a few take-aways. 1) that concern is not only reasonable, but also is a true problem. 2) prepending the first value does have some consequence, but in general is probably fine.

Let me iterate what I have discerned:
The first point of iteration should be ideally where any work is done, however... getting that first value MUST occur in the same isolation domain as the access to the object. Furthermore the first value may not be desirable in all circumstances. Lastly, holding that value might incur some sort of lifecycle impact.

Options:

  1. Hold the Observable type and defer access to the first value until the first point of iteration.
    This option does not address the missing first value problem but also poses a retain cycle issue.
  2. Grab the initial value immediately at construction.
    This option does address the "first value" problem but comes at a cost of extra overhead of storage. This also has an interesting alteration to the protocols; it forces the values(for:) to be isolated; because what happens if the Observable type is @MainActor bound? The keypath based access MUST follow the same rules as the property in question.

In my view option 1 might have an attractive characteristic that it defers access, but since it does not solve the issue it is disqualified.

Option 2 does validly solve the issue. It could be a option to the values(for:) but to be honest if folks don't want that first value... they could just say values(for:...).dropFirst(). But it means that the method for getting values must share the same isolation as the type. Which in the grand scheme of things is probably a reasonable alteration (having it be nonisolated was only really a bonus and not really a core part of the API).

That was all a very long winded way of stating; I agree that providing the first value can be accomplished, and should be part of the behavior for values(for:).

This now brings us to changes(for:). To me it would be strange that values(for:) had an "initial value" but changes(for:) did not. So for the same reasoning that should also have an "initial value". The initial would be claiming that the change went from "observing nothing" to "observing all the items". Meaning that it would emit a change containing all tracked properties of that observation.

I have some additional responses/musing/mulling-over of the synchronous versus asynchronous part of things but I will follow up later with that when my thoughts are a bit more congealed with that.

The behavioral change outlined here will be included with an update to the proposal. As well as an update to the implementation.

Thanks for all of the discussion so far - this is definitely the type of feedback that makes things much more robust and I appreciate your patience with me on navigating all of the feedback.

26 Likes

Is the following going to be improved in the new observation machinery?

class Model: ObservableObject {
    @Published var state = 0
    @Published var unrelated = 0
    
    init() {
        Timer.scheduledTimer(withTimeInterval: 1/100, repeats: true) { [self] _ in
            unrelated += 1
        }
        Timer.scheduledTimer(withTimeInterval: 1, repeats: true) { [self] _ in
            state += 1
        }
    }
}

var bodyCount = 0

struct ContentView: View {
    @StateObject private var model = Model()
    
    var body: some View {
        bodyCount += 1
        return VStack {
            Text(String(model.state))
        }
    }
}

In this example body is called 100 times a second, disregarding the fact that ContentView doesn't depend upon "unrelated" variable.

@Philippe_Hausler is the API available in the latest nightly-main toolchain?

The view in this case (when the model side would be using @Observable instead of ObservableObject) will only update once a second, not 100 times a second.

And it would be written as such:

@Observable final class Model {
    var state = 0
    var unrelated = 0
    
    init() {
        Timer.scheduledTimer(withTimeInterval: 1/100, repeats: true) { [self] _ in
            unrelated += 1
        }
        Timer.scheduledTimer(withTimeInterval: 1, repeats: true) { [self] _ in
            state += 1
        }
    }
}

var bodyCount = 0

struct ContentView: View {
    @State private var model = Model()
    
    var body: some View {
        bodyCount += 1
        return VStack {
            Text(String(model.state))
        }
    }
}
5 Likes

Hi @Philippe_Hausler,
In your example you mark the Model with the @State property wrapper. This differs from the similar example in the proposal text.

Will it need @State?