SE-0395: Observability

tcldr · 2023-04-14T18:20:43.910Z

Jon_Shier:

As I understand it this is just a limitation of the current implementation, not a fundamental requirement of Swift's async model. The compiler could be enhanced to see that the actors in both cases are the same and elide the hop. If it could be guaranteed in all situations, would that provide the behavior you need?

The short answer is yes. If, as you say, it could be guaranteed in all situations, then I think that would be reasonable.

The longer answer is that I'm sceptical we'll reach that point.

There's already some work being taken on around custom executors, and from what I've seen even custom executors will be unable to elide that initial hop.

There's also the issue that a Task's closure will need to execute synchronously, or at least within the current event loop cycle to ensure that no events are lost between the Task being created and then run.

The other concern is one of ergonomics. If we do indeed make it to that point it would be a shame if it required a bunch of syntax for what should, arguably, be the simplest use case.

Then there's the question of how useful the await annotation becomes. Is this a potential suspension point or not?

Finally, I would question whether or not we're shoe-horning the async mechanisms into areas where they may not be the best fit. These are great tools, but Swift concurrency and async sequences are supposed to make things simpler but if we're creating a bunch of syntax to control actor flow, how much simpler have we actually ended up making things?

To a man with a hammer, everything looks like a nail

But in principal, yes.

Alejandro_Martinez · 2023-04-14T18:50:18.761Z

I’ve been following this proposal since it’s early stages and although I think is very good I’m still concerned with the ‘flickering UIs’ problem that others have mentioned plenty of times and that I can’t explain any better.

I’m afraid that if this is shipped without solving that concern applications with bad UX will start proliferating leaving Swift in a bad place. We have to remember not everybody uses SwiftUI (which if I read correctly will use the api that doesn’t have this issue?). Furthermore even if a ‘future direction’ can solve this I find that a very bad situation because those hypothetical apps with poor UX will still be out and developers won’t be able to fix it for years because we are tied to OS versions (unless the solution can be back deployed which we can’t assume).

On top if this my concern is that I’m not sure if this is solvable by this proposal since as mentioned above is an intrinsic part if Swift Concurrency, not specific to Observation. So unless 1) swift concurrency in general offers a solution for this situations or 2) we get an observation api that is not and async sequence; I’m not sure this proposal gets a big yes from me.

On another topic, I don’t find the fact that you have to mark dependencies for computed properties manually any good. Developers will expect to work out of the box and needing the boilerplate is unfortunate.

I hope we can get this solved because the rest of the proposal looks AWESOME! Thanks for putting the time and effort on this, to the author and all the community giving feedback.

QuinceyMorris · 2023-04-15T01:01:53.598Z

I feel like I inadvertently released a flickery UI monster, especially because (it turned out) my concern wasn't the one that others picked up on. I see three different issues being raised, and somewhat conflated.

The first is the question of whether executor hops in delivering observations are regarded as harmful to performance. This seems to be the main concern of @tcldr's comments. This isn't a concern that motivates me, since "solving" it smells a little bit like premature optimization, but I agree it's an issue worth keeping an eye on.
@gwendal.roue raised the issue of transitioning from synchronous to asynchronous code, which can producing ugly results, although it isn't a performance issue. I don't think SE-0395 creates this problem, or even makes it worse. I do think this is a problem which can be solved at a higher level.
The issue I intended to raise was forcing a UI to display successive values, even when the value has already changed by time a change reaches the display. This problem is arguable worse when all the code is synchronous.

Problem #3 arises when an observer starts processing a change, and the value changes again during the processing. Since the observer is committed to some synchronous update, it has to finish with that before getting around to displaying the correct thing.

The point I was trying to make is that for an "observation" system, being faithful to the original mutation events can be undesirable, and prompt propagation of coalesced changes can be very desirable. This differs from a "publication" system, where the identity and integrity of the original events is much more important.

tcldr · 2023-04-15T06:48:39.971Z

I borrowed one of your terms (Flickery UIs) as it succinctly conveys a symptom which we can, apparently, expect from both our issues.

QuinceyMorris:

The first is the question of whether executor hops in delivering observations are regarded as harmful to performance. This seems to be the main concern of @tcldr's comments. This isn't a concern that motivates me, since "solving" it smells a little bit like premature optimization, but I agree it's an issue worth keeping an eye on.

Performance is just one of the issues that concerns me. The main issue for me is that the asynchronous sequences will deliver stale state on same actor observations making it unnecessarily hard to reason about the current state of the system. ‘Flickery UI’s’ will also be caused due to the event being delivered after the current event loop cycle, missing the current animation transaction deadline.

gwendal.roue · 2023-04-15T13:27:39.739Z

QuinceyMorris:

I see three different issues being raised, and somewhat conflated.

I agree with you.

I have written a thorough reply in Some suggestions for Swift Observation.

gwendal.roue · 2023-04-15T13:44:24.835Z

Ben_Cohen:

This specific concern still seems to be a rephrasing of the "how do I avoid missing changes". Would this be addressed by the addition of an "initial + subscribe" capability as @QuinceyMorris described as being there in KVO? Notwithstanding @Jon_Shier's other reasons why observed ties knowing they are being observed being useful. Apologies if I'm missing some subtly here.

Please see Some suggestions for Swift Observation for a thorough reply.

tcldr · 2023-04-16T11:02:31.452Z

I wanted to share some further thoughts that I originally wrote up in a separate thread to hopefully clarify my position:

In the context of SE-0395, it seems to me that, the intended primary purpose seems to be 'fulfilling the needs of GUI apps' as that was the context given by the proposal – but perhaps the aim was more general.

For me, I would like a cohesive solution, the parts of which can be used together seamlessly; using SE-0395's proposed ObservationTracking mechanism, should work well when also using the ObservedChanges/ObservedValues mechanism.

One example of what I mean by this is that: if I have some View directly observing a property on each of ObservableA and ObservableB using the ObservationTracking mechanism, then ObservableA also directly observes some property on ObservableB (a dependent property) using whatever the ObservedChanges/ObservedValues mechanism becomes, they all need to be able to synchronise in the current event loop cycle.

As SE-0395 is currently proposed, when ObservableB is updated, triggering both an observation event on the View and ObservableA, the View will receive its event in the current event loop cycle, while ObservableA will receive its update (via an async sequence) in some future event loop cycle. So ultimately the View will receive updates for this single update across multiple event loop cycles.

diagram734×862 7.6 KB

Cue 'Flickery UI's.

This is what I mean when I'm talking about invariants. (In this context, at least.) I realise that invariants can still be temporarily broken in the current frame/event loop cycle, but I think that's a lot easier to reason about than invariants broken across multiple frames/event loop cycles.

I do wonder if the attempt to combine both 'fulfilling the needs of GUI apps' and inter-actor observation in the same mechanism is perhaps too ambitious. Not quite satisfying either goal. Perhaps focusing the proposal on a more specific goal would lead to a better overall solution. Then, as part of a future direction, it can be considered whether or not the mechanism can be successfully expanded to achieve inter-actor observation without compromising the existing primary purpose.

tcldr · 2023-04-18T12:08:06.717Z

I've been thinking some more about the problem above and I'm wondering if perhaps I'm missing something obvious here. Here's some code which more or less models the diagram above, but leans on withTracking instead of the async sequence bits:

I have a View (SomeView) observing a property on each of ObservableA and ObservableB.

The property being observed on ObservableB is called directlyObservedProperty and, as the name suggests is being observed directly by the view.

The property being observed on ObservableA is called dependentProperty and it itself is updated via an observation on ObservableB using the ObservationTracking mechanism. Specifically, when indirectlyObservedProperty changes.

In code:


struct SomeView: View {
  
  @StateObject private var observableA = ObservableA()
  
  var body: some View {
    VStack {
      Text("\(observableA.observableB.directlyObservedProperty)")
      Text("\(observableA.dependentProperty)")
    }
  }
}

@Observable final class ObservableA {
  var dependentProperty = "Some Value"
  let observableB = ObservableB()
  init() {
    resetObservationOnB()
  }
  
  func resetObservationOnB() {
    ObservationTracking.withTracking {
      self.dependentProperty = "\(observableB.indirectlyObservedProperty). Woohoo!"
    } onChange: { [weak self] in
      self?.resetObservationOnB()
    }
  }
}

@Observable final class ObservableB {
  var directlyObservedProperty = "Some Value"
  var indirectlyObservedProperty = "Some Value"
  func updateBothProps() {
    self.directlyObservedProperty = "I'm changing"
    self.indirectlyObservedProperty = "And so am I!"
  }
}

My questions is, if you'll be gracious enough to humour me @Philippe_Hausler:

Is this a supported use of the withTracking mechanism? Can I assume that when updateBothProps() is called on ObservableB, the update to the directlyObservedProperty and dependentProperty will occur in the same view update/transaction? Do you see any issues arising from using this kind of chained/indirect observation pattern?

kiel · 2023-04-18T12:30:34.319Z

In that example, would it be possible to write a unit test for observable A without a handful of arbitrary waits?

Philippe_Hausler · 2023-04-18T17:54:54.176Z

I have a feeling that is a bit more complicated than it needs to be:

struct SomeView: View {
  @State private var observableA = ObservableA() // @StateObject is for `ObservableObject`

  var body: some View {
    VStack {
      Text("\(observableA.observableB.directlyObservedProperty)")
      Text("\(observableA.dependentProperty)")
    }
  }
}

@Observable final class ObservableA {
  var dependentProperty: String { observableB.indirectlyObservedProperty }
  let observableB = ObservableB()
}

@Observable final class ObservableB {
  var directlyObservedProperty = "Some Value"
  var indirectlyObservedProperty = "Some Value"
  func updateBothProps() {
    self.directlyObservedProperty = "I'm changing"
    self.indirectlyObservedProperty = "And so am I!"
  }
}

So lets suss that apart a bit:

Consider when the body is executed for SomeView. There are a few property accesses that happen. First the \ObservableA.observableB happens then \ObservableB.directlyObservedProperty, and then \ObservableA.dependentProperty but executing that ALSO hits \ObservableB.indirectlyObservedProperty.

The tracking then is installed as observing those things. When the first one changes out of that set of key paths and instances the view is flagged as needing update and the next pass of the rendering engine comes along and re-evaluates the view accordingly.

That means that when updateBothProps is invoked two of the properties given in the list of observations change (the first one invalidates the view).

tcldr · 2023-04-18T18:26:22.108Z

That's interesting.

I was overcomplicating for the sake of the example – this isn't the way i'd structure something this simple – but I must say I think my reservations regarding intra-actor communication are beginning to evaporate as the capabilities of the withTracking mechanism become clearer to me.

I'm still interested in the question of whether chained observations as originally described could work, though. I can imagine there could be situations where this would be useful. Perhaps there's some other action that needs to be taken when indirectlyObservedProperty is updated. Would a chained observation as described work?

Philippe_Hausler · 2023-04-18T18:27:19.901Z

tcldr:

originally described could work, though

To be clear; yes that could work if you needed it to.
Perhaps with some slight modifications - particularly it would probably need to re-evaluate things on the next run loop tick in there somewhere.

tcldr · 2023-04-18T18:35:02.984Z

Well, in that case... consider me converted!

Personally, I think I'd still be cautious about using the ObservedChanges/ObservedValues mechanism for same actor communication. But as I can no longer think of any reason why you'd use ObservedChanges/ObservedValues over the withTracking mechanism for same actor communication – it's a non issue. And, as I say above, for inter-actor observation I think the ObservedChanges/ObservedValues mechanism will suit me well.

Can't wait to take it for a test drive.

tcldr · 2023-04-18T19:07:34.448Z

Apologies, just noticed the edit on your post there.

Philippe_Hausler:

Perhaps with some slight modifications - particularly it would probably need to re-evaluate things on the next run loop tick in there somewhere.

I think that's what my concern would be – if an observation became spread across multiple run loop cycles – that's going to cause the Flickery UIs. Granted, the use-cases might be considered 'advanced'. But I can imagine withTracking being useful outside of just the UI render tree.

It would be really nice if we could get chained usage synchronised!

michelf · 2023-04-18T19:17:25.016Z

I note almost all the examples are made with final class. What happens with a non-final class? Can the subclass properties be observed too? If not, can you add @Observable on subclasses?

Philippe_Hausler · 2023-04-18T20:25:43.526Z

The final is used because the protocol requirements having Self in the keypath require that the types be final. There are ways that could be relaxed, however doing so may have some pretty steep impact - namely creating ambiguity over what keypath it is referring to; either the subclass or the superclass.

As currently proposed the requirement is that classes must be final.

michelf · 2023-04-18T20:47:53.916Z

So if you have a class hierarchy, you can only add @Observable on leaf classes (which must be final). That's a limitation I could not find in the proposal. Thank you for clarifying.

Now I wonder what this means for properties of a base class. Do they become observable when @Observable is applied on the derived class? I don't see how that'd work, so I suppose there are other ways to make them observable, perhaps by overriding them in the derived class?

I'm trying to see if @Observable can work (reasonably) with a class hierarchy, and it would appear the answer is mostly no. To me this is a very major downside of this proposal.

Philippe_Hausler · 2023-04-18T20:53:31.214Z

macros only apply to the properties that are members to the application, so it does not apply to superclasses (since macros don't mutate code, they only add to it)

tera · 2023-04-18T21:50:27.678Z

Philippe_Hausler:

macros only apply to the properties that are members to the application, so it does not apply to superclasses (since macros don't mutate code, they only add to it)

That's quite serious limitation IMHO. As far as I understand the previous version of the pitch was not using macros and it didn't have this limitation, right? Can you list all pros and cons the new macro based implementation has compared to the one that doesn't use macros.

michelf · 2023-04-18T23:28:01.775Z

I guess my followup question is whether it's possible to manually implement observability in those cases. Perhaps like this:

class Base {
   var value: Int // non-observable value in base class
}

@Observable
final class Derived: Base {
   // making Base.value observable in derived class
   override var value: Int {
      get {
         _$observationRegistrar.access(self, keyPath: \.value)
         return super.value
      }
      set {
         _$observationRegistrar.withMutation(self, keyPath: \.value) {
            super.value = newValue
         }
      }
   }
}

Would this work? It's neither convenient nor pretty, but it's good to know there's an escape route when you've painted yourself in a corner.