[Pitch] Observation (Revised)

[Callum (Callum T)]

This is a great revision and I’m really liking the direction this is going. Thanks.

---

While observation not working for multi-layered keypaths (\.account.name) is unfortunate, it is an easy boundary to explain. Sounds like something that would be good to explore in the future.

---

@Philippe_Hausler

For read-only computed properties, an author can add the static dependencies(of:) method to claim additional key paths as part of their observation.

I have some concern over the lack of automatic support for computed properties. It feels like something that should just work, or have similarly easy support like the main macro synthesis provides.

Could we generate the contents of the dependencies(of:) method using macros? Either from inspecting the computed property’s body, or via some annotation macro just to move the contents of the static method’s switch statement closer to the source. I guess the issue with all of that would be computed properties in extensions!

At the moment I’m not able to come up with a better solution here, but it would be ideal if we could reduce the friction of having to maintain a separate list of key paths in a different area of the source code from where the canonical information lies.

[FranzBusch]

Thanks for pitching this again! I think it looks amazing already but I have a few questions

Lifetime implications of observable objects

Both the ObservedChanges and ObservedValues types require the underlying object to be alive to produce new values. Are they retaining the subject or expect it be retained somewhere else. I think we should call this out in the documentation.

Isolation

I understand the reason why we want to add this since it allows callers of the ObservedChanges sequence to coalesce transactions. However, I am not entirely convinced this composes nicely in the broader ecosystem. As @tcldr already pointed out as soon as you add any other algorithm to the async sequence you are incurring hops again.

We have recently explored the custom actor executor feature in NIO and it already brings us substantially close to what we want. However, there is one missing piece which is having a way to tell a task what the global executor is for its scope. The problem that we are seeing in NIO and also here is that the AsyncSequence.AsyncIterator.next() method forces a hop right now on to the global executor. If we would be able to override the global executor on a task basis we wouldn't hop anymore. As far as I can see this would eliminate the need for the isolation parameter on ObservedChanges.

This is quite important to me because I think we are setting a bad precedent here in saying that root AsyncSequences or in general AsyncSequences should allow for an isolation opt-in. IMO we should rather get task executors so we can make this work generically everywhere and give the user a way to control where their code is run.

await withTaskGroup(Void.self) { group
    group.addTask(executor: MainActor.shared.unownedExectuor) {
        let someObservable = SomeObservable()
        let changes = someObservable
            .changes(for: [\.someProperty, \.someOtherProperty])
            .map {  }

        for await element in changes {
            // This runs on the MainActor
        }
    }
}

Castiness of the AsyncSequences

Could you specify and document what the behaviour of the ObservedChanges and ObservedValues sequences are. Can multiple iterators be created? If so what values do iterators get that are created delayed?

Location of API

This API will be housed in a package; outside of the standard library.

This was already brought up-thread from my understanding this library will be living in the toolchain after all. I would love if we could get a statement from the core team when a new API should live in the toolchain vs in a separate package. Furthermore, if we add new Swift modules to the toolchain how they integrate with the evolution process.
Lastly, if we land this in the toolchain directly we are bound to have API/ABI stability going forward. I would love if we could first play around with this within a package and get some larger feedback before integrating it into the toolchain.

Overall, I am very happy with how this shaped up and I a lot of usefulness in these APIs to fill a missing gap in the Concurrency story.

[twittemb (Thibault Wittemberg)]

Hi thanks for the pitch.

@FranzBusch

Castiness of the AsyncSequences

Could you specify and document what the behaviour of the ObservedChanges and ObservedValues sequences are. Can multiple iterators be created? If so what values do iterators get that are created delayed?

I'm also interested in having more details about that. Will the AsyncSequences be simulcast / broadcast ?

[Alejandro (Alejandro Alonso)]

@FranzBusch

Furthermore, if we add new Swift modules to the toolchain how they integrate with the evolution process.

I may be misremembering, but what I remember (and what my opinion is) is that because these are public imports that come by default with Swift, they must go through evolution.

[Karl]

The thing that I'm not sure about is composition. Let's say I have 2 observable objects, representing parts of my model, and I want to combine them in a new observable object which exposes data from each of them (possibly transforming their values along the way).

It seems that I would be expected to manually write out those dependencies:

@Philippe_Hausler

For read-only computed properties, an author can add the static dependencies(of:) method to claim additional key paths as part of their observation.

This essentially means that I need to write my code twice, which then becomes a very likely source of bugs - every time I update a computed property, I need to remember to update its entry in this Single Giant List else updates sometimes fail to trigger. I don't like this in KVO, and rather than copying it, I think we should consider it a major defect that observation in Swift should improve on.

This design also seems to be incompatible with extensions - if I add a computed property in an extension, I will also need to go back and update the Single Giant List. That hurts the organisation of my code and makes mistakes even more likely, and in some cases it may not even be possible - for instance, extensions across modules, or computed properties in a different file and marked as (file)private.

Also, is dependencies(of:) called recursively? Can keypath A declare a dependency on keypath B, which itself depends on keypath C? Or does keypath A need to list every leaf node of its dependency tree? Are dependencies also limited to one keypath component, or can they refer to keypaths in other objects (e.g. \.accountName depends on \.account.name)?

Also, is this limited to reference types, or types with reference semantics? It can be useful to wrap multiple reference types in a struct for composition purposes, and the resulting struct actually has reference semantics. Non-copyable structs also have reference semantics and should conceptually be able to support observation (although this particular design looks like it may require copyability; I'm not sure - it's possible that we can't express the lifetime of noncopyable structs well enough to support this design).

[Morten_Bek_Ditlevsen]

Thank you for the clear and concise explanation of the issues with computed properties.

I also worry that this will become a source of bugs. In the original pitch I got the feeling that converting to using Observable rather than ObservableObject would basically just be removing @Published and @ObservedObject and voila: now you have a more performant version of your SwiftUI code.

But if the computed properties need boilerplate to get to work with observation - and if the compiler doesn’t somehow warn you that you probably intended for a computed property to partake in observation then I think this will lead to bugs.

Also I think that most people wouldn’t expect a refactor from a read-only property to a computed property could introduce errors, but now this is no longer true.

[Philippe_Hausler]

@Karl I think there might be some parts here that need a bit more clarification:

The dependencies are only there for the express purpose of the AsyncSequence forms of observation. If we were to attempt to track all potential impact of computed getter style properties then that would either a) lead to a cascade of practically observing the entire thing (which puts us back into the realms of how ObservableObject works today) or b) we have to solve the halting problem (which I can safely say is outside of the realms of this project).

But more accurately to address your worry about this for SwiftUI - since the withTracking does graph based access those observations don't need dependencies to be expressed. For example:

@Observable final class Foo {
  var bar: Int = 0
  var baz: Int { return bar + 1 }
}

If this were used in a SwiftUI context (which uses withTracking) as such:

struct MyView: View {
  let foo = Foo()
  var body: some View {
    Text(foo.baz, format: .number)
    Button("Increment") { foo.bar += 1 }
  }
}

During the render of the body of MyView the property of foo.baz is accessed which does not have any sort of macro effect. But... that property accesses self.bar which does! That means that when MyView.body is accessed the observation tracking knows that the instance of Foo has \.bar used. In turn after the body is rendered it then knows because the access list is non-nil (being that it collected that during the execution of the apply closure of withObservation it then adds tracking observations to all of those specific instances for those specific key paths. Subsequently then when the Button is pressed it will trigger that change of the mutation of bar which in turn triggers the observer for SwiftUI to mark the view as having its attributes invalidated and require rendering.

The issue with observing then becomes when you use .changes(for: \.baz) or .values(for: \.baz) - which that does need some sort of indication that \.baz means that it really is needing to observe the TrackedProperties of [\.baz, \.bar].

That all being said - one potential future direction is that we could introduce some sort of metadata to indicate that directly on the declaration of var baz as @Observable(dependencies: \.bar) or something like that. But that particular change can be done at a later date and is additive to the proposal.

@Morten_Bek_Ditlevsen Per your concern about the interaction between property wrappers and observation: I am currently looking into how we might be able to make that work - There are some limitations on how macros don't yet let us alter functions that are kinda in the way to do it more generally, but there is a chance that I might be able to pull a rabbit out of the hat to make computed mutable properties (which property wrappers effectively are) to work.

[Philippe_Hausler]

You bring up some interesting areas here: there are some nuances that definitely need some extra explanation -

Lifetimes: The ObservedChanges and ObservedValues do NOT retain the observables. They retain part of the underpinnings of the registrar (specifically the registrar's context). This means observing changes or values from an Observable means the asynchronous sequence of elements runs for as long as the observable exists.

Isolation: That analysis is a touch off - the isolation determines when the trailing edge of the transaction is completed and NOT the actor upon the next function can be called. This means that it awaits space within that actor to schedule things. So if you tack on a map or something else that is fine. Even if that is asynchronous in the mapping function; the value is emitted beyond the point of transaction completion.

From what I understand of that code snippet isn't it isomorphic to this?

await withTaskGroup(Void.self) { group
    group.addTask { @MainActor in
       ...

From a transactional ending indication basis that is fine.

Per the "castiness" - multiple iterators can be created, the iterators are forbidden from being Sendable so sharing that is flat-out denied. If multiple iterators are created that will be as if you had multiple independent observations; each one is serviced.

[tcldr]

@Philippe_Hausler

This means that it awaits space within that actor to schedule things. So if you tack on a map or something else that is fine.

My understanding is that the next() call of the map operator will always run on the default executor. If it isn't annotated to run on a specific actor or GAIT, an async function will always be executed on the default executor/shared thread pool. Therefore, whether or not map is called within a closure that is annotated @MainActor is immaterial, Swift concurrency will always execute map's next() method on the default executor as it isn't specifically annotated (and can't be specifically annotated) to run anywhere else.

Also, IIUC, the fact that observations need to 'await space' on their actors suggests to me that it's unlikely observation events will be received in the same event loop cycle that they're fired. I can see this being an issue for smooth observation of something like a mouse pointer. Bunched events arriving at the same time will have the appearance of dropped frames. In my opinion, this is a critical requirement for an observation implementation to be used in a UI context.

To be clear: the issue is actor-hopping. While it's apparent that observation events must occur asynchronously, the propagation of those events from Observable to Observer really should be synchronous by default. I question whether Swift concurrency is the correct mechanism to do this right now.

[tcldr]

Thinking about this a little more: I think the issue that concerns me is that while this design seems suitable for inter-actor observation, it seems unsuitable for intra-actor observation. And seeing as this pitch has been presented in the context of being suitable as a UI observation mechanism, the same-actor properties of this mechanism will be key.

In other observation mechanisms like Combine/KVO it is possible for events from Observer to Observable to be propagated synchronously. This property means that UI events can be delivered and processed in the same event loop cycle that they're generated.

Without this property, UI will feel unresponsive and lacking synchronisation. For example, a shadow effect designed to track your finger will, at best, lag slightly behind and at worst appear to drop frames as multiple observation events bunch together. Or, a touch of a button will appear to have a delay before it's highlighted. Or, a screen with a loading state will often show a flash of a spinner even if its content has been previously cached. Or, Observers that should have their effects synchronised with other Observers will appear to react independently and with random delay... etc.

The reason for this is that both ObservedChanges and ObservedValues are asynchronous sequences. And despite the fact that a View and an Observable might both exist on the @MainActor, the fact that the Observable communicates to the View via an asynchronous sequence means that it will always (whether or not map is used) take the following route:

MainActor -> Shared Thread Pool -> MainActor

At minimum this means that events generated in the current event loop cycle, won't be delivered until the next event loop cycle. And at worst, it means that events won't be delivered until some unknown event loop in the future.

As mentioned above, this is because any non-annotated async function will always execute on the shared thread pool. And as both ObservedChanges and ObservedValues are asynchronous sequences with non-annotated next() calls this is the route all events must take. (IIUC.)

Frankly though, even if they didn't, even if the event propagation was going through the equivalent of DispatchQueue.main.async { }, that would still be too laggy for a UI observation mechanism.

In other words: while Observation events may occur asynchronously, that shouldn't imply that the propagation of those events from Observable to Observer should be asynchronous as well.

For responsive UI, event propagation should very much be synchronous.

[tclementdev]

This could cause state issues too, for example with async propagation it could be possible to click a button and find that the state disallows it:

Button("Click") {
    observableObject.disabled // could be true
}
.disabled(observableObject.disabled)

Async observability in general seems pretty brittle because the entire state of the program might not relate anymore to the change that the observer is receiving.

[FranzBusch]

@Philippe_Hausler

Lifetimes: The ObservedChanges and ObservedValues do NOT retain the observables. They retain part of the underpinnings of the registrar (specifically the registrar's context). This means observing changes or values from an Observable means the asynchronous sequence of elements runs for as long as the observable exists.

Thanks for clarifying. Could we make sure to put this into the documentation so that people know they have to retain the subject as well.

@Philippe_Hausler

Isolation: That analysis is a touch off - the isolation determines when the trailing edge of the transaction is completed and NOT the actor upon the next function can be called. This means that it awaits space within that actor to schedule things. So if you tack on a map or something else that is fine. Even if that is asynchronous in the mapping function; the value is emitted beyond the point of transaction completion.

I think this would result in the same behaviour. In your current implementation, you are hopping onto the actor which allows you to hit the next suspension point. In the case of task executor the actor would call next on the sequence and directly get a result if there are elements left or it would suspend. In the case of suspension it would resume once a new value has been produced and the actor itself hit a suspension point to resume its continuation.

@Philippe_Hausler

From what I understand of that code snippet isn't it isomorphic to this?

await withTaskGroup(Void.self) { group
    group.addTask { @MainActor in
       ...

No not at all. If we expand your snippet a bit more and iterate an async sequence in the child task we get totally different behaviour.

await withTaskGroup(Void.self) { group
    group.addTask { @MainActor in
        for await element in someSequence {}
    }

What happens here is that for each call to next on the iterator we are going to hop off the MainActor and then back on it again. This is due to the fact that currently every non isolated method is hopping to the global executor. If we would get task executors the following code wouldn't hop at all

await withTaskGroup(Void.self) { group
    group.addTask(executor: MainActor.shared.executor {
        for await element in someSequence.map{}.interspersed().debounced() {}
    }

This generalises nicely because it allows you to override the global executor from a given child task downwards. I also believe this could solve the coalescing of transaction problems that you are tackling here, but @John_McCall can probably speak more to that.

@Philippe_Hausler

Per the "castiness" - multiple iterators can be created, the iterators are forbidden from being Sendable so sharing that is flat-out denied. If multiple iterators are created that will be as if you had multiple independent observations; each one is serviced.

Thanks for expanding. So it is a broadcast async sequence. Might be worth calling out in the docs as well. How is the distribution mechanism working? Are we waiting for all iterators to consume an element before we send out the next one or does every iterator have it's own buffer?

[tcldr]

@FranzBusch

This is due to the fact that currently every non isolated method is hopping to the global executor. If we would get task executors the following code wouldn't hop at all

await withTaskGroup(Void.self) { group
    group.addTask(executor: MainActor.shared.executor {
        for await element in someSequence.map{}.interspersed().debounced() {}
    }

Would this allow the event to be delivered in the same event loop cycle that it is sent?

My concern is that even if events are guaranteed to be delivered on the next event loop cycle on the specified executor (i.e. rough equivalence to DispatchQueue.main.async), you'll still get UI glitches as well as the state issues described by @tclementdev . That would be a step backwards from KVO/Combine from a UI perspective.

[FranzBusch]

@tcldr

Would this allow the event to be delivered in the same event loop cycle that it is sent?

My concern is that even if events are guaranteed to be delivered on the next event loop cycle on the specified executor (i.e. rough equivalence to DispatchQueue.main.async), you'll still get UI glitches as well as the state issues described by @tclementdev . That would be a step backwards from KVO/Combine from a UI perspective.

I don't want to derail this pitch here too much with this discussion but I don't think this would be sync by nature. What would most likely happen is that your root async sequence has a continuation from its consumer (Consumer is on the MainActor). Now the sequence is producing a new value (also from the MainActor). The sequence would be calling continuation.resume which would enqueue the resumption on the executor. Basically a queue.async {}.

The custom executor pitch from @ktoso is outlining a few rules that a serial executor has to follow. One of those rules is:

If the executor is a serial executor, then the execution of all jobs must be totally ordered : for any two different jobs A and B submitted to the same executor with enqueue(_:) , it must be true that either all events in A happen-before all events in B or all events in B happen-before all events in A .

Since in our case, the job that calls continuation.resume is still running. The executor can only run the Job that created the continuation afterwards. @ktoso please correct me if I said something wrong here.

[tcldr]

@FranzBusch

The sequence would be calling continuation.resume which would enqueue the resumption on the executor. Basically a queue.async {}.

Yeah, I think from a UI perspective then, unfortunately even this would end up causing the issues described above.

[Philippe_Hausler]

@tcldr

The reason for this is that both ObservedChanges and ObservedValues are asynchronous sequences. And despite the fact that a View and an Observable might both exist on the @MainActor, the fact that the Observable communicates to the View via an asynchronous sequence

SwiftUI does not use the AsyncSequence parts - it uses the ObservationTracking which has the leading edge (willSet) and manages its own end of transaction (which is the inspiration for how the main actor bound changes(for:) works). I went into some detail about that in the last section of the proposal.

[tcldr]

@Philippe_Hausler

SwiftUI does not use the AsyncSequence parts - it uses the ObservationTracking which has the leading edge (willSet) and manages its own end of transaction (which is the inspiration for how the main actor bound changes(for:) works). I went into some detail about that in the last section of the proposal.

I’m not sure how this would solve the problem of intra-actor observation. If I have two MainActor model objects in a parent-child relationship, and I want the parent to observe changes to the child, I would surely reach for changes(for:) which yields an async sequence.

If so, I’d still run into synchronisation issues because by the time my event has been delivered via a round-trip to the Default Executor/Shared Thread pool the event data could be stale. Also, any change I make based on that event would now be delayed by however long the event took to arrive.

Or, are you saying that we should perform intra-actor observation using ObservationTracking.withTracking(_:onChange:)? If so, how?

It isn’t clear (to me at least) what ObservationTracking.withTracking(_:onChange:) does. The example given shows a render() method, and then iteration through some Observable s that is scoped viaObservationTracking.withTracking(_:onChange:), and if there is a change to an Observable through that iteration, onChange is called exactly once? How would you use this for intra-actor observation?

[gwendal.roue (Gwendal Roué)]

I also have concerns with the fact that observation is async-only, and that the leading-edge is willSet-only.

The willSet-based behavior of @Published is a gotcha of ObservableObject, because it publishes before the system is in the interesting state (interesting for the end user). It is frequently mentioned here, or maybe on Twitter or Mastadon, or Stackoverflow - but basically people are surprised, and not happy, do discover that @Published publishes too early.

The async sequences create their own set of questions as well:

Do they produce a value even if the property is not changed? (I already asked above).

How is it possible to avoid a race between a modifier task and an observing task, when one wants to make sure that observation has started before the first change is performed?

Variant of the previous point: it seems that it is impossible to both start observation, and start performing changes from a single synchronous function.

---

I have a little experience in designing observation apis that address common use cases met by application developers. The GRDB SQLite toolkit provides two kinds of observation: ValueObservation and DatabaseRegionObservation. In particular:

• People often want to handle the "current" value and all its future states (I write "current" within quotes because people do not always have a clear idea of what a "current" value is in the context of a database). To this end, ValueObservation always start with an initial value.
• People often want to handle the "current" value right away, synchronously, because they are not interested in waiting for the initial value, and do not want to write any single line of code that handles this undesired waiting state, or even have a strong desire that their program is unable to express this waiting state. For those users, it is possible to start ValueObservation synchronously.
• For reasons that would be too long to describe here, ValueObservation can coalesce multiples changes together. People often do not care, as long as they eventually get the "latest" value (and they do get it).
• ValueObservation can be used as an AsyncSequence.
• Less people want to process absolutely all changes without any coalescing. More precisely: all transactions that impact the value. Those people should use DatabaseRegionObservation. Yes, observing values is not the same as observing transactions. People can't choose the Dispatch queue were transactions are notified (so that they can act before another transaction takes place). And since DatabaseRegionObservation notifies transactions instead of values, it does not notify anything until the first change.
• With both types, the moment at which users know for sure that the observation has started is clear. There is no race between modifiers and observers.

If you're interested, maybe click on the two above links to the documentation of those two types: you'll find a precise description of their behaviors and intended uses. Those types have been available for several years now, have been improved multiple times from user feedback, and they cover a great deal of observation needs by application developers. Oh, last word: there is support for "willSet" as well, with the low-level TransactionObserver.

[jenox (Christian Schnorr)]

@Philippe_Hausler

In order to provide scoped observation, the ObservationTracking type provides a method to capture accesses to properties within a given scope and then call out upon the first change to any of those properties.

public struct ObservationTracking {
    public static func withTracking<T>(
        _ apply: () -> T, 
        onChange: @autoclosure () -> @Sendable () -> Void
    ) -> T
}

The withTracking method takes two closures where any access to any property within the apply closure will indicate that property on that specific instance should participate in changes informed to the onChange closure.

When is onChange called when an observable property read in apply is mutated with respect to that mutation? Asynchronously? Synchronously before the mutation? Synchronously after the mutation? SwiftUI in particular would require this to be called with willSet semantics, as it may need to flush pending changes in case the change that's about to be made cannot be coalesced due to incompatible animation settings.

[Morten_Bek_Ditlevsen]

@Philippe_Hausler

During the render of the body of MyView the property of foo.baz is accessed which does not have any sort of macro effect. But... that property accesses self.bar which does! That means that when MyView.body is accessed the observation tracking knows that the instance of Foo has \.bar used. In turn after the body is rendered it then knows because the access list is non-nil (being that it collected that during the execution of the apply closure of withObservation it then adds tracking observations to all of those specific instances for those specific key paths. Subsequently then when the Button is pressed it will trigger that change of the mutation of bar which in turn triggers the observer for SwiftUI to mark the view as having its attributes invalidated and require rendering.

Hi @Philippe_Hausler ,
Thanks for clarifying that withTracking doesn't require the dependencies to be expressed. That's awesome!

Another question about withTracking compared to the changes/values apis: It is listed as a future direction to support key paths with more than one layer of components.

How would withTracking work with nested Observable objects - or structs inside of an Observable object? Would that still just work based on the tracking you explained above?

Hos does ObservationTracking actually track the changes? Will the implementation of it be written in Swift?