[Pitch] Observation

One of the great shortcomings of KVO (imho) was the inability to precisely control the thread model. AKA:

  • on what thread am i suppose to register my observation ?
  • on what thread am i going to receive the observed call ? the thread doing the update, the thread i used to register my observation ?

A very common example is the UI (such as a view controller) observing a model being mutated on the background thread.

How would that new approach work regarding to threading / actor, etc ?

async methods declare the threading context they need, rather than the thing calling them handling it.

If another party modifies .seconds, this view will not be notified that .time changes, right? I saw @Jon_Shier suggesting a mechanism whereby changes to one property triggers a change notification for another property, similar to keyPathsForValuesAffectingValue(forKey:), and you say that’s under consideration. I think that will probably be necessary.

Related to that, how are property wrappers to be handled? An observation of the wrapped property must directly or indirectly observe changes to the wrapper’s backing property as well. Are there degenerate cases to account for? For example, is a property wrapper required to have a stored property? Along those lines, does the first evaluation of a lazy property count as a change for the purposes of observation? I’d say “no” to that.

Could you describe this in a little more detail? What happens to the observation of \A.b.c.d when the instance at c (c1) is replaced by a different instance, c2? Does the observer continue to receive changes to c1.d1, or does the observer start receiving changes to c2.d2 instead? I believe I would prefer the latter myself.

I wouldn’t want that. In the example of \A.b.c.d, if B were a struct but C was an observable object, I would certainly still want to be able to observe changes to d, given how ubiquitous value types are in Swift. If the value at b changes, that goes back to my previous question about transitivity—that’s something that needs to be considered regardless of whether B is a reference or value type.

Finally, some questions about concurrency and actors:

  • Can observable objects conform to Sendable, when they must mutate to retain their observers?
  • If an observable object is an actor (probably the easiest answer), do observers’ change handlers execute in the actor’s isolation context or are the handlers nonisolated? Probably the latter, but that makes referring back to the observed object’s properties tricky—you might want to ship a value in the parameter of a change handler rather than having them () -> ().
  • The implementation may have to deal with get async throws properties.
1 Like

I was under the impression that this would trigger a new update cycle.

@Philippe_Hausler im also curious about threading. Is this ONLY for single threaded stuff or is there a plan around async stuff?

Another quirk around concurrency is atomicity. Observing properties individually and aggregating the resulting AsyncSequences can yield a time series that may be temporally inconsistent with atomic writes to the object that touch any combination of these observed properties.

Sure, as evidenced by practices of FRP I've observed over the past years, it has been worked around through. either:

  • throttling/debouncing in the time domain (at the cost of delay + async hops, while a slim theoretical chance of inconsistency still remains);
  • packing related information into the same stored property (e.g. a struct); or
  • just outright accepting the temporal effect.

Supporting an observation on multiple keypaths could be a solution, though that would depend on variadic generics. But while this solves the consumer-side issue, it imposes a challenging expectation on the producer (the type/subject being observed) to have to carefully dance around their lock usage, since a write access to a property can now have a side effect of reading neighbouring properties for multi-keypath observation fulfilment.

With this, I am honestly unsure about the value proposition of generalising this, except for continued support for SwiftUI usage, and perhaps a strong desire to improve ergonomics of Swift ORM frameworks for reactive UI usages. However, these frameworks and also SwiftUI (so far) have dodged most concurrency issues through thread confinement and/or reliance on the event loop, which does not seem to be a compatible concept with the new Swift Concurrency world, where the generalised, "Swiftifed" KVO is supposed to work with.

Perhaps most importantly, streams of immutable values have been trending upwards over the conventional "big mutable model objects + KVO on properties" pattern, evidently with the Composable Architecture. I am not able to imagine a strongly convincing use case to resurrect full-blown KVO from the exiled land of "for Cocoa compatibility only".

10 Likes

Observation of an entity implies that entity has identity. That means that things that are observable are inherently reference types.

Is there a way to also allow for tracking struct properties with this proposal too? Perhaps an API that's similar to @dynamicMemberLookup for Self that Observable: AnyObject composes on top of. It would be great to be able to track generic changes to a struct via KeyPaths.

i.e.

struct Thing {
    let one: Int
    var two: Bool
    var three: String
    
    subscript<T>(storedProperty property: KeyPath<Self, T>) -> T {
        get {
            self[keyPath: property]
        }
        set(newValue) {
            print("setting value to \(newValue)")
            self[keyPath: property] = newValue
        }
    }
}

Perhaps it's a whole separate conversation / proposal.

final class ExampleObservable: ObservableObject {
    @Published private var minutes = 1
    @Published private var seconds = 0

    var time: String {
        String(format: "%.2ld:%.2ld", self.minutes, self.seconds)
    }
}

Do I understand correctly that calling observable.changes(for: \.time) will observe minutes and seconds.

Does that mean that implementation needs to call a getter for times to discover that it depends on minutes and seconds?

class A: ObservableObject {
    @Published private var foo = 1
}

class B: ObservableObject {
    var a: A
    var bar: Int { a.foo + 2 }
}

class C: ObservableObject {
    var a: A
    var baz: Int { a.foo + 3 }
}

class D: ObservableObject {
    var b: B
    var c: C
    var qux: Int { b.bar * c.baz }
}

Do I understand that under the hood only the stored properties are observed, so diamond dependency is not a problem. When observing D.qux and A.foo changes, will I receive two notifications or only one?

And also +1 to concerns shared by @Anders_Ha. I would prefer to have some way to batch changes. I would not be ok with API that requires me to explicitly enumerate key paths I’m about to modify:

final class ExampleObservable: ObservableObject {
    @Published private var minutes = 1
    @Published private var seconds = 0

    func tick() {
        batchChanges([\.minutes, \.seconds]) {
            if seconds == 59 {
                seconds = 0
                minutes += 1
            } else {
                seconds += 1
            }
        }
    }
}
1 Like

An easy way to batch updates is to structure your mutable state as a single value which is mutated. That allows you to mutate individual properties and batch updates by performing inout mutations. Frankly, I think most observable state should be structured like this. Independent but related observable properties are one of the big causes of complexity and bugs in reactive systems. TCA structures its state like this and it works really well.

final class MyObservable: Observable {
  struct State {
    var string = ""
    var int = 0
  }

  var state = State()

  // Each mutation triggers separate observation.
  func performMutation() {
    state.string = "newString"
    state.int += 1
  }
    
  
  func performBatchedMutation() {
    batch {
      $0.string = "batched"
      $0.int += 10
    }
  }

  // Each mutation triggers one observation.
  private func batch(performing mutation: (inout State) -> Void) {
    mutation(&state)
  }
}

It occurs to me that this sort of structure should really have the observable state be private with dynamic member mutation, so we should really make that customization as easy as possible.

4 Likes

I love the idea of moving away from @Published, but I kind of like continuing to annotate my dependencies in SwiftUI.

I sometimes pass ObservableObjects around as simple vars for performance reasons (parent doesn't need to update, but child does, or passes it to UIKit). What would happen in this contrived case if it were converted to Observable? Obviously in this example there wouldn't be any serious issues, but could there be?

class Foo: ObservableObject {
	@Published var bar = false
}

struct ContentView: View {
	@State var foo = Foo() // Intentionally *not* observing in this view
	var body: some View {
		FooUser(foo: self.foo) {
			if self.foo.bar {
				Color.red
			}
		}
	}
}

struct FooUser<Content: View>: View {
	@ObservedObject var foo: Foo
	var content: Content // Not a closure!
	
	init(foo: Foo, @ViewBuilder content: () -> Content) {
		self.foo = foo
		// foo.bar will be accessed when calling `content()`
		// Does this cause ContentView to start observing foo.bar?
		// Could that matter?
		self.content = content()
	}
	
	var body: some View {
		if self.foo.bar {
			self.content
		}
	}

}

OffTopic: keep in mind that unlike StateObject, State is not lazy initialized. That said, every time a ContentView is created you spawn a new Foo object until it gets discarded when the first Foo object gets re-injected by the framework.

1 Like

TL;DNR - Was replacing the Combine Publisher with an AsyncStream considered?

In an ObservableObject an @Published is used to wrap your property into a publisher. This forces your code to depend on the Combine framework and adds the limitations as mentioned by the author.
Isn't an AsyncStream the modern concurrency equivalent of a combine publisher?

What about a StreamableObject equivalent of ObservableObject?
One whose parameters can be marked with @Streamed instead of @Published?
Then you could use an @StreamedObject in place of an @ObservedObject.

Was this considered? It seems like it would be the closest to a drop in replacement for Combine to modern concurrency.

final class Model: StreamedObject {
    @Streamed private var minutes = 1
    @Streamed private var seconds = 0

    var time: String {
        String(format: "%.2ld:%.2ld", self.minutes, self.seconds)
    }
    
    func incrementMinutes() {
        minutes += 1
    }
}

struct ExampleView: View {
    @StreamedObject var model = Model
    
    var body: some View {
        Text(model.time)

        Button {
            model.incrementMinutes()
        } label: {
            Text("Increment Minutes")  
        } 
    }
}

SwiftUI would use the AsyncStream under the hood to await changes, in the same way that it now uses publishers.
Outside of SwiftUI you could observe the model changes using modern concurrency:

let model = Model()

for await time in model.$time {
    print(time)
}

In short, the same goals when using ObservableObject, @ObservedObject, and @Published would be accomplished by using an AsyncStream instead of a Combine.Publisher. (Thereby removing the dependency on the Combine framework)

StreamableObject would use a continuation for objectWillChange instead of how an ObservableObject uses a publisher.

And the @Streamed would use an AsyncStream as its projected value similar to how the @Published uses a Publisher.

All current functionality and familiar patterns would remain. The solution would just no longer use publishers "under the hood".

The model.time property would trigger a UI update in exactly the same way as it would with an @Published on and ObservableObject.

Just wondering if this was considered. Maybe this is too small scope of a change?

1 Like

I have the exact same question. In fact we implemented exactly what you are talking about here, and even named the property wrapper @Streamed. It would be good to know if this was considered. It does seem simpler than the proposal in many ways.

Observation itself can be registered on any thread/queue/actor, however changes to values of the observed thing must manage their own safety to the value being changed (being observable does not magically make something that wasn't thread safe to be thread safe).

The observed changes via the Observer types being passed in as a registered observer must (because of ordering of will/did change events) be on the same thread/queue/actor the change originated from. However observing the trailing edge via the AsyncSequence of changes is asynchronous so therefore it is on whatever actor it is doing the iteration.

Correct. The consideration for keyPathsForValuesAffectingValue(forKey:) is more for a nuanced case of adding observers for a given field (not for the general UI case since that already handles those).

For those fields any setter of the type wrapped accessor should trigger the change in both the general changes (for UI) as well as for the specific keypath observation case.

When c2 is set the old observation is removed from c1 and a new observation is installed. So it works as you prefer/expect.

That case works as you expect.

There is nothing stopping them from doing so. However that does come at the cost of managing the cases where you have a class and dealing with mutations on many tasks.

This is not nearly as easy as you might expect... Actors run afoul from an isolation perspective with type wrappers. To support them with that mechanism we need some sort of way of saying "I inherit isolation from a specific instance". We don't yet have this mechanism; so in the initial versions actors won't be supported for automatic default implementation access.

This is in the realms of the type wrapper feature; whatever that supports this will. However I expect that likely will "just work". However @hborla or @xedin might be more versed with that nuance.

At the point we have variadic generics I think it would definitely be possible to implement; as it stands I have access to the right stuff internally to the ObservationList to build the change sets from the fields but emitting the changed values and their key paths would either mean type erasure or wait for variadics (which I am more keen to do since that seems like a better fit for a future addition).

That is almost precisely the type wrapper feature.

That is the case that is not yet supported without the whole keyPathsForValuesAffectingValue(forKey:) addition.

That pattern was definitely something I closely looked at for how SwiftUI works. I have not found a single case yet that would impact poorly. Since it only invalidates views for things that are accessed in the scope of body it means that even if you use an intermediary view to pass the Observable thing down it does not in correctly update the upper level views.

This is something that I ran across. It is something that can be a bit tricky and I have talked at length about the options of revisiting this specifically for Observable things.

Yes it was one of the considerations. However the wins don't really out weigh the impact of task creation. Effectively with that route we have to spin up tons of small Task objects to do the iteration and that cost ends up out weighing any wins we could end up getting.

Can you expand on that? Is spinning up a Task expensive? Does this only apply in the SwiftUI context or whenever you observe an AsyncSequence?

Tasks are not free to create. They cost for their allocation and other resources too. If there are designs to avoid creating tons of root tasks (not child tasks) they should be avoided if where you are doing so would be performance sensitive. It isn't as costly as creating a pthread each time, but it isn't as cheap as testing an optional or just creating a closure. From a performance standpoint it is better to deal in structured concurrency and limit unstructured tasks to specific minimums.

4 Likes

I believe this question remains for @MainActor. Do observers’ change handlers execute in the main actor’s context or are they nonisolated and execute any Task they spin up on a background thread?

Change handlers do not re-thread anything. They execute on the actor/thread/queue properties are changed upon. However actions taken from SwiftUI may (which I still have on my docket to verify designs about) correct that to the main actor. That hopefully will side-step the issue with @Published failing when assigned from non-main actor sources.

2 Likes

But in my example, the access would happen in ContentView.body, during the construction of FooUser not during FooUser.body. So are you just saying you don't expect that access to have a large impact on performance? (And I admit, I struggle to come up with a simple example of this that's not obviously buggy.)

I'm definitely concerned about some sort of keyPathsForValuesAffectingValue being needed for computed properties or functions to work properly with SwiftUI. With @Published those seem more explicit. Is tracking individual property accesses strictly necessary instead of just tracking any change to self? It almost feels like premature optimization.