[Pitch] Observation

I am not sure how that would interact tbh; the mechanism isn't anything all that wild - it just uses key path based access. Which move-only/non-copyable types might pose a problem with when used in conjunction with type wrappers.

@xedin do you have any insights on how that will interact with type wrappers? if not do you happen to know who would be able to answer that?

Can it? yes it could... the information is definitely there... however this was a point that KVO has had some issue with; @David_Smith might be able to give better context on why that is a good idea or a bad idea per history with KVO.

There's a number of issues we ran into with KVO, all around concurrency:

  • If you're using locks to protect your state, you want to send KVO notifications outside the lock (recursive locks aren't sufficient because they can just dispatch_sync to another queue and then deadlock), which means if you send will- and did-change and want to avoid sending change notifications when nothing changed you need to do:

    lock
    check if the new value is different from the old value
    unlock
    will change
    lock
    change
    unlock
    did change
    

    and even that is thorny because someone else could come in when you unlock to send will change, and set a different new value, like this:

    thread 1: check if the new value is different from the old value
    thread 2: set the same new value as thread 1 is trying to
    thread 1: send will change
    recipient: look up current (supposedly pre-change) value
    thread 1: set new value
    thread 1: send did change
    recipient: look up new value, see it's the same as the old value
    developer of recipient: why is this framework so buggy
    

    (or the reverse situation where we decide to not notify because it's redundant, and it becomes non-redundant between the will and the did)

  • If you're getting the updated value "from the outside" and don't have it cached, then you may simply be unable to do this. This came up with iCloud UserDefaults, where we could get a change notification from another device, and by the time it made it to the client process the old value was gone.

  • With manual willChange/didChange, it's possible for developers to mis-nest them, like

    willChange: A
    willChange: B
    didChange: A
    didChange: B
    

    which greatly complicates the internal machinery for passing state between matched will and did pairs. The KVO test suite has a bunch of truly horrendous tests for edge cases like "what happens if many threads simultaneously mis-nest will/did while reentrantly setting the property being observed from inside the observation callback and occasionally throwing and catching exceptions?"

Most of these issues matter way less when you're only using observation for UI updates on the main thread of course. I don't think these are necessarily fatal flaws, but it's worth thinking things through carefully.

19 Likes

By my count this is the 4th iteration of observables in Swift (3 different KVO apis + ObservableObject) but I’m sure this time it will be the last :grin:.

Lot’s of good stuff in here.

Sounds like both value type changes and references to other observable objects can both be observed through the root observer? Value types because that’s how they already work: mutating a property on a struct triggers a mutation on the container. Observables through explicit support. That is an excellent addition and a common pitfall with the current ObservableObject model. That would just leave non-observable reference types. Would trying to observe a key path that included a non-observable cause an error?

Would this support computed properties and/or observing the entire object?

I don’t know if this is a use case worth worrying about, but I do occasionally use properties on ObservedObject that I explicitly do not want to be published, mostly for performance reasons with SwiftUI. For instance storing the scroll offset that I need when a button is tapped, but doesn’t need to trigger a render update. I suppose I could wrap that in a separate non-observable class.

Shouldn’t subjectDidSet include the previous value?

Am I understanding this correctly: ObservationTracking would enable SwiftUI to only update when properties that a view accessed changed? Huge if true!

One common pitfall with ObservableObject I’ve seen is that updates must be on the main thread/queue/actor but there is nothing in the protocol to enforce that. I’ve taken to marking all ObservableObjects as MainActor which might be the right approach, but is there any mechanism here that might address that? In an ideal world I would think an observable could operate on any context and that it would be be the observer that enforced the context it needed (which is how AsyncSequences work).

Correct; provided any reference types conform to Observable.

That part I am not sure about, it might be best served to no-op the observation and only let upper level items trigger the change and perhaps generate a runtime issue. (which seems like the right move to me)

The previous value can be kept track of by the observer itself and since the methods are mutating that value can be updated as a structural storage; which should be pretty efficient.

This particular facet was something I purposefully left out of the pitch; my prototypes as it stands have the potential at making that "just work", right now it works... but when updated off the main queue animations get a bit wonky. At worst; it could emit a runtime diagnostic and behave much like it is today (requiring the observed field to be main actor bound). But since the coupling is loose in SwiftUI in general I am not sure there is a way to enforce that via the type system.

1 Like

What about wrapping the data mode in an observer type with dynamic member lookup? API authors could add a typealias for the new type if it’s too verbose. Also, I agree that the current wrapper approach for ObservableObject isn’t ideal but I don’t think users are really confused by computed properties publishing events.

The problem with dynamic member lookup is that will only be usable for public fields and the private computed fields won't contribute to change events. So it could satisfy part of the API but the other part (the part that interacts with SwiftUI to determine fields contributing to the rendering of views) would not work as extensively and would leave holes that would not update the UI when one would expect them to.

2 Likes

Curious about fileprivate and internal. Seems like it should be consistent with the visibility model at all levels rather than just a special case on private?

All access control works as the access to the field by virtue of the ability to create key paths. For adding an observer; externally you can only create key paths to the visibility of that field - unlike KVO which allows strings to be passed observing things that are potentially beyond the visibility that you ought to be able to access.

5 Likes

Thanks.

One of the great shortcomings of KVO (imho) was the inability to precisely control the thread model. AKA:

  • on what thread am i suppose to register my observation ?
  • on what thread am i going to receive the observed call ? the thread doing the update, the thread i used to register my observation ?

A very common example is the UI (such as a view controller) observing a model being mutated on the background thread.

How would that new approach work regarding to threading / actor, etc ?

async methods declare the threading context they need, rather than the thing calling them handling it.

If another party modifies .seconds, this view will not be notified that .time changes, right? I saw @Jon_Shier suggesting a mechanism whereby changes to one property triggers a change notification for another property, similar to keyPathsForValuesAffectingValue(forKey:), and you say that’s under consideration. I think that will probably be necessary.

Related to that, how are property wrappers to be handled? An observation of the wrapped property must directly or indirectly observe changes to the wrapper’s backing property as well. Are there degenerate cases to account for? For example, is a property wrapper required to have a stored property? Along those lines, does the first evaluation of a lazy property count as a change for the purposes of observation? I’d say “no” to that.

Could you describe this in a little more detail? What happens to the observation of \A.b.c.d when the instance at c (c1) is replaced by a different instance, c2? Does the observer continue to receive changes to c1.d1, or does the observer start receiving changes to c2.d2 instead? I believe I would prefer the latter myself.

I wouldn’t want that. In the example of \A.b.c.d, if B were a struct but C was an observable object, I would certainly still want to be able to observe changes to d, given how ubiquitous value types are in Swift. If the value at b changes, that goes back to my previous question about transitivity—that’s something that needs to be considered regardless of whether B is a reference or value type.

Finally, some questions about concurrency and actors:

  • Can observable objects conform to Sendable, when they must mutate to retain their observers?
  • If an observable object is an actor (probably the easiest answer), do observers’ change handlers execute in the actor’s isolation context or are the handlers nonisolated? Probably the latter, but that makes referring back to the observed object’s properties tricky—you might want to ship a value in the parameter of a change handler rather than having them () -> ().
  • The implementation may have to deal with get async throws properties.
1 Like

I was under the impression that this would trigger a new update cycle.

@Philippe_Hausler im also curious about threading. Is this ONLY for single threaded stuff or is there a plan around async stuff?

Another quirk around concurrency is atomicity. Observing properties individually and aggregating the resulting AsyncSequences can yield a time series that may be temporally inconsistent with atomic writes to the object that touch any combination of these observed properties.

Sure, as evidenced by practices of FRP I've observed over the past years, it has been worked around through. either:

  • throttling/debouncing in the time domain (at the cost of delay + async hops, while a slim theoretical chance of inconsistency still remains);
  • packing related information into the same stored property (e.g. a struct); or
  • just outright accepting the temporal effect.

Supporting an observation on multiple keypaths could be a solution, though that would depend on variadic generics. But while this solves the consumer-side issue, it imposes a challenging expectation on the producer (the type/subject being observed) to have to carefully dance around their lock usage, since a write access to a property can now have a side effect of reading neighbouring properties for multi-keypath observation fulfilment.

With this, I am honestly unsure about the value proposition of generalising this, except for continued support for SwiftUI usage, and perhaps a strong desire to improve ergonomics of Swift ORM frameworks for reactive UI usages. However, these frameworks and also SwiftUI (so far) have dodged most concurrency issues through thread confinement and/or reliance on the event loop, which does not seem to be a compatible concept with the new Swift Concurrency world, where the generalised, "Swiftifed" KVO is supposed to work with.

Perhaps most importantly, streams of immutable values have been trending upwards over the conventional "big mutable model objects + KVO on properties" pattern, evidently with the Composable Architecture. I am not able to imagine a strongly convincing use case to resurrect full-blown KVO from the exiled land of "for Cocoa compatibility only".

10 Likes

Observation of an entity implies that entity has identity. That means that things that are observable are inherently reference types.

Is there a way to also allow for tracking struct properties with this proposal too? Perhaps an API that's similar to @dynamicMemberLookup for Self that Observable: AnyObject composes on top of. It would be great to be able to track generic changes to a struct via KeyPaths.

i.e.

struct Thing {
    let one: Int
    var two: Bool
    var three: String
    
    subscript<T>(storedProperty property: KeyPath<Self, T>) -> T {
        get {
            self[keyPath: property]
        }
        set(newValue) {
            print("setting value to \(newValue)")
            self[keyPath: property] = newValue
        }
    }
}

Perhaps it's a whole separate conversation / proposal.

final class ExampleObservable: ObservableObject {
    @Published private var minutes = 1
    @Published private var seconds = 0

    var time: String {
        String(format: "%.2ld:%.2ld", self.minutes, self.seconds)
    }
}

Do I understand correctly that calling observable.changes(for: \.time) will observe minutes and seconds.

Does that mean that implementation needs to call a getter for times to discover that it depends on minutes and seconds?

class A: ObservableObject {
    @Published private var foo = 1
}

class B: ObservableObject {
    var a: A
    var bar: Int { a.foo + 2 }
}

class C: ObservableObject {
    var a: A
    var baz: Int { a.foo + 3 }
}

class D: ObservableObject {
    var b: B
    var c: C
    var qux: Int { b.bar * c.baz }
}

Do I understand that under the hood only the stored properties are observed, so diamond dependency is not a problem. When observing D.qux and A.foo changes, will I receive two notifications or only one?

And also +1 to concerns shared by @Anders_Ha. I would prefer to have some way to batch changes. I would not be ok with API that requires me to explicitly enumerate key paths I’m about to modify:

final class ExampleObservable: ObservableObject {
    @Published private var minutes = 1
    @Published private var seconds = 0

    func tick() {
        batchChanges([\.minutes, \.seconds]) {
            if seconds == 59 {
                seconds = 0
                minutes += 1
            } else {
                seconds += 1
            }
        }
    }
}
1 Like

An easy way to batch updates is to structure your mutable state as a single value which is mutated. That allows you to mutate individual properties and batch updates by performing inout mutations. Frankly, I think most observable state should be structured like this. Independent but related observable properties are one of the big causes of complexity and bugs in reactive systems. TCA structures its state like this and it works really well.

final class MyObservable: Observable {
  struct State {
    var string = ""
    var int = 0
  }

  var state = State()

  // Each mutation triggers separate observation.
  func performMutation() {
    state.string = "newString"
    state.int += 1
  }
    
  
  func performBatchedMutation() {
    batch {
      $0.string = "batched"
      $0.int += 10
    }
  }

  // Each mutation triggers one observation.
  private func batch(performing mutation: (inout State) -> Void) {
    mutation(&state)
  }
}

It occurs to me that this sort of structure should really have the observable state be private with dynamic member mutation, so we should really make that customization as easy as possible.

4 Likes

I love the idea of moving away from @Published, but I kind of like continuing to annotate my dependencies in SwiftUI.

I sometimes pass ObservableObjects around as simple vars for performance reasons (parent doesn't need to update, but child does, or passes it to UIKit). What would happen in this contrived case if it were converted to Observable? Obviously in this example there wouldn't be any serious issues, but could there be?

class Foo: ObservableObject {
	@Published var bar = false
}

struct ContentView: View {
	@State var foo = Foo() // Intentionally *not* observing in this view
	var body: some View {
		FooUser(foo: self.foo) {
			if self.foo.bar {
				Color.red
			}
		}
	}
}

struct FooUser<Content: View>: View {
	@ObservedObject var foo: Foo
	var content: Content // Not a closure!
	
	init(foo: Foo, @ViewBuilder content: () -> Content) {
		self.foo = foo
		// foo.bar will be accessed when calling `content()`
		// Does this cause ContentView to start observing foo.bar?
		// Could that matter?
		self.content = content()
	}
	
	var body: some View {
		if self.foo.bar {
			self.content
		}
	}

}

OffTopic: keep in mind that unlike StateObject, State is not lazy initialized. That said, every time a ContentView is created you spawn a new Foo object until it gets discarded when the first Foo object gets re-injected by the framework.

1 Like