Since type wrappers aren’t an accepted feature, are any other implementation paths you have considered? E.g. could it hook into runtime functions or somehow employ reflection to achieve the same API without any code-generation feature?
I like the looks of this, but it feels like it's missing something (or maybe it's just me).
For example, let's say I have an object that's receiving network messages. I might have previously used NotificationCenter to post an event that can be observed whenever a new message arrived along with an object that represents the message, but no external properties in the streaming object may have needed to change for this to occur, so I couldn't refactor this to use Observable, right?
In my experience it's not uncommon for an object that's doing some kind of event processing like this to still have useful state properties like isConnected which would be a good fit for Observable. But since it doesn't seem like Observable can support event delivery that isn't tied to a property state change, it seems like one would have to mix and match both Observable and NotificationCenter or maybe Combine (which I'm not very familiar with) or somehow roll their own custom observer management stuff, or something else... just for the message delivery part of the object - which seems like a shame.
There definitely have been a number of iterations, reflection and proxies were an early sketch, but that ends up falling apart when dealing with computed properties and change handlers in SwiftUI.
Another early design included property wrappers to accomplish things too. This replicated some of the same problems with ObservableObject.
I think you might be able to, however it might be able to be made a bit easier to do so if some of the notification mechanism is split out from the ObservationList.setMember call. This could use a slight bit of noodling to make that easier; I will get back to ya on that one.
2 Likes
The problem with observability/KVO is that it makes a program a lot harder to understand. Because what looks like a simple property mutation may actually hide a lot of side-effects. This is already a problem to some extent with property wrappers and didSets (but still manageable) but this seems like it would be even harder to spot: having to first realize that a property might have side-effect, having to jump to definition, then having to look at the enclosing class (or one of its extension?) to see if it is Observable, then having to look for potential changes(for:) calls across the codebase. We spend a huge amount of time reading code, other's people code or our own past-self code, back in the days KVO was considered harmful by a number of us because it could make the codebase incomprehensible. Alas I don't have a solution, I'm worried about hidden side-effects, they are usually not a good thing.
11 Likes
I like the proposal!
Found a bit suspicious the use of private to prevent observation: immediate reaction to that – perhaps sometimes there'll be a need to make some variable non private (for unrelated reasons!) but then it would suddenly change it to be observable which might break things.
Unmanaged, implicit observability can lead to complex systems. Making such a capability explicit will help with that issue and Apple has a management solution in SwiftUI. But really it's up to what you make of it. A lot of the issues around KVO weren't necessarily the complexity of the observations themselves but what you did with them. We've learned a lot about that as reactive systems have become more popular. Plus KVO was just so unreliable (there was no single way to make a property KVO-compliant) it was hard to build larger systems around. A system with a well structured, reliable, explicit implementation should be far easier to get under control, especially with what we know now.
As for the pitch itself, I really appreciate the structured combination of Options and separate willSet and didSet. I do think .compareEquality should be something more like .removeDuplicates, as .compareEquality gives no indication of what it actually does (does it give me an equality result?). I also think some of the APIs could be more fluent, and I dislike get as a function prefix, it's redundant. (I also saw it pop up in the runtime attributes pitch.) So I would spell the getMember API as:
public func member<Member>(
on subject: Subject,
forProperty propertyKeyPath: KeyPath<Subject, Member>,
storingAt storageKeyPath: KeyPath<Storage, Member>,
using storage: Storage
) -> Member
There are other, similar changes, I would make (like setMember) but I'll save those for another revision.
One thing I would like to see here is performance, especially runtime CPU and memory impact. If there's no implementation yet I understand that's unknown, but I'd still like to get a feel for the goals here. I would also like see a bit more discussion about observation lifetimes, as that's always been one of the biggest issues with KVO and stream observation systems.
8 Likes
I feel uneasy too, but I also feel like forcing such a strict separation between observable and non-observable state might be good for managing complex systems of observations. I can imagine an object which encapsulates all of its observable state in a subtype exposed through a property (with dynamic key path lookup possibly). That doesn't quite help in situations where systems require the object itself to be Observable, like SwiftUI would, but that may also be a good thing, as it forces you to be very careful about what you put in the system in the first place.
One other bit of functionality to think about is easy composition of sibling observables. Something like observableA.onChange(trigger: observableB) so you can influence emission without having to internally tie the observed objects together.
5 Likes
I think there is a slight misunderstanding about that portion - the public/private portion is about adding observers directly to properties. Basically it is built upon being able to observe via key path. Since externally you cannot form key paths from private things it prevents the observation of those items. However change tracking on the other hand does interact with any field since it never exposes the key path or the value of those fields, only the fact they have an access and that those accessed fields changed.
That difference is why the ObservationTracking.Fields has no way to extract the key paths put into it. Because else wise you could leak out implementation details.
Observation and change tracking are inherently related but observation is the external interface, change tracking is both the public and internal field access.
3 Likes
The problem is that there seems to be no way to tell whether a property change has side-effects. Today you can jump to definition and see if a property has a wrapper or a didSet. Then you can read the code and see what it's doing (such as notifying observers). But here, how can you tell? Furthermore once you eventually realize the enclosing class is Observable, how do you find the observers?
ObservationTracking is really cool and I think I almost get there api but can you explain how to use it?
There are a few ways ObservationTracking can be used. I will step through them from the perspective of how they work in relation to SwiftUI.
public static func withTracking(_ apply: () -> Void) -> ObservationTracking?
The withTracking(_:) API defines an execution context in which fields will be gathered for tracking changes. This function will setup a storage location for an access list of fields, execute the passed in apply function and then clear out that access list. If anything was stored into the access list, then a tracking structure is created and returned and if nothing was accessed then nil is returned.
public static func registerAccess<Subject: Observable>(propertyKeyPath: PartialKeyPath<Subject>, subject: Subject)
During the call of the closure passed to withTracking(_:) any calls to registerAccess(propertyKeyPath: subject:) will test to see if there is storage for the access list, create one if needed, and then add the key path and subject into the access list; indicating that an access to an Observable was fired during the execution of that closure passed into withTracking(_:). This function and the withTracking(_:) are very light-weight if no active tracking is present, and also quite performant if there is. Basically both of these functions are intended to handle performance critical areas.
public func addChangeHandler(options: Options = [.willSet, .compareEquality], _ handler: @Sendable @escaping () -> Void)
The addChangeHandler(options: _:) function registers change handlers for each accessed field that was detected by calling addChangeHandler(for:options:,_:) on each Observable. These change handlers will invalidate on the first fire of any of the handlers. By default (since this API is designed for the use with UI systems and allow for transaction animations) it registers on the leading edge, the .willSet, and will enforce equality checks where they are applicable to determine the changes that occur (i.e. if a property is set to a value that is claimed as == it does not count as a change provided that value adopts Equatable).
public func invalidate()
If no change has been fired (and there is no consequence for doing it post-facto of a change) the invalidate() function can be invoked to remove all handlers that were registered.
Putting this all together:
func calculateBody(viewInvalidation: Invalidator, _ buildBody: () -> Void) {
if let tracking = ObservationTracking.withTracking({
buildBody()
}) {
tracking.addChangeHandler {
viewInvalidation.trigger()
}
}
}
1 Like
To circle back on this: I think with some minor modifications this use case can be addressed.
If the ObservationList drops the generic requirement for the storage and instead has the get/set methods changed to the following signatures:
public func getMember<Member>(
_ subject: Subject,
propertyKeyPath: KeyPath<Subject, Member>,
_ fetch: () -> Member
) -> Member
public func setMember<Member>(
_ subject: Subject,
propertyKeyPath: KeyPath<Subject, Member>,
oldValue: Member,
newValue: Member,
update: (Member) -> Void
)
That would allow for external systems to work as you described more easily. The added benefit of the removal of the extra generic parameter is also attractive. So far this looks like a good solution but I need to test it to make sure it satisfies all the performance and behaviors we want to address. If it works both for the general use and the specialized example you mentioned I think it is something worth considering as an alteration to the pitch.
1 Like
So is the intention then that swiftUI would call withTracking on every body execution and its like a one time use thing?
I guess swiftUI would need to hold on to tracking here?
Also, I think you said doing tracking does NOT retain the objects right?
Would this track ANY Observable that happens in withTracking?
Correct.
In my current builds it does not.
The ObserverTracking does incur a retain currently for as long as you hold that structure (you can think of the tracking as an immutable list of Observables and sets of fields to them). However the change trackers and such are not causing a retain.
Yup, here is an example:
final class MyModel2: Observable {
var test: String = "test"
var rotation = 0.0
}
final class MyModel: Observable {
var test = "test"
var foo = MyModel2()
var test2: String = ""
}
struct External {
var model = MyModel()
}
struct ContentView: View {
let external = External()
var body: some View {
VStack {
Image(systemName: "globe")
.imageScale(.large)
.foregroundColor(.accentColor).rotationEffect(.degrees(external.model.foo.rotation))
Text(external.model.foo.test)
Button("Change") {
withAnimation {
external.model.foo.rotation += 45
}
}
}
.padding()
}
}
In that example the External structure holds objects externally to the field list of the view. So the view itself has no knowledge of any of the Observables it might contain. But because the scope of body is wrapped in the apply of the ObservationTracking.withTracking it means that the accesses to the fields of .foo, .test and .rotation end up tracking the instances of external.model and external.model.foo for those fields (on those respective instances).
Since we know that the closure executed in withTracking is NOT async we can know that the same thread is accessed in all field accesses. So under the hood it uses a thread local slot to store the list. And since we are getting registrations of access via the registerAccess function those can be uniquely stored in that list of accessors. Specifically accessing the same field 10 times only adds one entry.
Hopefully that sheds some light into the details on how that particular system works.
4 Likes
Would that change to something else if we altered the API to look more like the example I illustrated for @BigZaphod? They practically become fetch/update; which honestly are decent names. The intent of the previous names was to indicate that they are intended for use with get/set in the type wrapper subscripts for building your own type wrapper.
I may have to take your word for it that this addresses the ability to send an event, because I don't think I quite get it yet! One thing I'm not sure about is how the Observable object itself gets access to its own ObservationList in the first place - maybe that was mentioned or I'm missing it in the API, though. The other is that if I did have access to the list, it seems like there would still need to be a "dummy" key path for observers to subscribe to in order to get the "new message arrived" event, right?
That is from the #Future Directions section (it is placed there since we need to make a few alterations to the type wrapper proposal to wire this all together). The type wrapper DefaultObservable gets applied to the protocol. This means that any adopter (by default) will be wrapped up in a type wrapper. This means that all field access to any stored item is funneled into that type wrapper instance.
Lets say you have an observable defined as such:
final class MyModel: Observable {
var name: String = "test"
init() { }
}
The type wrapper effectively transforms this to:
final class MyModel: Observable {
struct $Storage {
var name: String
}
var storage: DefaultObservable<Self, $Storage>
var name: String {
get {
storage[wrappedSelf: self, propertyKeyPath: \MyModel.name, storageKeyPath: \$Storage.name]
}
set {
storage[wrappedSelf: self, propertyKeyPath: \MyModel.name, storageKeyPath: \$Storage.name] = newValue
}
}
init() {
storage = DefaultObservable(for: Self.self, storage: Storage(name: ""))
}
}
The DefaultObservable then has an ivar of both the $Storage and the ObservationList. So it can manage the access and fetching stuff out of the storage as well as telling the observation list when things are set.
5 Likes
Ah ha - that's what I was missing! Thanks for that. It's starting to make more sense now.
How does this interact with non-copyable types? I see eg. ObservedChanges.Element has newValue, which wouldn't be possible if Member is non-copyable. IIRC, the non-copyable types proposal simply disallows them from participating in generics at this stage, so does that mean they simply can't be observed, even if all I want to know is "that they changed" rather than "how they changed"?
KVO has the option to send the old value with a change, as well as the new one. This makes certain kinds of change observation easier, since both relevant values are "right there", and one doesn't have to go back to the object to retrieve whichever of old/new is missing. ObservedChanges.Element contains only the newValue, which definitely makes some things harder. Can it / should it also offer an oldValue property?
2 Likes