Observation
Introduction
Making apps that are responsive often requires mechanisms to observe changes that are applied from a model to drive changes elsewhere in the presentation of that data. The observer pattern allows a subject to maintain a list of observers and notifies them of specific or general state changes. This has the advantage of not directly coupling objects together; an object that is observed needs no information about the observer other than it is an observer. In addition to the decoupled nature it also has an implicit distribution of the updates across potential multiple observers.
This design pattern is a well traveled path by many languages and Swift has an opportunity to provide a robust, type safe, and performant mechanism for offering this pattern as a low level uniform system. This proposal defines what an observable reference is, what an observer needs to conform to, and the connection between those.
Motivation
There are already a few mechanisms for observation in Swift. Some of these include Key Value Observing (KVO), or ObservableObject; but those are relegated to in the case of KVO to just NSObject descendants, and ObservableObject requires using Combine which is restricted to Darwin platforms and does not leverage language features like async/await or AsyncSequence. By taking experience from those existing systems we can build a more generally useful feature that applies to all Swift reference types; not just those that inherit from NSObject and have it work cross platform with using the advantages from low level language features like async/await.
Prior art
KVO
Key Value Observing in Objective-C has served that model well but is limited to just systems based on inheritance to NSObject. The APIs only offer the intercepting of events (meaning that the callout to the inform of the changes is between the will/did events). KVO has great flexibility with the granularity of events but lacks in composability. Observers for KVO must inherit from NSObject and rely on the objc runtime to track the changes that occur. Even though the interface for KVO has been updated to utilize the more modern Swift keypath constructs that are strongly typed, under the hood it is still stringly typed events.
Combine
Combine's ObservableObject produces changes on the leading edge of the will/did events and all delivered values are before the value did set. Albeit this serves SwiftUI well, it is restrictive for non SwiftUI usage and can be surprising to developers first encountering that restriction. ObservableObject also requires that all observed properties to be marked as @Published to interact with the change events. In most cases this requirement is applied to every single property and becomes redundant to the developer; meaning folks writing an ObservableObject conforming type must repeatedly (with little to no true gained clarity) annotate each property - in the end this results in a fatigue of the meaning of what is a participating item or not.
Proposed solution
A formalized observer pattern needs to support the following:
- Marking a reference type as "observable"
- Tracking changes within an instance of an observable type
- The ability to observe and utilize those changes from somewhere else, e.g. another type
A type can declare itself as observable simply by conforming to the Observable protocol:
final class MyObject: Observable {
var someProperty: String = ""
var someOtherProperty = 0
}
Unlike ObservableObject and @Published, none of the fields of the type need to be individually marked as observable. Instead, all fields of the observable type are implicitly observable. Since creating key paths is limited to the visibility of the fields this means that the control of observation is relegated to the visibility of the fields. If a developer needs to restrict the observation of a field, it can be marked as private which prevents external access to the key path from being constructed and consequently observation.
The Observable protocol includes a set of extension methods to handle observation. In the simplest, most common case, a client can use the changes(for:) method to observe changes to that field for a given instance.
func processChanges(_ object: MyObject) async {
for await change in object.changes(for: \.someProperty) {
print(change.newValue)
}
}
This allows users of this protocol to be able to observe the changes to specific values either as a distinct step in the chain of change events or as an asynchronous sequence of change events. The advantage of the changes is specifically type safe since it can only be changes applied from one specific field.
object.someProperty = "hello"
// prints "hello" in the awaiting loop
object.someOtherProperty += 1
// nothing is printed
Detailed design
Observation of an entity implies that entity has identity. That means that things that are observable are inherently reference types. Structural types that are not rooted in a reference semantic storage do not have a well formed concept of external observation of changes. However, if the structure is a member variable of a reference type, descendant key paths to specific values passing through structural types with a root of a reference type do make sense as being an observable field.
In order to mark a reference type as being able to be observed Swift has an existing mechanism to do so: protocols. This observable protocol will only have two requirements; 1) that the type is a reference type and 2) that it has it's state changes observed. The requirement for being a reference type is only present to enforce the concept of reference semantics; if a structural type was observable it would immediately violate the concept of it's value-type-ness. The second requirement is the root functionality that being observable is defined by.
public protocol Observable: AnyObject {
associatedtype Observation
func addObserver<Member>(_ observer: some Observer<Self, Member>, for keyPath: KeyPath<Self, Member>) -> Observation
func addChangeHandler(for fields: ObservationTracking.Fields<Self>, options: ObservationTracking.Options, _ handler: @Sendable @escaping () -> Void) -> Observation
func removeObservation(_ observation: Observation)
}
extension Observable {
public func changes<Member>(for keyPath: KeyPath<Self, Member>) -> ObservedChanges<Self, Member> { ... }
}
For cases where developers want to offer change events directly but do not want to manage the list of observers themselves a default storage type is included. The ObservationList type is specifically crafted to allow for composition of observation with other systems. It provides type safe and concurrency safe addition and removal of observations and custom storage accessors for manipulating fields.
public struct ObservationList<Subject: Observable, Storage> {
public func addObserver<Member>(
_ observer: some Observer<Subject, Member>,
for keyPath: KeyPath<Subject, Member>
) -> ObservationTracking.Token
public func addChangeHandler(
for fields: ObservationTracking.Fields<Self>,
options: ObservationTracking.Options = [.willSet, .compareEquality],
_ handler: @Sendable @escaping () -> Void
) -> ObservationTracking.Token
public func removeObservation(_ observation: ObservationTracking.Token)
public func getMember<Member>(
_ subject: Subject,
propertyKeyPath: KeyPath<Subject, Member>,
storageKeyPath: KeyPath<Storage, Member>,
storage: Storage
) -> Member
public func setMember<Member>(
_ subject: Subject,
propertyKeyPath: KeyPath<Subject, Member>,
storageKeyPath: WritableKeyPath<Storage, Member>,
storage: inout Storage,
newValue: Member
)
The concept of tracking changes is the fundamental mechanism in which any Observable type is able to provide changes to observe. By default the concept of observation is transitively forwarded; observing a keypath of \A.b.c.d means that if the field .b is Observable that is registered with an observer to track \B.c.d and so on. This means that graphs of observations can be tracked such that any set of changes are forwarded as an event.
Observations come in a few different forms. An observer can be just an iteration of events over time of the post change event of one specific field or any field of the storage of the Observable instance. Alternatively, it can be an observation of the events before a value is changed. These two categories are the concept of didSet and willSet either specifically targeted at a particular field or as any potential field.
Capturing the will and did set events can be then modeled by a protocol.
public protocol Observer<Subject: Observed> {
mutating func subjectWillSet<Member>(_ keyPath: KeyPath<Subject, Member>, newValue: Member)
mutating func subjectDidSet<Member>(_ keyPath: KeyPath<Subject, Member>, newValue: Member)
}
extension Observer {
public mutating func subjectWillSet<Member>(_ keyPath: KeyPath<Subject, Member>, newValue: Member) { }
public mutating func subjectDidSet<Member>(_ keyPath: KeyPath<Subject, Member>, newValue: Member) { }
}
A more formal set of changes for either a given field or any field can then be modeled as change event values over time. In swift the language model of that type of values asynchronously over time is represented as AsyncSequence.
public struct ObservedChanges<Observed: Observable, Member> {
public struct Element {
public var newValue: Member
public var keyPath: KeyPath<Subject, Member>
}
}
extension ObservedChanges: Sendable where Observed: Sendable, Member: Sendable { }
extension ObservedChanges: AsyncSequence {
public struct Iterator: AsyncIteratorProtocol {
public mutating func next() async -> Element?
}
public func makeAsyncIterator() -> Iterator
}
@availability(*, unavailable)
extension ObservedChanges.Iterator: Sendable { }
A key area of focus that other systems can face issues for is lifetime management. The ownership model for Observed types will strongly own observers for the lifetime of their registration. Observations and the AsyncSequences do not maintain strong references to the Observed instance. At the point of iteration of the ObservedChanges will return nil from their iterator when the reference to the Observed instance has become weakly loaded as nil. In short observation via iteration terminates when the observed object is no longer valid. That means in the common case there is no opportunity for invalid or stalled observations.
Since all observations have hooks into both the read as well as the write this allows for the affordance to track just the changes to properties that were accessed within a given scope. Unlike other systems, that observation can be done efficiently and with key options to grant pruning behaviors that are not easily approachable in a straightforward mechanism. The access can track only the accessed properties, determine if events need to be fired on the leading edge (willSet) or trailing edge (didSet) and account for equatability (setting the same value). In the case for UI or other widgeting systems it is commonly tracked on the leading edge, values read during the construction of the main contents of a view, and equatable values being set over to the same value are ideally not fired for a change event.
public struct ObservationTracking: @unchecked Sendable {
public struct Token: Hashable, Sendable {
public init()
}
public struct Fields<Subject: Observable>: Hashable, @unchecked Sendable {
public init(keyPaths: Set<PartialKeyPath<Subject>>)
}
public struct Options: OptionSet, Sendable {
public var rawValue: Int
public init(rawValue: Int)
public static var willSet: Options { get }
public static var didSet: Options { get }
public static var compareEquality: Options { get }
}
public func addChangeHandler(options: Options = [.willSet, .compareEquality], _ handler: @Sendable @escaping () -> Void)
public func invalidate()
public static func registerAccess<Subject: Observable>(propertyKeyPath: PartialKeyPath<Subject>, subject: Subject)
public static func withTracking(_ apply: () -> Void) -> ObservationTracking?
}
The ObservationTracking mechanism is the primary interface designed for the purposes to interoperate with SwiftUI. Views will register via the withTracking method such that if in the execution of body any field is accessed in an Observable that field is registered into the access set that will be indicated in the handler passed to the addChangeHandler function. If at any point in time that handler needs to be directly invalidated the invalidate function can be invoked; which will remove all change handlers registered to the Observable instances under the tracking.
SDK Impact (a preview of SwiftUI interaction)
Using existing systems like ObservableObject there are a number of edge cases that can be surprising unless developers really have an in-depth view to SwiftUI. Formalizing observation can make these edge cases considerably more approachable by reducing the complexity of the different systems needed to be understood.
The following is adapted from the Fruta sample app, modified for clarity:
final class Model: ObservableObject {
@Published var order: Order?
@Published var account: Account?
var hasAccount: Bool {
return userCredential != nil && account != nil
}
@Published var favoriteSmoothieIDs = Set<Smoothie.ID>()
@Published var selectedSmoothieID: Smoothie.ID?
@Published var searchString = ""
@Published var isApplePayEnabled = true
@Published var allRecipesUnlocked = false
@Published var unlockAllRecipesProduct: Product?
}
struct SmoothieList: View {
var smoothies: [Smoothie]
@ObservedObject var model: Model
var listedSmoothies: [Smoothie] {
smoothies
.filter { $0.matches(model.searchString) }
.sorted(by: { $0.title.localizedCompare($1.title) == .orderedAscending })
}
var body: some View {
List(listedSmoothies) { smoothie in
...
}
}
}
The @Published identifies each field that participates to changes in the object, however it does not provide any differentiation for those changes. This means that from SwiftUI's perspective, a change to order effects things using hasAccount. This unfortunately means that there are additional layouts, rendering and updates created. The proposed API can not only reduce some of the @Published repetition but also simplify the SwiftUI view code too!
The previous example can then be written as:
final class Model: Observable {
var order: Order?
var account: Account?
var hasAccount: Bool {
return userCredential != nil && account != nil
}
var favoriteSmoothieIDs = Set<Smoothie.ID>()
var selectedSmoothieID: Smoothie.ID?
var searchString = ""
var isApplePayEnabled = true
var allRecipesUnlocked = false
var unlockAllRecipesProduct: Product?
}
struct SmoothieList: View {
var smoothies: [Smoothie]
var model: Model
var listedSmoothies: [Smoothie] {
smoothies
.filter { $0.matches(model.searchString) }
.sorted(by: { $0.title.localizedCompare($1.title) == .orderedAscending })
}
var body: some View {
List(listedSmoothies) { smoothie in
...
}
}
}
The advantages of course don't stop at the only reading dependent property access. In the case for accessing searchString; if the value does not actually change - e.g. if it starts off as "blueb" and is then set again to "blueb" it is an equal value in the access so it does not fire a second time. Whereas ObservableObject today, does emit two events even if the value is the same; since it does not have any value in the objectWillChange publisher that information is lost. That information is intrinsically built in to Observable meaning that SwiftUI can utilize the .compareEquality option to ensure that duplicate updates do not impact performance.
There are some other interesting differences that come up; for example - tracking observation of access within a view can be applied to an Array, an Optional, or even a custom type. This opens up new and interesting ways developers can utilize SwiftUI more easily.
This is a potential future direction for SwiftUI but is not part of this proposal.
Source compatibility
This proposal is additive and provides no impact to existing source code.
Effect on ABI stability
This proposal is additive and no impact is made upon existing ABI stability. This does have implication to the inlinability and back-porting of this feature. In the cases where it is determined to be performance critical to the distribution of change events the methods will be marked as inlinable.
Effect on API resilience
This proposal is additive and no impact is made upon existing API resilience.
Location of API
It is an open question if this functionality should live at the standard library layer or in a package above it. There are considerations for each home for these APIs. If it were to live in the standard library (and Concurrency library) that would simplify some of the ABI requirements for interoperating with keypaths. The functionality is the logical progression of the fundamental language constructs of willSet/didSet. Putting this into a library above the standard library does maintain some level of isolation of functionality. The names picked for the types and protocols are relatively specific to the jobs they serve and pose only trivial impact of potential global namespace costs. If it is chosen that this lives in a package above the standard library; it only has minor operating system requirements but does require that there are particular additions to the key path types for handling splitting and checking if a key path has a given prefix.
extension PartialKeyPath {
func hasPrefix(_ other: PartialKeyPath<Subject>) -> Bool
}
extension AnyKeyPath {
struct Component: Equatable {
var type: Any.Type
}
var components: [Component] { get }
init?(components: [Component])
}
Future Directions & Default Implementation
A default implementation can be accomplished in generality by a type wrapper that intercepts the modifications of any field on the Observable . The type wrapper DefaultObservable provides default implementations for the type where the associated Observation type is ObservationTracking.Token. This means that developers have the flexibility of an easy to use interface that progressively allows for more and more detailed control: starting from mere annotation that grants general observability, progressing to delegation to a storage mechanism that manages registering and unregistering observers, to full control of observation.
@typeWrapper
public struct DefaultObservable<Subject: Observable, Storage> {
public init(for subject: Subject.Type, storage: Storage)
public subscript<Member>(
wrappedSelf subject: Subject,
propertyKeyPath property: KeyPath<Subject, Member>,
storageKeyPath keyPath: KeyPath<Storage, Member>
) -> Member { get }
public subscript<Member>(
wrappedSelf subject: Subject,
propertyKeyPath property: ReferenceWritableKeyPath<Subject, Member>,
storageKeyPath keyPath: WritableKeyPath<Storage, Member>
) -> Member { get set }
}
@DefaultObservable
extension Observable where Observation == ObservationTracking.Token {
public func addObserver<Member>(_ observer: some Observer<Self, Member>, for keyPath: KeyPath<Self, Member>) -> Observation { ... }
public func addChangeHandler(for fields: ObservationTracking.Fields<Self>, options: ObservationTracking.Options, _ handler: @Sendable @escaping () -> Void) -> Observation { ... }
public func removeObservation(_ observation: Observation) { ... }
}
This definitely provides an attractive default experience such that developers do not need to implement their own access and observation tracking. However it does require a few minor modifications to the type wrapper proposal, namely of which; the application of a type wrapper on an extension of a protocol.
Further improvements might also include the ability to observe multiple properties with one observer, this would require a few external changes but internally the systems to observe should remain the same. This would allow for the future improvement to all Observable conforming types to gain the ability to add multiple change observations to produce an async sequence of changes.
for await change in someObject.changes(for: \.property1, \.property2) {
}
This requires some advanced features using variadic generics and is within the scope of considerations.
Alternatives considered
Under consideration for this API is to offer equivalents of keyPathsForValuesAffectingValueForKey: and automaticallyNotifiesObserversForKey:.
The observation APIs could be redesigned to return a value indicating automatic removal like change handlers. This requires developers to understand the implications of a boolean (or similar) return value, which may not be immediately obvious. Also this could be filed into options passed in with the observer.
The change handler APIs could instead of taking a closure could take an Observer that is required to have a signature of some Observer<Self, Self> in that it passes the keypath of \Self.self and self as the new value. Granted this reduces the surface area, it is quite strange of a syntax and is not immediately intuitive of an answer.
Acknowledgments
- Holly Borla - For providing fantastic ideas on how to implement supporting infrastructure to this pitch
- Pavel Yaskevich - For tirelessly iterating on prototypes for supporting compiler features
- Rishi Verma - For bouncing ideas and helping with the design of integrating this idea into other work
- Kyle Macomber - For connecting resources and providing useful feedback
- Matt Ricketson - For helping highlight some of the inner guts of SwiftUI
Related systems
- Swift
Combine.ObservableObject - Objective-C Key Value Observing
- C#
IObservable - Rust
Trait rx::Observable - Java
Observable - Kotlin
observable
Edits:
Updated attributions
