# \[Pitch\] Observation

**URL:** <https://forums.swift.org/t/pitch-observation/62051>\
**Category:** Pitches\
**Created:** [December 14, 2022, 6:07pm UTC](https://forums.swift.org/t/pitch-observation/62051 "2022-12-14T18:07:26Z")\
**Posts on this page:** 1\
**Showing post:** 1

<div class="post-metadata">

**Author:** ![Philippe\_Hausler](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/philippe_hausler/32/22569_2.png) [@Philippe\_Hausler](https://forums.swift.org/u/Philippe_Hausler)\
**Post date:** [December 14, 2022, 6:07pm UTC](https://forums.swift.org/t/pitch-observation/62051/1 "2022-12-14T18:07:26Z")

</div>

# Observation

## Introduction

Making apps that are responsive often requires mechanisms to observe changes that are applied from a model to drive changes elsewhere in the presentation of that data. The observer pattern allows a subject to maintain a list of observers and notifies them of specific or general state changes. This has the advantage of not directly coupling objects together; an object that is observed needs no information about the observer other than it is an observer. In addition to the decoupled nature it also has an implicit distribution of the updates across potential multiple observers.

This design pattern is a well traveled path by many languages and Swift has an opportunity to provide a robust, type safe, and performant mechanism for offering this pattern as a low level uniform system. This proposal defines what an observable reference is, what an observer needs to conform to, and the connection between those.

## Motivation

There are already a few mechanisms for observation in Swift. Some of these include Key Value Observing (KVO), or `ObservableObject`; but those are relegated to in the case of KVO to just `NSObject` descendants, and `ObservableObject` requires using Combine which is restricted to Darwin platforms and does not leverage language features like async/await or `AsyncSequence`. By taking experience from those existing systems we can build a more generally useful feature that applies to all Swift reference types; not just those that inherit from `NSObject` and have it work cross platform with using the advantages from low level language features like async/await.

### Prior art

#### KVO

Key Value Observing in Objective-C has served that model well but is limited to just systems based on inheritance to `NSObject`. The APIs only offer the intercepting of events (meaning that the callout to the inform of the changes is between the will/did events). KVO has great flexibility with the granularity of events but lacks in composability. Observers for KVO must inherit from NSObject and rely on the objc runtime to track the changes that occur. Even though the interface for KVO has been updated to utilize the more modern Swift keypath constructs that are strongly typed, under the hood it is still stringly typed events.

#### Combine

Combine's `ObservableObject` produces changes on the leading edge of the will/did events and all delivered values are before the value did set. Albeit this serves SwiftUI well, it is restrictive for non SwiftUI usage and can be surprising to developers first encountering that restriction. `ObservableObject` also requires that all observed properties to be marked as `@Published` to interact with the change events. In most cases this requirement is applied to every single property and becomes redundant to the developer; meaning folks writing an `ObservableObject` conforming type must repeatedly (with little to no true gained clarity) annotate each property - in the end this results in a fatigue of the meaning of what is a participating item or not.

## Proposed solution

A formalized observer pattern needs to support the following:

- Marking a reference type as "observable"
- Tracking changes within an instance of an observable type
- The ability to observe and utilize those changes from somewhere else, e.g. another type

A type can declare itself as observable simply by conforming to the `Observable` protocol:

```swift
final class MyObject: Observable {
  var someProperty: String = ""
  var someOtherProperty = 0
}

```

Unlike `ObservableObject` and `@Published`, none of the fields of the type need to be individually marked as observable. Instead, all fields of the observable type are implicitly observable. Since creating key paths is limited to the visibility of the fields this means that the control of observation is relegated to the visibility of the fields. If a developer needs to restrict the observation of a field, it can be marked as `private` which prevents external access to the key path from being constructed and consequently observation.

The Observable protocol includes a set of extension methods to handle observation. In the simplest, most common case, a client can use the `changes(for:)` method to observe changes to that field for a given instance.

```swift
func processChanges(_ object: MyObject) async {
  for await change in object.changes(for: \.someProperty) {
    print(change.newValue)
  }
}

```

This allows users of this protocol to be able to observe the changes to specific values either as a distinct step in the chain of change events or as an asynchronous sequence of change events. The advantage of the changes is specifically type safe since it can only be changes applied from one specific field.

```swift
object.someProperty = "hello" 
// prints "hello" in the awaiting loop
object.someOtherProperty += 1
// nothing is printed

```

## Detailed design

Observation of an entity implies that entity has identity. That means that things that are observable are inherently reference types. Structural types that are not rooted in a reference semantic storage do not have a well formed concept of external observation of changes. However, if the structure is a member variable of a reference type, descendant key paths to specific values passing through structural types with a root of a reference type do make sense as being an observable field.

In order to mark a reference type as being able to be observed Swift has an existing mechanism to do so: protocols. This observable protocol will only have two requirements; 1) that the type is a reference type and 2) that it has it's state changes observed. The requirement for being a reference type is only present to enforce the concept of reference semantics; if a structural type was observable it would immediately violate the concept of it's value-type-ness. The second requirement is the root functionality that being observable is defined by.

```swift
public protocol Observable: AnyObject {
  associatedtype Observation
  
  func addObserver<Member>(_ observer: some Observer<Self, Member>, for keyPath: KeyPath<Self, Member>) -> Observation
  func addChangeHandler(for fields: ObservationTracking.Fields<Self>, options: ObservationTracking.Options, _ handler: @Sendable @escaping () -> Void) -> Observation
  func removeObservation(_ observation: Observation)
}

extension Observable {
  public func changes<Member>(for keyPath: KeyPath<Self, Member>) -> ObservedChanges<Self, Member> { ... }
}

```

For cases where developers want to offer change events directly but do not want to manage the list of observers themselves a default storage type is included. The `ObservationList` type is specifically crafted to allow for composition of observation with other systems. It provides type safe and concurrency safe addition and removal of observations and custom storage accessors for manipulating fields.

```swift
public struct ObservationList<Subject: Observable, Storage> {
  public func addObserver<Member>(
    _ observer: some Observer<Subject, Member>, 
    for keyPath: KeyPath<Subject, Member>
  ) -> ObservationTracking.Token 
  
  public func addChangeHandler(
    for fields: ObservationTracking.Fields<Self>, 
    options: ObservationTracking.Options = [.willSet, .compareEquality], 
    _ handler: @Sendable @escaping () -> Void
  ) -> ObservationTracking.Token
  
  public func removeObservation(_ observation: ObservationTracking.Token)
  
  public func getMember<Member>(
    _ subject: Subject,
    propertyKeyPath: KeyPath<Subject, Member>,
    storageKeyPath: KeyPath<Storage, Member>,
    storage: Storage
  ) -> Member
  
  public func setMember<Member>(
    _ subject: Subject,
    propertyKeyPath: KeyPath<Subject, Member>,
    storageKeyPath: WritableKeyPath<Storage, Member>,
    storage: inout Storage,
    newValue: Member
  )

```

The concept of tracking changes is the fundamental mechanism in which any `Observable` type is able to provide changes to observe. By default the concept of observation is transitively forwarded; observing a keypath of `\A.b.c.d` means that if the field `.b` is `Observable` that is registered with an observer to track `\B.c.d` and so on. This means that graphs of observations can be tracked such that any set of changes are forwarded as an event.

Observations come in a few different forms. An observer can be just an iteration of events over time of the post change event of one specific field or any field of the storage of the `Observable` instance. Alternatively, it can be an observation of the events before a value is changed. These two categories are the concept of `didSet` and `willSet` either specifically targeted at a particular field or as any potential field.

Capturing the will and did set events can be then modeled by a protocol.

```swift
public protocol Observer<Subject: Observed> {
  mutating func subjectWillSet<Member>(_ keyPath: KeyPath<Subject, Member>, newValue: Member)
  mutating func subjectDidSet<Member>(_ keyPath: KeyPath<Subject, Member>, newValue: Member)
}

extension Observer {
  public mutating func subjectWillSet<Member>(_ keyPath: KeyPath<Subject, Member>, newValue: Member) { }
  public mutating func subjectDidSet<Member>(_ keyPath: KeyPath<Subject, Member>, newValue: Member) { }
}

```

A more formal set of changes for either a given field or any field can then be modeled as change event values over time. In swift the language model of that type of values asynchronously over time is represented as `AsyncSequence`.

```swift
public struct ObservedChanges<Observed: Observable, Member> {
  public struct Element {
    public var newValue: Member
    public var keyPath: KeyPath<Subject, Member>
  }
}

extension ObservedChanges: Sendable where Observed: Sendable, Member: Sendable { }

extension ObservedChanges: AsyncSequence {
  public struct Iterator: AsyncIteratorProtocol {
    public mutating func next() async -> Element?
  }
  
  public func makeAsyncIterator() -> Iterator
}

@availability(*, unavailable)
extension ObservedChanges.Iterator: Sendable { }

```

A key area of focus that other systems can face issues for is lifetime management. The ownership model for `Observed` types will strongly own observers for the lifetime of their registration. Observations and the AsyncSequences do not maintain strong references to the `Observed` instance. At the point of iteration of the `ObservedChanges` will return nil from their iterator when the reference to the `Observed` instance has become weakly loaded as nil. In short observation via iteration terminates when the observed object is no longer valid. That means in the common case there is no opportunity for invalid or stalled observations.

Since all observations have hooks into both the read as well as the write this allows for the affordance to track just the changes to properties that were accessed within a given scope. Unlike other systems, that observation can be done efficiently and with key options to grant pruning behaviors that are not easily approachable in a straightforward mechanism. The access can track only the accessed properties, determine if events need to be fired on the leading edge (willSet) or trailing edge (didSet) and account for equatability (setting the same value). In the case for UI or other widgeting systems it is commonly tracked on the leading edge, values read during the construction of the main contents of a view, and equatable values being set over to the same value are ideally not fired for a change event.

```swift
public struct ObservationTracking: @unchecked Sendable {
  public struct Token: Hashable, Sendable {
    public init()
  }
  
  public struct Fields<Subject: Observable>: Hashable, @unchecked Sendable {
    public init(keyPaths: Set<PartialKeyPath<Subject>>)
  }
  
  public struct Options: OptionSet, Sendable {
    public var rawValue: Int

    public init(rawValue: Int)

    public static var willSet: Options { get }
    public static var didSet: Options { get }
    public static var compareEquality: Options { get }
  }
  
  public func addChangeHandler(options: Options = [.willSet, .compareEquality], _ handler: @Sendable @escaping () -> Void)
  public func invalidate()
  
  public static func registerAccess<Subject: Observable>(propertyKeyPath: PartialKeyPath<Subject>, subject: Subject)
  
  public static func withTracking(_ apply: () -> Void) -> ObservationTracking?
}

```

The `ObservationTracking` mechanism is the primary interface designed for the purposes to interoperate with SwiftUI. Views will register via the `withTracking` method such that if in the execution of `body` any field is accessed in an `Observable` that field is registered into the access set that will be indicated in the handler passed to the `addChangeHandler` function. If at any point in time that handler needs to be directly invalidated the `invalidate` function can be invoked; which will remove all change handlers registered to the `Observable` instances under the tracking.

## SDK Impact (a preview of SwiftUI interaction)

Using existing systems like `ObservableObject` there are a number of edge cases that can be surprising unless developers really have an in-depth view to SwiftUI. Formalizing observation can make these edge cases considerably more approachable by reducing the complexity of the different systems needed to be understood.

The following is adapted from the [Fruta sample app](https://developer.apple.com/documentation/swiftui/fruta_building_a_feature-rich_app_with_swiftui), modified for clarity:

```swift
final class Model: ObservableObject {
    @Published var order: Order?
    @Published var account: Account?
    
    var hasAccount: Bool {
        return userCredential != nil && account != nil
    }
    
    @Published var favoriteSmoothieIDs = Set<Smoothie.ID>()
    @Published var selectedSmoothieID: Smoothie.ID?
    
    @Published var searchString = ""
    
    @Published var isApplePayEnabled = true
    @Published var allRecipesUnlocked = false
    @Published var unlockAllRecipesProduct: Product?
}

struct SmoothieList: View {
    var smoothies: [Smoothie]
    @ObservedObject var model: Model
    
    var listedSmoothies: [Smoothie] {
        smoothies
            .filter { $0.matches(model.searchString) }
            .sorted(by: { $0.title.localizedCompare($1.title) == .orderedAscending })
    }
    
    var body: some View {
        List(listedSmoothies) { smoothie in
            ...
        }
    }
} 

```

The `@Published` identifies each field that participates to changes in the object, however it does not provide any differentiation for those changes. This means that from SwiftUI's perspective, a change to `order` effects things using `hasAccount`. This unfortunately means that there are additional layouts, rendering and updates created. The proposed API can not only reduce some of the `@Published` repetition but also simplify the SwiftUI view code too!

The previous example can then be written as:

```swift
final class Model: Observable {
    var order: Order?
    var account: Account?
    
    var hasAccount: Bool {
        return userCredential != nil && account != nil
    }
    
    var favoriteSmoothieIDs = Set<Smoothie.ID>()
    var selectedSmoothieID: Smoothie.ID?
    
    var searchString = ""
    
    var isApplePayEnabled = true
    var allRecipesUnlocked = false
    var unlockAllRecipesProduct: Product?
}

struct SmoothieList: View {
    var smoothies: [Smoothie]
    var model: Model
    
    var listedSmoothies: [Smoothie] {
        smoothies
            .filter { $0.matches(model.searchString) }
            .sorted(by: { $0.title.localizedCompare($1.title) == .orderedAscending })
    }
    
    var body: some View {
        List(listedSmoothies) { smoothie in
            ...
        }
    }
} 

```

The advantages of course don't stop at the only reading dependent property access. In the case for accessing `searchString`; if the value does not actually change - e.g. if it starts off as "blueb" and is then set again to "blueb" it is an equal value in the access so it does not fire a second time. Whereas `ObservableObject` today, does emit two events even if the value is the same; since it does not have any value in the `objectWillChange` publisher that information is lost. That information is intrinsically built in to `Observable` meaning that SwiftUI can utilize the `.compareEquality` option to ensure that duplicate updates do not impact performance.

There are some other interesting differences that come up; for example - tracking observation of access within a view can be applied to an Array, an Optional, or even a custom type. This opens up new and interesting ways developers can utilize SwiftUI more easily.

This is a potential future direction for SwiftUI but is not part of this proposal.

## Source compatibility

This proposal is additive and provides no impact to existing source code.

## Effect on ABI stability

This proposal is additive and no impact is made upon existing ABI stability. This does have implication to the inlinability and back-porting of this feature. In the cases where it is determined to be performance critical to the distribution of change events the methods will be marked as inlinable.

## Effect on API resilience

This proposal is additive and no impact is made upon existing API resilience.

## Location of API

It is an open question if this functionality should live at the standard library layer or in a package above it. There are considerations for each home for these APIs. If it were to live in the standard library (and Concurrency library) that would simplify some of the ABI requirements for interoperating with keypaths. The functionality is the logical progression of the fundamental language constructs of willSet/didSet. Putting this into a library above the standard library does maintain some level of isolation of functionality. The names picked for the types and protocols are relatively specific to the jobs they serve and pose only trivial impact of potential global namespace costs. If it is chosen that this lives in a package above the standard library; it only has minor operating system requirements but does require that there are particular additions to the key path types for handling splitting and checking if a key path has a given prefix.

```swift
extension PartialKeyPath {
  func hasPrefix(_ other: PartialKeyPath<Subject>) -> Bool
}

extension AnyKeyPath {
  struct Component: Equatable {
    var type: Any.Type
  }
  
  var components: [Component] { get }
  init?(components: [Component])
}

```

## Future Directions & Default Implementation

A default implementation can be accomplished in generality by a [type wrapper](https://forums.swift.org/t/pitch-type-wrappers/60019) that intercepts the modifications of any field on the `Observable `. The type wrapper `DefaultObservable` provides default implementations for the type where the associated `Observation` type is `ObservationTracking.Token`. This means that developers have the flexibility of an easy to use interface that progressively allows for more and more detailed control: starting from mere annotation that grants general observability, progressing to delegation to a storage mechanism that manages registering and unregistering observers, to full control of observation.

```swift
@typeWrapper
public struct DefaultObservable<Subject: Observable, Storage> {
  public init(for subject: Subject.Type, storage: Storage)
  public subscript<Member>(
    wrappedSelf subject: Subject,
    propertyKeyPath property: KeyPath<Subject, Member>,
    storageKeyPath keyPath: KeyPath<Storage, Member>
  ) -> Member { get }
  public subscript<Member>(
    wrappedSelf subject: Subject,
    propertyKeyPath property: ReferenceWritableKeyPath<Subject, Member>,
    storageKeyPath keyPath: WritableKeyPath<Storage, Member>
  ) -> Member { get set }
}

@DefaultObservable
extension Observable where Observation == ObservationTracking.Token { 
  public func addObserver<Member>(_ observer: some Observer<Self, Member>, for keyPath: KeyPath<Self, Member>) -> Observation { ... }
  public func addChangeHandler(for fields: ObservationTracking.Fields<Self>, options: ObservationTracking.Options, _ handler: @Sendable @escaping () -> Void) -> Observation { ... }
  public func removeObservation(_ observation: Observation) { ... }
}

```

This definitely provides an attractive default experience such that developers do not need to implement their own access and observation tracking. However it does require a few minor modifications to the type wrapper proposal, namely of which; the application of a type wrapper on an extension of a protocol.

Further improvements might also include the ability to observe multiple properties with one observer, this would require a few external changes but internally the systems to observe should remain the same. This would allow for the future improvement to all `Observable` conforming types to gain the ability to add multiple change observations to produce an async sequence of changes.

```swift
for await change in someObject.changes(for: \.property1, \.property2) {
}

```

This requires some advanced features using variadic generics and is within the scope of considerations.

## Alternatives considered

Under consideration for this API is to offer equivalents of `keyPathsForValuesAffectingValueForKey:` and `automaticallyNotifiesObserversForKey:`.

The observation APIs could be redesigned to return a value indicating automatic removal like change handlers. This requires developers to understand the implications of a boolean (or similar) return value, which may not be immediately obvious. Also this could be filed into options passed in with the observer.

The change handler APIs could instead of taking a closure could take an Observer that is required to have a signature of `some Observer<Self, Self>` in that it passes the keypath of `\Self.self` and `self` as the new value. Granted this reduces the surface area, it is quite strange of a syntax and is not immediately intuitive of an answer.

## Acknowledgments

- [Holly Borla](https://github.com/hborla) - For providing fantastic ideas on how to implement supporting infrastructure to this pitch
- [Pavel Yaskevich](https://github.com/xedin) - For tirelessly iterating on prototypes for supporting compiler features
- Rishi Verma - For bouncing ideas and helping with the design of integrating this idea into other work
- [Kyle Macomber](https://github.com/kylemacomber) - For connecting resources and providing useful feedback
- [Matt Ricketson](https://github.com/ricketson) - For helping highlight some of the inner guts of SwiftUI

## Related systems

- [Swift `Combine.ObservableObject`](https://developer.apple.com/documentation/combine/observableobject/)
- [Objective-C Key Value Observing](https://developer.apple.com/documentation/objectivec/nsobject/nskeyvalueobserving?language=objc)
- [C# `IObservable`](https://learn.microsoft.com/en-us/dotnet/api/system.iobservable-1?view=net-6.0)
- [Rust `Trait rx::Observable`](https://docs.rs/rx/latest/rx/trait.Observable.html)
- [Java `Observable`](https://docs.oracle.com/javase/7/docs/api/java/util/Observable.html)
- [Kotlin `observable`](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.properties/-delegates/observable.html)

## Edits:

Updated attributions

---

_[View the full topic](https://forums.swift.org/t/pitch-observation/62051)._