[Pitch] Observation (Revised)

Observation

Changes

  • Pitch 1: Initial pitch
  • Pitch 2: Previously Observation registered observers directly to Observable, the new approach registers observers to an Observable via a ObservationTransactionModel. These models control the "edge" of where the change is emitted. They are the responsible component for notifying the observers of events. This allows the observers to focus on just the event and not worry about "leading" or "trailing" (will/did) "edges" of the signal. Additionally the pitch was shifted from the type wrapper feature over to the more appropriate macro features.
  • Pitch 3: The Observer protocol and addObserver(_:) method are gone in favor of providing async sequences of changes and transactions.

Suggested Reading

Introduction

Making responsive apps often requires the ability to update the presentation when underlying data changes. The observer pattern allows a subject to maintain a list of observers and notify them of specific or general state changes. This has the advantages of not directly coupling objects together and allowing implicit distribution of updates across potential multiple observers. An observable object needs no specific information about its observers.

This design pattern is a well-traveled path by many languages, and Swift has an opportunity to provide a robust, type-safe, and performant implementation. This proposal defines what an observable reference is, what an observer needs to conform to, and the connection between a type and its observers.

Motivation

There are already a few mechanisms for observation in Swift. These include key-value observing (KVO) and ObservableObject, but each of those have limitations. KVO can only be used with NSObject descendants, and ObservableObject requires using Combine, which is restricted to Darwin platforms and does not use current Swift concurrency features. By taking experience from those existing systems, we can build a more generally useful feature that applies to all Swift reference types, not just those that inherit from NSObject, and have it work cross-platform with the advantages from language features like async/await.

The existing systems get a number of behaviors and characteristics right. However, there are a number of areas that can provide a better balance of safety, performance, and expressiveness. For example, grouping dependent changes into an independent transaction is a common task, but this is complex when using Combine and unsupported when using KVO. In practice, observers want access to transactions, with the ability to specify how transactions are interpreted.

Annotations clarify what is observable, but can also be cumbersome. For example, Combine requires not just that a type conform to ObservableObject, but also requires each property that is being observed to be marked as @Published. Furthermore, computed properties cannot be directly observed. In reality, having non-observed fields in a type that is observable is uncommon.

Throughout this document, references to both KVO and Combine will illustrate what capabilities are benefits and can be incorporated into the new approach, and what drawbacks are possible to solve in a more robust manner.

Prior Art

KVO

Key-value observing in Objective-C has served that model well, but is limited to class hierarchies that inherit from NSObject. The APIs only offer the intercepting of events, meaning that the notification of changes is between the willSet and didSet events. KVO has great flexibility with granularity of events, but lacks in composability. KVO observers must also inherit from NSObject, and rely on the Objective-C runtime to track the changes that occur. Even though the interface for KVO has been updated to utilize the more modern Swift strongly-typed key paths, under the hood its events are still stringly typed.

Combine

Combine's ObservableObject produces changes on the leading edge of the willSet/didSet events, and all values are delivered before the value is set. While this serves SwiftUI well, it is restrictive for non-SwiftUI usage and can be surprising to developers first encountering that restriction. ObservableObject also requires all observed properties to be marked as @Published to interact with change events. In most cases, this requirement is applied to every single property and becomes redundant to the developer; folks writing an ObservableObject conforming type must repeatedly (with little to no true gained clarity) annotate each property. In the end, this results in meaning fatigue of what is or isn't a participating item.

Proposed solution

A formalized observer pattern needs to support the following capabilities:

  • Marking a type as observable
  • Tracking changes within an instance of an observable type
  • Observing and utilizing those changes from somewhere else, e.g. another type

In addition, the design and implementation should meet these criteria:

  • Observable types are easy to annotate (without fatigue of meaning)
  • Access control should be respected
  • Adopting the features for observability should require minimal effort to get started
  • Using advanced features should progressively disclose to more complex systems
  • Observation should be able to handle more than one observed member at once
  • Observation should be able to work with computed properties that reference other properties
  • Observation should be able to work with computed properties that process both get and set to external storage
  • Integration of observation should work in transactions of graphs and not just singular objects

We propose a new standard library module named Observation that includes the protocols, types, and macros to implement such a pattern.

Primarily, a type can declare itself as observable simply by using the @Observable macro annotation:

@Observable public final class MyObject {
    public var someProperty: String = ""
    public var someOtherProperty = 0
    fileprivate var somePrivateProperty = 1
}

The @Observable macro declares and implements conformance to the Observable protocol, which includes a set of extension methods to handle observation. In the simplest case, a client can use the changes(for:) method to observe changes to a specific property for a given instance.

func processChanges(_ object: MyObject) async {
    for await value in object.values(for: \.someProperty) {
        print(value)
    }
}

This allows users of Observable types to observe changes to specific values or an instance as a whole as asynchronous sequences of change events. The changes(for:) method provides type safety, since it only provides the changes to one specific property.

object.someProperty = "hello" 
// prints "hello" in the awaiting loop
object.someOtherProperty += 1
// nothing is printed

Observable objects can also provide changes grouped into transactions, which coalesce any changes that are made between suspension points. Transactions are delivered isolated to an actor that you provide, or the main actor by default.

func processTransactions(_ object: MyObject) async {
    for await change in objects.changes(for: [\.someProperty, \.someOtherProperty]) {
        print(myObject.someProperty, myObject.someOtherProperty)
    }
}

Unlike ObservableObject and @Published, the properties of an @Observable type do not need to be individually marked as observable. Instead, all stored properties are implicitly observable.

For read-only computed properties, an author can add the static dependencies(of:) method to claim additional key paths as part of their observation. This is similar to the mechanism that KVO uses to provide additional key paths that have effects to key paths.

extension MyObject {
    var someComputedProperty: Int { 
        somePrivateProperty + someOtherProperty
    }

    nonisolated static func dependencies(
        of keyPath: PartialKeyPath<Self>
    ) -> TrackedProperties<Self> {
        switch keyPath {
        case \.someComputedProperty:
            return [\.somePrivateProperty, \.someOtherProperty]
        default:
            return [keyPath]
        }
    }
}

Since all access to observing changes is by key path, visibility keywords like public and private determine what can and cannot be observed. Unlike KVO, this means that only members that are accessible in a particular scope can be observed. This fact is reflected in the design, where transactions are represented as TrackedProperties instances, which allow querying for the changed key paths, but not their iteration.

// ✅ `someProperty` is publicly visible
object.changes(for: \.someProperty)
// ❌ `somePrivateProperty` is restricted to `private` access
object.changes(for: \.somePrivateProperty) 
// ✅ `someComputedProperty` is visible; `somePrivateProperty` isn't accessible in returned `TrackedProperties` instances
object.changes(for: \.someComputedProperty) 

Detailed Design

The Observable protocol, @Observable macro, and a handful of supporting types comprise the Observation module. As described below, this design allows adopters to use terse, straightforward syntax for simple cases, while allowing full control over the details the implementation when necessary.

Observable protocol

The core protocol for observation is Observable. Observable-conforming types define what is observable by registering changes and provide asynchronous sequences of transactions and changes to individual properties, isolated to a specific actor.

protocol Observable {
    /// Returns an asynchronous sequence of change transactions for the specified
    /// properties.
    nonisolated func changes<Isolation: Actor>(
        for properties: TrackedProperties<Self>,
        isolatedTo: Isolation
    ) -> ObservedChanges<Self, Isolation>
      
    /// Returns an asynchronous sequence of changes for the specified key path.
    nonisolated func values<Member: Sendable>(
        for keyPath: KeyPath<Self, Member>
    ) -> ObservedValues<Self, Member>
      
    /// Returns a set of tracked properties that represent the given key path.
    nonisolated static func dependencies(
        of keyPath: PartialKeyPath<Self>
    ) -> TrackedProperties<Self>
}

The first two protocol requirements need to be implemented by conforming types, either manually or by using the @Observable macro, described below. These two methods make use of ObservationRegistrar (also described below) to track changes and provide async sequences of changes and transactions.

In addition to these protocol requirements, Observable types must implement the semantic requirement of tracking each access and mutation to observable properties. This tracking is also provided by using the @Observable macro, or can be implemented manually using the access and withMutation methods of a registrar.

The Observable protocol also implements extension methods that provide convenient access to transactions isolated to the main actor or tracking just a single key path.

extension Observable {
    /// Returns an asynchronous sequence of change transactions for the specified
    /// key path, isolated to the given actor.
    nonisolated func changes<Member, Isolation: Actor>(
        for keyPath: KeyPath<Self, Member>,
        isolatedTo: Isolation
    ) -> ObservedChanges<Self, Delivery>
        
    /// Returns an asynchronous sequence of change transactions for the specified
    /// properties, isolated to the main actor.
    nonisolated func changes(
        for properties: TrackedProperties<Self>
    ) -> ObservedChanges<Self, MainActor.ActorType>
      
    /// Returns an asynchronous sequence of change transactions for the specified
    /// key path, isolated to the main actor.
    public nonisolated func changes<Member>(
        for keyPath: KeyPath<Self, Member>
    ) -> ObservedChanges<Self, MainActor.ActorType>
    
    // Default implementation returns `[keyPath]`.
    public nonisolated static func dependencies(
        of keyPath: PartialKeyPath<Self>
    ) -> TrackedProperties<Self>
}

The default implementation for dependencies(of:) returns a TrackedProperties type constructed with the given key path. This function is expected to be implemented in types when read only computed key paths are used, as seen in the someComputedProperty example above.

Macro Synthesis

In order to make implementation as simple as possible, the @Observable macro automatically synthesizes conformance to the Observable protocol, transforming annotated types into a type that can be observed. The @Observable macro does the following:

  • declares conformance to the Observable protocol
  • adds the required Observable method requirements
  • adds a property for the registrar
  • adds a storage abstraction for access tracking
  • changes all stored properties into computed properties

Since all of the code generated by the macro could be manually written, developers can write their own implementation when they need more fine-grained control.

@Observable final class Model {
    var order: Order?
    var account: Account?
  
    var alternateIconsUnlocked = false
    var allRecipesUnlocked = false
  
    init() { }
  
    func purchase(alternateIcons: Bool, allRecipes: Bool) {
        alternateIconsUnlocked = alternateIcons
        allRecipesUnlocked = allRecipes
    }
}

Expanding the macro for the previous example results in the following:

final class Model: Observable {
    internal let _registrar = ObservationRegistrar<Model>()
    
    public nonisolated func changes<Isolation: Actor>(
        for properties: TrackedProperties<Model>, 
        isolatedTo: Delivery
    ) -> ObservedChanges<Model, Isolation> {
        _registrar.changes(for: properties, isolation: isolation)
    }

    public nonisolated func values<Member: Sendable>(
        for keyPath: KeyPath<Model, Member>
    ) -> ObservedValues<Model, Member> {
        _registrar.changes(for: keyPath)
    }

    private struct Storage {
        var order: Order?
        var account: Account?
  
        var alternateIconsUnlocked = false
        var allRecipesUnlocked = false
    }
  
    private var _storage = Storage()
  
    var order: Order? {
        get { 
            _registrar.access(self, keyPath: \.order)
            return _storage.order
        }
        set {
            _registrar.withMutation(self, keyPath: \.order) {
                _storage.order = newValue
            }
        }
    }
  
    var account: Account? {
        get { 
            _registrar.access(self, keyPath: \.account)
            return _storage.account
        }
        set {
            _registrar.withMutation(self, keyPath: \.account) {
                _storage.account = newValue
            }
        }
    }

    var alternateIconsUnlocked: Bool {
        get { 
            _registrar.access(self, keyPath: \.alternateIconsUnlocked)
            return _storage.alternateIconsUnlocked
        }
        set {
            _registrar.withMutation(self, keyPath: \.alternateIconsUnlocked) {
                _storage.alternateIconsUnlocked = newValue
            }
        }
    }

    var allRecipesUnlocked: Bool {
        get { 
            _registrar.access(self, keyPath: \.allRecipesUnlocked)
            return _storage.allRecipesUnlocked
        }
        set {
            _registrar.withMutation(self, keyPath: \.allRecipesUnlocked) {
                _storage.allRecipesUnlocked = newValue
            }
        }
    }
  
    init() { }
  
    func purchase(alternateIcons: Bool, allRecipes: Bool) {
        alternateIconsUnlocked = alternateIcons
        allRecipesUnlocked = allRecipes
    }
}

TrackedProperties

When observing changes to a type, there may be associated side effects to members that are not publicly visible. The TrackedProperties type allows for internal key paths to be included in a transaction without being exposed beyond their visibility.

public struct TrackedProperties<Root>: ExpressibleByArrayLiteral, @unchecked Sendable {
    public typealias Element = PartialKeyPath<Root>
    public typealias ArrayLiteralElement = PartialKeyPath<Root>
        
    public init()
    
    public init(_ sequence: some Sequence<PartialKeyPath<Root>>)
      
    public init(arrayLiteral elements: PartialKeyPath<Root>...)
      
    public func contains(_ member: PartialKeyPath<Root>) -> Bool
      
    public mutating func insert(_ newMember: PartialKeyPath<Root>) -> Bool
  
    public mutating func remove(_ member: PartialKeyPath<Root>)
}

extension TrackedProperties where Root: Observable {
    public init(dependent: TrackedProperties<Root>)
}

ObservationRegistrar

ObservationRegistrar is the default storage for tracking and providing access to changes. The @Observable macro synthesizes a registrar to handle these mechanisms as a generalized feature. By default, the registrar is thread safe and must be as Sendable as containers could potentially be; therefore it must be designed to handle independent isolation for all actions.

public struct ObservationRegistrar<Subject: Observable>: Sendable {
    public init()
      
    public func access<Member>(
        _ subject: Subject, 
        keyPath: KeyPath<Subject, Member>
    )
      
    public func willSet<Member>(
        _ subject: Subject, 
        keyPath: KeyPath<Subject, Member>
    )
      
    public func didSet<Member>(
        _ subject: Subject, 
        keyPath: KeyPath<Subject, Member>
    )
      
    public func withMutation<Member, T>(
        of subject: Subject, 
        keyPath: KeyPath<Subject, Member>, 
        _ mutation: () throws -> T
    ) rethrows -> T
      
    public func changes<Isolation: Actor>(
        for properties: TrackedProperties<Subject>, 
        isolatedTo: Isolation
    ) -> ObservedChanges<Subject, Isolation>
      
    public func values<Member: Sendable>(
        for keyPath: KeyPath<Subject, Member>
    ) -> ObservedValues<Subject, Member>
}

The access and withMutation methods identify transactional accesses. These methods register access to the ObservationTracking system for access and identify mutations to the transactions registered for observers.

ObservationTracking

In order to provide scoped observation, the ObservationTracking type provides a method to capture accesses to properties within a given scope and then call out upon the first change to any of those properties.

public struct ObservationTracking {
    public static func withTracking<T>(
        _ apply: () -> T, 
        onChange: @autoclosure () -> @Sendable () -> Void
    ) -> T
}

The withTracking method takes two closures where any access to any property within the apply closure will indicate that property on that specific instance should participate in changes informed to the onChange closure.

@Observable final class Car {
    var name: String
    var awards: [Award]
}

let cars: [Car] = ...

func render() {
    ObservationTracking.withTracking {
        for car in cars {
            print(car.name)
        }
    } onChange {
        scheduleRender()
    }
}

In the example above, the render function accesses each car's name property. When any of the cars change name, the onChange closure is then called on the first change. However, if a car has an award added, the onChange call won't happen. This design supports uses that require implicit observation tracking, such as SwiftUI, ensuring that views are only updated in response to relevant changes.

ObservedChanges and ObservedValues

The two included asynchronous sequences provide access to transactions based on a TrackedProperties instance or to changes based on a key path, respectively. The two sequences have slightly different semantics.

The ObservedChanges sequence is the result of calling changes(for:isolatedTo:) on an observable type and passing one or more key paths as a TrackedProperties instance. The isolating actor that you pass (or the main actor, by default) determines how changes are coalesced. Any changes between suspension points on the isolating actor, whether to one property or multiple properties, only provide a single transaction event. Sequence elements are ObservedChange instances, which you can query to see if a specific property has changed. When the observed type is Sendable, an ObservedChange also includes the observed subject, in order to simplify accessing the updated properties.

The ObservedValues sequence, on the other hand, is the result of calling values(for:), passing a single key path to observe. Instead of coalescing changes in reference to an isolating actor, ObservedValues provides changed values that are coalesced at each suspension during iteration. Since the iterator isn't Sendable, that behavior implicitly isolates changes to the current actor.

public struct ObservedChange<Subject: Observable>: @unchecked Sendable {
    public func contains(_ member: PartialKeyPath<Subject>) -> Bool
}

extension ObservedChange where Subject: Sendable {
    public var subject: Subject { get }
}

/// An asynchronous sequence of observed changes.
public struct ObservedChanges<Subject: Observable, Delivery: Actor>: AsyncSequence {
    public typealias Element = ObservedChange<Subject>
  
    public struct Iterator: AsyncIteratorProtocol {
        public mutating func next() async -> Element?
    }
    
    public func makeAsyncIterator() -> Iterator
}

extension ObservedChanges: @unchecked Sendable { }
@available(*, unavailable)
extension ObservedChanges.Iterator: Sendable { }

/// An asynchronous sequence of observed changes.
public struct ObservedValues<Subject: Observable, Element: Sendable>: AsyncSequence {  
    public struct Iterator: AsyncIteratorProtocol {
        public mutating func next() async -> Element?
    }
    
    public func makeAsyncIterator() -> Iterator
}

extension ObservedValues: @unchecked Sendable where Subject: Sendable { }
@available(*, unavailable)
extension ObservedChanges.Iterator: Sendable { }

SDK Impact (a preview of SwiftUI interaction)

Using existing systems like ObservableObject there are a number of edge cases that can be surprising unless developers really have an in-depth view to SwiftUI. Formalizing observation can make these edge cases considerably more approachable by reducing the complexity of the different systems needed to be understood.

The following is adapted from the Fruta sample app, modified for clarity:

final class Model: ObservableObject {
    @Published var order: Order?
    @Published var account: Account?
    
    var hasAccount: Bool {
        return userCredential != nil && account != nil
    }
    
    @Published var favoriteSmoothieIDs = Set<Smoothie.ID>()
    @Published var selectedSmoothieID: Smoothie.ID?
    
    @Published var searchString = ""
    
    @Published var isApplePayEnabled = true
    @Published var allRecipesUnlocked = false
    @Published var unlockAllRecipesProduct: Product?
}

struct SmoothieList: View {
    var smoothies: [Smoothie]
    @ObservedObject var model: Model
    
    var listedSmoothies: [Smoothie] {
        smoothies
            .filter { $0.matches(model.searchString) }
            .sorted(by: { $0.title.localizedCompare($1.title) == .orderedAscending })
    }
    
    var body: some View {
        List(listedSmoothies) { smoothie in
            ...
        }
    }
} 

The @Published identifies each field that participates to changes in the object, however it does not provide any differentiation for those changes. This means that from SwiftUI's perspective, a change to order effects things using hasAccount. This unfortunately means that there are additional layouts, rendering and updates created. The proposed API can not only reduce some of the @Published repetition but also simplify the SwiftUI view code too!

The previous example can then be written as:

@Observable final class Model {
    var order: Order?
    var account: Account?
    
    var hasAccount: Bool {
        return userCredential != nil && account != nil
    }
    
    var favoriteSmoothieIDs = Set<Smoothie.ID>()
    var selectedSmoothieID: Smoothie.ID?
    
    var searchString = ""
    
    var isApplePayEnabled = true
    var allRecipesUnlocked = false
    var unlockAllRecipesProduct: Product?
}

struct SmoothieList: View {
    var smoothies: [Smoothie]
    var model: Model
    
    var listedSmoothies: [Smoothie] {
        smoothies
            .filter { $0.matches(model.searchString) }
            .sorted(by: { $0.title.localizedCompare($1.title) == .orderedAscending })
    }
    
    var body: some View {
        List(listedSmoothies) { smoothie in
            ...
        }
    }
} 

There are some other interesting differences that come up; for example - tracking observation of access within a view can be applied to an Array, an Optional, or even a custom type. This opens up new and interesting ways developers can utilize SwiftUI more easily.

This is a potential future direction for SwiftUI but is not part of this proposal.

Source compatibility

This proposal is additive and provides no impact to existing source code.

Effect on ABI stability

This proposal is additive and no impact is made upon existing ABI stability. This does have implication to the inlinability and back-porting of this feature. In the cases where it is determined to be performance critical to the distribution of change events the methods will be marked as inlinable.

Effect on API resilience

This proposal is additive and no impact is made upon existing API resilience.

Location of API

This API will be housed in a package; outside of the standard library.

Future Directions

The initial implementation will not track changes for key paths that have more than one layer of components. For example, key paths such as \.account would work, but \.account.name would not. This feature would be possible as soon as the standard library offers a mechanism to iterate components of a key path. Since there is no way to determine this yet, key paths that have more than one component will never observe any changes.

Another area of focus for future enhancements is support for observable actor types. This would require specific handling for key paths that currently does not exist for actors.

Finally, once variadic generics are available, an observation mechanism could be added to observe multiple key paths as an AsyncSequence.

Alternatives considered

An earlier consideration instead of defining transactions used direct will/did events to the observer. This, albeit being more direct, promoted mechanisms that did not offer the correct granularity for supporting the required synchronization between dependent members. It was determined that building transactions are worth the extra complexity to encourage developers using the API to consider what models for transactionality they need, instead of thinking just in terms of will/did events.

Another design included an Observer protocol that could be used to build callback-style observer types. This has been eliminated in favor of the AsyncSequence approach.

The ObservedChange type could have the Sendable requirement relaxed by making the type only conditionally Sendable and then allowing access to the subject in all cases; however this poses some restriction to the internal implementations and may have a sendability hole. Since it is viewed that accessing values is most commonly by one property the values AsyncSequence fills most of that role and for cases where more than one field is needed to be accessed on a given actor the iteration can be done with a weak reference to the observable subject.

Acknowledgments

  • Holly Borla - For providing fantastic ideas on how to implement supporting infrastructure to this pitch
  • Pavel Yaskevich - For tirelessly iterating on prototypes for supporting compiler features
  • Rishi Verma - For bouncing ideas and helping with the design of integrating this idea into other work
  • Kyle Macomber - For connecting resources and providing useful feedback
  • Matt Ricketson - For helping highlight some of the inner guts of SwiftUI

Related systems

22 Likes

Very cool, excited to see this latest iteration!

To be clear—is the addition or removal of an @Observable attribute an ABI-compatible change? (It seems like the answer is "no" for removal but perhaps "yes" for addition to non-frozen types?)

Not to shut down any discussion, but if this can be implemented entirely in a library, does that place it formally outside the scope of the Swift evolution process? We don't, e.g., govern Swift Collections via the evolution process.

3 Likes

Adding or removing the macro application (provided the conformance and implementations remain the same) is not ABI breaking. Removing without keeping conformance or implementations would be breaking iidc - the commentary on the effect of API resilience is more to do with the impact of adding this pitch into the swift ecosystem. We are not pitching any alteration of existing API in the standard libraries.

Technically it could be outside of that - but because of the potential massive impact to how folks write stuff for SwiftUI I figured it would be best to include this into the evolution process. Quite honestly, the community has a lot of great feedback that would be very valuable to this feature and it definitely can have some really impactful perspective changes for how folks interoperate with Swift. Furthermore it is reasonable feedback that this should perhaps be considered to be in the default imports - but our initial thoughts (and feedback from the community) was to start conservatively with a module.

4 Likes

Yeah I think this section has always been a little unclear—IIRC it's been replaced in the latest version of the proposal template with an "implications of adoption" section that more directly asks about what long-term commitments authors are making if they choose to use the feature.

Yeah definitely don't want to discourage discussion about this feature, just wanted to understand if there were any fundamental language changes being proposed (beyond macros) to support this pitch!

Is it supposed to be object.values(for: \.someProperty) or object.changes(for: \.someProperty)? There seems to be a bit of inconsistency there.

The pitch has both: values(for:) emits an AsyncSequence of values for a given key path, changes(for:) emits an AsyncSequence of changes for a given set of tracked properties.

Just to clarify, I think the intention for this is that it is a new module that is part of the shipping toolchain who will require an explicit import Observation, similar to RegexBuilder and Distributed. This is not going to be a "package" that one can declare as a dependency (using .package) in SwiftPM.

6 Likes

What happens if you don’t do this? Will the listener to \.someComputedProperty just never receive any updates?

Would it be possible to call the getter when creating the observation and detect which properties are accessed, then dynamically use that to subscribe to updates? (You would have to re-check this each time the property is changed in order to ensure that nothing is missed behind e.g. if conditions that change).


Should the dependencies(of:) method be documented as intended to be a pure function (such that calling it for a given key path will always return the same value)?


How does the macro handle let-bound properties? Do they stay on the top level object or are they also moved into the Storage struct?


I worry that this could lead to people unintentionally running tasks on the main actor. Could there be a better way of maintaining these guarantees without this potential performance issue? (Maybe each type could define its own actor, and actors could use their own internal actor?

That makes sense! This is the bit that was confusing me:

1 Like

Regarding actor isolation: I think it's great that there is a way to isolate observed changes to arrive on a specific actor, this seems it will be especially important in a UI context. I do have a couple of questions regarding this:

  • With other observation techniques it was possible to guarantee that an observation event fired on the main actor (previously the main dispatch queue) would be called in the same event loop cycle as the event that triggered it (e.g. via KVO/KVC or Combine). This is/was essential for avoiding animation glitches. Is this same guarantee provided here?
  • Will there be a way to cascade isolation to the actor originally isolated to if follow-on operators like map/filter are used? It's my understanding that Swift's concurrency system will – unless the method is annotated – always execute non-isolated/annotated async methods on the default executor. So, if you were to follow up your call to object.values(for: \.someProperty) with .map { SomeStruct($0) }, as things stand, that map call may unexpectedly shift your concurrency context to the default executor. That would probably be unexpected/undesirable in a UI context causing the previously mentioned animation glitches due to context switching. I'm especially interested in how usage marries with the advice that has been given to date, which is to avoid frequent context switching when using Swift concurrency, which may manifest as an issue if observing something like the user's mouse position for example.
2 Likes

Thanks for the new pitch!

I have a question about the async sequences. Is the first value the current one (when the iteration starts), or only on the first change that happens after iteration has started?

4 Likes

I'll parse the pitch later, but here's a smallish nit pick. Could the synthesized storage type be called and stored in a different place?

Instead of _storage: Storage could we make it less likely to collide with existing code and make it _observableStorage: ObservableStorage?

8 Likes

Looks really interesting pitch and a great example of macros. I just want to check that this doesn't effect the didSet/willSet of observed properties?

3 Likes

Very nice iteration of the pitch, I really like it!

A comment about the pitch text itself: I find it a bit weird that there is no mention of willSet/didSet since that's really the only "language" feature related to observability. The two mention are part of Objective-C and a framework, not really part of the Swift language. Although I totally get their mention since a lot of learnings come from there it would be good to mention the language native way of observing changes and how this pitch relates to that.

A comment about the SwiftUI example: the userCredential property used in the hasAccount computation doesn't seem to be defined.


I find a bit weird that a goal is:

But then you need to do this for something as common as computed properties:

I'm not fond of this distinction, I would expect any property to be observable no matter if it's computed or stored. And what would happen if you want to observe the computed property but is not declared as a dependency? I assume the keypath will be accessible so it will just silently ignore updates?

I think this becomes very important in the hypothetical integration with SwiftUI. With the code in the example I would expect the body of my view to be updated if I used the hasAccount computed property. If that's not the case it can be a very subtile source of issues.


Besides this I really like this pitch, the integration with async sequences and actor isolation is very nice to see.

I also loved the withTracking explanation, it's like seeing a glimpse of how SwiftUI works under the hood. I could imagine SwiftUI wrapping the calls to body with the apply closure. Eye opening really.

And I'm very glad we're using a language tool (macros) that seems to be covering a lot of ground in multiple pitches :D

Kudos to everybody involved!

10 Likes

What does this mean? Is this talking about the case where e.g. several properties change at once but you only really want a single notification?

1 Like

Good so far!

I'm curious why prior art does not mention Swift's getters and setters or property wrappers, which are both good ways to set up forms of observation that do not require a class to be used?

Or does Observable work with structs?

1 Like

Still haven't read the proposal in detail, but the _storage property kept me thinking. This isn't about the Observation but rather about a limitation we face that is reserved to the Swift compiler. I have not followed the macro proposals in details either so excuse me if I'm going to unintentionally rehash something. Could the new macros be used to rebrand @propertyWrapper attribute as a macro? If that could theoretically work out, then can we open up the creation of $ prefixed type members via macros? That would allow us to spawn a more unique name for _storage (e.g. $observableStorage) and possibly allow its exposure when needed.

cc @Douglas_Gregor I'd like to hear your opinion on the idea of allowing $ type members via macros.

1 Like

More thoughts:

Could we have a semi-opt-out functionality? I can imagine that we could permit the user to manually provide the Storage type. Then the macro will ignore all stored properties on the observable type itself and only synthesis the stored properties into the parent type. That gives us some fine grain control when needed and it also provides us the ability to expose the storage itself when needed via its access modifier.

Some examples:

@Observable
struct S {
  public struct Storage {
    public var a: Int
    internal var b: Int
    internal var c: Int
    private var d: Int
  }

  private var b: Int
  public var c: Int

  public var e: Int
}

// desugars to
struct S: Observable {
  public struct Storage {
    public var a: Int
    internal var b: Int
    internal var c: Int
    private var d: Int
  }

  // storage property exposed as `Storage` is public.
  public _storage: Storage // `public $observableStorage` would be nicer

  // synthesized as public
  public var a: Int {
    get { 
      _registrar.access(self, keyPath: \.a)
      return _storage.order
    }
    set {
      _registrar.withMutation(self, keyPath: \.a) {
        _storage.order = newValue
      }
    }
  }

  // access level through that property reduced from internal to private
  private var b: Int {
    get { 
      _registrar.access(self, keyPath: \.b)
      return _storage.order
    }
    set {
      _registrar.withMutation(self, keyPath: \.b) {
        _storage.order = newValue
      }
    }
  }

  // access level through that property is increased from internal to public
  public var c: Int {
    get { 
      _registrar.access(self, keyPath: \.c)
      return _storage.order
    }
    set {
      _registrar.withMutation(self, keyPath: \.c) {
        _storage.order = newValue
      }
    }
  }

  // synthesized as private
  private var d: Int {
    get { 
      _registrar.access(self, keyPath: \.d)
      return _storage.order
    }
    set {
      _registrar.withMutation(self, keyPath: \.d) {
        _storage.order = newValue
      }
    }
  }

  public var e: Int // has no synthesis, as it's explicitly not part of `Storage`
}

The stored property e cannot be observed. b and c property declarations are still consumed by the macro, as they are part of the explicit Storage type. a and d are projected from Storage into the S type and their access modifier is mirrored with the access modifier inside Storage.

If we could make something like this work, it would provide some great control over this functionality, but only when needed and not as a default requirement. It also shows that the storage type could be potentially exposed, if we want to allow that, and that it should probably have a better name which isn't prefixed by an underscore. If exposing the storage property is a no-go then ignore that part of my suggestion, but we might still want to provide an explicit Storage type declaration for the previously mentioned control.

1 Like

A great pitch! I'd definitely love this addition into the language.

My concern is about its use SwiftUI. If I understand correctly, you expect the View body to get reevaluated – under what conditions? Whenever anything in the model changes, or only when model.searchString changes?

With the former, the performance would get only worse than now.

With the latter, I have an issue here that this may be unexpected – there could be some variables that you do not want to get View to be redrawn with.

Also, the general paradigm in SwiftUI behavior is to use a PropertyWrapper for parts of code that can cause View reevaluation. For that, the SwiftUI code is quite easy to reason about but the proposed does break that – with us never being sure whether var could cause our view to reevaluate on its own.

I was fondly reminded of this document introducing key-value observing while reading this.

I could imagine these switch statements grow to an unwieldy wall of key paths. The document above speaks to the keyPathsForValuesAffecting<Key> convention. Any chance the observation machinery could look for the more precisely named method before hitting dependencies(of:)? e.g.: going with the example class, could it check for static var dependenciesOfSomeComputedProperty: TrackedProperties<Self>).

Really great pitch! Keep up the exciting work.

2 Likes