SE-0395: Observability

Hi Swift community,

The review of SE-0395: Observability begins now and runs through April 24, 2023.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to the review manager. When emailing the review manager directly, please keep the proposal link at the top of the message.

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

  • What is your evaluation of the proposal?
  • Is the problem being addressed significant enough to warrant a change to Swift?
  • Does this proposal fit well with the feel and direction of Swift?
  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available at

https://github.com/apple/swift-evolution/blob/main/process.md

Thank you,

Ben Cohen
Review Manager

13 Likes

To be clear here, this API includes no control over when observation events are triggered (willSet vs. didSet)? Since all observations are async that may not be an issue, but the async nature will certainly require adjustment. Additionally, is the exact timing of event emission documented directly anywhere? It seems implied that it's only emitted at suspension points or when access is complete but it's not completely clear. On a related note, the example about suspension points seems at odds with the generated code example directly afterward. What mechanism is withMutation using to apparently coalesce changes between suspension points into a single change?

1 Like

I'm trying to read the proposal and understand the capabilities of generated dependencies(of:). It is noted in the Future Directions section:

Future macro features could potentially permit refinements to the dependencies(of:) implementation generated by the @Observable macro, to more accurately select the stored properties that each computed property depends on.

But it doesn't explain what the current limitations are. It'd be nice to have some clarification around to better understand potentially unsupported cases.

Additionally, in the explanation of the synthesis, I'm having trouble understanding why the following would be generated for the PropertyExample

nonisolated static func dependencies(
    of keyPath: PartialKeyPath<Self>
) -> TrackedProperties<Self> {
    let storedProperties: [PartialKeyPath<Self>] = [\.b, \.c]
    switch keyPath {
    case \.a:
        return TrackedProperties(storedProperties + [keyPath])
    default:
        return [keyPath]
    }
}
@Observable final class PropertyExample {
    var a: Int {
        willSet { print("will set triggered") }
        didSet { print("did set triggered") }
    }
    var b: Int = 0
    var c: String = ""
}

Does adding property observers willSet/didSet change the generation this way? It isn't intuitive to me that add willSet/didSet would result in a triggering as a change for all changes of a and b.

I might be misunderstanding things, and it could be helped by expanding the details a bit on how dependencies(of:) is generated.

I think that example has a copy/paste mistake in it - the case where synthesis of dependencies break down is when you have a manual get/set for the property. In those cases the dependencies by default claim that all non-computed properties are a dependency.

willSet/didSet do not interfere with synthesis; they act just as stored properties (modulo lifting the willSet/didSet into the synthesis of the setter).

Properties that have willSet and didSet property observations are supported. For example, the @Observable macro on the PropertyExample type here:

@Observable final class PropertyExample {
   var a: Int {
       willSet { print("will set triggered") }
       didSet { print("did set triggered") }
   }
   var b: Int = 0
   var c: String = ""
}

...transforms the property a as follows:

var a: Int {
   get {
       _$observationRegistrar.access(self, keyPath: \.a)
       return _$observationStorage.a 
   }
   set {
       print("will set triggered")
       _$observationRegistrar.withMutation(of: self, keyPath: \.a) {
           _$observationStorage.a = newValue
       }
       print("did set triggered")
   }
}

It seems unclear to me how the transformation handles oldValue in didSet. I’m also not sure if implementing willSet and didSet in this way works identically to that without observability.

1 Like

I think these are covered pretty explicitly in the "ObservedChanges and ObservedValues" section? Each mutation schedules an event emission, and each event will at delivery time include all changes since the last event. If the observation and mutation are on the same actor this inherently means that event emission can happen only at suspension points.

If that's the case, shouldn't the ImageDescription example be updated to call out the fact that the example is only true if the observation is in the same isolation? Is there an alternate mechanism we're supposed to use to support atomic updates in different isolations? I suppose we can always create an inout method to do the mutations in, which should make them atomic to the containing observable, but if users aren't told when that's necessary it can become a source of bugs (as noted in the proposal).

1 Like

I think the ImageDescription example is invalid. The class isn't Sendable or isolated to a global actor, so it simply can't have an async function that mutates its properties in the first place. Making it Sendable but not isolated would require adding locking and that locking would provide the atomicity. Making it isolated would mean that the observation has to happen on the same actor as the mutation.

If that's truly the case, it might lead developers down paths that produce more changes than intended, potentially even going so far as to cause poor performance and the cause will be unclear.

It might be worth considering either

  1. disabling observation for computed properties unless manual dependencies(of:) enables it (which is the NSKeyValueObserving approach
  2. disabling synthesis of dependencies(of:) (i.e. requiring manually implementing it to compile)

To me, this seems like a new sharp edge in SwiftUI performance, and it'd be nice to avoid that. Of course, it's difficult to say how either of the 3 directions (my 2 plus what is in the proposal) would scale in the real world.

1 Like

If that's the case that's a huge limitation. I would expect that observations being async in the first place would allow observation of Sendable values to cross isolation boundaries, if nothing else. I guess I don't fundamentally understand what role isolation is really playing here, especially since that's not really vocabulary the community uses.

1 Like

My first impression from reading this is that the conceptual model is very reminiscent of how Realm's async notifications work today, and that's something we've felt has largely worked out well in practice. Our design is a bit more complicated because we also integrate things like rerunning database queries on a background thread into the notification system and we can do things like produce copies of non-Sendable objects isolated to arbitrary actors, but overall this looks a lot like what I'd have come up with if I was designing a new Swift-native version of it.

I need to give it a few more reads and actually dig into the finer details, but so far I think there is at least some sort of answer in the proposal to all of the complications we ran into.

1 Like

note: the dependencies have no bearing on SwiftUI usage - it doesn't need them to work.

1 Like

Observable sounds like a good candidate and motivation to witness macro. Is this a future direction?

2 Likes

I guess we can set aside any future use of this by SwiftUI.

I think it’d be surprising if one created a type with @Observable that has 500 stored properties and 1 computed property that combines the values of 2 stored properties and after some investigation of performance issues they find out that in their program, while modifying the 498 other stored properties, they were inadvertently triggering updates on some distant subsystem observing that one computed property.

I feel like opt in would be a better default because it forces the decision on the developer. By forcing the decision, it becomes very clear what it happen, and clarity over brevity is important.

Also, if someone writes a computed property that depends on data from a different type entirely, even this very aggressive approach won’t actually result in changes being emitted when one expect it to by “magic”.

Where is the link to the implementation of this? I don't find it at the start of the proposal page where the standard template usually includes it.

(Side note, the very first self-link to SE-0395 is also broken.)

1 Like
  • What is your evaluation of the proposal?

Overall, I'm in support of having observation provided out-of-the-box by the language. I have a few design questions.

  • Is the problem being addressed significant enough to warrant a change to Swift?

Yes; the main library-specific solution available today (ObservableObject) has a handful of shortcomings summarized in the proposal.

  • Does this proposal fit well with the feel and direction of Swift?

For the most part, yes—a couple areas in this proposal stand out as quirks:

  • Behavioral discrepancies between stored properties and initializers for @Observable classes compared to non-@Observable classes;
  • Key path limitations hindering natural composition (multi-component key path observation; actor property key paths).

I've included a few more detailed thoughts in the section below.

  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

I'm comfortable with both KVO and Combine (read: have written bugs with both systems). This proposal thoughtfully learns from each.

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

A thorough read of this proposal iteration, and a casual following of earlier pitch phases.


My specific notes & questions are as follows:

  1. class vs. @Observable class — Observable class stored properties and initializers have a few surprising behaviors which don't align with standard classes:
  • Required type annotations on stored properties: an unfortunate quirk unique to these classes; would be great to see this experience inform the ongoing design of macros.
  • Generated memberwise initializer: this is a welcome convenience in principle, but feels unusual given the analogous feature does not exist for standard classes.
    • In particular: how does the synthesized memberwise initializer interact with a public @Observable class? Suppressing this synthesis for public struct types is deliberate to avoid unintentional API changes when a type evolves to gain new properties.
  1. dependencies(of:) — Is there any design which would permit an extension computed property defined by a different module to become observable? Consider the following:
// Module ImageCore
@Observable public class ImageDescription {
    public var width: Double
    public var height: Double

    public var area: Double { width * height }

    nonisolated static func dependencies(
        of keyPath: PartialKeyPath<Self>
    ) -> TrackedProperties<Self> {
        switch keyPath {
        case \.area:
            return [\.width, \.height]
        default:
            return [keyPath]
        }
    }
}

// Client App
extension ImageDescription {
    var isSquare: Bool { width == height }
    // Unable to specify dependencies of \.isSquare
}

Given how common client-written extensions are, this would be great to support.

  1. ObservationTracking — is type an empty namespace? Was a free function (e.g. withObservationTracking(_:onChange:)) considered?

  2. ObservedChange — could/should this type provide access to the equivalent of oldValue (and even newValue for convenience / disambiguity) for each changed key path?

  3. Multi-component key paths\.account.name not being observable feels like a footgun if silent; this is an unfortunate missing case of feature composition. Assuming this is not easily enforceable at compile-time, could a runtime check against the number of components in the key path trap if observation is attempted?

3 Likes

Hello, thanks for the updated pitch. In the willSet/didSet section, it looks like the @Observable macro will generate invalid code if willSet and didSet declare the same local variable:

@Observable final class PropertyExample {
    // INPUT
    var a: Int {
        willSet { let x = "foo"; ... }
        didSet { let x = "bar"; ... }
    }

    // TRANSFORMED
    var a: Int {
        get {
            _$observationRegistrar.access(self, keyPath: \.a)
            return _$observationStorage.a 
        }
        set {
            let x = "foo"; ...
            _$observationRegistrar.withMutation(of: self, keyPath: \.a) {
                _$observationStorage.a = newValue
            }
            // ❌ Invalid redeclaration of 'x'
            let x = "bar"; ...
        }
    }
}

A quick solution would be to wrap the willSet and didSet transformations in their own separate do blocks.

3 Likes

This comment from the second pitch has been left unaddressed.

To sum up: user code has no way to know when observation has actually started, so user code has no way to know when it can start modifying a value with the guarantee that changes will be notified. In other words: the proposal, as is, has an unaddressed problem of "missing frames".

This is a fundamental flaw in the proposal, in my opinion.

Maybe this race can be avoided with withTracking - but I'm not sure this is ergonomic, or even possible.

Pretty strong -1 for me, until practical use cases have been considered and studied in-depth. For example, I suggest considering use cases of UIKit/AppKit applications, because SwiftUI is not the only GUI framework used by developers.

13 Likes

Can I try rephrasing your question purely in terms of the observer, rather than from the perspective of when another part of the code can know when it is able to modify the value? i.e.

How can an observer both read the initial value, and subscribe to updates, without risking missing updates in-between?

seems simpler to discuss, unless it misses some key use case (that could not be covered merely by building on top of a solution to the purely in-terms-of-the observer phrasing of the question above).

Note, in your pitch comment you wave away discussion of actors:

Maybe some @MainActor decorations have to be added - but this is the gist.

But you cannot do this. In Swift, actors and suspension points are fundamental to solving these problems, and are key to the working of parts of this proposal. The code snippet above that comment is incomplete without being explicit about how isolation is achieved on the initial read.

3 Likes