[Second review] SE-0395: Observability

I believe init accessors (currently in pitch) would allow @Observable to relax the default value restriction.

7 Likes

Thanks, I'll look at the pitch for init accessors :-)

I'd suggest that the Observability proposal is amended so that the default value requirement is moved out of the proposal in a much more assertive way. For example:

Initializers

In the current implementation, the @Observable macro requires that all stored properties have a default value. This helps observable types rely on definitive initialization, use the implicitly generated initializers, and define additional initializers in extensions.

This default value requirement is not part of the proposal. It could be relaxed in a future version; see the Future Directions section for more information.

To clarify the intent; when init accessors land I am intending to have a PR land in conjunction that adds the init accessors to Observable synthesis and removes that compilation error requiring all values have an initial value.

To be honest; I have a feeling that a number of other similar model type things could utilize the init accessors to achieve similar things without requiring a bare initializer; to me that pitch (no matter the spelling that is settled on) seems like a really nice improvement for a number of use cases - and is a really ingenious solution to the problem space.

3 Likes

Bumping my questions since they seem to have been lost in the fray.

First off, I do like that we are descoping the proposal. This makes it both easier to review and focuses more on the immediate use-case that we want to support.

I do have some questions/feedback on the updated pitch though:

Observable marker protocol

The proposal says Observable is a marker protocol does this mean it acts similar to Sendable where there is no runtime type information available? How is the semantic requirement enforced?

withObservationTracking

  1. Does it ever make sense to have an async apply closure? (I think not but we should call it out in the proposal)
  2. The proposal doesn't state if onChange is only triggered once or multiple times? (From the code it looks like once since we are recursively calling the render method)

Sendability

The proposal shows the following code snippet:

@Observable public class Person: Sendable {    
    internal var firstName = ""
    internal var lastName = ""
    public var age: Int?
    
    public var fullName: String {
        "\(firstName) \(lastName)"
    }
    
    public var friends: [Person] = []
}

How is this class Sendable? From the expanded macro implementation it doesn't look like the macro is making it Sendable. How do we envision this to work? Furthermore, the example that uses the Person class then in the onChange method is spawning an unstructured Task in a @Sendable closure so we really require the type to be Sendable.

The storage requirement was dropped.

No longer needed since the storage is no longer there.

You can read the source here. Summary it emits to the list of currently tracked items the context of the registrar and keypath being accessed for future observation triggering from a given change.

That is not scoped so it is not a problem

Nope, because that isn't what is happening there.

That is not valid to apply to a computed property - you must write those manually (if the storage/mutation is external from the type and does not interplay with any composition of existing stored properties). If however the properties are composed from other stored properties then nothing special needs to be done.

Those closures cannot be asynchronous.

That happens only once.

Hmm @nnnnnnnn that looks like a misstep - that cannot and probably should not be Sendable.

I thought so, but it points to a larger question. Can you actually make your Observable class Sendable?

yes it can be; but some extra care might need to be taken since classes with mutable state are not trivially made to be Sendable.

Taking the Person example (without friends since that would be kinda lengthy to really show):

@Observable public final class Person: Sendable {    
  struct State {
    var firstName: String
    var lastName: String
    var age: Int?
    var fullName: String {
      "\(firstName) \(lastName)"
    }
  }

  let state: OSAllocatedUnfairLock<State>

  internal var firstName: String {
    get {
      access(keyPath: \.firstName)
      return state.withLock { $0.firstName }
    }
    set {
      withMutation(keyPath: \.firstName) {
        state.withLock { $0.firstName = newValue }
      }
    }
  }
  internal var lastName: String {
    get {
      access(keyPath: \.lastName)
      return state.withLock { $0.lastName }
    }
    set {
      withMutation(keyPath: \.lastName) {
        state.withLock { $0.lastName = newValue }
      }
    }
  }
  public var age: Int? {
    get {
      access(keyPath: \.age)
      return state.withLock { $0.age }
    }
    set {
      withMutation(keyPath: \.age) {
        state.withLock { $0.age = newValue }
      }
    }
  }

  init(firstName: String, lastName: String, age: Int?) {
    state = OSAllocatedUnfairLock(State(firstName: firstName, lastName: lastName, age: age))
  }

  public var fullName: String {
    access(keyPath: \.firstName)
    access(keyPath: \.lastName)
    return state.withLock { $0. fullName }
  }
}

Most of the boiler plate for this is caused by the requirements of Sendable; hence why it is often good to only mark reference types as Sendable if you absolutely must - there are often better ways to approach that.

2 Likes

That doesn’t really answer my question, so I’ll ask it another way. Why does this proposal require calling access(_:) on read, while KVO does not?

This is unfortunate, and I think it is further evidence that Observable does not qualify as a “marker protocol”.

As I mentioned upthread, the way to make something sendable is to simply add a conformance to Sendable. The implication of Observable being a protocol with no requirements is that someone could just as easily add a conformance to Observable and expect observation to function. Instead it will silently fail in a way the type system won’t catch.

A better design would only allow conformance to Observable by using the @Observable macro. If that’s not possible, then perhaps we must wait until we have an “observation sequence” type that @Observable macro could use for a generated property.

2 Likes

The access tracking is the crux to which the registration for automatic SwiftUI updates hangs upon. KVO was built for a legacy system of UI that did not interoperate with changes in that way; instead it relies upon swizzling class isa's to notify of events and kvo bindings to accomplish bidirectional connections. The merits of which are well outside of the scope of this proposal. In short - if KVO was re-designed today it likely would track access just like this does.

To be fair Sendable has the same issue - it does not know if the type is actually safe to be used in multiple isolation domains. It only checks the conformances of composition - which is a compilation feature somewhat orthogonal to the nature of it being a protocol.

The basis for a non-marker protocol is that it must be the root definition of how that type is used. Since the observation is defined as internal to the type (which is required for ensuring the type is not leaking out implementation details like how KVO/KVC can by accessing, mutating, or observing private ivars) it cannot be a requirement. Since requirements are public interfaces by public conformance to a protocol. This leaves the only remaining utility as a marker because it has no requirements. That marker allows for the affordance of restricting the types used to construct APIs.

I think this is an overly strong position, but I also think the following alternative satisfies it:

public struct ObservationSequence<T> {
    private init(_: ObservationGuts) { }
}

public protocol Observable {
    public func changes<T>(to: Keypath<Self, T>) -> ObservationSequence<T>()
}

extension Observable {
    public func access<T>(_: Keypath<Self, T>)
    public func withMutation<T>(of: Keypath<Self, T>, do: () -> Void)
}

In this case, you interact with a value’s observability by calling its changes(to:) method. Because ObservationSequence has a private initializer, only the @Observable macro could actually implement conformance to the Observable protocol, thus avoiding the confusion inherent in the marker protocol approach.

Aren't macros expanded at compile-time? If ObservationSequence used a private init, how would that be called by the macro? You still can't use a private init.

What happens if observed stored properly is mutated inside the apply closure? Would onChange be called inside apply? SwiftUI currently triggers an assertion, but if we want to support chaining of observables, it might need be allowed. Maybe if there is a mutation, stored properly should be excluded from access list? Treated as output, not input?

Why onChange is Sendable? If we can access stored properly inside the apply, that means that apply and entire call to withTracking() is happening inside the corresponding isolation context. When properly is mutated and onChange is called, we are in the same isolation context. So calls to the withTracking() and onChange() happen within the same isolation context.

1 Like

Thinking about the Sendable stuff some more. Shouldn't an Observable be Sendable out of the box? What is the value of an Observable object that is not Sendable, especially when the onChange closure is @Sendable? Should we add two APIs depending on if the Observable is Sendable the closure becomes @Sendable or not?

1 Like

For me, the major use case of Observable would be same actor observation for UI related updates. No need for Observable to be Sendable in this case. But I agree, in that case, I'm not sure what benefits would be derived from the onChange closure being Sendable – the willSet timing becomes meaningless across an actor boundary.

Longer term, for inter-actor observation, I would hope to see a nonisolated API added to Observable that yields a Sendable type.

I do see same actor observation, more specifically MainActor observation, to be very common. Maybe we can get away with only supporting this in the beginning.

I don't think yielding Sendable type is enough for inter-actor observation. Most likely any actor that observes an Observable wants to actually get the Observable which means the Observable must be Sendable. Maybe at this point the Observable should be an actor instead of a class.

If it's a GAIT (i.e. @MainActor) it should be fine, too. Then at least the Observable can be 'passed around' to retrieve some kind of Sendable observable sequence from a non-isolated API. So for an inter-actor Observable it just needs to be a) Observable and b) conform to Sendable directly, or conform via virtue of being an Actor or GAIT (@MainActor, etc).

My point being, Observable doesn't need to specify Sendability conformance directly. If programmers want inter-actor observation they just need to make sure their Observable type is also Sendable. (Assuming the appropriate nonisolated API on Observable.)

In the suggested reading section, the link to attached macros leads toa 404 page FYI.
This is the broken link: https://github.com/DougGregor/swift-evolution/blob/attached-macros/proposals/nnnn-attached-macros.md

I do agree on this point. The combination of Observable + Sendable should unlock inter-actor observation.

I am unsure about this one. Why would you need the API to be nonisolated? Presumably for inter-actor we can use AsyncSequences since we do have hops anyhow.

1 Like