[Second review] SE-0395: Observability

Hi Swift community,

The second review of SE-0395: Observability begins now and runs through June 12, 2023.

The proposal has been amended from the first review, primarily to subset out some functionality in order to focus on an Observable marker protocol and the withTracking(_:changes:) function, and adds more focus on supporting observation for subclasses. The asynchronous values(for:) and changes(for:) methods have been removed, and are now discussed in future directions.

A diff for the recent revision of the proposal can be found here.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to the review manager. When emailing the review manager directly, please keep the proposal link at the top of the message.

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

  • What is your evaluation of the proposal?
  • Is the problem being addressed significant enough to warrant a change to Swift?
  • Does this proposal fit well with the feel and direction of Swift?
  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available at

https://github.com/apple/swift-evolution/blob/main/process.md

Thank you,

Ben Cohen
Review Manager

9 Likes
  • If @Observable is a “marker protocol” with “no formal requirements”, how is ObservationTracker the “required storage for tracking accesses and mutations”? What enforces that requirement? Should it not be a formal requirement of the protocol? (Edit: contrast this with Sendable, which is a marker protocol explicitly because you can just tag something with the protocol to make it Sendable.)
  • What does ObservableStorage.access(_:) do? Is it necessary to implement some sort of ordering for multithreaded observers? If so, given get { defer { foo() } access(\.bar) }, does foo() occur within or outside of the scope of access(_:)? Would it be clearer if access(_:) were scoped like withMutation(_:)?
  • Can I use @ObservationTracked on a computed property to automatically wrap its getter and setter in access(_:)/withMutation(_:)? Or do I have to write those out manually?
2 Likes

Thanks for putting this together. I think it makes sense to push the async parts to a future direction – and has the benefit of simplifying the review.

One question that spills over from the last review: how does the withTracking mechanism handle chained observations?

Take the example of an external module that exposes a public @Observable type (ExternalObservable) . Perhaps it is observed by both a) an internal Observable (InternalObservable) that, on observing a change, updates some internal state, and b) a SwiftUI View that observes the same property.

Now, if that View is also observing the state on InternalObservable, can we be guaranteed that the updates will occur in the same run-loop cycle/animation transaction?

Some code to illustrate the question:

// View

struct SomeView: View {

  @State private var model = InternalObservable()

  var body: some View {
	// Here, We're observing one property on the external 
	// Observable directly, and one property indirectly via 
	// the Internal Observable as an intermediary.
	//
	// If we cause the properties to be updated at exactly the
	// same time (via calling `updateBothProps() for instance)
	// we want the view update for both properties to occur at 
	// exactly the same time, too (the same animation 
	// transaction/view update/run loop cycle.)
    VStack {
      Text("\(model.external.directlyObservedProperty)")
      Text("\(model.dependentProperty)")
    }
  }
}


// Internal Observable

@Observable
final class InternalObservable {

  var dependentProperty = "Some Value"
  let external = ExternalObservable()

  init() {
    resetObservationOnExternalObservable()
  }
  
  // We're observing the external Observable, and updating
  // some local state which in-turn is observed by the view.
  // Yes, it would be simpler as a computed property, but 
  // perhaps we want to limit view updates, or alter the 
  // observation, or some other view invariant we wish to 
  // maintain.
  func resetObservationOnExternalObservable() {
    ObservationTracking.withTracking {
      self.dependentProperty = "\(external.indirectlyObservedProperty). Woohoo!"
    } onChange: { [weak self] in
      self?.resetObservationOnExternalObservable()
    }
  }
}

// Some other module

@Observable
public final class ExternalObservable {

  public var directlyObservedProperty = "Some Value"
  public var indirectlyObservedProperty = "Some Value"
	
  // Calling this method affects both text fields in the View.
  // Hopefully the changes are guaranteed to be coalesced into
  // a single view update!
  public func updateBothProps() {
    self.directlyObservedProperty = "I'm changing"
    self.indirectlyObservedProperty = "And so am I!"
  }
}

Obviously this is a contrived example and can be simplified using a simple computed property, but hopefully works for illustration purposes.

The feeling I got from the last thread was that this wasn't quite as simple as one might hope:

I'd be interested to know if the revised implementation resolves this.

I have a few questions, sorry if they have already been answered in the other thread, I couldn't find it.

  • is there a way to cancel a withObservationTracking() observation before onChange is ever called?
  • if we access the changed value from within onChange, will we see the old value or the new value?
  • I'm curious and confused about the @autoclosure for the onChange parameter, what benefit does wrapping the closure in another closure provide here?

So that chaining won't be needed first off; the indirect value when composed with other values will be tracked. So the code is considerably more simple than you might have expected:

// View

struct SomeView: View {

  @State private var model = InternalObservable()

  var body: some View {
    VStack {
      Text("\(model.external.directlyObservedProperty)")
      Text("\(model.dependentProperty)")
    }
  }
}


// Internal Observable

@Observable
final class InternalObservable {

  var dependentProperty: String {
    "\(external.indirectlyObservedProperty). Woohoo!"
  }
  let external = ExternalObservable()

  init() { }
}

// Some other module

@Observable
public final class ExternalObservable {

  public var directlyObservedProperty = "Some Value"
  public var indirectlyObservedProperty = "Some Value"
	
  // Calling this method affects both text fields in the View.
  // The changes _are_ guaranteed to be coalesced into
  // a single view update!
  public func updateBothProps() {
    self.directlyObservedProperty = "I'm changing"
    self.indirectlyObservedProperty = "And so am I!"
  }
}

However if you do have a need to (in non SwiftUI scenarios) to do something similar then the use for the withObservationTracking (note; it got changed to a free floating function).

func resetObservationOnExternalObservable() {
    ObservationTracking.withTracking {
      self.dependentProperty = "\(external.indirectlyObservedProperty). Woohoo!"
    } onChange: { [weak self] in
      // The onChange is fired on the willSet side of things - so the reset needs to be called asynchronously to prevent a recursion and also update after the property changes.
      Task { @MainActor in
        self?.resetObservationOnExternalObservable()
      }
    }
  }

Internally yes there is a way to do this; but that is not exposed since it would be a fairly complicated story about how one cancels the actual instance of observation.

The onChange occurs on the leading edge (willSet).

This is done for performance reasons; it ensures that the callers that are not seeing any accesses within the scope do not get penalized for creating the onChange closure until it is needed. The withObservationTracking function is designed to handle very high volume and frame rate sensitive contexts.

1 Like

I thought about this but creating the wrapper closure also needs to retain/copy all the variables captured by the inner closure. What makes this more performant?

3 Likes

Yeah, I think the willSet thing definitely means that the answer to my question is: no, due to the async call, it can't be chained without breaking view invariants.

It would be nice if we could have a synchronous didSet variant.

Something like:

public func withObservationTracking<T>(
    _ apply: () -> T, 
    willSet: @autoclosure () -> @Sendable () -> Void = {},
    didSet: @autoclosure () -> @Sendable () -> Void = {}
) -> T

This would allow: Updating a dependent property that, if my understanding is correct, can still be applied synchronously – in time to take part in the current view transaction.

3 Likes

Just wanted to bump an unanswered question from the first review, but the proposal still doesn't mention how observability plays with dynamic member lookup, and subscripts in general. Can some light be shed here?

3 Likes

Those types of things would need to be done manually - since macros have no capability of editing the function body of the subscript accessor.

It would need to take the form of something like this.

subscript (...) -> ... {
  get { 
    access(<keypath>)
    ...
  }
  set {
    withMutation(of: <keypath>) {
       ...
    }
  }

While I'm neither against nor pro macros or another particular mechanism like type wrappers, I'd like to notice that the super important and profound feature like observation might require some unique traits we might not have readily available in the language yet; it's worth to go from "what we need" to "how we do it" instead of first choosing any particular implementation method.

For what it is worth; I know how the usage of those would work from an observation standpoint - it was included within the exploration I did to understand the extent of the AsyncSequence based prototypes. Basically the territory of that is known but requires a decent amount of work to complete - and in the interim it is better to give the common use case as a solid feature than to push the language too far beyond experimental noodling.

In the spirit of design by progressive disclosure; it does make sense to have a more advanced use case like dynamic member lookup to require slightly more advanced usages for observation.

Because observable types generally use the implicitly generated initializers, the @Observable macro requires that all stored properties have a default value. This guarantees definitive initialization, so that additional initializers can be added to observable types in an extension.

For the project I’m working on, majority of ObservableObject’s have their dependencies on injected through initializer. This constraint will lead to a lot of IUO, and will discourage adoption of the feature.

Would it be possible to have an initializer that initializes underscored properties?

@Observable class Foo {
    var k: Int

    init(k: Int) {
        self._k = k
    }
}
5 Likes

This is a pretty important feedback, thank you. Forcing all properties to have a default value have several undesired consequences:

  • Some teams will use IUO properties (var foo: Foo!), as mentioned by @Nickolas_Pohilets. That's acceptable but not ideal.
  • Some teams will not use IUO (because they avoid the ! sigil) and initialize invalid instances instead (dummy default values that don't go well together in objects that should enforce inner invariants), because they fear ! more than bogus instances and internal inconsistency bugs :person_shrugging: That's a consequence of how some people are, and of the strictness of some coding styles. I'm concerned that the proposal fosters the creation of such invalid instances.
  • Some teams or libraries will require an initializer without any argument init() on top of @Observable, because the proposal makes it possible. Requiring "default instance" initializers is a trend that I find very concerning. It is already fostered by property wrappers, because PW only work from instances. Some libraries use PW as providers of "static" information, and they don't hesitate requiring "default instance" initializers in order to access this "static" information from the default instance. When a major one like Vapor/Fluent does this ("Models must have an empty initializer method"), this is a strong signal that this is an acceptable design. Well, this is questionable design IMHO, and I'm concerned that @Observable pours more water into this bucket.

@Philippe_Hausler, I'd be interested in learning about what is missing from the language that would allow this proposal to not require default values (and avoid the undesired consequences described above).

16 Likes

I wanted to share a little more context for one of the reasons why I think the ability to chain Observables is important. The pitch states that one of the problems with Published is that it triggers unnecessary view updates.

The @Published attribute identifies each field that participates in changes in the object, but it does not provide any differentiation or distinction as to the source of changes. This unfortunately results in additional layouts, rendering, and updates.

I agree. However, the pitched proposal has the same problem but from a slightly different angle.

For example, imagine an Observable that emits rapid-fire user input events:

protocol MouseObservable: Observable {
  var position: CGPoint { get }
}

If we want to create a view that switches views when the mouse reaches a threshold of 100 pixels, we could do this:

struct MouseView: View {
  
  @State private var mouse = MouseObservable()

  var body: some View {
    if mouse.position.x > 100 {
      ViewA()
    }
    else {
      ViewB()
    }
  }
}

However, this means that a view update will be triggered and reevaluated every time the mouse position updates. Every pixel. X or Y. This seems in contradiction of the above quoted aim to avoid unnecessary ‘additional layouts, rendering, and updates’.

Even if we were to create a computed property:

extension MouseObservable {
	var isXGreaterThan100: Bool { mouse.position.x > 100 } 
}

And use that in our view instead – the very ‘additional layouts, rendering, and updates’ that we are trying to avoid still get triggered!

If there were an Observable API that offered a synchronous, trailing-edge, didSet based update it would be possible to get around this using an intermediary Observable. The intermediary observable can then perform direct observation on the mouse, and only update its own properties when there’s a relevant change:

final class ViewModel: Observable {

  private let mouse = MouseObservable()
  var isXGreaterThan100 = false

  init() {
    reobserveMouse()
  }

  func reobserveMouse() {
    // ⚠️ this API differs to the one proposed.
    // It uses a 'trailing edge' observation rather than the
    // 'leading edge' observation specified in the proposal.
    withObservationTracking {
      if isXGreaterThan100 != mouse.position.x > 100 {
        isXGreaterThan100.toggle()
	  }
    } didChange /* trailing edge */ : { [weak self] in
      self?.reobserveMouse()
    }
  }
}

Now, isXGreaterThan100 only gets updated when the condition actually changes. In addition, as it’s called synchronously, it’s guaranteed to be in the same animation transaction as any other properties dependent on the mouse position. View invariants are maintained. And of course, unnecessary ‘additional layouts, rendering, and updates’ are avoided’ .

Also, I’m not necessarily wedded to the didSet mechanism being part of the withObservationTracking API – just something that facilitates this capability. Arguably a simpler synchronous mechanism (vs. the previously proposed async mechanisms) for this would be more ergonomic.

final class ViewModel: Observable {

  private let mouse = MouseObservable()
  var isXGreaterThan100 = false
  private var observation: ObservationToken?

  init() {
    observation = mouse.observe(\.position, edges. [.initial, .didChange]) {
	  if isXGreaterThan100 != mouse.position.x > 100 {
        isXGreaterThan100.toggle()
	  }
	}
  }
}

In summary, my feeling is that, if the aim of this API is to provide control of how and when view updates are triggered for SwiftUI, we need an API that gives us some control of how and when those observation events are fired. We need to be able to get initial, willSet and didSet values so that we can properly orchestrate property updates, and crucially, we need to receive them synchronously in order that we can tie in to the current view update/event loop cycle/animation transaction.

6 Likes

I have a suggestion for even better API:

class CachedObservable<T: Equatable>: Observable {
    init(_ block: @escaping () -> T)
    var value: T { get }
}

@Observable
class ViewModel {
  private let mouse: MouseObservable
  private let isXGreaterThan100Observable: CachedObservable<Bool>
  var isXGreaterThan100: Bool { isXGreaterThan100Observable.value }

  init(mouse: MouseObservable) {
    self.mouse = mouse
    isXGreaterThan100Observable = CachedObservable {
        mouse.position.x > 100
    }
  }
}

I would love to have this available as a macro, but not sure what would be the most ergonomic way to avoid capturing self:

@Observable
class ViewModel {
  private let mouse: MouseObservable

  @CachedObservable
  var isXGreaterThan100: Bool { mouse.position.x > 100 }

  init(mouse: MouseObservable) {
    self.mouse = mouse
  }
}
1 Like

Yeah, that could be a useful macro – but alas this proposal doesn't support it!

Good example. We need dozens of such use cases / users stories and validate observation designs on them.

This particular example prompts for either:

class MouseObservable: Observable {
	var position: CGPoint {
		didSet {
			greaterThan100 = position > 100
		}
	}
	SOME_OBSERVARION_MARKER var greaterThan100: Bool
}

or:

class MouseObservable: Observable {
	var position: CGPoint
	SOME_OBSERVARION_MARKER var greaterThan100: Bool {
		position > 100
	}
}

with the fields without a marker not triggering view recalculation.

1 Like

I think State types are a common one. I've seen lots of ObservableObjects that look something like:

final class Model: ObservableObject {

  enum ViewState {
    case initial
    case ready(SomeBigInternalStateStructWithLotsOfNestedProperties)
    case error(Error)
  }

  @Published private var state = ViewState.initial

  // ⚠️ re-computed by the view each time any value
  // on state gets updated – nested or otherwise
  var someViewConvenience: Bool {
    guard case .ready(let props) = state else { return false }
    return props.someNestedComputedProp
  }
}

Rebuilding this as an Observable isn't going to yield any better performance:

final class Model: Observable {

  enum ViewState {
    case initial
    case ready(SomeBigInternalStateStructWithLotsOfNestedProperties)
    case error(Error)
  }

  private var state = ViewState.initial

  // ⚠️ _still_ re-computed by the view each time any value
  // on state gets updated – nested or otherwise
  var someViewConvenience: Bool {
    guard case .ready(let props) = state else { return false }
    return props.someNestedComputedProp
  }
}

I think markers could be one approach, but I think that could eventually get quite noisy. Also, in the example above MouseObservable could very well be in a third-party module, so it would need to work via extensions.

I'd be more inclined to have a type returned by some kind of changes(on:edges:) method on Observables that can be adapted with chained operators – an extremely lightweight reactive interface. That would give room for things like uniquing, but also bridging to the asynchronous world.

final class ViewModel: Observable {

  private let mouse = MouseObservable()
  var isXGreaterThan100 = false
  private var observation: ObservationToken?

  init() {
    observation = mouse
      .observe(\.position, edges. [.initial, .didChange])
      .map { $0.x > 100 }
      .distinct()
      .assign(to: isXGreaterThan100)
  }
}

You could streamline it so it's just the essentials (no back pressure, no throwing, no generic Failure param, no Dispatch queue jumping, no temporal operators, etc.) – so it's really simple for people to pick up. If they need the rest – bridge it to an AsyncSequence via an async operator.

1 Like

The reason I don't like this is because it is too manual and error prone: very easy to forget to do that (or do it wrong).