[Second review] SE-0395: Observability

The CoW semantics of the registrar are very close to this but I agree that it would be better to push that part out and only have the macro apply to class types for now.

The proposal itself does not specify any restriction other than enum (due to lack of stored properties) and actor (due to keypath incompatibilities) as restricted type categories. It would be only a small amount of modification to adjust the proposal to include a restriction to class types for the macro (and add in the blurb about future directions).

The draft implementation I have for that has a clear direction to move to supporting structures in an ABI compatible manner so this seems like a clear move to me.

2 Likes

If two or more modules have the same name for different types I'd expect to see a compilation error like "ambiguous name, use module prefix to disambiguate" rather than compiler "arbitrarily" picking up one type; so the error message spelling you are getting feels like a bug.

IRT SwiftUI.Observable: normally it's features or quirks or bugs are off topic on this forum which is focused on issues related to Swift language and its standard library.

The reexport part is beyond the scope of the proposal. However if we could apply some sort of disfavored overload type attributes to a marker protocol that would be reasonable to me: however I’m not sure that exists.

I think it's more correct to say that making the observer set CoW solves some of the problems of storing the set as part of the value but has no answer for the others. There's no incremental refinement of CoW that will make x = y stop overwriting the observers of x with the observers of y. Saying it's "close" makes it sound like there's a path for getting there from here, and there isn't.

5 Likes

Was there any further discussion regarding a trailing edge variant of the API, @Philippe_Hausler?

There seems to be a genuine need here. If Observable is to be a Swift language type – rather than just a data flow accessory for SwiftUI – it seems like a big omission not to include it.

The willSet variant is great for allowing SwiftUI to perform its 'before changes' render pass and schedule its 'after changes' render pass for the end of the current event loop cycle, but for anything else that needs to happen between these two points – we're stuck.

Really, the onChange parameter of the withObservationTracking(_:onChange:) function should be called willChange to better reflect the semantics of the API.

A participant in the old pitch thread laid out their frustration with the current API:

As I summarised in the pitch thread, the two recommendations we have right now are to a) use computed properties, or b) use this pattern:

However, method A suffers from unnecessary view refreshes, while method B suffers from an async hop i.e the observation event happens after the 'after changes' SwiftUI snapshot. It misses SwiftUI's deadline for changes in the current transaction. It's also for this reason that an asynchronous sequence based API wouldn't help us here either.

To make this work how we want it, we really need Observable to generate trailing edge (didSet) synchronous notifications.

This would allow dependent observers to make any changes they need to make within SwiftUI's deadline for the current event loop.

It doesn't need to be anything huge for this version of the proposal, something as simple as changing withObservationTracking(_:onChange:) to withObservationTracking(_:willChange:didChange:) could lay the foundation.

11 Likes

Thanks. What would be a proper channel to express these concerns and get someone from the core team / SwiftUI team to consider the grave impact this might have on many apps?

Just to clarify - I was not expressing frustration so much as asking a question. It is, however, something I would love to have addressed.

Daniel

1 Like

SwiftUI issues are best communicated as bug reports (feedback) to Apple. That said, the appropriate folks are already aware of the issue in this case.

5 Likes

That was probably me projecting then. :slight_smile: Fingers crossed we get a resolution.

hmmmm

var x = 42 {
    didSet {
        print("value changed from \(oldValue) to \(x)")
    }
}

Here "x" acts as location.

Do you think it is impractical to have observation in a language with value types only, say, "Val"? cc @dabrahams

Adding observing accessors to an individual variable is extrinsic to the value stored in the variable and doesn't contradict my point at all. If you add that to a struct member, though, it's probably going to have a lot of unwanted non-value-like behavior.

2 Likes

Please clarify what do you mean by that?

struct S {
	var x = 42 {
		didSet {
			print("S.x changed from \(oldValue) to \(x)")
		}
	}
}

var s = S() {
    didSet {
        print("s value changed from \(oldValue) to \(s)")
    }
}

Edit:

Conceptually, for any arbitrary deep value type variable I may want to track, say:

s.a.b[123].c["key"]

e.g. if I were to do it by hand, I'd "po" this value type value to a console, do "step over" or "step into", "po" again, and manually compare the old and the new values: if the two are different I know there's a change - in that case do whatever observation is suppose to do and remember the new value, then repeat – so this is possible, at least conceptually.

The next step to somehow automate the process. E.g. this:

startObserving(s.a.b[123].c["key"]) { newValue in
    print("value changed")
}

Where:

@discardableResult
func startObserving<T: Equatable>(
    _ get: @autoclosure @escaping () -> T,
    changed: @escaping (T) -> Void
) -> Timer {
    var old = get()
    changed(old)
    return Timer.scheduledTimer(withTimeInterval: 0.01, repeats: true) { _ in
        let new = get()
        if old != new {
            old = new
            changed(new)
        }
    }
}

Which is obviously non optimal but will do as a proof of concept.

And "the cherry on top of the cake" would be having this code hidden / autogenerated, a-la in SwiftUI:

let v = s.a.b[123].c["key"]
Text(v)

// the machinery knows about this dependency and will call this again if there's a change.

Most forms of observation are an expression of reference semantics, with associated "spooky action at a distance." It's almost in the definition of the pattern: you change this thing over here, and as a consequence any number of other things change over there, due to whatever observations have been set up.

In a pure value semantics world, you find other ways to solve the problems being addressed by observation. /cc @Alvae

3 Likes

The onChange argument has a broad semantic scope, making it unclear whether it will be called just once or repeatedly. It also lacks clarity on whether it will be called upon the first willSet, first didSet, or last didSet (as part of a transaction-like behavior).

To address these concerns and provide more precise information, the signature of the withObservationTracking function could be modified as follows:

public struct Observation {
  var willSet: Task<Void> // or `var willSet: Void { get async { ... } }
}

public func withObservationTracking<T>(
  _ apply: () throws -> T
) rethrows -> (T, Observation)

This API is self-explanatory: it is clear that it observes the "willSet" event and that it will only be triggered once.
Additionally, as a nice bonus, the result of the call can be used in the handling of the "willSet" event.

Are you suggesting that a mere "didSet" on a value type variable brings us to reference semantics?

I'm extremely curious to see examples of these other ways.

Me too, fwiw.

Not by itself, but if that didSet accesses any global variables, class instances, or shared closures with mutable captures, it does… and in my experience if it doesn't do any of those things, it's not very useful beyond printf debugging.

I'm extremely curious to see examples of these other ways.

Because it's an anti-pattern, I never think, "oh, I want an observer here; I'll have to find some other way to accomplish the same thing,” so it's hard for me to come up with specific examples. It might be easier if you describe what you use observation for.

The general form of the answer to your question is usually as follows: the thing being observed is part of some whole data structure that also contains the state that gets updated upon observation (this data structure might be as big as all the state of your program if you've made a mess of things, of course). The observation encodes a relationship between two parts of your state. Instead of hiding that relationship in the incidental algorithm created by the observation chain, make it a first-class invariant of your data structure and maintain it explicitly using a first-class algorithm.

HTH,
Dave

6 Likes

Bird's-eye view: you have this giant data structure that could well describe the state of the app, and you want to have a function to be magically called when needed to reflect the change in a particular subset of the state:

func foo() {
    let v = s.a.b[123].c["key"]
    print(v) // "UI"
}

An extremely inefficient workaround I outlined above would be to use polling:

var oldValue = ...

func timer() { // called every 1/100 second
    let value = s.a.b[123].c["key"]
    if oldValue != value {
        oldValue = value
        foo()
    }
}

I don't understand what you mean, can you show a pseudo code?

I think that your example introduces the kind of reference semantics @dabrahams was talking about.

You have s which represents some large state and a function foo that, if something happens to some part of s, is allowed to perform side effects. So an incidental algorithm has been created which can't be reasoned about in a value-oriented way. That's because calling a function (inout T) -> U on some part of s, might cause foo to get called and access some unrelated part; we've lost local reasoning.

var s: BinaryTree = ...

func foo() {
  // note: merely using `s` here already implies reference semantics
  print(s.left)
}

func bar(_ p: inout BinaryTree.Node) { ... }

// `foo` may or may not be called, there's no way to reason about what
// parts of `s` are actually accessed.
bar(&s.right)

If we further allow foo to mutate anything (putting aside inconsequential side effects, like logging), then we've brought back spooky actions.

var s: BinaryTree = ...
var t: [Int] = ...

func foo() { t.removeAll() }

func bar(_ p: inout BinaryTree.Node) { ... }

if !t.isEmpty {
  bar(&s.right)
  print(t[0]) // hmm...
}

In a world of value semantics, one would call bar on the whole s and let it figure out if it also wants to do something on s.left. That way, we can reason about all the parts that the operation may access by just looking at a single signature and a single function call, ignoring any other code.

2 Likes

For the purposes of this discussion we could limit "foo" to not change anything, so this is perfect:

func foo() {
    print(s.left) // poor man's UI
}

and the idea of an ideal observation is that "foo" above should not be called for:

bar(&s.right)

because it's "observing" a different part of the structure that's not getting changed. (In practice if I run this observation machinery by a trigger on "s" variable's didSet "foo" will be called, and with the above timer based approach "foo" won't be called; let's ignore these differences for now.)

Can you show a pseudocode for this as well?