SE-0258: Property Delegates

[Douglas_Gregor]

@tanner0101

Discoverability could be helped in other ways that don't need to make the syntax more verbose. For example, option clicking on the attribute could inform you that it's a property delegate and link you to the implementation / doc blocks.

FWIW, this should fall out of the compiler 'walking' the custom attributes. When the new toolchain is available, it should be testable.

Doug

[Douglas_Gregor]

@anandabits

Having used the toolchain, the $ property is pretty discoverable, as it shows up in the autocomplete list for the enclosing type.

But it's not discoverable when writing enclosingType.delegatingProperty. , does it? The delegate's API would be more discoverable if it were available as $ -prefixed identifiers off of the main property. I'm not positive that's a good idea but as I mentioned in my review, I do think it is worthy of more discussion than it has received.

Apologies for repeating myself, but we can make the delegate members visible via autocomplete and go back to insert the '$' in the right place.

Doug

[Douglas_Gregor]

@Jon_Hull

The advantage of a post-fix operator, as opposed to a shadow variable (i.e. foo$ instead of $foo) is actually pretty substantial (despite being almost the same syntactically):

• The mental model is easier to understand. I am doing something to the property itself as opposed to referencing a different invisible variable. (e.g. myProp$.reset() is resetting the property)
• When we do get composition, it will naturally compose to allow deeper access (e.g. myProp$.deeperFunc() )

Tl;dr: We need to spend a little more time thinking through the details of the syntax. One more round. I don't feel like it is fully baked yet...

FWIW, before posting my original pitch, I went through and prototyped a design that involved a postfix operator that could be repeated to expose different 'levels' of the composition. People found it too clever, and one problem with postfix operators is that they're actually less discoverable that (say) a separate property. You can pull the same tricks I mentioned before where you provide the API of the delegate type on the original property in code completion (inserting the $ where appropriate), but having a desugaring to a separate property is a simpler overall mental model. "This syntax with the @Foo desugars to ."

Doug

[Douglas_Gregor]

@zwaldowski

Additionally, I think the trend for Swift has been moving away from using concrete protocols and toward more ad hoc protocols, because protocols aren't (and won't ever be, IMO) expressive enough for our syntactic tie-ins.

This isn’t true as recently as SE-0228. Having a named protocol in the stdlib even if it can’t describe its requirements there is far preferable to me. A protocol gives users something to Cmd-Click on, where they immediately see the feature in the larger context of the language feature as crafted by the Core Team and Apple’s technical writers. This has immense value for people learning of the feature. That property delegates are pitched as something a Swift user might use daily has me expecting absolutely nothing less.

I consider this a tooling problem, where Cmd-Click on an attribute should give you documentation about what it does. Amusingly, with custom attributes, we'll get this behavior for free because custom attributes refer to types---which already have all of this infrastructure. Documentation for @Lazy would be easy to find that for lazy.

Doug

[Douglas_Gregor]

@Jon_Hull

• We should have a plan to allow parameters in the attribute (e.g. @Delegate(foo: 1, bar: "") ). Are those inits? Some other form of declaration? We don't have to implement it now, but we should know where we are headed.

This is supported; see the UserDefaults example and this section.

@Jon_Hull

Furthermore, when you think about it, the real problem isn't whether to expose the storage or not, but that we really want to expose only a few methods of the delegate and keep the rest private to the enclosing type. That is something our current access system can't quite spell.

I don't agree with this. The delegate type can control its API via the normal access control rules. It shouldn't have different API when you have a "normal" instance of the type vs. when it's a delegate for another type, because that breaks the "it's just a type" model.

Doug

[Douglas_Gregor]

@DaveZ

What do you think about allowing people to declare type variables? That would solve some problems here. For example, instead of @UnsafeMutablePointer @Atomic var x = y being problematic (because attributes are unordered), one could write var x : Atomic<UnsafeMutablePointer<_>> = y where _ in a type context means "type variable".

I think this would be a nice improvement to the language that composes well with property delegates. It stands alone for, e.g. something like:

let d: Dictionary<_, Double> = ["one": 1, "pi": 3.14159]   // infer the Key type to be String

@DaveZ

How likely is it that the reference storage types could be implemented (or ever be implementable) as property delegates?

I don't know that we could intercept the reference-counting operations well enough, but it's possible that we could pull some Builtin tricks in the standard library to do this.

Doug

[hartbit (David Hart)]

@Douglas_Gregor

If there is no meaningful API for users of a delegate other than value there is no reason to have $foo available at all. One direction I suggested we might consider is not even synthesizing a $foo backing storage if the delegate does not implement delegateValue . This would require delegates to opt-in to having a visible backing storage. The flaw with this design, as with the proposed design of delegateValue , is that it breaks out-of-line initialization.

Alternatively, one could make the $foo private in such cases. That gives you direct initialization and hides it from the outside world.

I haven't had time to write a full review for this proposal but I'm fully behind it. I'd just like to point out that for certain use cases, it's very important for the backing storage to be synthesised as a normal property. It allows it to be discoverable at runtime through Mirror, greatly improving its usefulness. For example, the snippet I posted to Twitter depends on it being accessible.

[griotspeak (TJ Usiyan)]

Is there any chance that these could be used on function arguments? @CalledOnlyOnce could be implemented this way. Compile time warnings would be better but…

[DaveZ (David Zarzycki)]

@Douglas_Gregor

DaveZ:

How likely is it that the reference storage types could be implemented (or ever be implementable) as property delegates?

I don't know that we could intercept the reference-counting operations well enough, but it's possible that we could pull some Builtin tricks in the standard library to do this.

Ya, the reference storage type question was mostly to probe the potential of the proposal and possible future directions for the reference storage types. Thanks.

[anandabits (Matthew Johnson)]

@Douglas_Gregor

Just to be clear, does something like the private(storage) future direction meet your criteria here, or do you feel that there is a deeper problem with the design that needs to be addressed?

That direction was my suggestion prior to the introduction of delegateValue. It solved the access control issue fine in that design. It’s less clear to me in the delegateValue design because we haven’t identified a solution for out-of-line initialization when the type of delegateValue is different than the type of the property delegate itself.

If we find a solution for out-of-line initialization it might or might not be sufficient. For example, we could say $$foo is used for out-of-line initialization only and is therefore always private and possibly not even available outside of initialization. If we do that then a single additional modifier for $foo would be sufficient but parameterizing it with storage might be misleading since the modified value isn’t always the storage. Unfortunately delegate would be just as misleading. We would need to use the rather verbose delegateValue in order to be 100% clear.

@Douglas_Gregor

I think we can provide a more natural CopyOnWrite wrapper using generalized accessors , since modify is able to do the uniqueness check and efficiently yield the unique value. I've been meaning to try it with the toolchain, because _modify is in the compiler already. Should the property delegates proposal examples depend on this experimental feature? I guess it could.

Good to hear that there is a better option. I don’t think the proposal needs to depend on an experimental feature. What I am looking for is motivation for delegateValue and I don’t see a compelling motivation in the examples in the proposal. I think it’s an interesting feature but it introduces complexity and edge cases into the design. It should be well motivated.

@Douglas_Gregor

Huh, I thought I had addressed this. The intent is that Box is for creating storage, and Ref is about referring to something that's either directly in a Box , formed through some other get / set pair (say, a computed value), or derived via key path from another Ref thing.

You did reply to me and I let it drop in the discussion thread. But as I continue to think about it I really don’t see how directly exposing the box makes the design better. As far as I can tell, unless there is a reason users need to access a Box directly there is no reason for it to be part of the user-facing API.

Since your design hides the Box altogether it isn’t at all clear to me what advantage there is for users to have to know to use @Box for a delegate that is directly initialized with a value instead of just using @Ref everywhere. It could be that this is just a difference of opinion. But if that is the case it is not a strong motivating example for delegateValue. A stronger motivating example would provide a use case that is unambiguously better off with the use of the feature.

Please don’t misunderstand: I’m not trying to argue against the feature in principle. I just think we need to understand what the use cases are better than we do in order to be confident that we have the right design. It introduces edge cases I’m not entirely happy with. This makes me wonder if it is the right approach or if there might be alternatives that come to mind when we better understand the problems we’re trying to solve with it.

@Douglas_Gregor

I liked your idea of doing Definite Initialization for the $ property based on init(initialDelegateValue:) . Haven't had time to implement it yet, but it seems like a reasonable addition here.

Even if it were implemented, this solution still only covers some additional use cases. It would not provide out-of-line initialization when the delegateValue is not of type Self - as with @Box for example.

@Douglas_Gregor

I'm fairly strongly against this direction, because delegate types are normal types. The "weirdness" should be in accessing the backing storage of the property that has a delegate; once you have that value, it's just a normal value of the delegate type.

That’s fair. However, it’s worth pointing out that the introduction of delegateValue introduced its own kind of “weirdness” (which is probably why it wasn’t clear to me on the first reading). A property delegate is no longer a straightforward backing storage property of the kind we can write manually today. It is able to provide a stand-in for itself.

@Douglas_Gregor

FWIW, autocomplete can support this design, too, by inserting the '$' earlier. There's some precedent here with insertion of ? in someOptional.<completion shows members of the wrapped type>.

Good to know!

@Douglas_Gregor

It is smart enough to do that through init(initialValue:) . As mentioned above, it could do the same for an init(initialDelegateValue:) .

Right, but that still leaves us with delegates that don’t support out-of-line initialization.

One final question: what prompted you to add delegateValue to the second draft of the proposal? There must have been a reason but I haven’t heard it stated clearly. Why was this feature selected for implementation now instead of making it a possible future direction (and instead of designing and implementing one of the other future directions that have more clearly understood use cases)?

[s-k (Stephen Kockentiedt)]

@John_McCall

What is your evaluation of the proposal?

A huge +1 from me! I like the @ syntax. I like the $ syntax. It will enable many patterns we can't yet imagine. I think the feature is easy to understand as it all simply comes down to plain Swift types, which we all understand.

I also hope follow-on proposals for access control and access to self and the property's keyPath will follow soon.

@John_McCall

Is the problem being addressed significant enough to warrant a change to Swift?

Yes, absolutely. It will remove so much boilerplate code as we will learn to put this feature to good use.

@John_McCall

Does this proposal fit well with the feel and direction of Swift?

Yes, it does. In my opinion, this is the first proposal that enables macro-like functionality in Swift. So, naturally, some new syntax like @Lazy and $property has to be introduced. This may feel unfamiliar and therefore not 'swifty' to some but I absolutely think this is the right direction.

@John_McCall

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

I haven't used a feature like this in another language.

@John_McCall

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

I have followed both pitch threads pretty closely and have read multiple versions of the proposal.

[Douglas_Gregor]

@griotspeak

Is there any chance that these could be used on function arguments? @CalledOnlyOnce could be implemented this way.

We could extend the model to function parameters, but we have to decide what the caller and ABI do. Does the caller always provide a value of the delegate type, or do we go through the init(initialValue:) when it’s available? Is the ABI in terms of the delegate type or the stated parameter type?

Doug

[anandabits (Matthew Johnson)]

@Douglas_Gregor

We could extend the model to function parameters, but we have to decide what the caller and ABI do. Does the caller always provide a value of the delegate type, or do we go through the init(initialValue:) when it’s available? Is the ABI in terms of the delegate type or the stated parameter type?

My initial reaction is that the caller should always provide a value of the wrapped type and you go through init(initialValue:). The question in my mind is whether init(initialValue) runs on the caller side (ABI in terms of the delegate) or is synthesized at the top of the function body (ABI in terms of the initial value). Maybe I’m just stating the same question in a different way...

Is there an advantage of leaving ABI in terms of the wrapped type? It seems like this would allow the delegate to be changed without breaking ABI - it becomes an implementation detail. Callers don’t have a way to interact directly with the delegate so there probably isn’t an advantage of exposing the delegate type in the ABI, is there?

[Douglas_Gregor]

@anandabits

Is there an advantage of leaving ABI in terms of the wrapped type? It seems like this would allow the delegate to be changed without breaking ABI - it becomes an implementation detail. Callers don’t have a way to interact directly with the delegate so there probably isn’t an advantage of exposing the delegate type in the ABI, is there?

Probably not. Delegate types aren't part of the public interface or ABI for properties.

inout becomes really interesting here, although presumably it's fine... we put the value into the delegate type, then write it back at the end.

Don't forget that there are delegate types without an init(initialValue:); those would presumably have to pass via the delegate type.

Doug

[Douglas_Gregor]

@anandabits

Since your design hides the Box altogether it isn’t at all clear to me what advantage there is for users to have to know to use @Box for a delegate that is directly initialized with a value instead of just using @Ref everywhere. It could be that this is just a difference of opinion. But if that is the case it is not a strong motivating example for delegateValue . A stronger motivating example would provide a use case that is unambiguously better off with the use of the feature.

There are conceptually three things going on with

@Box var foo: Int

(1) the actual storage for the value (a Box), (2) the value as you see it (produced by value, accessed by foo), and (3) an abstract reference to a value (produced by delegateValue, accessed by $foo). There could be different storage strategies for (1) with a common abstract-referencing scheme via (3). The ability to use key path member lookup on (3) is really nice from this abstract-reference approach.

@anandabits

One final question: what prompted you to add delegateValue to the second draft of the proposal? There must have been a reason but I haven’t heard it stated clearly. Why was this feature selected for implementation now instead of making it a possible future direction (and instead of designing and implementing one of the other future directions that have more clearly understood use cases)?

Eh, sorry. I really wanted to get something out for discussion that was implemented enough for people to poke at it, and I overestimated how much time it would take to get this bit into shape (and underestimated how much heartburn it would cause as I evolved the proposal). I find the Ref / Box thing very interesting because it allows us abstract away the storage completely, and it was the last bit of "control" that I felt like delegate type authors needed to craft their APIs. For example, you could literally have an empty storage type and use Mirror (as @hartbit notes) to figure out what properties were declared.

@anandabits

Right, but that still leaves us with delegates that don’t support out-of-line initialization.

Personally, I'm happy with the private $$foo approach here. We're way out in the margins where you don't want direct initialization and have a delegateValue.

Doug

[anandabits (Matthew Johnson)]

@Douglas_Gregor

Don't forget that there are delegate types without an init(initialValue:) ; those would presumably have to pass via the delegate type.

Or maybe just invalid as parameter delegates. What is the value of using a delegate if callers have to provide a value of the delegate type? All a parameter delegate would do is immediately wrap an argument, prevent the body of the function from using the direct argument value, and allow the implementation to avoid saying .value manually. Delegates that don’t support init(initialValue:) seem of dubious value in the parameter context.

[zwaldowski (Zachary Waldowski)]

@Douglas_Gregor

I consider this a tooling problem, where Cmd-Click on an attribute should give you documentation about what it does.

The evolution processes of other languages and projects require some attempt at documentation and learnability — even just one-liner headerdocs. Apple's own API review processes do as well. I have to feel like creating a one-off construct that can't have documentation added to it according to current language rules and punting on how that gets settled would be frowned upon in those communities.

The Swift community shouldn't have to beg to get bare minimum documentation.

If the proposal were accepted as-is, and continued to spell its main feature addition as a mystery meat attribute, the UX for a developer who hasn't already read this proposal looks pretty unpleasant. How do I know this feature exists in the first place? Maybe I'll monkey-see-monkey-do everything based on @Lazy, assuming I find it in the first place. How would I ever know about the different forms of init(initialValue:), or whatever we come to do about subscripting with self?

[anandabits (Matthew Johnson)]

@Douglas_Gregor

(1) the actual storage for the value (a Box ), (2) the value as you see it (produced by value , accessed by foo ), and (3) an abstract reference to a value (produced by delegateValue , accessed by $foo ). There could be different storage strategies for (1) with a common abstract-referencing scheme via (3). The ability to use key path member lookup on (3) is really nice from this abstract-reference approach.

Yeah, I can see the moving parts. It just isn't clear to me what benefit users get from (1) being a visible part of the API surface. Can you elaborate on what the different storage strategies might be and why users would want control over that? It might help motivate this capability better.

@Douglas_Gregor

I find the Ref / Box thing very interesting because it allows us abstract away the storage completely, and it was the last bit of "control" that I felt like delegate type authors needed to craft their APIs.

I find the feature and the ability to abstract away storage interesting. And I don't doubt that it will be useful in some way. I just feel like it might be good to understand more concrete use cases in hand before committing to a design. As with other features, there is no harm in making it a future direction if we have uncertainty about this part of the proposal.

@Douglas_Gregor

For example, you could literally have an empty storage type and use Mirror (as @hartbit notes) to figure out what properties were declared.

Is there a post by @hartbit that goes into more detail on this? This sounds interesting but the details are pretty murky (for me anyway).

@Douglas_Gregor

Personally, I'm happy with the private $$foo approach here. We're way out in the margins where you don't want direct initialization and have a delegateValue .

So $$foo would always be synthesized as private for properties with a delegate that implements delegateValue? Or would this identifier be limited to initialization only?

This approach certainly plugs the hole in capabilities. Maybe it's the best solution. I just don't feel confident in that yet.

[AlexanderM (Alexander Momchilov)]

Boy, I'm not a really fan of how the @attribute syntax stuck. It started as implementation details/hacks, and now it's creeping into ever more domains (dynamic calculability, inalienability, property delegation, and more that I don't even know about, I can't keep up).

Before we embark this property delegate route (which I'm really excited about!), I think we need to take a second and define and clean up the details of attributes.

1. How are they spelled?
2. How are declarations with multiple attributes spelled?
3. How do multiple declarations compose?
4. Can attributes be de-magic-ified, and exposed as simple protocols (perhaps with some intrinsic magic types where necessary, but otherwise generally available for end user user), Rust/Java style?
5. What runtime capabilities do they offer?
6. What "hooks" do they offer, that users can plug into?

3 is seems particularly important to me. What if I want a lazy atomic, a thread local delayed mutable, a (delayed) immutable IBOutlet, etc.?

[Douglas_Gregor]

Your hyperbole detracts from your message, but I'll nonetheless try to answer.

@zwaldowski

The evolution processes of other languages and projects require some attempt at documentation and learnability — even just one-liner headerdocs. Apple's own API review processes do as well. I have to feel like creating a one-off construct that can't have documentation added to it according to current language rules and punting on how that gets settled would be frowned upon in those communities.

The Swift community shouldn't have to beg to get bare minimum documentation.

Proposals serve as the documentation while we're going through the evolution process. Some time after acceptance, @propertyDelegate will be documented in the TSPL with other attributes. That will be augmented by blog posts elsewhere, more fleshed-out examples, and wider adoption of the feature that will bring more awareness. This is how I learn about features in any language I poke around with. If you find this approach inadequate, start another thread with constructive ideas to improve the situation.

@zwaldowski

If the proposal were accepted as-is, and continued to spell its main feature addition as a mystery meat attribute, the UX for a developer who hasn't already read this proposal looks pretty unpleasant. How do I know this feature exists in the first place? Maybe I'll monkey-see-monkey-do everything based on @Lazy , assuming I find it in the first place. How would I ever know about the different forms of init(initialValue:) , or whatever we come to do about subscripting with self ?

Discoverability is straightforward:

• Code completion after @ will show you the @propertyDelegate attribute. A web search will find relevant documentation, blog posts, examples, etc.
• Code completion after @ will show you property delegate types you can choose from. Try one and see what it does. Their definitions will be clearly marked with @propertyDelegate; web search that to find more information.
• QuickHelp and Jump to Definition will work on any use of property delegates you see in your code. The documentation comments will show you how to use that specific property delegate.

This really isn't different from any other part of the language, so either you have a wildly different understanding of this feature's obvious tooling support than I do, or you have general complaints about Swift's documentation and tooling that belong in a separate thread.

Doug