SE-0253: Static callables

[Ben_Cohen]

@Chris_Lattner3

Ben, I don't understand your concern about closures.

Yeah, sorry, I left that statement dangling.

My concern is that if we adopt this proposal, we will have two similar but different concepts: callable structs with stored properties, and closures that capture variables. I'd like to see some discussion of this as part of the proposal, including considering the possibility of unifying them, even if it's to conclude that they don't need to be related and that having two similar but different things is OK.

[anandabits (Matthew Johnson)]

I think you have me convinced that either the call declaration or the call func syntax @Chris_Lattner3 just presented would be the best way to go (for reasons described below). I don’t have a strong preference between these, beyond wanting to choose the option that best supports future enhancements and evolution of the language.

@dan-zheng

A more serious problem is: naming call-syntax delegate methods becomes very tricky. Consider this example:

struct MachineLearningModel {
    var weight: Float
    var bias: Float
    // Since the method is non-mutating, it should be named `applied(to:)`,
    // not the imperative `apply`.
    @callable func applied(to input: Float) -> Float { ... }
}

let model = MachineLearningModel(...)
let input = ...

// Direct method references now have proper naming.
model.applied(to: input)
// But call-syntax has an awkward label! This is killer.
model(to: input)

// The name `func applied(_ input: Vector<Float>)` fixes the call-syntax,
// but makes direct references un-Swifty:
model.applied(input)

// `monoid.combine(_:)` violates Swift naming guidelines.
// Same problem would apply to `monoid.combined(with:)`.

AFAICT, this naming difficulty exists for any approach involving call-syntax delegate methods with custom names.

This is indeed troublesome and could lead to some awkward call-site syntax. If we also supported @callable func _ it would be possible to leave @callable off of the original method and write a forwarding @callable wrapper that fixes up the argument labels in an intentional manner. But that defeats the benefit in syntax of not having to write a forwarding wrapper. Further, some programmers may not think carefully before adding @callable and would therefore end up with an awkward API.

@ebg

This seems pretty minor and solvable. How about a callable automatically removes the parameter name?

@callable func applied(to input: Float) -> Float { ... }

The compiler gives you:
model.applied(to: input)
model(input)

This is not a viable direction. The compiler would not be able to distinguish between intentional labels and unintentional labels. It would have to remove them all. We definitely do not want callable syntax to be unable to use argument labels.

[anandabits (Matthew Johnson)]

@Ben_Cohen

My concern is that if we adopt this proposal, we will have two similar but different concepts: callable structs with stored properties, and closures that capture variables. I'd like to see some discussion of this as part of the proposal, including considering the possibility of unifying them, even if it's to conclude that they don't need to be related and that having two similar but different things is OK.

An example of discussion in this direction: @Joe_Groff has suggested that we could eventually have a way to constrain generic arguments to be @callable. With that in place we might consider updating many APIs that currently take functions to be generic over a callable type with a specified signature. This is related to implicit conversion to function types, but may be easier for the compiler to optimize than APIs that take a closure, even when they’re used with a callable type via implicit conversion.

[Joe_Groff]

Yeah, the natural unification I see (which to be clear, is not a reason to hold up this proposal in particular, which IMO seems fine and useful on its own) would be to make it so that today's function types become existential types for functions as generic constraints, similar to how the Fn* traits work in Rust. Closures then become synthesized types with a call operator to conform to the constraint, and types with explicit call operators can also conform.

[anandabits (Matthew Johnson)]

@Joe_Groff

Yeah, the natural unification I see (which to be clear, is not a reason to hold up this proposal in particular, which IMO seems fine and useful on its own) would be to make it so that today's function types become existential types for functions as generic constraints, similar to how the Fn* traits work in Rust.

I was thinking the syntax might piggyback on the opaque parameter syntax you described elsewhere, something like this:

extension Optional {
   func map<T>(_ transform: some (Wrapped) throws -> T) rethrows -> T? { ... }
}

And like you said, the existing function types would be the existential types for these callable constraints.

[Ben_Cohen]

@Joe_Groff

(which to be clear, is not a reason to hold up this proposal in particular, which IMO seems fine and useful on its own)

I take the opposite view, which is that the low language-complexity-to-win ratio on this proposal as it stands suggests we should take a step back and make sure a fuller proposal addresses these point as well, instead of accepting this small part and hoping everything works out OK for future directions.

[hartbit (David Hart)]

@dan-zheng

But I strongly feel that we should evaluate "the best syntax for call-syntax delegate methods" at face-value: focusing on which syntax best fits into Swift and temporarily putting aside implementation costs. In particular, I feel more attention should be given to spelling and Swift naming guidelines.

I agree with you, but I still feel like the type attribute is the best option, despite any implementation cost. Here are a few additional points:

Concerning the type attribute

We should not forget that this solution has some important advantages:

• It doesn't introduce any new syntax.
• It makes a parallel with @dynamicCallable by using a similarly named type-attribute to enable it.

Both those advantages make the feature that much easier to teach and for people to understand. The solution in the proposal or the named call-syntax introduce new syntax which has a higher chance of confusing users.

@dan-zheng

• The name for call-syntax delegate methods (e.g. func call ) is not first-class in the language, while their call site syntax is.

That's also the case with @dynamicCallable.

@dan-zheng

Subscripts have existed for a long time in Swift. Perhaps this is not an apt comparison, but imagine subscripts didn’t exist, and there’s a new proposal to introduce them. I would support a subscript declaration over alternatives like @subscript func(...) because it's more natural.

I don't follow this reasoning. First of all, subscripts feel more different from functions (or properties for that matter) than static callable to me. Both can be seen as syntactic sugar for function calls, but subscripts is syntactic sugar for two functions with very specific arguments, while static callable is syntactic sugar for any function.

[David_Sweeris]

@Ben_Cohen

@Joe_Groff

(which to be clear, is not a reason to hold up this proposal in particular, which IMO seems fine and useful on its own)

I take the opposite view, which is that the low language-complexity-to-win ratio on this proposal as it stands suggests we should take a step back and make sure a fuller proposal addresses these point as well, instead of accepting this small part and hoping everything works out OK for future directions.

What’s the deadline for this feature? Given that Swift 5 is still fairly new, I’d guess that there’s time to explore the idea more fully.

[anandabits (Matthew Johnson)]

@hartbit

We should not forget that this solution has some important advantages:

• It doesn't introduce any new syntax.
• It makes a parallel with @dynamicCallable by using a similarly named type-attribute to enable it.

Both those advantages make the feature that much easier to teach and for people to understand. The solution in the proposal or the named call-syntax introduce new syntax which has a higher chance of confusing users.

Of course this syntax suffers from the same argument label issue described uptrhead as any syntax based on ordinary methods will.

Beyond that, how do you envision a type attribute supporting static static callable (i.e. callable metatypes). Repeating what I said in the discussion thread:

The type-level attribute is the only option that would require additional syntax to support static callable. The attribute might be @staticCallable , in which case we would have a confusing mix of @callable , @dynamicCallable and @staticCallable . I suspect this would not be desirable and result in metatypes just not becoming callable.

[hartbit (David Hart)]

@anandabits

Beyond that, how do you envision a type attribute supporting static static callable (i.e. callable metatypes).

I saw you mention that previously but I’m not sure how ‘static’ static callables would be used? Aren’t they ambiguous with a call to the initializer when used the literal meta-type?

Otherwise, I do agree with one argument of yours: the fact that a function level attribute does allow naming the function as makes most sense for the API in case you do want to call it explicitly.

[dan-zheng]

@hartbit

Concerning the type attribute

We should not forget that this solution has some important advantages:

• It doesn't introduce any new syntax.
• It makes a parallel with @dynamicCallable by using a similarly named type-attribute to enable it.

Both those advantages make the feature that much easier to teach and for people to understand. The solution in the proposal or the named call-syntax introduce new syntax which has a higher chance of confusing users.

I feel that static callable behavior and @dynamicCallable are essentially orthogonal language features: the use cases and generality of these features are very different.

In particular, @dynamicCallable serves a niche purpose: it was designed specifically for language interoperability, and "rewriting argument labels as string keys" is a highly peculiar behavior. This nicheness justifies the explicit @dynamicCallable type attribute and the informal dynamicallyCall method requirements: they are designed to be unmistakable.

By comparison, static callable behavior is more general - multiple use cases are mentioned in the proposal and in the pitch thread. Thus, I don't think the @dynamicCallable design should serve as a model for the static callable design.

[anandabits (Matthew Johnson)]

@hartbit

I saw you mention that previously but I’m not sure how ‘static’ static callables would be used? Aren’t they ambiguous with a call to the initializer when used the literal meta-type?

‘They wouldn’t cause ambiguity any more than initializer overloads would. In fact as I have said previously, I think initializers are really a special case of metatype callable which has specific implementation requirements. There is no good reason to limit metatypes to only be callable via initializers.

The advantage of static callable is that the return type does not need to be Self. It is probably the case that when the return type is Self an initializer would be used, but there are other reasons you might want a metatype to be callable.

[dan-zheng]

I don't quite agree that a @callable type attribute and informally required func call methods are easier to teach and understand than a first-class call declaration (or call func(...) methods).

---

@hartbit

Subscripts have existed for a long time in Swift. Perhaps this is not an apt comparison, but imagine subscripts didn’t exist, and there’s a new proposal to introduce them. I would support a subscript declaration over alternatives like @subscript func(...) because it's more natural.

I don't follow this reasoning. First of all, subscripts feel more different from functions (or properties for that matter) than static callable to me. Both can be seen as syntactic sugar for function calls, but subscripts is syntactic sugar for two functions with very specific arguments, while static callable is syntactic sugar for any function.

I see your perspective. My point was that "callable behavior" is closer to the first-class behavior of "subscriptability" than the niche behavior of @dynamicCallable. Thus, callable behavior is better modeled using a first-class declaration kind (or unnamed instance method, which is relatively first-class), rather than a niche type attribute and informal required methods.

[beccadax (Becca Royal-Gordon)]

@dan-zheng

Subscripts have existed for a long time in Swift. Perhaps this is not an apt comparison, but imagine subscripts didn’t exist, and there’s a new proposal to introduce them. I would support a subscript declaration over alternatives like @subscript func(...) because it's more natural.

I don't think this is an apt comparison because functions flat-out cannot express the semantics necessary for a subscript. If they could—for instance, if we had inout return values which caused a function body to have accessors—and you were proposing a new subscript declaration today, I would just as strongly argue for func subscript(…) -> inout Foo instead of subscript(…) -> Foo.

@dan-zheng

As stated in the proposal:

We feel this approach is not ideal because:

• A marker type attribute is not particularly meaningful. The call-syntax delegate methods of a type are what make values of that type callable - a type attribute means nothing by itself. There's an unfortunate edge case that must be explicitly handled: if a @staticCallable type defines no call-syntax delegate methods, an error must be produced.
• The name for call-syntax delegate methods (e.g. func call ) is not first-class in the language, while their call site syntax is.

I read this argument in the proposal and was not convinced.

First, let me concede this: Both of these points have some merit. Having an attribute does introduce an edge case where the type is marked callable but it doesn't declare any call(…) methods. (This might make sense if you intend to declare call(…) methods in constrained extensions, but whatever.) And the declaration site would read better with a dedicated declaration kind for this specific use case.

(I also don't think a call declaration would be confusing. People will probably be able to guess that something called "call" which looks kind of like a function definition makes the type callable, and even if they don't, Google will tell them what it is.)

But when I put these two benefits on one side of the scale, and put the cost of "the implementation is ten times more complicated" on the other side of the scale, I don't think the balance tips towards the benefits. These are small upsides compared to the bugs that will inevitably be introduced to the compiler, the resources that will be spent to update tooling, and the need for future language designs to consider and implement interactions with this new declaration. Swift is already a pretty big language; when we make it even bigger, we should take efforts to grow it no more than necessary.

If this was something people were going to write often, the calculus might be different, but I don't think anyone has suggested that declaring a call will be anything but a niche feature. If this was a first step down a road to something better, I think we could justify it, but I haven't seen that roadmap. As it stands, I just don't see benefits from the call declaration commensurate with the costs.

[dan-zheng]

@beccadax

But when I put these two benefits on one side of the scale, and put the cost of "the implementation is ten times more complicated" on the other side of the scale, I don't think the balance tips towards the benefits. These are small upsides compared to the bugs that will inevitably be introduced to the compiler, the resources that will be spent to update tooling, and the need for future language designs to consider and implement interactions with this new declaration. Swift is already a pretty big language; when we make it even bigger, we should take efforts to grow it no more than necessary.

If this was something people were going to write often, the calculus might be different, but I don't think anyone has suggested that declaring a call will be anything but a niche feature. If this was a first step down a road to something better, I think we could justify it, but I haven't seen that roadmap. As it stands, I just don't see benefits from the call declaration commensurate with the costs.

Thanks for your perspective! Implementation cost is certainly a real, pragmatic concern. I can say that I'm interested in working with code owners and devoting significant effort towards a smooth call declaration integration.

I would like to mention again that further extensions of callable behavior in Swift can justify a more first-class call, so there are future-facing considerations:

@dan-zheng

Similarly, I feel call declarations also fit naturally in Swift. With time, call declarations could be extended with more first-class functionality and further integrated into the language:

• Values defining call declarations can be converted to function-typed values.
• Closures can be defined as nominal types with a call declaration, with bound variables represented as stored properties.

@rxwei and I are definitely willing to flesh out these directions, if (more) people feel in-depth future designs are important for the acceptance of this initial proposal.

---

@beccadax: I noticed you specifically used the "type attribute alternative" in your comparison:

@beccadax

Having an attribute does introduce an edge case where the type is marked callable but it doesn't declare any call(…) methods. [...]

I wonder, do you favor the type attribute approach over alternatives (like unnamed instance methods) that require some compiler changes, but much less changes than needed for adding a new declaration?

[Avi]

@dan-zheng

@rxwei and I are definitely willing to flesh out these directions, if (more) people feel in-depth future designs are important for the acceptance of this initial proposal.

I will chime in here as a user of Swift. I feel that without implicit conversion to function types, this feature will remain niche amongst Swift developers. We'd be using callable types created by others, but the boilerplate necessary to use them in common situations would dissuade creation of new callable types in our own code.

It is common in C++ to use such types with collection classes from the stdlib, as filtering and sorting predicates, for example. Do you foresee adding overloads for map(), sorted(by:), etc?

[jawbroken]

@dan-zheng

Here is my opinion on some alternate syntaxes, assuming the following goals:
…

• We want to support direct references to call-syntax delegate methods to avoid the necessity of thunks like { foo($0) } .

I'm not convinced this should be one of the main goals. e.g. I don't think there's a way to directly reference subscripts (for unrelated good reasons), which is one of the closest analogues to the current pitch. Dropping this goal would solve the issue with naming/argument labels that you bring up, and the lack of direct referencing can be relatively easily worked around using thunks or by writing a forwarding method (or, hypothetically, some sort of future implicit conversion to function types). This would also make some of the alternative implementations more attractive here (e.g. unnamed instance methods).

If the current approach in the proposal is chosen, I think all these things with similar declarations (init, subscript, call) would preferably behave similarly. This raises the same problem that was bought up in the pitch thread though, i.e. currently you can't name an instance method subscript or init, but call is proposed to behave differently because it's an attractive word for an unrelated instance method and this causes source compatibility concerns. This suggests to me that, if the current approach is chosen, a different keyword to call should probably be used so conflicts are less likely. Or, alternatively, the restriction could be lifted on init and subscript, but I don't foresee a lot of appetite for that because it would cause trouble when people accidentally use func init instead of init, etc.

[Jon_Hull]

@Chris_Lattner3

Also, it seems that there is a ton of confusion on this thread - without responding point by point, would people understand and feel more comfortable about this if the operator was spelled with a decl modifier as:

call func() {... }

This eliminates the declaration side grammar issues (because declaration modifiers "just work" in the grammar, without requiring a keyword), makes it clear that these can appear anywhere an instance func is, etc.

I definitely feel better about that direction, but my main concern is how to refer to it as a function. The myCallable.call feels a bit weird because we are still stealing the ability to have a function named call. Making the declaration func call() fixes that, but then we still have this magic function name call that we have to know is special.

I still really think func _ () is the best I have seen so far, but I would be certainly be ok with call func(). You still have to learn the concept somehow, but you won't accidentally trip over the magic name. In both of these cases, we have the question of how to reference the function without calling it.

As I said above, I don't actually think .call solves this problem for any of the syntax choices, because:

• It overloads .call (what if you have func call()?)
• You have to know the magic name
• It is special, but it feels like a normal member reference
• It can't disambiguate between different call signatures
• It breaks the metaphor of the type itself being the callable thing for the caller

We should strive for something which doesn't break the metaphor that our type acts as a function, and allows us to disambiguate between different signatures when needed. In that respect, I really do believe that as ()->() is the most natural syntax for this. It just says: "treat this type like this particular function signature", and that is totally in line with the meaning of a callable. I really think anyone (fluent in Swift) coming across it in code will be able to grok the meaning without having to look anything up. Compare that to .call which most people will assume at first glance is a member reference.

I really believe that as ()->() still needs to work, even if we choose another syntax like .call. At the very least, we will need it to disambiguate.

[Jon_Hull]

@anandabits

An example of discussion in this direction: @Joe_Groff has suggested that we could eventually have a way to constrain generic arguments to be @callable . With that in place we might consider updating many APIs that currently take functions to be generic over a callable type with a specified signature. This is related to implicit conversion to function types, but may be easier for the compiler to optimize than APIs that take a closure, even when they’re used with a callable type via implicit conversion.

I think (in the long term) the most natural way to do this is to allow callable functions to be part of a protocol:

protocol MyProtocol {
    func _ () -> Int
    func anotherFunction()
}

struct<T:MyProtocol> {
    func getAnIntFromT(_ t:T) -> Int {
        return t() //I know I can call this because it is defined in the protocol
    }
}

[Jon_Hull]

@anandabits

@hartbit

I saw you mention that previously but I’m not sure how ‘static’ static callables would be used? Aren’t they ambiguous with a call to the initializer when used the literal meta-type?

‘They wouldn’t cause ambiguity any more than initializer overloads would. In fact as I have said previously, I think initializers are really a special case of metatype callable which has specific implementation requirements. There is no good reason to limit metatypes to only be callable via initializers.

The advantage of static callable is that the return type does not need to be Self. It is probably the case that when the return type is Self an initializer would be used, but there are other reasons you might want a metatype to be callable.

To add to that, a popular use case would be as a factory initializer (i.e. potentially returning a different type)