SE-0253: Static callables

[DeFrenZ (Davide De Franceschi)]

@Joe_Groff

Yeah, the natural unification I see (which to be clear, is not a reason to hold up this proposal in particular, which IMO seems fine and useful on its own) would be to make it so that today's function types become existential types for functions as generic constraints, similar to how the Fn* traits work in Rust. Closures then become synthesized types with a call operator to conform to the constraint, and types with explicit call operators can also conform.

This would be so powerful on so many levels... I've found myself in the past to shave off syntax from some utility/structure until getting to the point of "everything is a function anyway " but unable to add sugar nor introspection if I were to actually use functions.

A few examples I remember thinking about in this way are (using one of the proposed syntax)

typealias Result<T> = () throws -> T
extension Result {
  func map <U> (_ transform: (T) throws -> U) rethrows -> Result<U> {
    let transformedValue = try transform(self())
    return { transformedValue }
  }
}

typealias Continuation<T> = (T -> Void) -> Void
extension Continuation {
  func then(_ block: (T) -> Void) {
    self(block)
  }
  func map <U> (_ transform: (T) -> U) -> Continuation<U> {
    return { block in
      self { value in
        block(transform(value)) 
      }
    }
  }
}

[michelf (Michel Fortin)]

What is your evaluation of the proposal?

I do feel like we're loosing semantic clarity about what is a function and I don't find the example very convincing.

Is the problem being addressed significant enough to warrant a change to Swift?

Personally, I don't feel the need for this. I'm quite happy making closure and passing around function types. I'm not too fond of the idea of "values representing functions" that aren't function types. I like to have clear lines between some concepts instead of "everything can act like anything".

I find it makes little sense to be able to "call" a Parser, or "call" a Model, or "call" a Polynomial. The goal of these examples seems entirely about saving some typing at the call site, but you can already bind the function you want to a variable:

let eval = Polynomial(coefficients: [2, 3, 4]).evaluated
print(eval(2)) // => 24

Perhaps if the syntax for declaring this wasn't called call I'd be more inclined to accept the non-call use cases. Something more operator-like such as func ()(param: Int) comes to mind, where () acts as an operator name like in func ==(a: Int, b: Int).

I also worry about this being later allowed on metatypes later and it becoming confusing with an initializer call. Although I don't think this is part of this proposal.

Does this proposal fit well with the feel and direction of Swift?

I'm a bit puzzled about how this interacts (or not) with SE-0111 which removed the type system significance of function argument labels in function types. It was hinted at the time that variables of a function type would eventually get "compound names", allowing the labels to become part of the variable's name.

There's no way I can see compound names work with static callables though. Argument names and overloading in call makes static callable types fundamentally different from a function type.

So I'm not too sure where this new feature would lead us to in term of consistency. It seems to me static callables are moving back to the old days where argument labels were part of the type. Should we revert some parts of SE-0111 to improve consistency?

Or perhaps we could make argument names non-significant in call (like they are for operators) which would allow callable types to have compound names. That would also require forbidding overloading for call or else the compound names would make no sense. That'd bring callable types closer to a function type.

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

C++ has operator() which is quite similar to this and allows one to write functors. I mostly stopped using functors in C++ as soon as lambdas appeared (C++ name for a closure). I still have to use them for some APIs that expects to be passed functors to be overloaded (or templated) by argument type like a visitor pattern made entirely static through templates (which wouldn't work in Swift).

D has opCall which to my knowledge is pretty much always used in conjunction with static to work around limitations of struct constructors.

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Read the proposal, most of the review thread, and gave it some thought.

[dan-zheng]

@Avi

I will chime in here as a user of Swift. I feel that without implicit conversion to function types, this feature will remain niche amongst Swift developers. We'd be using callable types created by others, but the boilerplate necessary to use them in common situations would dissuade creation of new callable types in our own code.

Thanks for your feedback! I realized this isn't stated clearly in the proposal, but I think implicit conversion (enabling array.map(foo) where foo's type defines a call member) is a natural direction for extending the current proposal.

However, I feel that the impacts of implicit conversion (on compiler performance and the entire type system) are unknown and require exploration - it's not a bygone conclusion that such a feature is definitely desirable. Here's a comment by @Joe_Groff on the "key path expressions as functions" proposal implementation:

Implicit conversions impact the entire type system, and we've set up an expectation that they hold for dynamic casts too, complicating the runtime implementation as well.

To start going down the rabbit hole: would a simple reference like foo resolve to the callable nominal type or resolve to a call member function type?

---

I think this initial proposal is fine as it stands. Exploring implicit conversion makes sense if this proposal is accepted and after details are fleshed out.

[dan-zheng]

@Avi

It is common in C++ to use such types with collection classes from the stdlib, as filtering and sorting predicates, for example. Do you foresee adding overloads for map() , sorted(by:) , etc?

Since types with call-syntax delegate methods are not generalized by a protocol (rationale here), I don't think it's possible to overload higher-order functions to accept such types in place of function arguments.

Usage with higher-order functions is supported via direct call member references for now (e.g. array.map(foo.call)), and potentially via function-type conversions in the future.

[dan-zheng]

@Jon_Hull

As I said above, I don't actually think .call solves this problem for any of the syntax choices, because:

• It overloads .call (what if you have func call() ?)
• You have to know the magic name
• It is special, but it feels like a normal member reference
• It can't disambiguate between different call signatures
• It breaks the metaphor of the type itself being the callable thing for the caller

I'm not quite convinced of these problems.
What is a use case where a type would define both func call and a call member?

I also wonder when/how often it's necessary to disambiguate overloaded call members via explicit casting. I don't particularly see a reason to store let x = foo.call in an intermediate variable. When applying foo(...) directly, overload resolution based on arguments' types should be sufficient (unless result types are overloaded).

---

@Jon_Hull

It just says: "treat this type like this particular function signature", and that is totally in line with the meaning of a callable. I really think anyone (fluent in Swift) coming across it in code will be able to grok the meaning without having to look anything up. Compare that to .call which most people will assume at first glance is a member reference.

I do generally agree with this sentiment of uniformly treating callables like functions.

Is your idea to support explicit casting to function types via as, and to not support implicit conversion? If I understood you correctly, that is an interesting direction and avoids implicit conversion complications.

---

@michelf

Personally, I don't feel the need for this. I'm quite happy making closure and passing around function types. I'm not too fond of the idea of "values representing functions" that aren't function types. I like to have clear lines between some concepts instead of "everything can act like anything".

I find it makes little sense to be able to "call" a Parser , or "call" a Model , or "call" a Polynomial .

Thanks for your perspective! I think @allevato made some great points in the pitch thread that are relevant to your response:

@allevato

I will grant that the facta that Swift has context-capturing first-class functions means that many of the uses of operator() from early C++ (until lambdas were introduced) can be duplicated that way instead, but there's still value in being able to treat state and application as a single unit . Today in Swift, you only have three options:

• You pass the state and call an explicit function, losing the notational benefits that you want

• You pass the state's member function, losing the ability to query or mutate the state itself because it's captured by the function and inaccessible otherwise

• You pass both, obfuscating the API

A call operation resolves this nicely for the problem domains that benefit from it.

Context-capturing functions and closures are great for certain use cases. But mutable state and application are often encapsulated as nominal types with instance methods. Call-syntax sugar for instance methods representing "function application" is valuable.

---

@michelf

I find it makes little sense to be able to "call" a Parser , or "call" a Model , or "call" a Polynomial . The goal of these examples seems entirely about saving some typing at the call site, but you can already bind the function you want to a variable:

let eval = Polynomial(coefficients: [2, 3, 4]).evaluated

print(eval(2)) // => 24

Instead of using the verb "call", I would think in terms of "application": applying a parser, machine learning model, or polynomial function. Application (of the primary, function-like behavior) of these constructs does make sense.

---

@michelf

I'm a bit puzzled about how this interacts (or not) with SE-0111 which removed the type system significance of function argument labels in function types. It was hinted at the time that variables of a function type would eventually get "compound names", allowing the labels to become part of the variable's name.

This proposal simply presents a syntactic sugar: foo(x: 1, 2, 3) is rewritten as an instance member application foo.call(x: 1, 2, 3). Such sugars are not related to the type system significance of function argument labels.

[jayton (Jens Ayton)]

@dan-zheng

I think this initial proposal is fine as it stands. Exploring implicit conversion makes sense if this proposal is accepted and after details are fleshed out.

I disagree. Without implicit conversion, or a way to explicitly abstract over both functions and callables, this feature is likely to become a frustrating trap. Users hoping to avoid verbosity by using callables will be frustrated when they can’t pass them to higher order functions. People writing functional libraries will be tempted to add callable support in common use cases, leading to extra boilerplate code and a fragmented landscape where callables and HOF sometimes work together and sometimes don’t.

Callables as proposed will get a reputation as a design mistake, and people will avoid them, even after attempts are made to rectify the situation. This will be especially bad if they come with the conceptual and implementation overhead of a special decltype.

[michelf (Michel Fortin)]

@dan-zheng

This proposal simply presents a syntactic sugar: foo(x: 1, 2, 3) is rewritten as an instance member application foo.call(x: 1, 2, 3) . Such sugars are not not related to the type system significance of function argument labels.

Exactly. As soon as argument labels enter the picture, it no longer behave like a function type. I'm not sure that's right.

[Avi]

I understand that implicit conversion could wait for a future proposal, but if it's not the desired end-goal, and remains merely a possibility, I reiterate that this will remain a niche feature, at best.

I refer to jayton's reply, as he expressed my sentiments precisely.

SE-0253: Static callables

I disagree. Without implicit conversion, or a way to explicitly abstract over both functions and callables, this feature is likely to become a frustrating trap. Users hoping to avoid verbosity by using callables will be frustrated when they can’t pass them to higher order functions. People writing functional libraries will be tempted to add callable support in common use cases, leading to extra boilerplate code and a fragmented landscape where callables and HOF sometimes work together and somet…

[anthonylatsis]

I would like to urge the authors and community to consider Chris' idea of implementing this as a modifier; it offers the combined advantages of an attribute and the proposed call(...) syntax while also having a balanced benefit to complexity ratio.

call func parse(...)

A modifier is a lot more flexible than an attribute that must be placed on the type's primary definition. It allows for making types callable retroactively and still mitigates all implementation difficulties many of us are concerned about. As @anandabits and the proposal mentions, call is not an equally suitable name for all domains, especially if one decides to use partial application. A modifier (an attribute as well) perfectly coexists with a method name. Speaking of future directions, a modifier can be placed on both static and instance methods, while it is unclear how a single attribute would work with regular callables, static ones and the combined case.

[anandabits (Matthew Johnson)]

@Jon_Hull

I think (in the long term) the most natural way to do this is to allow callable functions to be part of a protocol:

This is not a long-term thing. This will be supported immediately. But I think we also need support for callable constraints. We don’t want to have to create protocols CallableWithThisSpecificSignature and declare conformances in order to use callable in generic contexts where the alternative would be a closure (and the potential overhead associated with those).

[dan-zheng]

I'd like to emphasize some naming difficulties regarding "named call-syntax delegate methods", stated above. Do you have ideas for addressing these difficulties?

@dan-zheng

A more serious problem is: naming call-syntax delegate methods becomes very tricky. Consider this example:

struct MachineLearningModel {
    var weight: Float
    var bias: Float
    // Since the method is non-mutating, it should be named `applied(to:)`,
    // not the imperative `apply`.
    @callable func applied(to input: Float) -> Float { ... }
}

let model = MachineLearningModel(...)
let input = ...

// Direct method references now have proper naming.
model.applied(to: input)
// But call-syntax has an awkward label! This is killer.
model(to: input)

// The name `func applied(_ input: Vector<Float>)` fixes the call-syntax,
// but makes direct references un-Swifty:
model.applied(input)

// `monoid.combine(_:)` violates Swift naming guidelines.
// Same problem would apply to `monoid.combined(with:)`.

AFAICT, this naming difficulty exists for any approach involving call-syntax delegate methods with custom names.

This problem doesn't seem to for call-syntax delegate methods ending with ing:

// call func parsing(_ input: String), name is okay
let sexp = parser.parsing("(+ 1 2)")
let sexp = parser("(+ 1 2)")
// call func applying(_ input: Float), not a great name
let prediction = model.applying(2)
let prediction = model(2)

But does exist for methods ending in ed:

// call func parsed(from input: String), name is okay
let sexp = parser.parsed(from: "(+ 1 2)")
let sexp = parser(from: "(+ 1 2)")
// call func applied(to input: Float), name is okay
let prediction = model.applied(to: 2)
let prediction = model(to: 2)

I think ed names sometimes make more sense than ing names.

For example, "function applied to an input" makes sense, but "function applying an input" doesn't. It's more like "user is applying the function to an input".

[anthonylatsis]

@dan-zheng

I'd like to emphasize some naming difficulties regarding "named call-syntax delegate methods", stated above. Do you have ideas for addressing these difficulties?

Off the top of my head, you could name the method appliedTo(_ input:). I think differently, but I even remember someone from the team mentioning this style is more in line with the Guidelines, probably in one of the huge threads on bikeshedding stuff like any and all for Sequence. Regarding alternatives, I don't think anything in the direction of exceptionally allowing to omit argument labels is a reasonable approach.

[anandabits (Matthew Johnson)]

@dan-zheng

I think ed names sometimes make more sense than ing names.

For example, "function applied to an input" makes sense, but "function applying an input" doesn't. It's more like " user is applying the function to an input".

I agree.

@dan-zheng

I'd like to emphasize some naming difficulties regarding "named call-syntax delegate methods", stated above. Do you have ideas for addressing these difficulties?

If we go in this direction I think we need to support unnamed @callable methods and trust people to write an unnamed callable method that forwards to the base method with appropriate labels instead of making the base method itself callable.

I'm not sure how this would play out in practice. If we have support for opaque callable parameters and / or implicit conversion most use cases will be in generic and existential contexts where labels are already erased. As others have pointed out, without these features static callable is of much less utility. I'm not sure how often people will actually write code where they invoke a call on a concrete callable type. If it turns out to be rare to write invocations against concrete callable types people might not think or care much about what the labels on their callable are and the compiler won't encourage them.

The primary benefit of requiring an unnamed call declaration or method is that it forces the programmer to consider the appropriate labels for a call where the base name of the variable storing the callable could be anything. This is significantly different than the considerations that apply to naming ordinary methods. It is a reasonably strong argument in favor of the proposed syntax or some variation of it based on methods that are not allowed to have a base name.

[beccadax (Becca Royal-Gordon)]

@dan-zheng

wonder, do you favor the type attribute approach over alternatives (like unnamed instance methods) that require some compiler changes, but much less changes than needed for adding a new declaration?

If foo.call is going to refer to the call entry point, I think func call is better than any of the halfway-between syntaxes because you don't need to know anything new to understand what foo.call means.

If not, I'm fine with any solution that doesn't require a new Decl subclass or a disruptive change to the libSyntax grammar. For instance, I think func _(…) could probably be fit into the existing grammar (though I haven't tried it) and you could represent it as a FuncDecl with an empty base name.

@Joe_Groff

Yeah, the natural unification I see (which to be clear, is not a reason to hold up this proposal in particular, which IMO seems fine and useful on its own) would be to make it so that today's function types become existential types for functions as generic constraints, similar to how the Fn* traits work in Rust. Closures then become synthesized types with a call operator to conform to the constraint, and types with explicit call operators can also conform.

Are you imagining a design similar to C++ lambdas? Because that design (IMHO) composes poorly with C++ auto by making it infer a type that's specific to the exact closure, and I think that would end up happening in Swift variable declarations too unless we hacked around it somehow. var fn = { foo() }; if bar { fn = { baz() } } works today and needs to keep working in the future.

[Joe_Groff]

I agree we'd want to keep the existing behavior for closures; without thinking too deeply about it (and not wanting to start a whole new design discussion amid an active review for a different feature), maybe the anonymous lambda type would only get exposed in the context of a free generic type variable in a call.

[anandabits (Matthew Johnson)]

@Joe_Groff

maybe the anonymous lambda type would only get exposed in the context of a free generic type variable in a call.

Wouldn't this also work using opaque type syntax for local variables (so we don't lose the type info when we want it)?

let callable: some (Int, Int) -> Int = makeCallable()

[Joe_Groff]

If we want to keep exploring the "callable constraint" idea, maybe we should spin off a new design thread?

[rxwei (Richard Wei)]

Agreed. I believe that function type constraints can justify the first-class'ness of call-syntax delegate methods. It is clear to me that many people want to see a fully thought-out solution that covers function constraints and implicit conversion to functions types. Maybe a larger proposal can start from function type constraints and have call-syntax delegate methods be a required part of it.

[Michael_Ilseman]

• What is your evaluation of the proposal?

Big +1 to the idea, unsure of the new keyword vs attribute.

I don't see this as analogous to other cases where we have keywords: subscript and computed variables can have setters/accessors and so they're fundamentally different than methods. This seems to be more like a method and so an attribute feels appropriate. I don't have stronger reasoning beyond "feels" at this point.

This is a weakly held opinion and I fully trust the proposal authors to debate with @beccadax and @jrose and the core team to figure out the right approach.

• Is the problem being addressed significant enough to warrant a change to Swift?
• Does this proposal fit well with the feel and direction of Swift?

Yes on both counts, I view this as directly analogous to tuples vs structs. Tuples are very convenient for transient structures such a multiple return values. But as you want to use, store, and pass that same kind of structure in more places, it deserves a proper name by making it a struct. At that point, conformances etc., can be added. The "smell" to consider making this shift is when you start making a typealias for your tuples and multiple functions over them. You gain advantages of structs at the cost of lighter-weight syntax (but hopefully this will continue to be improved).

Closures (especially non-escaping) are very convenient ways to bundle transient functions-with-state to pass around for local operations like map and withFoo. But, as you want to use, store, and pass that same kind of function-with-state around in more places, it deserves a proper name by making it a callable struct. Again, conformances, etc., can be added and the "smell" comes when you want to store closures of a certain shape or use typealiases. Again, you gain power of structs at the cost of the lighter-weight closure syntax.

• If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

C++ has operator(), but I feel this proposal is more akin to nominal closures than C++.

• How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Quick reading.

[Jon_Hull]

@dan-zheng

I also wonder when/how often it's necessary to disambiguate overloaded call members via explicit casting. I don't particularly see a reason to store let x = foo.call in an intermediate variable. When applying foo(...) directly, overload resolution based on arguments' types should be sufficient (unless result types are overloaded).

It is one of those things that won't come up very often, but you really want a way to solve it when it happens. One time when it will break is when you have multiple call functions for a type and multiple overloads of the function which it is being passed to. Which one does it use?

I guess a work around could be:

let temp:()->() = myCallable.call
myOverloadedFunc(temp)

I still think that the following is more elegant though (and we don't burn .call):

myOverloadedFunc(myCallable as ()->())

To me the intent is much clearer to the reader here...

@dan-zheng

I'm not quite convinced of these problems.
What is a use case where a type would define both func call and a call member?

It would be admittedly rare, but why introduce sharp edges to the language when we can avoid it?

@dan-zheng

I do generally agree with this sentiment of uniformly treating callables like functions.

Is your idea to support explicit casting to function types via as , and to not support implicit conversion? If I understood you correctly, that is an interesting direction and avoids implicit conversion complications.

Yes, as would allow you to explicitly cast to a specific function type. Even if implicit conversion becomes available later, it will still be useful in edge cases where you need to disambiguate which call signature you want when there are multiple matches.

If you still want the magic of not having to think about (or write out) the particular function signature, then we could also support myCallable as func for the cases where there is no ambiguity.

That combined with @Chris_Lattner3's call func() would allow us to avoid any overlap/confusion around .call