SE-0253: Static callables

I like this direction. I would prefer to not have .call available on my callable types.

3 Likes

FWIW: the analog of foo as () -> Void is foo.call as () -> Void. foo.call is just like a normal method reference.

This is a fair point; supporting func call is necessary for source compatibility (and also because there are methods that genuinely should be named call).


Actually, I've come to strongly agree that direct references of call-syntax delegate methods via foo.call is undesirable and the wrong direction.

The eventual direction is not to support direct call member references, but to support conversions of callables to function-typed values. Supporting explicit casting via as seems noncontroversial, and implicit conversion requires more exploration.

call declarations should be represented as "instance methods with a special-case name", not "instance methods with the name 'call'".


Thread participants seem to agree with this direction. @rxwei or I will update the proposal to remove foo.call direct references, and instead advocate creating { foo(...) } thunks in the short term.

6 Likes
  • What is your evaluation of the proposal?

I like the concept of the proposal, but there are many points that I'd like to be addressed before it is accepted, so I am currently -1.

  • Is the problem being addressed significant enough to warrant a change to Swift?

Yes, the problem being addressed is enough to warrant a change even though I think this will be a very niche feature.

  • Does this proposal fit well with the feel and direction of Swift?

No, I don't think so. Introducing a new keyword with a completely different syntax is very inconvenient. I understand the desire of making callables feel like a first-class citizen language feature, but I think it's too much in this case. One thing that worries me is also the following part:

The rest of the call declaration grammar and semantics is identical to that of function declarations – it supports the same syntax for:

  • Access levels.
  • Generic parameter clauses.
  • Argument labels.
  • Return types.
  • throws and rethrows .
  • mutating .
  • where clauses.

If the declaration is so similar, why not just create a special case that would handle functions defined as func call(...) to be callable? I like the Scala implementation, where the syntax is the same as defining any other function.
I would be happy to accept a @staticCallable keyword which would be much easier to understand and associate to the already implemented @dynamicCallable. This proposal seems to be a follow-up of @dynamicCallable, so I don't see why this approach is so radically different. The two language features would be pretty similar, so as a Swift user I would expect them to work pretty similarly from the syntax perspective.

  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

I briefly worked with Scala lately, but haven't really explored apply(...) that much.

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Read the proposal and forum thread.

I'll preface this with a general comment that I haven't thought through the details completely w.r.t. swift's type system (in particular generic erasure vs opaque existentials).

I think it's worth pointing out the design that Java took here when introducing the Functional interfaces in java.util.function java.util.function (Java Platform SE 8 )

It subsumes call-ables as described in the pitch (where lambdas are also callable) and allows many other expressive patterns to be captured as as proper Types / Interfaces. e.g. Consumer, Supplier, UnaryOperator, BinaryOperator, Predicate... (note, I'm not suggesting this proposal needs to consider syntactic operator overloading).

What's interesting about a well designed set of call-able / functional interfaces (which I think Python struggles to model well with a single overloaded notion of 'call-able') is that it ends up being parsimonious with several other components. A few that come to mind:

i) Work queues / task executors / things that can run units of work concurrently (e.g. ExecutorService in java)

ii) "stream programing" (a la RX Observables) or java 8 style java.util.stream (inspired somewhat by Guy Steele's work on Fortress) where you might want to map/filter/reduce but in parallel (i.e. you need to be able to express the concept of a Spliterator which is much easier to express if you also have a Consumer.

iii) the motivating case you pointed out with tensor flow

iv) and obviously the standard map / filter / reduce / foreach idioms

v) rust has gone as far as create a similar type-trait for a callable that's a default constructor or factory method.

As a concrete example, let's pretend to revise swift's map using the Java terminology (I'm not suggesting these need to be the names of the various protocols).

  • map could be expressed as taking a Function<R,T> rather than the a specific function type
  • filter in terms of a Predicate
  • reduce in terms of a stateless BinaryOperator (the accumulator) or more likely some variant of the triple (Supplier identity, BiFunction<U, T, U> accumulator, BinaryOperator combiner) (with variations for mutable, in-place reduction steps.

We have been talking about generalizing this so that the generics system would support callable constraints on generic types. We would use a function signature in a generic constraint position to do this. That way instead of having to define protocols like Function, BinaryOperator, Predicate, etc we would be able to just do things like the:

  • map<Result>(_ transform: some (Element) throws -> Result) rethrows -> [Result]
  • filter(_ predicate: some (Element) throws -> Bool) rethrows -> [Element]

In these examples some is potential future syntax that comes from the opaque result type discussions. It is shorthand for something like the following:

  • map<Result, Transform: (Element) throws -> Result>(_ transform: Transform) rethrows -> [Result]
  • filter<Predicate: (Element) throws -> Bool>(_ predicate: Predicate) rethrows -> [Element]
3 Likes

Yup, that makes sense and is basically what I was trying to hint at (I also followed the opaque result type discussion quite closely but stayed quiet on that one). I picked on Java because it originally didn't even have lambdas and they extended the language with Lambdas and other functional idioms and did a fairly good job of it.

The main question I still have with generic constraints as you described them when considering both lambdas and a hypothetical callable (I agree with Ben here, I think they both need to be considered together), if you need to use a (Element) throws -> Result type elsewhere (e.g. a data mapping situation either from db record or json object to a domain object) does that other bit of code just repeat the function signature and the compiler figures out that the callable struct can be used in both the map and data mapper or is there value in capturing the concept as some sort of generic type alias?

1 Like

I haven't had much time to think about this, so I haven't read all the comments, but I'll jot down a few more thoughts:

  • One advantage of having a dedicated name for call functions (whether call(…), func _(…), or whatever) is that there is only one thing the compiler has to look up when you say foo(). If the functions can be arbitrarily named, the compiler has to crawl through all members of the type, or we have to start keeping a separate index. (The latter's not that bad but it increases the complexity of the compiler in other ways.)

  • I don't think this is a universally held opinion, but personally I think being able to say obj.foo (or even obj.foo(of:)) as shorthand for { obj.foo($0) } has worked out really poorly, especially when in an escaping context. For classes, it captures the self value in a way that's non-obvious, making it harder to get a sense of your object graph when reading code. For value types, it doesn't work with mutating operations. The version that only has the base name breaks when other overloads of foo get defined. (This shows up the most with init.) So I'd much rather end up in a space with the "implicit conversion" (wrap in a closure) than a space where foo.call becomes common, and without that I think writing the explicit { foo($0) } is fine.

  • I still really want to see information about the ABI of call declarations, or an explicit acknowledgement that call will not be supported when deploying against a Swift 5.0 runtime (along with a commitment to verifying that in the compiler).

9 Likes

Puts compiler implementer hat on
One of the reasons I like having an attribute on the type definition itself (and not a member) is that the type checker doesn't having to go digging through all of the members of the type to determine whether it might be something it can call. It can check the attributes and, if not @callable, quickly conclude that a value of that type is (or is not) callable. It sounds like a small thing, but the type checker makes this "can potentially call this thing?" determination a lot when it's type checking an expression, and particularly in large Swift modules, having to parse the bodies of type definitions in other source files just to establish whether there is a @callable method there can add up to a compile-time performance problem.

Doug

4 Likes

This is the kind of issue where we as a community have to trust the core team to make the right decision. If this makes a “line in the sand” difference (like with union types) then so be it. But otherwise I don’t think it leads to the best programmer model.

Thanks for your perspective!

This is interesting. I think the "spirit" of call declarations is to make a first-class declaration kind, just like how init and subscript are first-class declaration kinds - this is core.

I definitely agree that the ability to quickly conclude whether a type is callable is valuable. But on the other hand, declarations don't define "semantics" in the same way that protocol requirements denote semantic requirements.

It might also be valuable to know, at a glance, whether a type is subscriptable, but "the fact that a type defines a certain declaration kind" isn't modeled as an explicit type annotation, it's an implicit quality of the type because it defines a subscript. Same for first-class call declarations.

1 Like

As Jordan notes, having a special declaration kind or a name (init, func _, func call, etc.) ameliorate my concern somewhat.

FWIW, I have the same concerns that Brent expresses about introducing a new declaration kind. I've said it elsewhere (e.g., in the custom attributes pitch), but introducing a new declaration kind has a very high cost, both in cognitive load and in the implementation. I strongly prefer that we make these normal functions with a type-level attribute to tag it.

Doug

1 Like

I see, that makes sense. (Finding call-syntax delegate methods is more efficient.)

I would like to reiterate some naming difficulties with using "normal named methods" to represent call-syntax delegate methods. If you haven't read it already, please read this post if you have some time. I would be curious about your thoughts.

TLDR: it is difficult to name "normal named methods" in a way such that both 1) applications using direct method references and 2) call-syntax applications are idiomatic.

struct Model {
    @callDelegate func applied(to input: Float) -> Float { ... }
}
model.applied(to: x) // this is fine.
model(to: x) // but leads to this, which is not fine.

// Dropping the argument label makes call-syntax applications idiomatic:
// `func apply(_ input: Float) -> Float` enables `model(x)`.
// But this violates naming guidelines (imperative verbs imply mutability).

Unnamed call-syntax delegate methods make way more sense, something like:

  • Underscore name: @callDelegate func _(...).
    • Requiring @callDelegate methods to have no name (_) may be good. I think that is my ideal alternative.
  • Extra parsing rules to support no name at all: @callDelegate func(...).
    • I dislike the idea of adding new special case grammar for func declarations.
2 Likes

This whole proposal is about objects that can act as something else, isn‘t it?
That is a concept which is quite general, and I wonder wether we may want a general solution:
There has been a proposal for objects to act as an Optional (Introducing `Unwrappable`, a biased unwrapping protocol), and I guess many people would be happy about a way to do calculations with implicit conversions (Float acting as Double...).

Afaik, there are no plans to add such forwarding to Swift, but I‘m also not aware of a decision that those capabilities are unsolicited, so imho it‘s an alternative worth to be considered.

If there is only one way to name the call function (be it func _ or func call or whatever), it's not really an issue: developers that want a prettier direct method reference syntax can provide a separate API with that fully-elaborated name.

Doug

3 Likes

Yes, providing separate APIs with a fully-elaborated name is fine. Not exposing poorly-named call-syntax delegate methods is my main concern, which seems solved by using unnamed call-syntax delegate methods (call(...), @callDelegate func _(...), etc).

Pretty sure we're on the same page, being extra pedantic to leave nothing tacit. :slight_smile:


Minor update: I anticipate @rxwei or I will have time to update the proposal today, so that direct references via foo.call are no longer supported, and to fill in details about ABI impact. Thanks again for everyone's feedback! I think discussion has been very fruitful.

2 Likes

Will putting the attribute on the type ever allow conditionally callable types? Think something like:

@callable
extension Future where  Value: Callable {
    call func<Arg>(_ arg: Future<Arg>) -> Future<Value.CallableResult<Arg>> {
        return self.then { fn in
            arg.then(fn)
        }
}

The interoperation of this proposal and generics is what's caused most doubt in me. People have asked whether it'll be possible to use callability as a constraint (like I also did above with Value: Callable), but I think conditional conformance is equally important.

Even if it won't be possible to use callables in the generic context in the first design, I think it'd be a shame if the chosen syntax made it impossible to extend the design in the future.

Conditionally callable types are (naturally) possible with first-class call declarations: just define a conditional conformance containing a call declaration. You can do the same with subscripts today.

Conceptually, the definition of "call-syntax delegate methods" are what make values callable - a type attribute is a non-essential marker (that comes with type-checking efficiency benefits, and arguably clarity benefits, as mentioned by @Douglas_Gregor).


We argue against a Callable protocol representing types whose values are callable:

Rather than creating a Callable protocol, a more principled and general direction is to implement function types using protocols with required call-syntax delegate methods. See Rust and Scala for precedent in other languages.

AFAICT, this didn't seem to work in the attached implementation. I tried the macOS toolchain, but maybe I didn't try hard enough.

Defining call members in constrained extensions worked for me. I used the macOS toolchain.

extension Array where Element == (Int) -> Int {
  call(_ input: Int) -> Int {
    return reduce(input) { result, fn in fn(result) }
  }
}

let chainedFunctions: [(Int) -> Int] = [
  { $0 * $0 }, // square,
  { $0 + 1 },  // then add one.
]
for i in 1..<10 {
  print(chainedFunctions(i))
  // 2, 5, 10, 17, 26, 37, 50, 65, 82
}

Toolchains:

2 Likes