@unrequired in function signature for optional closure parameters

From a perspective of a framework writer it is important to provide clear and understandable documentation for the correct usage of their APIs. Swift allows us to write functions that are at some extent self documenting, and clearly define what the function will actually do.

since swift 3, closures in function parameters are by default nonescaping: you pass a closure as function parameter, and you know that this closure will be synchronously used within the function scope before this function returns.

declaring the closure to be @escaping as function parameter we say:

A closure is said to escape a function when the closure is passed as an argument to the function, but is called after the function returns. When you declare a function that takes a closure as one of its parameters, you can write @escaping before the parameter’s type to indicate that the closure is allowed to escape.

Quote from Swift documentation

All good so far.

func nonescaping(closure: () -> Void)
func escaping(closure: @escaping () -> Void)

What about optional closures?

func foo(closure: (() -> Void)? = nil)

Here we are saying that this closure parameter is not essential for the function to work, but what about its escaping nature?

In this case the closure is implicitly escaping because of the definition of Optionality in swift. But is that closure intended to escape the actual function scope? Is this closure executed before or after the function return?

Just by reading the function signature this is unknown, and we are forced by the compiler to assume that it is escaping the function (because it is assigned as associated value of the some case of Optional)

While we can argue that technically the closure is in fact escaping the function, maybe it is not the intention of the framework writer.

From this point of view

func foo(closure: (() -> Void)? = nil) {
    ....
    closure?()
    ...
}

Is the same as

func foo(closure: @escaping () -> Void = { }) {
    ....
    closure()
    ...
}

except that closure is not escaping foo in the second case, and if it wasn't for the definition of Optional it wasn't escaping foo in the first case either.

A solution to explicitly declare escapability of closures also in optional cases is by providing a default valid closure and not using Optional

func foo(closure: () -> Void = { }) {
    ....
    closure()
    ...
}

but it some cases, this can lead to an ugly and hard to read function signature

func foo(closure: (Data?, Response?, Error?) -> Void = { _, _, _ in }) {
    ....
    closure(data, response, nil)
    ...
}

and sometimes can also be error prone

func foo(closure: () -> Int = { /* what should I return here? */ }) {
    ....
    //how to know that it is default?
    ?? = closure()
    ...
}

@unrequired keyword

An idea might be to stop this ambiguity for optional closures by introducing a new keyword @unrequired

var closure: (()->Int)?
func escaping(closure: @escaping @unrequired () -> Int) {
    ...
    self.closure = closure
    ...
}
func nonEscaping(closure: @unrequired () -> Int) {
    ...
    let value: Int
    if closure is Empty {
        value = 0
    } else { value = closure() }
    ...
}

that would be equivalent to

func escaping(closure: @escaping  () -> Int = Empty)
func nonEscaping(closure: () -> Int = Empty)

This would avoid the need of using the existent Optional eliminating the ambiguity for escapability of the closure in relation to the function.

For the definition of Empty I'll need your help. :smiley: Ideally it should be a special Type like Never, that can represent Any type of empty closure and returns Never if the closure has a return type different than Void

3 Likes

If you improve some existing compiler machinery, you can do it today without any new keywords! And fixing these problems would help in general, not just in this situation!

import Foundation
struct Response { }

func empty() -> Void {}
func empty<A>(_: A) -> Void {}
func empty<A, B>(_: A, _: B) -> Void {}
func empty<A>() -> A {
    fatalError()
}
func empty<A, B>(_: B) -> A {
    fatalError()
}
func empty<A, B, C>(_: B, _: C) -> A {
    fatalError()
}
func empty<A, B, C, D>(_: B, _: C, _: D) -> A {
    fatalError()
}

func foo(closure: ((Data?, Response?, Error?) -> Void) = empty) {
    let data = Data()
    let response = Response()
    if closure == (empty as (Data?, Response?, Error?) -> Void) { // two problems: `==` escapes the closure, and compiler doesn't know which `empty` without explicit type information
        closure(data, response, nil)
    } else {
        print("is empty")
    }
    
}
func bar(closure: (String) -> Void = empty) { // problem: compiler doesn't realize that void-returning version is a closer match
    closure("asdf")
}
bar {
    print($0)
}
bar()
foo { a, b, c in
    print(b)
}
foo()
2 Likes

I'm not sure which would be the best way to approach the problem (if the community shares the idea that there actually is a problem).

What I know is that I saw this @escaping behaviour with optional closures creating some confusion in the community of young swift developers. Whoever ignores the nature of Optional wonders why @escaping is specified whenever the function will use the closure after return except for cases where the closure is optional, in which the usage of the closure by the function is unspecified without additional documentation.

The statement "by default closure are nonescaping" and the missing @escaping for Optional closures causes confusion.

This seems so obvious to me now, but I honestly had no idea why optional closures were treated differently.

1 Like

Yeah, I had no idea that Optional changed the default. It would be nice to require @escaping if the closure can’t be nonescaping because of Optional and have the compiler give a warning at a minimum.

3 Likes

It's not exactly the default that changes. Technically it actually do escape, because the closure is assigned to the some case of Optional, therefore it is escaping. it's pretty much like assigning the closure to a var. Except the fact that in this case it's producing the side effect to make the closure escaping also if it is not within the function scope.

It’s not escaping, though, is it? That’s the point underlying this entire thread. It’s merely that the compiler today is assuming it’s escaping the function’s scope because the compiler’s not actually looking at the closure’s lifetime / use within the function.

It’s easy to understand why lots of people get confused by this, as a result - it’s plainly clear to a human that the closure never leaves the function; it seems like the compiler is just being “dumb”.

IMO it’s making Optional a leaky abstraction. Optional is supposed to be just a more explicit and robust way for the language to handle nulls. That it happens to be implemented as a ‘real’ struct technically like any other feels like an implementation detail that should be irrelevant. Using an optional closure should be completely equivalent to a hypothetical Optionaless version of Swift:

func doSomething(with closure: (Int, Int) -> Bool) {
  if closure != null {
    closure()
  }
}

Thus attempting to address this by requiring a nonEscaping annotation (or similar) just exacerbates the problem.

(also, just writing that Javaesq example makes me so happy that Swift does have Optional, unlike the NullPointerException hell of Java and similar languages - optionals are maybe my favourite feature of Swift)

10 Likes

That’s just reimplementing Optional though, isn’t it? It’s a recipe for confusion to have empty in this manner when we already have nil.

Is it an implementation detail though? I think it’s perfectly valid for Optional to be implemented as an enum and for those enum semantics to be carried over to Optional.

¯\_(ツ)_/¯ I just implemented what's described in the first post

Why? What’s the benefit to the coder in this case? I get that it’s helpful and sensible to reuse existing language concepts to expose useful functionality around optionals in other situations, as opposed to some bespoke syntax for it, but in this case what is it adding?

Put another way, though, I don’t see this as a problem with Optional specifically per se, but rather it goes back to the compiler not knowing and/or acting on the closure’s actual lifetime. One naively wouldn’t expect to have to use escaping in this case either, because the closure isn’t escaping:

struct Stubs {
  let getVersion: () -> String
}

func loadPlugin(_ stubs: Stubs) {
  print(“App version is: {stubs.getVersion()}”)
}

loadPlugin(Stubs(getVersion: { “1.0” }))

Yes, you can declare a Stubs instance and use it in a way that it does escape any given context, but that’s not what’s actually happening in the code above. So you can hopefully understand why a user is confused by the compiler’s behaviour.

Having a closure in a struct doesn’t actually say anything about how it is used - it doesn’t technically determine the lifetime of the struct or its contents.

To be clear, it’s easy to sympathise with and understand why the compiler is being conservative, of course, and I’m not sure if there is a better way. This is the age-old philosophical question between what is and what can. Nonetheless it is a usability problem for the language.

1 Like

Swift at least tries to make it a detail in some circumstances. Otherwise you'd have to write .some(thing) everywhere instead of just thing. Thanks of those implicit conversions, you don't need as much awareness of what is optional and what isn't, and it feels more like a language feature than just a type in the stdlib.

There's some other special behaviors around Optional. For instance Optional<DerivedObject> will implicitly convert to Optional<BaseObject>, which is not possible for regular generic types. The language would be clearly worse if this didn't work.

I don't feel like it'd be wrong to have one more special case for allowing optional closures to be non-escaping. Optional is a special and privileged type in this language and people already expect much more from it than what a regular enum can provide.

2 Likes

Thanks, I had never realised this issue existed!

I don't appreciate much how @unrequired would create a redundant way of expressing optionality in Swift, and IMO there is nothing wrong per se with the Optional type being an enum.

I think the issue we encounter is that while the special compiler support the Optional type benefits from is a good thing (eg: gives us the nice T? syntax instead of Optional<T>, avoids us from explicitly setting optional properties to nil or .none) it still has some gaps that need to be filled, one I have in mind is this one @codafi recently mentioned.

IMO we should either build into the compiler the additional logic required to handle this specific case for when closures are optional, or another option could be to introduce a new attribute that would also be applicable to other types, which could be useful on the ones that are primarily containers.

The attribute could be named something such as @propagatesEscape that would be used this way:

@propagatesEscape
enum Optional<Wrapped> { ... }

But also on other types:

@propagatesEscape
struct Array<Element> { ... }

This way, any time you would use them with a type requiring the explicit @escaping attribute, they would themselves start requiring it if they are escaped, for instance:

struct Foo<T> {
    var x: [(Int)->(String)]
    
    init(x: [(Int)->(String)]) {
        self.x = x // Assigning non-escaping parameter 'x' to an escaping container type
    }
}

struct Bar<T> {
    var x: ((Int)->(String))?
    
    init(x: ((Int)->(String))?) {
        self.x = x // Error: Assigning non-escaping parameter 'x' to an escaping container type
    }
}

The correct way to write this would now be to do that:

struct Bar<T> {
        var x: ((Int)->(String))?
        
        init(x: @escaping ((Int)->(String))?) {
            self.x = x
        }
}

What do you think about this? I don't know what drawbacks this approach might have.

This would be a source breaking change, so maybe it could be made a warning during one or two years before becoming an error?

5 Likes

While I clearly see how the compiler can optimise Optional case, I don’t see how propagate escape would work in the Array case. You will still have to create an heap storage and copy your closure inside it, and by definition it breaks the nonescaping optimization.

I’m more in favor of a special handling of Optional by the compilers.

It's not just reusing syntax, language constructs for the sake of the compiler, but also for the sake of humans. Optional is an enum in every sense, and makes it easy and coherent to reason about. An Optional is either something or it is nothing. It has two specific and distinct cases. It's not an implementation detail, but a core concept, mapping perfectly to the human mental model of the thing it models.

That said, I agree that sometimes a closure is wrapped in a way that makes it not really escape, and that this makes sense to somehow model. Wether it is important enough to spend time and effort to implement the concept of escapability to other types in addition to closures, is up to others to decide.

I have personally not been hit by neither a cognitive nor a performance penalty for the lack of such a feature.

3 Likes

I always thought @escaping was more about giving a hint to the users about the consequences it has because of ARC rather than something the compiler needs to be manually told about. As far as I can tell, the compiler is always able to determine when you should mark a closure as escaping, so except maybe for some lower level cases, it could probably have evolved in a way that made it implicit while it would have still been able to make the same optimisations, but of course we don't want to loose the clarity/safety @escaping offers.

The Swift language guide states that:

A closure is said to escape a function when the closure is passed as an argument to the function, but is called after the function returns.

In this sense, this makes the concept of @escaping rather independent of what wether it is heap allocated or not.

EDIT: Just realised what I said before my edit about heap allocation is wrong, it will still be considered as escaping by the compiler if you assign the property to some class / struct in the init body, but based on the definition given by the guide, this looks more like some lacking analysis capability by the compiler?

So if marking a closure @escaping is mainly about telling the user the closure might be executed later and warning them about the risk of reference cycle, why not extending the concept for the container types that might allow it to escape in practice?

3 Likes

I like it.

1 Like

This sound reasonable. That said, I'm not sure the concept is common enough to warrant a language extension.

I would be glade to be proven wrong, but I actually never saw a function that take a container of escapable closures.

And by the way, putting a closure into a collection already make is non escapable, as the collection lifespan can't be guarantee to not escape the current function.

Yes I noticed this, didn't know it was the case, it would be nice if there were more guarantees.

Speaking of this, making the concept clean and enabling the compiler to make the right proofs looks like a harder challenge than I first thought...

I thought long about this problem... I was thinking that I actually don't even like the wrapping parenthesis around the function to make it optional (()->())? It's kinda confusing. The more I think about it, the more I believe that the annotation @unneded or similar makes sense.

We want to treat closures as variable types, but there are innegabile differences between closures and normal types. For start we don't need to annotate an Int as @escaping, we don't need to wrap it around parentheses to make it optional and memory rules around closures are different than normal types.

I think that closures in function parameters should not be allowed to be optional and the annotation @unneded should be used instead.

This, or the compiler should do extra magic around Optional to let us annotate the escapability of closures. Right now, what the documentation says about this swift feature

A closure is said to escape a function when the closure is passed as an argument to the function, but is called after the function returns.

is just not true, and confusing for new generation of swift developers.

1 Like