[Proposal] Make non-escaping closures the default


(Trent Nadeau) #1

https://github.com/tanadeau/swift-evolution/blob/make-noescape-default/proposals/XXXX-make-noescape-default.md

# Make non-escaping closures the default

* Proposal: [SE-NNNN](NNNN-name.md)
* Author: [Trent Nadeau](https://github.com/tanadeau)
* Status: **Awaiting review**
* Review manager: TBD

## Introduction

The current default of closure arguments to functions (i.e., arguments to
functions that themselves have function type such as `(T) -> U`) is to be
"escaping", meaning they can escape the function body such as saving it to
a field in a struct or a global variable. In order to say that a closure
argument cannot possibly escape the function body ("non-escaping"), the
developer must explicitly add an `@noescape` annotation to the argument
type.

This proposal switches the default to be non-escaping and requires an
`@escaping` annotation if a closure argument can escape the function body.
Since the escaping case can be statically detected, this annotation can be
added via an error with a fixit. Other annotations that have consequences
for escape semantics (e.g., `@autoclosure(escaping)`) will be altered to
make use of the new `@escaping` annotation.

Swift-evolution threads: [Discussion thread topic for that proposal (TBD)](
http://news.gmane.org/gmane.comp.lang.swift.evolution)

## Motivation

Per Chris Lattner [on swift-evolution](
https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160530/019880.html
):

To provide some more details, this approach has the following advantages:

- Most functional algorithms written in pure Swift will benefit because

they are naturally noescape. The core team feels that this will reduce the
boilerplate involved with writing these algorithms.

- The compiler has enough logic in it to provide a great QoI experience

when a developer doesn’t think about escaping, and tries to escape a
closure - it can provide a fixit that suggests adding @escaping.

- Recent changes (to disallow escaping closures to close over an inout

parameter) are pushing the language to prefer noescape closures. noescape
closures have also always been the preferred default, since they eliminate
a class of retain cycle issues.

- "@autoclosure(escaping)" can be simplified and standardized to

"@autoclosure @escaping

## Detailed design

The `@noescape` annotation is removed from the language. The compiler will
emit an error with a fixit to remove the annotation.

The compiler will emit an error if a closure argument is found to possibly
escape the function body. In order to silence the warning, the developer
must add, manually or via fixit, the `@escaping` annotation to the argument
type.

The compiler's semantic analysis implementation can be simplified as the
more constrained escaping case that conflicts with other attributes is now
no longer the default.

The standard library should be changed to use the new default whenever
possible by removing all uses of `@noescape` and only adding `@escaping`
where the compiler detects the need.

### Imported C/Objective-C APIs

Per the Core Team, most Cocoa closure/block parameters are escaping (e.g.,
delegates). As such the Clang importer will automatically add the
`@escaping` annotation to closure/block parameters encountered in imported
Objective-C APIs unless they are explicitly marked with the Clang
`((noescape))` attribute. This will also be done with imported C APIs with
function pointer or block parameters.

## Impact on existing code

Existing code using the `@noescape` attribute will need to be migrated to
remove the attribute since it will be the default. In addition, the
compiler will need to detect escaping closures that are not marked with
`@escaping` and create an error with a fixit to add the required attribute.

Uses of `@autoclosure(escaping)` must be changed to `@autoclosure
@escaping`.

There should be few, if any, changes required for uses of Cocoa APIs as
these will be mostly marked as `@escaping`, and escaping closure arguments
are *more* constrained than non-escaping ones.

## Future directions

The `@noescape(once)` annotation proposed in [SE-0073](
https://github.com/apple/swift-evolution/blob/master/proposals/0073-noescape-once.md)
would, if some future version is accepted, just become `@once`.

## Alternatives considered

Leave the `@noescape` attribute and existing semantics as they are now.

## Acknowledgements

Thanks to Chris Lattner, **TBD**, and anyone else who reviewed and
contributed to this proposal.

···

--
Trent Nadeau


(Ben Rimmington) #2

Trent Nadeau wrote:

### Imported C/Objective-C APIs

Per the Core Team, most Cocoa closure/block parameters are escaping
(e.g., delegates). As such the Clang importer will automatically add
the `@escaping` annotation to closure/block parameters encountered in
imported Objective-C APIs unless they are explicitly marked with the
Clang `((noescape))` attribute. This will also be done with imported
C APIs with function pointer or block parameters.

"SE-0012: Add @noescape to public library API" has a list of Foundation
and CoreFoundation APIs, but that proposal is still awaiting scheduling.

<https://github.com/apple/swift-evolution/blob/master/schedule.md>

-- Ben


(Brent Royal-Gordon) #3

This proposal switches the default to be non-escaping and requires an `@escaping` annotation if a closure argument can escape the function body.

Is @escaping part of the function type syntax (like @autoclosure) or the parameter syntax (like inout)? It seems to me that there are places where you *do* want non-parameter closure variables to be non-escaping:

  func foo(closure: () -> Void) {
    let bar = closure // You should not be able to escape through `bar`
  }

But then there are also many places where you would have to write @escaping even though a non-escaping closure would be obviously nonsensical:

  struct Foo {
    var closure: () -> Void // Error; @escaping is mandatory here
    func method() -> () -> Void {...} // Same
  }

Requiring a boilerplate attribute like this is not really so great.

Existing code using the `@noescape` attribute will need to be migrated to remove the attribute since it will be the default. In addition, the compiler will need to detect escaping closures that are not marked with `@escaping` and create an error with a fixit to add the required attribute.

This becomes difficult when a protocol has a closure parameter; there's not necessarily a default implementation to examine for escaping, and even if there is one, the default may be a simple, non-escaping implementation for something which ought to allow escaping from other implementations. Similar concerns apply to non-final class members.

One way to address this issue would be to have the migrator conservatively mark non-@noescape closure parameters in protocol and class definitions as @escaping, but that might undermine the intent of automatically transitioning a lot of code to the new default.

(By the way, is an @escaping closure a subtype of a non-escaping closure?)

···

--
Brent Royal-Gordon
Architechies


(Jordan Rose) #4

I'm against this for library evolution reasons: if someone releases a version of their library that has a non-escaping closure and later discovers it needs to be escaping, they can't change it.

IIRC the counterpoint to this is that people were probably implicitly relying on it being non-escaping already, and that there aren't many cases where you'd want to do this anyway. My counter-counterpoint is that you might have some case, like dispatch_async (er, DispatchQueue.async) with a semaphore where you know the closure isn't escaping, but you need to treat it as one. Maybe that just means we need an asUnsafeEscapingClosure helper in the standard library.

I also think it might be useful to be able to annotate references as non-escaping (purely for performance reasons), and I can't see those being non-escaping by default. I know we don't want to block one change because of something else that might happen down the line, but still.

Jordan

···

On Jun 5, 2016, at 20:49, Trent Nadeau via swift-evolution <swift-evolution@swift.org> wrote:

https://github.com/tanadeau/swift-evolution/blob/make-noescape-default/proposals/XXXX-make-noescape-default.md

# Make non-escaping closures the default

* Proposal: [SE-NNNN](NNNN-name.md)
* Author: [Trent Nadeau](https://github.com/tanadeau)
* Status: **Awaiting review**
* Review manager: TBD

## Introduction

The current default of closure arguments to functions (i.e., arguments to functions that themselves have function type such as `(T) -> U`) is to be "escaping", meaning they can escape the function body such as saving it to a field in a struct or a global variable. In order to say that a closure argument cannot possibly escape the function body ("non-escaping"), the developer must explicitly add an `@noescape` annotation to the argument type.

This proposal switches the default to be non-escaping and requires an `@escaping` annotation if a closure argument can escape the function body. Since the escaping case can be statically detected, this annotation can be added via an error with a fixit. Other annotations that have consequences for escape semantics (e.g., `@autoclosure(escaping)`) will be altered to make use of the new `@escaping` annotation.

Swift-evolution threads: [Discussion thread topic for that proposal (TBD)](http://news.gmane.org/gmane.comp.lang.swift.evolution)

## Motivation

Per Chris Lattner [on swift-evolution](https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160530/019880.html):

> To provide some more details, this approach has the following advantages:
>
> - Most functional algorithms written in pure Swift will benefit because they are naturally noescape. The core team feels that this will reduce the boilerplate involved with writing these algorithms.
>
> - The compiler has enough logic in it to provide a great QoI experience when a developer doesn’t think about escaping, and tries to escape a closure - it can provide a fixit that suggests adding @escaping.
>
> - Recent changes (to disallow escaping closures to close over an inout parameter) are pushing the language to prefer noescape closures. noescape closures have also always been the preferred default, since they eliminate a class of retain cycle issues.
>
> - "@autoclosure(escaping)" can be simplified and standardized to "@autoclosure @escaping

## Detailed design

The `@noescape` annotation is removed from the language. The compiler will emit an error with a fixit to remove the annotation.

The compiler will emit an error if a closure argument is found to possibly escape the function body. In order to silence the warning, the developer must add, manually or via fixit, the `@escaping` annotation to the argument type.

The compiler's semantic analysis implementation can be simplified as the more constrained escaping case that conflicts with other attributes is now no longer the default.

The standard library should be changed to use the new default whenever possible by removing all uses of `@noescape` and only adding `@escaping` where the compiler detects the need.

### Imported C/Objective-C APIs

Per the Core Team, most Cocoa closure/block parameters are escaping (e.g., delegates). As such the Clang importer will automatically add the `@escaping` annotation to closure/block parameters encountered in imported Objective-C APIs unless they are explicitly marked with the Clang `((noescape))` attribute. This will also be done with imported C APIs with function pointer or block parameters.

## Impact on existing code

Existing code using the `@noescape` attribute will need to be migrated to remove the attribute since it will be the default. In addition, the compiler will need to detect escaping closures that are not marked with `@escaping` and create an error with a fixit to add the required attribute.

Uses of `@autoclosure(escaping)` must be changed to `@autoclosure @escaping`.

There should be few, if any, changes required for uses of Cocoa APIs as these will be mostly marked as `@escaping`, and escaping closure arguments are *more* constrained than non-escaping ones.

## Future directions

The `@noescape(once)` annotation proposed in [SE-0073](https://github.com/apple/swift-evolution/blob/master/proposals/0073-noescape-once.md) would, if some future version is accepted, just become `@once`.

## Alternatives considered

Leave the `@noescape` attribute and existing semantics as they are now.

## Acknowledgements

Thanks to Chris Lattner, **TBD**, and anyone else who reviewed and contributed to this proposal.

--
Trent Nadeau
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Anton Zhilin) #5

Trent Nadeau via swift-evolution <swift-evolution@...> writes:

https://github.com/tanadeau/swift-evolution/blob/make-noescape-

default/proposals/XXXX-make-noescape-default.md

-1 from me.

1. One must break API and ABI compatibility to add @escaping to an
existing function
2. @nonescaping case is actually quite as common as @escaping
3. Swift's lifetime system is not powerful enough, I'll explain below

Lazy map and filter will be marked as @escaping. Now consider the
following code:

func sum<T>(_ array: [T], transform: (T) -> Int) -> Int {
    return array.lazy.map(func).reduce(0, combine: +)
}

`transform` will be marked as @escaping, despite that `transform` never
actually escapes `sum`.

I believe that we should not accept nonescaping-by-default until we get
a powerful lifetime management system, as in Rust.

I think it makes sense to create a mini-proposal just for renaming of
@noescape to @nonescaping.

- Anton


(Trent Nadeau) #6

@escaping would be part of the parameter type just as @noescape is today.
Your foo(closure:) example wouldn't compile with my proposal, the same as
today if you marked the parameter with @noescape. Non-escaping function
parameters are only allowed to be called. They can't be assigned to
variables.

The current @noescape and the proposed @escaping can only be applied to the
types of function parameters so the code in your `struct Foo` example
wouldn't change.

Currently escaping and non-escaping closures are considered to be different
types so there is already a problem when a protocol requires a function
with a closure parameter. All conforming types must use the same
"escapiness" as the protocol.

···

On Tue, Jun 7, 2016 at 3:45 AM, Brent Royal-Gordon <brent@architechies.com> wrote:

> This proposal switches the default to be non-escaping and requires an
`@escaping` annotation if a closure argument can escape the function body.

Is @escaping part of the function type syntax (like @autoclosure) or the
parameter syntax (like inout)? It seems to me that there are places where
you *do* want non-parameter closure variables to be non-escaping:

        func foo(closure: () -> Void) {
                let bar = closure // You should not be able to
escape through `bar`
        }

But then there are also many places where you would have to write
@escaping even though a non-escaping closure would be obviously nonsensical:

        struct Foo {
                var closure: () -> Void // Error;
@escaping is mandatory here
                func method() -> () -> Void {...} // Same
        }

Requiring a boilerplate attribute like this is not really so great.

> Existing code using the `@noescape` attribute will need to be migrated
to remove the attribute since it will be the default. In addition, the
compiler will need to detect escaping closures that are not marked with
`@escaping` and create an error with a fixit to add the required attribute.

This becomes difficult when a protocol has a closure parameter; there's
not necessarily a default implementation to examine for escaping, and even
if there is one, the default may be a simple, non-escaping implementation
for something which ought to allow escaping from other implementations.
Similar concerns apply to non-final class members.

One way to address this issue would be to have the migrator conservatively
mark non-@noescape closure parameters in protocol and class definitions
as @escaping, but that might undermine the intent of automatically
transitioning a lot of code to the new default.

(By the way, is an @escaping closure a subtype of a non-escaping closure?)

--
Brent Royal-Gordon
Architechies

--
Trent Nadeau


(John McCall) #7

I'm against this for library evolution reasons: if someone releases a version of their library that has a non-escaping closure and later discovers it needs to be escaping, they can't change it.

IIRC the counterpoint to this is that people were probably implicitly relying on it being non-escaping already, and that there aren't many cases where you'd want to do this anyway.

Right. APIs are already semantically constrained in how they're allowed to use their closure arguments. Closure arguments inject arbitrary code, with arbitrary data access, into the callee; as a rule, the caller must know how the callee intends to use the closure, or its semantics will be grossly violated. You can't re-implement an existing API that always synchronously sub-invokes a closure to instead call the closure asynchronously or concurrently because it is completely reasonable for the caller to pass a closure that relies on being called synchronously or from at most one thread at once and/or within a fixed range of time. For example, the closure may modify a captured local variable, or it may it use a network connection that will be closed after the API returns. APIs that want to do this sort of thing have to reserve the right to do that (and even then, they may have binary compatibility limitations), in which case it is totally reasonable to expect them to express that in the type.

My counter-counterpoint is that you might have some case, like dispatch_async (er, DispatchQueue.async) with a semaphore where you know the closure isn't escaping, but you need to treat it as one. Maybe that just means we need an asUnsafeEscapingClosure helper in the standard library.

I agree that it's important to have some unsafe method of dodging the restriction. Apparently unsafeBitCast works right now, although it really shouldn't and we should come up with a better alternative.

John.

···

On Jun 9, 2016, at 3:43 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

I also think it might be useful to be able to annotate references as non-escaping (purely for performance reasons), and I can't see those being non-escaping by default. I know we don't want to block one change because of something else that might happen down the line, but still.

Jordan

On Jun 5, 2016, at 20:49, Trent Nadeau via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

https://github.com/tanadeau/swift-evolution/blob/make-noescape-default/proposals/XXXX-make-noescape-default.md

# Make non-escaping closures the default

* Proposal: [SE-NNNN](NNNN-name.md)
* Author: [Trent Nadeau](https://github.com/tanadeau)
* Status: **Awaiting review**
* Review manager: TBD

## Introduction

The current default of closure arguments to functions (i.e., arguments to functions that themselves have function type such as `(T) -> U`) is to be "escaping", meaning they can escape the function body such as saving it to a field in a struct or a global variable. In order to say that a closure argument cannot possibly escape the function body ("non-escaping"), the developer must explicitly add an `@noescape` annotation to the argument type.

This proposal switches the default to be non-escaping and requires an `@escaping` annotation if a closure argument can escape the function body. Since the escaping case can be statically detected, this annotation can be added via an error with a fixit. Other annotations that have consequences for escape semantics (e.g., `@autoclosure(escaping)`) will be altered to make use of the new `@escaping` annotation.

Swift-evolution threads: [Discussion thread topic for that proposal (TBD)](http://news.gmane.org/gmane.comp.lang.swift.evolution)

## Motivation

Per Chris Lattner [on swift-evolution](https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160530/019880.html):

> To provide some more details, this approach has the following advantages:
>
> - Most functional algorithms written in pure Swift will benefit because they are naturally noescape. The core team feels that this will reduce the boilerplate involved with writing these algorithms.
>
> - The compiler has enough logic in it to provide a great QoI experience when a developer doesn’t think about escaping, and tries to escape a closure - it can provide a fixit that suggests adding @escaping.
>
> - Recent changes (to disallow escaping closures to close over an inout parameter) are pushing the language to prefer noescape closures. noescape closures have also always been the preferred default, since they eliminate a class of retain cycle issues.
>
> - "@autoclosure(escaping)" can be simplified and standardized to "@autoclosure @escaping

## Detailed design

The `@noescape` annotation is removed from the language. The compiler will emit an error with a fixit to remove the annotation.

The compiler will emit an error if a closure argument is found to possibly escape the function body. In order to silence the warning, the developer must add, manually or via fixit, the `@escaping` annotation to the argument type.

The compiler's semantic analysis implementation can be simplified as the more constrained escaping case that conflicts with other attributes is now no longer the default.

The standard library should be changed to use the new default whenever possible by removing all uses of `@noescape` and only adding `@escaping` where the compiler detects the need.

### Imported C/Objective-C APIs

Per the Core Team, most Cocoa closure/block parameters are escaping (e.g., delegates). As such the Clang importer will automatically add the `@escaping` annotation to closure/block parameters encountered in imported Objective-C APIs unless they are explicitly marked with the Clang `((noescape))` attribute. This will also be done with imported C APIs with function pointer or block parameters.

## Impact on existing code

Existing code using the `@noescape` attribute will need to be migrated to remove the attribute since it will be the default. In addition, the compiler will need to detect escaping closures that are not marked with `@escaping` and create an error with a fixit to add the required attribute.

Uses of `@autoclosure(escaping)` must be changed to `@autoclosure @escaping`.

There should be few, if any, changes required for uses of Cocoa APIs as these will be mostly marked as `@escaping`, and escaping closure arguments are *more* constrained than non-escaping ones.

## Future directions

The `@noescape(once)` annotation proposed in [SE-0073](https://github.com/apple/swift-evolution/blob/master/proposals/0073-noescape-once.md) would, if some future version is accepted, just become `@once`.

## Alternatives considered

Leave the `@noescape` attribute and existing semantics as they are now.

## Acknowledgements

Thanks to Chris Lattner, **TBD**, and anyone else who reviewed and contributed to this proposal.

--
Trent Nadeau
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Chéyo Jiménez) #8

That proposal was rejected because the swift evolution is not the correct vehicle to drive changes to Apple frameworks.

···

On Jun 6, 2016, at 8:33 AM, Ben Rimmington via swift-evolution <swift-evolution@swift.org> wrote:

Trent Nadeau wrote:

### Imported C/Objective-C APIs

Per the Core Team, most Cocoa closure/block parameters are escaping
(e.g., delegates). As such the Clang importer will automatically add
the `@escaping` annotation to closure/block parameters encountered in
imported Objective-C APIs unless they are explicitly marked with the
Clang `((noescape))` attribute. This will also be done with imported
C APIs with function pointer or block parameters.

"SE-0012: Add @noescape to public library API" has a list of Foundation
and CoreFoundation APIs, but that proposal is still awaiting scheduling.

<https://github.com/apple/swift-evolution/blob/master/schedule.md>

-- Ben

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(John McCall) #9

This is a really good point. In the short term, we can address this with an unsafe
nonescaping-to-escaping construct, but we'll need a more complete answer eventually.

John.

···

On Jun 20, 2016, at 11:24 AM, Anton Zhilin via swift-evolution <swift-evolution@swift.org> wrote:
Trent Nadeau via swift-evolution <swift-evolution@...> writes:

https://github.com/tanadeau/swift-evolution/blob/make-noescape-

default/proposals/XXXX-make-noescape-default.md

-1 from me.

1. One must break API and ABI compatibility to add @escaping to an
existing function
2. @nonescaping case is actually quite as common as @escaping
3. Swift's lifetime system is not powerful enough, I'll explain below

Lazy map and filter will be marked as @escaping. Now consider the
following code:

func sum<T>(_ array: [T], transform: (T) -> Int) -> Int {
   return array.lazy.map(func).reduce(0, combine: +)
}

`transform` will be marked as @escaping, despite that `transform` never
actually escapes `sum`.


(Brent Royal-Gordon) #10

@escaping would be part of the parameter type just as @noescape is today. Your foo(closure:) example wouldn't compile with my proposal, the same as today if you marked the parameter with @noescape. Non-escaping function parameters are only allowed to be called. They can't be assigned to variables.

Okay, that does correct that issue. Although it raises a separate issue: a bare closure type now means something different in a parameter list than anywhere else.

Are generic types which happen to be functions in some particular use automatically @escaping? Are typealiases and associated types automatically @escaping?

Also, if `@escaping` is a part of the parameter list syntax (like `inout`) instead of the type syntax (like `@autoclosure`), would it make sense to drop its `@` sign to make them more syntactically similar?

···

--
Brent Royal-Gordon
Architechies


(Jordan Rose) #11

I don't buy this. If someone publishes an API that executes something on the current thread today and on a background queue tomorrow, that's totally fine if they never promised it would execute on a particular thread. If a client accidentally assumes an implementation detail is part of the interface, that's their fault, and always has been…though the library author might decide to continue supporting their use in the future in the interest of not making waves.

(Escaping/non-escaping by default doesn't affect this, by the way, but by the same token this argument doesn't really affect escaping/non-escaping by default.)

Jordan

···

On Jun 9, 2016, at 16:10, John McCall <rjmccall@apple.com> wrote:

On Jun 9, 2016, at 3:43 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I'm against this for library evolution reasons: if someone releases a version of their library that has a non-escaping closure and later discovers it needs to be escaping, they can't change it.

IIRC the counterpoint to this is that people were probably implicitly relying on it being non-escaping already, and that there aren't many cases where you'd want to do this anyway.

Right. APIs are already semantically constrained in how they're allowed to use their closure arguments. Closure arguments inject arbitrary code, with arbitrary data access, into the callee; as a rule, the caller must know how the callee intends to use the closure, or its semantics will be grossly violated. You can't re-implement an existing API that always synchronously sub-invokes a closure to instead call the closure asynchronously or concurrently because it is completely reasonable for the caller to pass a closure that relies on being called synchronously or from at most one thread at once and/or within a fixed range of time. For example, the closure may modify a captured local variable, or it may it use a network connection that will be closed after the API returns. APIs that want to do this sort of thing have to reserve the right to do that (and even then, they may have binary compatibility limitations), in which case it is totally reasonable to expect them to express that in the type.


(John McCall) #12

@escaping would be part of the parameter type just as @noescape is today. Your foo(closure:) example wouldn't compile with my proposal, the same as today if you marked the parameter with @noescape. Non-escaping function parameters are only allowed to be called. They can't be assigned to variables.

Okay, that does correct that issue. Although it raises a separate issue: a bare closure type now means something different in a parameter list than anywhere else.

@escaping is really a parameter-specific aspect of the outer function type, not an aspect of the parameter's type. It's just like something like NS_CONSUMED in Objective-C ARC.

John.

···

On Jun 7, 2016, at 7:25 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

Are generic types which happen to be functions in some particular use automatically @escaping? Are typealiases and associated types automatically @escaping?

Also, if `@escaping` is a part of the parameter list syntax (like `inout`) instead of the type syntax (like `@autoclosure`), would it make sense to drop its `@` sign to make them more syntactically similar?

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(John McCall) #13

Synchronous-but-off-thread is kind of a special case because it's only observable in very special ways, e.g. thread-local storage and the current thread ID. Concurrent (e.g. calling an enumeration callback on multiple threads simultaneously) and asynchronous (even if it comes back to the current queue) are absolutely something you have to know about as a caller. It is deeply unreasonable for an API to suddenly start invoking a closure asynchronously when it hasn't documented that it might do that (perhaps implicitly by obviously following some well-known pattern, e.g. calling the closure a completion handler); that would be a huge semantic and binary-compatibility problem.

Another line of argument: flipping the default is a huge boon to static analysis because (1) closure execution becomes ordered by default and (2) an escaping closure becomes a much more meaningful hint. For example, consider a use-after-free static analysis that sees this code:

  func foo(ptr: UnsafeMutablePointer<Int>) {
    bar { ptr[5] = 0 }
    ptr.dealloc()
  }

This analysis is currently blocked by this abstraction unless it knows something specific about 'bar'. If 'bar' marks its argument @noescape, then the analysis knows that this is safe; but if not, the analysis is unlikely to be willing to warn because it's quite likely that 'bar' is just missing an annotation and does not actually execute its closure asynchronously. However, if the polarity is flipped, the analysis wins on both ends: it can prove correctness in many more cases by default, and the cases where the closure actually escapes become much more suspicious, probably enough to warn by default.

John.

···

On Jun 9, 2016, at 4:15 PM, Jordan Rose <jordan_rose@apple.com> wrote:

On Jun 9, 2016, at 16:10, John McCall <rjmccall@apple.com <mailto:rjmccall@apple.com>> wrote:

On Jun 9, 2016, at 3:43 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

I'm against this for library evolution reasons: if someone releases a version of their library that has a non-escaping closure and later discovers it needs to be escaping, they can't change it.

IIRC the counterpoint to this is that people were probably implicitly relying on it being non-escaping already, and that there aren't many cases where you'd want to do this anyway.

Right. APIs are already semantically constrained in how they're allowed to use their closure arguments. Closure arguments inject arbitrary code, with arbitrary data access, into the callee; as a rule, the caller must know how the callee intends to use the closure, or its semantics will be grossly violated. You can't re-implement an existing API that always synchronously sub-invokes a closure to instead call the closure asynchronously or concurrently because it is completely reasonable for the caller to pass a closure that relies on being called synchronously or from at most one thread at once and/or within a fixed range of time. For example, the closure may modify a captured local variable, or it may it use a network connection that will be closed after the API returns. APIs that want to do this sort of thing have to reserve the right to do that (and even then, they may have binary compatibility limitations), in which case it is totally reasonable to expect them to express that in the type.

I don't buy this. If someone publishes an API that executes something on the current thread today and on a background queue tomorrow, that's totally fine if they never promised it would execute on a particular thread. If a client accidentally assumes an implementation detail is part of the interface, that's their fault, and always has been…though the library author might decide to continue supporting their use in the future in the interest of not making waves.


(Joe Groff) #14

This hardly ever goes well. There's a pretty long blood trail of blog posts about this kind of attempted evolution in Javascript land; http://blog.ometer.com/2011/07/24/callbacks-synchronous-and-asynchronous/ happens to be the one I have on hand this moment.

-Joe

···

On Jun 9, 2016, at 4:15 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

On Jun 9, 2016, at 16:10, John McCall <rjmccall@apple.com> wrote:

On Jun 9, 2016, at 3:43 PM, Jordan Rose via swift-evolution <swift-evolution@swift.org> wrote:

I'm against this for library evolution reasons: if someone releases a version of their library that has a non-escaping closure and later discovers it needs to be escaping, they can't change it.

IIRC the counterpoint to this is that people were probably implicitly relying on it being non-escaping already, and that there aren't many cases where you'd want to do this anyway.

Right. APIs are already semantically constrained in how they're allowed to use their closure arguments. Closure arguments inject arbitrary code, with arbitrary data access, into the callee; as a rule, the caller must know how the callee intends to use the closure, or its semantics will be grossly violated. You can't re-implement an existing API that always synchronously sub-invokes a closure to instead call the closure asynchronously or concurrently because it is completely reasonable for the caller to pass a closure that relies on being called synchronously or from at most one thread at once and/or within a fixed range of time. For example, the closure may modify a captured local variable, or it may it use a network connection that will be closed after the API returns. APIs that want to do this sort of thing have to reserve the right to do that (and even then, they may have binary compatibility limitations), in which case it is totally reasonable to expect them to express that in the type.

I don't buy this. If someone publishes an API that executes something on the current thread today and on a background queue tomorrow, that's totally fine if they never promised it would execute on a particular thread. If a client accidentally assumes an implementation detail is part of the interface, that's their fault, and always has been…though the library author might decide to continue supporting their use in the future in the interest of not making waves.

(Escaping/non-escaping by default doesn't affect this, by the way, but by the same token this argument doesn't really affect escaping/non-escaping by default.)


(Trent Nadeau) #15

When would be a good time to actual put out a PR to the swift-evolution
repo for this proposal? The feedback so far has been very light, but I'm
not sure if that's because everyone's gearing up for WWDC, if there's
little interest, or if it's uncontroversial.

···

On Wed, Jun 8, 2016 at 1:17 PM, John McCall <rjmccall@apple.com> wrote:

> On Jun 7, 2016, at 7:25 PM, Brent Royal-Gordon via swift-evolution < > swift-evolution@swift.org> wrote:
>> @escaping would be part of the parameter type just as @noescape is
today. Your foo(closure:) example wouldn't compile with my proposal, the
same as today if you marked the parameter with @noescape. Non-escaping
function parameters are only allowed to be called. They can't be assigned
to variables.
>
> Okay, that does correct that issue. Although it raises a separate issue:
a bare closure type now means something different in a parameter list than
anywhere else.

@escaping is really a parameter-specific aspect of the outer function
type, not an aspect of the parameter's type. It's just like something like
NS_CONSUMED in Objective-C ARC.

John.

> Are generic types which happen to be functions in some particular use
automatically @escaping? Are typealiases and associated types automatically
@escaping?
>
> Also, if `@escaping` is a part of the parameter list syntax (like
`inout`) instead of the type syntax (like `@autoclosure`), would it make
sense to drop its `@` sign to make them more syntactically similar?
>
> --
> Brent Royal-Gordon
> Architechies
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

--
Trent Nadeau


(Trent Nadeau) #16

Thanks for the very interesting design discussion so far. It's exactly what
I was wanting to see :slight_smile:

I'm going to incorporate some of this in the next draft of the proposal.
I'm also going to add that there should be an asUnsafeEscapingClosure(_:slight_smile:
helper that will convert a default/noescape closure to an escaping one. For
the initial implementation it could use unsafeBitCast(_:to:) to perhaps be
changed later.

···

On Thu, Jun 9, 2016 at 8:29 PM, John McCall <rjmccall@apple.com> wrote:

On Jun 9, 2016, at 4:15 PM, Jordan Rose <jordan_rose@apple.com> wrote:

On Jun 9, 2016, at 16:10, John McCall <rjmccall@apple.com> wrote:

On Jun 9, 2016, at 3:43 PM, Jordan Rose via swift-evolution < > swift-evolution@swift.org> wrote:

I'm against this for library evolution reasons: if someone releases a
version of their library that has a non-escaping closure and later
discovers it needs to be escaping, they can't change it.

IIRC the counterpoint to this is that people were probably implicitly
relying on it being non-escaping already, and that there aren't many cases
where you'd want to do this anyway.

Right. APIs are already semantically constrained in how they're allowed
to use their closure arguments. Closure arguments inject arbitrary code,
with arbitrary data access, into the callee; as a rule, the caller must
know how the callee intends to use the closure, or its semantics will be
grossly violated. You can't re-implement an existing API that always
synchronously sub-invokes a closure to instead call the closure
asynchronously or concurrently because it is completely reasonable for the
caller to pass a closure that relies on being called synchronously or from
at most one thread at once and/or within a fixed range of time. For
example, the closure may modify a captured local variable, or it may it use
a network connection that will be closed after the API returns. APIs that
want to do this sort of thing have to reserve the right to do that (and
even then, they may have binary compatibility limitations), in which case
it is totally reasonable to expect them to express that in the type.

I don't buy this. If someone publishes an API that executes something on
the current thread today and on a background queue tomorrow, that's totally
fine if they never promised it would execute on a particular thread. If a
client accidentally assumes an implementation detail is part of the
interface, that's their fault, and always has been…though the library
author might decide to continue supporting their use in the future in the
interest of not making waves.

Synchronous-but-off-thread is kind of a special case because it's only
observable in very special ways, e.g. thread-local storage and the current
thread ID. Concurrent (e.g. calling an enumeration callback on multiple
threads simultaneously) and asynchronous (even if it comes back to the
current queue) are absolutely something you have to know about as a
caller. It is deeply unreasonable for an API to suddenly start invoking a
closure asynchronously when it hasn't documented that it might do that
(perhaps implicitly by obviously following some well-known pattern, e.g.
calling the closure a completion handler); that would be a huge semantic
and binary-compatibility problem.

Another line of argument: flipping the default is a huge boon to static
analysis because (1) closure execution becomes ordered by default and (2)
an escaping closure becomes a much more meaningful hint. For example,
consider a use-after-free static analysis that sees this code:

  func foo(ptr: UnsafeMutablePointer<Int>) {
    bar { ptr[5] = 0 }
    ptr.dealloc()
  }

This analysis is currently blocked by this abstraction unless it knows
something specific about 'bar'. If 'bar' marks its argument @noescape,
then the analysis knows that this is safe; but if not, the analysis is
unlikely to be willing to warn because it's quite likely that 'bar' is just
missing an annotation and does not actually execute its closure
asynchronously. However, if the polarity is flipped, the analysis wins on
both ends: it can prove correctness in many more cases by default, and the
cases where the closure actually escapes become much more suspicious,
probably enough to warn by default.

John.

--
Trent Nadeau


(Trent Nadeau) #17

I've created the PR for the proposal at
https://github.com/apple/swift-evolution/pull/373.

Thanks again to everyone that has discussed this so far.

···

On Mon, Jun 20, 2016 at 8:18 PM, Joe Groff via swift-evolution < swift-evolution@swift.org> wrote:

> On Jun 9, 2016, at 4:15 PM, Jordan Rose via swift-evolution < > swift-evolution@swift.org> wrote:
>
>>
>> On Jun 9, 2016, at 16:10, John McCall <rjmccall@apple.com> wrote:
>>
>>> On Jun 9, 2016, at 3:43 PM, Jordan Rose via swift-evolution < > swift-evolution@swift.org> wrote:
>>>
>>> I'm against this for library evolution reasons: if someone releases a
version of their library that has a non-escaping closure and later
discovers it needs to be escaping, they can't change it.
>>>
>>> IIRC the counterpoint to this is that people were probably implicitly
relying on it being non-escaping already, and that there aren't many cases
where you'd want to do this anyway.
>>
>> Right. APIs are already semantically constrained in how they're
allowed to use their closure arguments. Closure arguments inject arbitrary
code, with arbitrary data access, into the callee; as a rule, the caller
must know how the callee intends to use the closure, or its semantics will
be grossly violated. You can't re-implement an existing API that always
synchronously sub-invokes a closure to instead call the closure
asynchronously or concurrently because it is completely reasonable for the
caller to pass a closure that relies on being called synchronously or from
at most one thread at once and/or within a fixed range of time. For
example, the closure may modify a captured local variable, or it may it use
a network connection that will be closed after the API returns. APIs that
want to do this sort of thing have to reserve the right to do that (and
even then, they may have binary compatibility limitations), in which case
it is totally reasonable to expect them to express that in the type.
>
> I don't buy this. If someone publishes an API that executes something on
the current thread today and on a background queue tomorrow, that's totally
fine if they never promised it would execute on a particular thread. If a
client accidentally assumes an implementation detail is part of the
interface, that's their fault, and always has been…though the library
author might decide to continue supporting their use in the future in the
interest of not making waves.
>
> (Escaping/non-escaping by default doesn't affect this, by the way, but
by the same token this argument doesn't really affect escaping/non-escaping
by default.)

This hardly ever goes well. There's a pretty long blood trail of blog
posts about this kind of attempted evolution in Javascript land;
http://blog.ometer.com/2011/07/24/callbacks-synchronous-and-asynchronous/
happens to be the one I have on hand this moment.

-Joe
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

--
Trent Nadeau


(John McCall) #18

When would be a good time to actual put out a PR to the swift-evolution repo for this proposal? The feedback so far has been very light, but I'm not sure if that's because everyone's gearing up for WWDC, if there's little interest, or if it's uncontroversial.

The initial feedback phase is intended for discussion which shapes the basic content of a proposal, e.g. exploring the goals, suggesting different ways of achieving those goals, etc. Specifically, it's not designed to determine whether people actually *like* the proposal; that's what formal review is for. In this case, I don't see much room for variation in the basic content: we have to flip the default rule, we have to provide some way to declare a closure as escaping, that way pretty much has to be a parameter attribute, the obvious spelling for that attribute is @escaping, etc. So if you're happy with the content of the proposal, just send the PR and we'll schedule some time to talk about it. That time will obviously not be WWDC week. :slight_smile:

John.

···

On Jun 9, 2016, at 3:17 PM, Trent Nadeau <tanadeau@gmail.com> wrote:

On Wed, Jun 8, 2016 at 1:17 PM, John McCall <rjmccall@apple.com <mailto:rjmccall@apple.com>> wrote:
> On Jun 7, 2016, at 7:25 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
>> @escaping would be part of the parameter type just as @noescape is today. Your foo(closure:) example wouldn't compile with my proposal, the same as today if you marked the parameter with @noescape. Non-escaping function parameters are only allowed to be called. They can't be assigned to variables.
>
> Okay, that does correct that issue. Although it raises a separate issue: a bare closure type now means something different in a parameter list than anywhere else.

@escaping is really a parameter-specific aspect of the outer function type, not an aspect of the parameter's type. It's just like something like NS_CONSUMED in Objective-C ARC.

John.

> Are generic types which happen to be functions in some particular use automatically @escaping? Are typealiases and associated types automatically @escaping?
>
> Also, if `@escaping` is a part of the parameter list syntax (like `inout`) instead of the type syntax (like `@autoclosure`), would it make sense to drop its `@` sign to make them more syntactically similar?
>
> --
> Brent Royal-Gordon
> Architechies
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution@swift.org <mailto:swift-evolution@swift.org>
> https://lists.swift.org/mailman/listinfo/swift-evolution

--
Trent Nadeau