[Draft] @selfsafe: a new way to avoid reference cycles


(Matthew Johnson) #1

# `@selfsafe`: a new way to avoid reference cycles

* Proposal: [SE-NNNN](NNNN-selfsafe.md)
* Authors: [Matthew Johnson](https://github.com/anandabits)
* Review Manager: TBD
* Status: **Awaiting review**

## Introduction

This proposal introduces the `@selfsafe` function argument attribute which together with a `withWeakSelf` property on values of function type. Together these features enable library authors to create APIs can be statically verified to never extend the lifetime of the `self` a function they take may have captured. This is accomplished by allowing the library implementation convert the function to a nearly identical function that is guaraneteed to have a `weak` capture of `self` and be a no-op after `self` is released.

Swift-evolution thread: []()

## Motivation

Accidentally forgeting to use weak references is a common problem and can easily lead to reference cycles. Some APIs are best designed such that users *cannot* extend the lifetime of `self` by escaping a closure that happens to strongly capture `self`. For example, `UIControl.addTarget(_:action:for:) does not retain the target, thereby preventing users from making the mistake of using a closure with an accidental strong reference. We can do something similar in Swift:

// in the library:
func addTarget<T: AnyObject>(_ target: T, action: T -> Int -> Void) {
   // store a weak reference to the target
   // when the action is fired call ref.map{ action($0)(42) }
}

// in user code:
class C {
   init() {
      addTarget(self, action: C.takesInt)
   }
   
   func takesInt(_ i: Int) {}
}

Both the library and the caller have to deal with a lot of details and boilerplate that we would prefer to avoid. The natural design in Swift would be to simply take an action function. Unfortunately if we do that we run into a problem:

// in the library
func addAction(_ f: Int -> Void) {
   // store a strong ref to f, which might include a strong ref to a captured self
   // later when the action is fired call f(42)
}

// in user code
class C {
   init() {
      addAction(takesInt)
      // oops! should have been: addAction{ [weak self] self?.takesInt($0) }
   }
   
   func takesInt(_ i: Int) {}
}

Here the syntax is much nicer, but unfortunately we have unintentionally extended the lifetime of `self`. The burden of ensuring `self` is not captured or captured weakly falls on users of the library.

It would very nice if it were possible to design an API that has weak capture semantics while still acheiving the more concise and Swifty syntax.

## Proposed Solution

This proposal introduces a read-only property on all function types: `withWeakSelf` as well as a `@selfsafe` function argument annotation. (This name for the annotation is a strawman - I would love to hear better ideas)

### `withWeakSelf`

`withWeakSelf` can be imagined as a property declared like the following:

extension T -> Void {
  var withWeakSelf: T -> Void { return // compiler magic }
}
extension T -> U {
  var withWeakSelf: T -> U? { return // compiler magic }
}
extension T -> U? {
  var withWeakSelf: T -> U? { return // compiler magic }
}

It returns a closure that is identical to itself in every respect except four:
1. If the context includes a strong or unowned `self` capture, that is converted to a weak capture.
2. If the function returns a non-`Void`, non-`Optional` type the return type is wrapped in an `Optional`.
3. The function returned by `withWeakSelf` is a no-op after `self` has been released. If the return type is non-`Void`, the function returns `nil` after `self` has been released.
4. Any addtional non-`self` context is released as soon as possible after `self` is released.

From these rules, it follows that `withWeakSelf` can be a no-op unless `self` is captured strong or unowned or the return type is non-`Void` and non-`Optional`.

Important note: any additional context the closure requies beyond `self` continues to be captured as it was originally.

### `@selfsafe`

When a function argument is annotated with `@selfsafe` two rules are in effect for the implementation:
1. The argument *is not* allowed to escape the function.
2. The closure returend by `withWeakSelf` *is* allowed to escape the function.

There should also be a warning if `argument.withWeakSelf` *does not* escape the function in any code paths (because the `@selfsafe` annotation is meaningless if it does not escape).

This allows a library to do something like the following (a simplified version of a hypothetical future method on `UIControl`):

   class UIControl {
   
      var action: () -> Void
   
      func setAction(_ action: @selfsafe () -> Void) {
         self.action = action.withWeakSelf
      }
   }

In the previous example the function declares the argument with the `@selfsafe` annotation. The compiler verifies that the `action` argument does not escape `setAction` and that `action.withWeakSelf` *does* escape. Because these verifications guarantee that the lifetime of `self` cannot be exteneded by calling `setAction`, the compiler can allow users to omit `self` if this function is called with a closure.

class UIViewController {
   func viewDidLoad() {
       // misc setup
       myControl.setAction(doSomething) // great! no problem creating a reference cycle.
       myOtherControl.setAction {
          // `self.doSomethingElse()` is not required.
          // The compiler knows we cannot create a reference cycle here.
          doSomethingElse()
       }
   }
   func doSomething() {}
   func doSomethingElse() {}
}

## Detailed design

TBD - If anyone can think of additional details that need to be spelled out please provide feedback in this thread.

## Source compatibility

No effect. This is a purely additive change.

## Effect on ABI stability

No effect. This is a purely additive change.

## Effect on API resilience

No effect. This is a purely additive change.

## Alternatives considered

I considered various ways of exposing the entire captured context or the captured `self` reference directly as properties of function types, along with access to an unbound version of the function. This turns out to be far more complex than necessary to solve the problem this proposal aims to solve and doesn't enable any interesting use cases I have thought of.


(Daniel Duan) #2

This reminded me of an idea I had long time ago which will have a similar effect: add a way to disable implicit captures for closures. FWIW.

···

On Feb 18, 2017, at 5:24 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

# `@selfsafe`: a new way to avoid reference cycles

* Proposal: [SE-NNNN](NNNN-selfsafe.md)
* Authors: [Matthew Johnson](https://github.com/anandabits)
* Review Manager: TBD
* Status: **Awaiting review**

## Introduction

This proposal introduces the `@selfsafe` function argument attribute which together with a `withWeakSelf` property on values of function type. Together these features enable library authors to create APIs can be statically verified to never extend the lifetime of the `self` a function they take may have captured. This is accomplished by allowing the library implementation convert the function to a nearly identical function that is guaraneteed to have a `weak` capture of `self` and be a no-op after `self` is released.

Swift-evolution thread: []()

## Motivation

Accidentally forgeting to use weak references is a common problem and can easily lead to reference cycles. Some APIs are best designed such that users *cannot* extend the lifetime of `self` by escaping a closure that happens to strongly capture `self`. For example, `UIControl.addTarget(_:action:for:) does not retain the target, thereby preventing users from making the mistake of using a closure with an accidental strong reference. We can do something similar in Swift:

// in the library:
func addTarget<T: AnyObject>(_ target: T, action: T -> Int -> Void) {
  // store a weak reference to the target
  // when the action is fired call ref.map{ action($0)(42) }
}

// in user code:
class C {
  init() {
     addTarget(self, action: C.takesInt)
  }

  func takesInt(_ i: Int) {}
}

Both the library and the caller have to deal with a lot of details and boilerplate that we would prefer to avoid. The natural design in Swift would be to simply take an action function. Unfortunately if we do that we run into a problem:

// in the library
func addAction(_ f: Int -> Void) {
  // store a strong ref to f, which might include a strong ref to a captured self
  // later when the action is fired call f(42)
}

// in user code
class C {
  init() {
     addAction(takesInt)
     // oops! should have been: addAction{ [weak self] self?.takesInt($0) }
  }

  func takesInt(_ i: Int) {}
}

Here the syntax is much nicer, but unfortunately we have unintentionally extended the lifetime of `self`. The burden of ensuring `self` is not captured or captured weakly falls on users of the library.

It would very nice if it were possible to design an API that has weak capture semantics while still acheiving the more concise and Swifty syntax.

## Proposed Solution

This proposal introduces a read-only property on all function types: `withWeakSelf` as well as a `@selfsafe` function argument annotation. (This name for the annotation is a strawman - I would love to hear better ideas)

### `withWeakSelf`

`withWeakSelf` can be imagined as a property declared like the following:

extension T -> Void {
 var withWeakSelf: T -> Void { return // compiler magic }
}
extension T -> U {
 var withWeakSelf: T -> U? { return // compiler magic }
}
extension T -> U? {
 var withWeakSelf: T -> U? { return // compiler magic }
}

It returns a closure that is identical to itself in every respect except four:
1. If the context includes a strong or unowned `self` capture, that is converted to a weak capture.
2. If the function returns a non-`Void`, non-`Optional` type the return type is wrapped in an `Optional`.
3. The function returned by `withWeakSelf` is a no-op after `self` has been released. If the return type is non-`Void`, the function returns `nil` after `self` has been released.
4. Any addtional non-`self` context is released as soon as possible after `self` is released.

From these rules, it follows that `withWeakSelf` can be a no-op unless `self` is captured strong or unowned or the return type is non-`Void` and non-`Optional`.

Important note: any additional context the closure requies beyond `self` continues to be captured as it was originally.

### `@selfsafe`

When a function argument is annotated with `@selfsafe` two rules are in effect for the implementation:
1. The argument *is not* allowed to escape the function.
2. The closure returend by `withWeakSelf` *is* allowed to escape the function.

There should also be a warning if `argument.withWeakSelf` *does not* escape the function in any code paths (because the `@selfsafe` annotation is meaningless if it does not escape).

This allows a library to do something like the following (a simplified version of a hypothetical future method on `UIControl`):

  class UIControl {

     var action: () -> Void

     func setAction(_ action: @selfsafe () -> Void) {
        self.action = action.withWeakSelf
     }
  }

In the previous example the function declares the argument with the `@selfsafe` annotation. The compiler verifies that the `action` argument does not escape `setAction` and that `action.withWeakSelf` *does* escape. Because these verifications guarantee that the lifetime of `self` cannot be exteneded by calling `setAction`, the compiler can allow users to omit `self` if this function is called with a closure.

class UIViewController {
  func viewDidLoad() {
      // misc setup
      myControl.setAction(doSomething) // great! no problem creating a reference cycle.
      myOtherControl.setAction {
         // `self.doSomethingElse()` is not required.
         // The compiler knows we cannot create a reference cycle here.
         doSomethingElse()
      }
  }
  func doSomething() {}
  func doSomethingElse() {}
}

## Detailed design

TBD - If anyone can think of additional details that need to be spelled out please provide feedback in this thread.

## Source compatibility

No effect. This is a purely additive change.

## Effect on ABI stability

No effect. This is a purely additive change.

## Effect on API resilience

No effect. This is a purely additive change.

## Alternatives considered

I considered various ways of exposing the entire captured context or the captured `self` reference directly as properties of function types, along with access to an unbound version of the function. This turns out to be far more complex than necessary to solve the problem this proposal aims to solve and doesn't enable any interesting use cases I have thought of.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Brent Royal-Gordon) #3

Both of these mechanisms are weirdly ad hoc. They involve the callee assuming things about the caller that are not necessarily correct—in particular, that the caller's `self` is going to, directly or indirectly, hold a strong reference to the callee's `self`.

For instance, suppose you've read too many design pattern books, and you're using the Command Pattern:

  class DeleteRecordCommand: Command {
    let record: Record
    
    func execute(with viewController: RecordViewController) {
      let alert = UIAlertController(title: "Really delete?", preferredStyle: .alert)
      
      alert.addAction(UIAlertAction(title: "Delete", style: .destructive) { _ in
        self.record.delete()
        viewController.performSegue(withIdentifier: "Cancel", sender: self)
      })
      alert.addAction(UIAlertAction(title: "Cancel", style: .cancel))
      
      viewController.present(alert)
    }
  }

Now, imagine that the `UIAlertAction` initializer used `@selfsafe`, hoping to prevent incorrect use of `self`. Well, that would be totally wrong in this case—it would weaken `self`, which *needs* to be strong, and leave `viewController` strong, when it's creating a retain cycle. `@selfsafe` didn't prevent a bug—it introduced another one, without any visible sign of it in the code.

*The problem is not `self`.* The problem is that you need to pay attention to memory management in Swift. And that's a tradeoff that's baked into the language.

···

On Feb 18, 2017, at 5:24 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

This proposal introduces the `@selfsafe` function argument attribute which together with a `withWeakSelf` property on values of function type. Together these features enable library authors to create APIs can be statically verified to never extend the lifetime of the `self` a function they take may have captured.

--
Brent Royal-Gordon
Architechies


(Florent Bruneau) #4

Hi Matthew,

I'm not convinced we should come with a new mechanism that would handle the capture of `self` differently from the other captures. I do understand that this would match the way Apple APIs work, but as a general purpose language, this looks weird to add some asymmetry in the language itself. If we want to come with a solution to make memory management easier, it should be applicable to any captured variable of the closures, not just `self`.

Moreover, having the `@selfsafe` attribute defined on the library side seems dangerous to me: it imposes a custom reference counting mechanism to apply on the user provided closure. This means that, in case the caller code does not match the reference counting pattern the library writer had in mind, the caller might require a strong reference on `self` (but maybe a weak reference on another captured variable) and the library would just become unusable at this point.

As soon as you receive a function as argument, I cannot see any good reason (but there may be some I missed) to make any assumption about what that function (maybe except about purity): the function might be or not be a closure, might capture or not capture variables and among those variables, there may be `self` or any other ref counted object. If you start making assumptions about the function you received, this might mean you want to receive an object that match a particular protocol, in which case you control the reference counting you apply on the received object (I do understand you cannot change Apple APIs, but you can build your own extension that will provide the safer API that would receive the object and do the weak capture itself).

···

Le 19 févr. 2017 à 02:24, Matthew Johnson via swift-evolution <swift-evolution@swift.org> a écrit :

# `@selfsafe`: a new way to avoid reference cycles

* Proposal: [SE-NNNN](NNNN-selfsafe.md)
* Authors: [Matthew Johnson](https://github.com/anandabits)
* Review Manager: TBD
* Status: **Awaiting review**

## Introduction

This proposal introduces the `@selfsafe` function argument attribute which together with a `withWeakSelf` property on values of function type. Together these features enable library authors to create APIs can be statically verified to never extend the lifetime of the `self` a function they take may have captured. This is accomplished by allowing the library implementation convert the function to a nearly identical function that is guaraneteed to have a `weak` capture of `self` and be a no-op after `self` is released.

Swift-evolution thread: []()

## Motivation

Accidentally forgeting to use weak references is a common problem and can easily lead to reference cycles. Some APIs are best designed such that users *cannot* extend the lifetime of `self` by escaping a closure that happens to strongly capture `self`. For example, `UIControl.addTarget(_:action:for:) does not retain the target, thereby preventing users from making the mistake of using a closure with an accidental strong reference. We can do something similar in Swift:

// in the library:
func addTarget<T: AnyObject>(_ target: T, action: T -> Int -> Void) {
  // store a weak reference to the target
  // when the action is fired call ref.map{ action($0)(42) }
}

// in user code:
class C {
  init() {
     addTarget(self, action: C.takesInt)
  }

  func takesInt(_ i: Int) {}
}

Both the library and the caller have to deal with a lot of details and boilerplate that we would prefer to avoid. The natural design in Swift would be to simply take an action function. Unfortunately if we do that we run into a problem:

// in the library
func addAction(_ f: Int -> Void) {
  // store a strong ref to f, which might include a strong ref to a captured self
  // later when the action is fired call f(42)
}

// in user code
class C {
  init() {
     addAction(takesInt)
     // oops! should have been: addAction{ [weak self] self?.takesInt($0) }
  }

  func takesInt(_ i: Int) {}
}

Here the syntax is much nicer, but unfortunately we have unintentionally extended the lifetime of `self`. The burden of ensuring `self` is not captured or captured weakly falls on users of the library.

It would very nice if it were possible to design an API that has weak capture semantics while still acheiving the more concise and Swifty syntax.

## Proposed Solution

This proposal introduces a read-only property on all function types: `withWeakSelf` as well as a `@selfsafe` function argument annotation. (This name for the annotation is a strawman - I would love to hear better ideas)

### `withWeakSelf`

`withWeakSelf` can be imagined as a property declared like the following:

extension T -> Void {
 var withWeakSelf: T -> Void { return // compiler magic }
}
extension T -> U {
 var withWeakSelf: T -> U? { return // compiler magic }
}
extension T -> U? {
 var withWeakSelf: T -> U? { return // compiler magic }
}

It returns a closure that is identical to itself in every respect except four:
1. If the context includes a strong or unowned `self` capture, that is converted to a weak capture.
2. If the function returns a non-`Void`, non-`Optional` type the return type is wrapped in an `Optional`.
3. The function returned by `withWeakSelf` is a no-op after `self` has been released. If the return type is non-`Void`, the function returns `nil` after `self` has been released.
4. Any addtional non-`self` context is released as soon as possible after `self` is released.

From these rules, it follows that `withWeakSelf` can be a no-op unless `self` is captured strong or unowned or the return type is non-`Void` and non-`Optional`.

Important note: any additional context the closure requies beyond `self` continues to be captured as it was originally.

### `@selfsafe`

When a function argument is annotated with `@selfsafe` two rules are in effect for the implementation:
1. The argument *is not* allowed to escape the function.
2. The closure returend by `withWeakSelf` *is* allowed to escape the function.

There should also be a warning if `argument.withWeakSelf` *does not* escape the function in any code paths (because the `@selfsafe` annotation is meaningless if it does not escape).

This allows a library to do something like the following (a simplified version of a hypothetical future method on `UIControl`):

  class UIControl {

     var action: () -> Void

     func setAction(_ action: @selfsafe () -> Void) {
        self.action = action.withWeakSelf
     }
  }

In the previous example the function declares the argument with the `@selfsafe` annotation. The compiler verifies that the `action` argument does not escape `setAction` and that `action.withWeakSelf` *does* escape. Because these verifications guarantee that the lifetime of `self` cannot be exteneded by calling `setAction`, the compiler can allow users to omit `self` if this function is called with a closure.

class UIViewController {
  func viewDidLoad() {
      // misc setup
      myControl.setAction(doSomething) // great! no problem creating a reference cycle.
      myOtherControl.setAction {
         // `self.doSomethingElse()` is not required.
         // The compiler knows we cannot create a reference cycle here.
         doSomethingElse()
      }
  }
  func doSomething() {}
  func doSomethingElse() {}
}

## Detailed design

TBD - If anyone can think of additional details that need to be spelled out please provide feedback in this thread.

## Source compatibility

No effect. This is a purely additive change.

## Effect on ABI stability

No effect. This is a purely additive change.

## Effect on API resilience

No effect. This is a purely additive change.

## Alternatives considered

I considered various ways of exposing the entire captured context or the captured `self` reference directly as properties of function types, along with access to an unbound version of the function. This turns out to be far more complex than necessary to solve the problem this proposal aims to solve and doesn't enable any interesting use cases I have thought of.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Derrick Ho) #5

What wrong with [unowned self]

···

On Sat, Feb 18, 2017 at 11:01 PM Daniel Duan via swift-evolution < swift-evolution@swift.org> wrote:

This reminded me of an idea I had long time ago which will have a similar
effect: add a way to disable implicit captures for closures. FWIW.

> On Feb 18, 2017, at 5:24 PM, Matthew Johnson via swift-evolution < > swift-evolution@swift.org> wrote:
>
> # `@selfsafe`: a new way to avoid reference cycles
>
> * Proposal: [SE-NNNN](NNNN-selfsafe.md)
> * Authors: [Matthew Johnson](https://github.com/anandabits)
> * Review Manager: TBD
> * Status: **Awaiting review**
>
> ## Introduction
>
> This proposal introduces the `@selfsafe` function argument attribute
which together with a `withWeakSelf` property on values of function type.
Together these features enable library authors to create APIs can be
statically verified to never extend the lifetime of the `self` a function
they take may have captured. This is accomplished by allowing the library
implementation convert the function to a nearly identical function that is
guaraneteed to have a `weak` capture of `self` and be a no-op after `self`
is released.
>
> Swift-evolution thread: []()
>
> ## Motivation
>
> Accidentally forgeting to use weak references is a common problem and
can easily lead to reference cycles. Some APIs are best designed such that
users *cannot* extend the lifetime of `self` by escaping a closure that
happens to strongly capture `self`. For example,
`UIControl.addTarget(_:action:for:) does not retain the target, thereby
preventing users from making the mistake of using a closure with an
accidental strong reference. We can do something similar in Swift:
>
> ```swift
> // in the library:
> func addTarget<T: AnyObject>(_ target: T, action: T -> Int -> Void) {
> // store a weak reference to the target
> // when the action is fired call ref.map{ action($0)(42) }
> }
>
> // in user code:
> class C {
> init() {
> addTarget(self, action: C.takesInt)
> }
>
> func takesInt(_ i: Int) {}
> }
> ```
>
> Both the library and the caller have to deal with a lot of details and
boilerplate that we would prefer to avoid. The natural design in Swift
would be to simply take an action function. Unfortunately if we do that we
run into a problem:
>
> ```swift
> // in the library
> func addAction(_ f: Int -> Void) {
> // store a strong ref to f, which might include a strong ref to a
captured self
> // later when the action is fired call f(42)
> }
>
> // in user code
> class C {
> init() {
> addAction(takesInt)
> // oops! should have been: addAction{ [weak self]
self?.takesInt($0) }
> }
>
> func takesInt(_ i: Int) {}
> }
> ```
>
> Here the syntax is much nicer, but unfortunately we have unintentionally
extended the lifetime of `self`. The burden of ensuring `self` is not
captured or captured weakly falls on users of the library.
>
> It would very nice if it were possible to design an API that has weak
capture semantics while still acheiving the more concise and Swifty syntax.
>
> ## Proposed Solution
>
> This proposal introduces a read-only property on all function types:
`withWeakSelf` as well as a `@selfsafe` function argument annotation.
(This name for the annotation is a strawman - I would love to hear better
ideas)
>
> ### `withWeakSelf`
>
> `withWeakSelf` can be imagined as a property declared like the following:
>
> ```swift
> extension T -> Void {
> var withWeakSelf: T -> Void { return // compiler magic }
> }
> extension T -> U {
> var withWeakSelf: T -> U? { return // compiler magic }
> }
> extension T -> U? {
> var withWeakSelf: T -> U? { return // compiler magic }
> }
> ```
>
> It returns a closure that is identical to itself in every respect except
four:
> 1. If the context includes a strong or unowned `self` capture, that is
converted to a weak capture.
> 2. If the function returns a non-`Void`, non-`Optional` type the return
type is wrapped in an `Optional`.
> 3. The function returned by `withWeakSelf` is a no-op after `self` has
been released. If the return type is non-`Void`, the function returns
`nil` after `self` has been released.
> 4. Any addtional non-`self` context is released as soon as possible
after `self` is released.
>
> From these rules, it follows that `withWeakSelf` can be a no-op unless
`self` is captured strong or unowned or the return type is non-`Void` and
non-`Optional`.
>
> Important note: any additional context the closure requies beyond `self`
continues to be captured as it was originally.
>
> ### `@selfsafe`
>
> When a function argument is annotated with `@selfsafe` two rules are in
effect for the implementation:
> 1. The argument *is not* allowed to escape the function.
> 2. The closure returend by `withWeakSelf` *is* allowed to escape the
function.
>
> There should also be a warning if `argument.withWeakSelf` *does not*
escape the function in any code paths (because the `@selfsafe` annotation
is meaningless if it does not escape).
>
> This allows a library to do something like the following (a simplified
version of a hypothetical future method on `UIControl`):
>
> ```swift
> class UIControl {
>
> var action: () -> Void
>
> func setAction(_ action: @selfsafe () -> Void) {
> self.action = action.withWeakSelf
> }
> }
> ```
>
> In the previous example the function declares the argument with the
`@selfsafe` annotation. The compiler verifies that the `action` argument
does not escape `setAction` and that `action.withWeakSelf` *does* escape.
Because these verifications guarantee that the lifetime of `self` cannot be
exteneded by calling `setAction`, the compiler can allow users to omit
`self` if this function is called with a closure.
>
> ```swift
> class UIViewController {
> func viewDidLoad() {
> // misc setup
> myControl.setAction(doSomething) // great! no problem creating a
reference cycle.
> myOtherControl.setAction {
> // `self.doSomethingElse()` is not required.
> // The compiler knows we cannot create a reference cycle here.
> doSomethingElse()
> }
> }
> func doSomething() {}
> func doSomethingElse() {}
> }
> ```
>
> ## Detailed design
>
> TBD - If anyone can think of additional details that need to be spelled
out please provide feedback in this thread.
>
> ## Source compatibility
>
> No effect. This is a purely additive change.
>
> ## Effect on ABI stability
>
> No effect. This is a purely additive change.
>
> ## Effect on API resilience
>
> No effect. This is a purely additive change.
>
>
> ## Alternatives considered
>
> I considered various ways of exposing the entire captured context or the
captured `self` reference directly as properties of function types, along
with access to an unbound version of the function. This turns out to be
far more complex than necessary to solve the problem this proposal aims to
solve and doesn't enable any interesting use cases I have thought of.
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Matthew Johnson) #6

This proposal introduces the `@selfsafe` function argument attribute which together with a `withWeakSelf` property on values of function type. Together these features enable library authors to create APIs can be statically verified to never extend the lifetime of the `self` a function they take may have captured.

Both of these mechanisms are weirdly ad hoc. They involve the callee assuming things about the caller that are not necessarily correct—in particular, that the caller's `self` is going to, directly or indirectly, hold a strong reference to the callee's `self`.

After I sent this draft last night I realized we can eliminate the `withWeakSelf` property on functions and just have the compiler perform the conversion when the argument is passed. This just requires restricting the `@safeself` annotation to `Void` and `Optional` returning functions. This will prevent unintended uses of that property and keep more of the implementation details hidden, so let's just focus on `@selfsafe`. (Also note: this name is a straw man, I would like to find something better)

Here's the logic that resulted in this idea:

1. Swift *already* acknowledges that it is far easier to create a reference cycle through captured strong references to `self` than any other way. This is why you have to explicitly say `self.` in escaping closures.

2. There are *already* APIs which are designed to capture an object weak and then call a method on it if it's still around when an event occurs. In fact this has been a trend in Apple's newer APIs. Users are able to learn the semantics of these APIs without a problem. In fact, users like the, because they solve a real problem by ensuring that3. object lifetime is not extended when using them.

3. Swift libraries don't tend to design APIs with weak callback semantics, probably because they are currently syntactically heavy for both users and libraries. This is true even when weak callback semantics would be beneficial for users and help prevent leaks (which can lead to crashes and other badly behaved apps).

4. There have been several ideas proposed to make weak capture easier to do on the call side but they haven't gone anywhere. The syntactic savings aren't that significant for callers and the burden is still on callers to get it right.

All of these facts indicate to me that it should be possible to design an API with weak callback semantics that does not impose a significant syntactic burden on callers. I am not as concerned about a possible syntactic burden on libraries, but clients should be able to pass `myMethod` or `{ myMethod(); myOtherMethod() }` and have it "just work". A small decoration like the `&` required for `inout` would be acceptable, but not much more than that.

For instance, suppose you've read too many design pattern books, and you're using the Command Pattern:

   class DeleteRecordCommand: Command {
       let record: Record
       
       func execute(with viewController: RecordViewController) {
           let alert = UIAlertController(title: "Really delete?", preferredStyle: .alert)
           
           alert.addAction(UIAlertAction(title: "Delete", style: .destructive) { _ in
               self.record.delete()
               viewController.performSegue(withIdentifier: "Cancel", sender: self)
           })
           alert.addAction(UIAlertAction(title: "Cancel", style: .cancel))
           
           viewController.present(alert)
       }
   }

Now, imagine that the `UIAlertAction` initializer used `@selfsafe`, hoping to prevent incorrect use of `self`. Well, that would be totally wrong in this case—it would weaken `self`, which *needs* to be strong, and leave `viewController` strong, when it's creating a retain cycle. `@selfsafe` didn't prevent a bug—it introduced another one, without any visible sign of it in the code.

You bring up a very important point. I was going to include a section on "what if users actually need a strong reference" and simply forgot to add that before I sent it out.

If users actually need a strong reference there are ways to handle that. First, if it is likely that a user might want self to be captured strong the API might choose to not use `@selfsafe`. If they *do* choose to use it the could also add an overload that does not use it and is disambiguated using some other means (base name or argument label). Finally, users can always add an extra closure wrapper {{ self.myMethod() }} to bypass the API's weak callback semantics. Here, `self` is captured by the inner closure rather than the the outer closure and the API only converts strong `self` references in the outer closure.

All of this aside, I think you make a great point that in the design I proposed it is possible for users to intend a strong reference and have it converted without any sign of that at the call site. What if we required a sigil at the call site for `selfsafe` like we do for `inout`? This would make it clear at the call site that the lifetime of `self` is not extended by the closure and it is only invoked if `self` is still around when an event occurs. I'm going to try to think of a good sigil to use for this.

*The problem is not `self`.* The problem is that you need to pay attention to memory management in Swift. And that's a tradeoff that's baked into the language.

Yes, and it's also something that we strive to handle automatically where possible and make as easy as possible to do correctly. I am *very* careful about capturing strong references to `self` and still make an accidental mistake every once in a while.

I am very confident that enabling the class of weak callback APIs I have described is a good idea (regardless of how the end result is designed). This would be a powerful tool in our tool belts and help make correct resource lifetime management easier than it is today.

···

Sent from my iPad
On Feb 19, 2017, at 2:15 AM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Feb 18, 2017, at 5:24 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

--
Brent Royal-Gordon
Architechies


(David Hart) #7

I must admit I find this proposal very weird.

• it only handles strong references to self. While the APIs we work with mostly direct us towards accidental references cycles including self in closures, I would want a compiler feature to be much more generic.
• I'm not a fan of "magically" fixing reference cycles in certain APIs. It creates special cases, inconsistencies this will create will make it that much worse for newcomers to understand reference cycles and for advanced users to reasons about reference cycles (always having to check the API to check if it handles this special magic).

···

On 19 Feb 2017, at 09:15, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 18, 2017, at 5:24 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

This proposal introduces the `@selfsafe` function argument attribute which together with a `withWeakSelf` property on values of function type. Together these features enable library authors to create APIs can be statically verified to never extend the lifetime of the `self` a function they take may have captured.

Both of these mechanisms are weirdly ad hoc. They involve the callee assuming things about the caller that are not necessarily correct—in particular, that the caller's `self` is going to, directly or indirectly, hold a strong reference to the callee's `self`.

For instance, suppose you've read too many design pattern books, and you're using the Command Pattern:

   class DeleteRecordCommand: Command {
       let record: Record
       
       func execute(with viewController: RecordViewController) {
           let alert = UIAlertController(title: "Really delete?", preferredStyle: .alert)
           
           alert.addAction(UIAlertAction(title: "Delete", style: .destructive) { _ in
               self.record.delete()
               viewController.performSegue(withIdentifier: "Cancel", sender: self)
           })
           alert.addAction(UIAlertAction(title: "Cancel", style: .cancel))
           
           viewController.present(alert)
       }
   }

Now, imagine that the `UIAlertAction` initializer used `@selfsafe`, hoping to prevent incorrect use of `self`. Well, that would be totally wrong in this case—it would weaken `self`, which *needs* to be strong, and leave `viewController` strong, when it's creating a retain cycle. `@selfsafe` didn't prevent a bug—it introduced another one, without any visible sign of it in the code.

*The problem is not `self`.* The problem is that you need to pay attention to memory management in Swift. And that's a tradeoff that's baked into the language.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Matthew Johnson) #8

This places the burden on users. The library is not able to offer a guarantee.

···

Sent from my iPad

On Feb 19, 2017, at 1:20 AM, Derrick Ho <wh1pch81n@gmail.com> wrote:

What wrong with [unowned self]

On Sat, Feb 18, 2017 at 11:01 PM Daniel Duan via swift-evolution <swift-evolution@swift.org> wrote:
This reminded me of an idea I had long time ago which will have a similar effect: add a way to disable implicit captures for closures. FWIW.

> On Feb 18, 2017, at 5:24 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:
>
> # `@selfsafe`: a new way to avoid reference cycles
>
> * Proposal: [SE-NNNN](NNNN-selfsafe.md)
> * Authors: [Matthew Johnson](https://github.com/anandabits)
> * Review Manager: TBD
> * Status: **Awaiting review**
>
> ## Introduction
>
> This proposal introduces the `@selfsafe` function argument attribute which together with a `withWeakSelf` property on values of function type. Together these features enable library authors to create APIs can be statically verified to never extend the lifetime of the `self` a function they take may have captured. This is accomplished by allowing the library implementation convert the function to a nearly identical function that is guaraneteed to have a `weak` capture of `self` and be a no-op after `self` is released.
>
> Swift-evolution thread: []()
>
> ## Motivation
>
> Accidentally forgeting to use weak references is a common problem and can easily lead to reference cycles. Some APIs are best designed such that users *cannot* extend the lifetime of `self` by escaping a closure that happens to strongly capture `self`. For example, `UIControl.addTarget(_:action:for:) does not retain the target, thereby preventing users from making the mistake of using a closure with an accidental strong reference. We can do something similar in Swift:
>
> ```swift
> // in the library:
> func addTarget<T: AnyObject>(_ target: T, action: T -> Int -> Void) {
> // store a weak reference to the target
> // when the action is fired call ref.map{ action($0)(42) }
> }
>
> // in user code:
> class C {
> init() {
> addTarget(self, action: C.takesInt)
> }
>
> func takesInt(_ i: Int) {}
> }
> ```
>
> Both the library and the caller have to deal with a lot of details and boilerplate that we would prefer to avoid. The natural design in Swift would be to simply take an action function. Unfortunately if we do that we run into a problem:
>
> ```swift
> // in the library
> func addAction(_ f: Int -> Void) {
> // store a strong ref to f, which might include a strong ref to a captured self
> // later when the action is fired call f(42)
> }
>
> // in user code
> class C {
> init() {
> addAction(takesInt)
> // oops! should have been: addAction{ [weak self] self?.takesInt($0) }
> }
>
> func takesInt(_ i: Int) {}
> }
> ```
>
> Here the syntax is much nicer, but unfortunately we have unintentionally extended the lifetime of `self`. The burden of ensuring `self` is not captured or captured weakly falls on users of the library.
>
> It would very nice if it were possible to design an API that has weak capture semantics while still acheiving the more concise and Swifty syntax.
>
> ## Proposed Solution
>
> This proposal introduces a read-only property on all function types: `withWeakSelf` as well as a `@selfsafe` function argument annotation. (This name for the annotation is a strawman - I would love to hear better ideas)
>
> ### `withWeakSelf`
>
> `withWeakSelf` can be imagined as a property declared like the following:
>
> ```swift
> extension T -> Void {
> var withWeakSelf: T -> Void { return // compiler magic }
> }
> extension T -> U {
> var withWeakSelf: T -> U? { return // compiler magic }
> }
> extension T -> U? {
> var withWeakSelf: T -> U? { return // compiler magic }
> }
> ```
>
> It returns a closure that is identical to itself in every respect except four:
> 1. If the context includes a strong or unowned `self` capture, that is converted to a weak capture.
> 2. If the function returns a non-`Void`, non-`Optional` type the return type is wrapped in an `Optional`.
> 3. The function returned by `withWeakSelf` is a no-op after `self` has been released. If the return type is non-`Void`, the function returns `nil` after `self` has been released.
> 4. Any addtional non-`self` context is released as soon as possible after `self` is released.
>
> From these rules, it follows that `withWeakSelf` can be a no-op unless `self` is captured strong or unowned or the return type is non-`Void` and non-`Optional`.
>
> Important note: any additional context the closure requies beyond `self` continues to be captured as it was originally.
>
> ### `@selfsafe`
>
> When a function argument is annotated with `@selfsafe` two rules are in effect for the implementation:
> 1. The argument *is not* allowed to escape the function.
> 2. The closure returend by `withWeakSelf` *is* allowed to escape the function.
>
> There should also be a warning if `argument.withWeakSelf` *does not* escape the function in any code paths (because the `@selfsafe` annotation is meaningless if it does not escape).
>
> This allows a library to do something like the following (a simplified version of a hypothetical future method on `UIControl`):
>
> ```swift
> class UIControl {
>
> var action: () -> Void
>
> func setAction(_ action: @selfsafe () -> Void) {
> self.action = action.withWeakSelf
> }
> }
> ```
>
> In the previous example the function declares the argument with the `@selfsafe` annotation. The compiler verifies that the `action` argument does not escape `setAction` and that `action.withWeakSelf` *does* escape. Because these verifications guarantee that the lifetime of `self` cannot be exteneded by calling `setAction`, the compiler can allow users to omit `self` if this function is called with a closure.
>
> ```swift
> class UIViewController {
> func viewDidLoad() {
> // misc setup
> myControl.setAction(doSomething) // great! no problem creating a reference cycle.
> myOtherControl.setAction {
> // `self.doSomethingElse()` is not required.
> // The compiler knows we cannot create a reference cycle here.
> doSomethingElse()
> }
> }
> func doSomething() {}
> func doSomethingElse() {}
> }
> ```
>
> ## Detailed design
>
> TBD - If anyone can think of additional details that need to be spelled out please provide feedback in this thread.
>
> ## Source compatibility
>
> No effect. This is a purely additive change.
>
> ## Effect on ABI stability
>
> No effect. This is a purely additive change.
>
> ## Effect on API resilience
>
> No effect. This is a purely additive change.
>
>
> ## Alternatives considered
>
> I considered various ways of exposing the entire captured context or the captured `self` reference directly as properties of function types, along with access to an unbound version of the function. This turns out to be far more complex than necessary to solve the problem this proposal aims to solve and doesn't enable any interesting use cases I have thought of.
>
> _______________________________________________
> swift-evolution mailing list
> swift-evolution@swift.org
> https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Matthew Johnson) #9

I must admit I find this proposal very weird.

I am going to remove the `withWeakSelf` property. This conversion can be handled behind the scenes as an implementation detail of passing an argument to a parameter that includes the `selfsafe` annotation. Hopefully it feels less weird after I do that.

• it only handles strong references to self. While the APIs we work with mostly direct us towards accidental references cycles including self in closures, I would want a compiler feature to be much more generic.

How would such a feature work? We already have special case handling for strong self capture by escaping closures in the language. I don't know of a way to do it more generally in a way that a library could meaningfully take advantage of. That said, I'm interested in hearing ideas if you have them.

• I'm not a fan of "magically" fixing reference cycles in certain APIs. It creates special cases, inconsistencies this will create will make it that much worse for newcomers to understand reference cycles and for advanced users to reasons about reference cycles (always having to check the API to check if it handles this special magic).

How do you feel about the Apple frameworks moving to weak callback semantics for many of their APIs? Everyone I've talked to *likes* this. Let's enable them to be implemented and used in a way that is more at home in Swift.

Would you feel better if we used a sigil at call sites like we do with `inout`? This would make it clear in the calling code that this semantic is in effect and would also make it clear when it is not in effect. If users tried to use an API incorrectly they would get a compiler error.

···

Sent from my iPad

On Feb 19, 2017, at 7:41 AM, David Hart <david@hartbit.com> wrote:

On 19 Feb 2017, at 09:15, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 18, 2017, at 5:24 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

This proposal introduces the `@selfsafe` function argument attribute which together with a `withWeakSelf` property on values of function type. Together these features enable library authors to create APIs can be statically verified to never extend the lifetime of the `self` a function they take may have captured.

Both of these mechanisms are weirdly ad hoc. They involve the callee assuming things about the caller that are not necessarily correct—in particular, that the caller's `self` is going to, directly or indirectly, hold a strong reference to the callee's `self`.

For instance, suppose you've read too many design pattern books, and you're using the Command Pattern:

  class DeleteRecordCommand: Command {
      let record: Record

      func execute(with viewController: RecordViewController) {
          let alert = UIAlertController(title: "Really delete?", preferredStyle: .alert)

          alert.addAction(UIAlertAction(title: "Delete", style: .destructive) { _ in
              self.record.delete()
              viewController.performSegue(withIdentifier: "Cancel", sender: self)
          })
          alert.addAction(UIAlertAction(title: "Cancel", style: .cancel))

          viewController.present(alert)
      }
  }

Now, imagine that the `UIAlertAction` initializer used `@selfsafe`, hoping to prevent incorrect use of `self`. Well, that would be totally wrong in this case—it would weaken `self`, which *needs* to be strong, and leave `viewController` strong, when it's creating a retain cycle. `@selfsafe` didn't prevent a bug—it introduced another one, without any visible sign of it in the code.

*The problem is not `self`.* The problem is that you need to pay attention to memory management in Swift. And that's a tradeoff that's baked into the language.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Brent Royal-Gordon) #10

1. Swift *already* acknowledges that it is far easier to create a reference cycle through captured strong references to `self` than any other way. This is why you have to explicitly say `self.` in escaping closures.

The reason you have to explicitly say `self` is that `self` is the only variable you can implicitly reference. That's disabled in closures, so it's on the same footing as other variables.

Other than that, the only reason `self` is more likely to cause a loop is that, in common Cocoa patterns, you're more likely to be handing the closure to something owned by `self`. But that's not *necessarily* the case, it just happens to be true in many cases.

2. There are *already* APIs which are designed to capture an object weak and then call a method on it if it's still around when an event occurs. In fact this has been a trend in Apple's newer APIs. Users are able to learn the semantics of these APIs without a problem. In fact, users like the, because they solve a real problem by ensuring that3. object lifetime is not extended when using them.

3. Swift libraries don't tend to design APIs with weak callback semantics, probably because they are currently syntactically heavy for both users and libraries. This is true even when weak callback semantics would be beneficial for users and help prevent leaks (which can lead to crashes and other badly behaved apps).

4. There have been several ideas proposed to make weak capture easier to do on the call side but they haven't gone anywhere. The syntactic savings aren't that significant for callers and the burden is still on callers to get it right.

I definitely agree this is a problem, but again, that doesn't mean `self` is the issue here. I think this is mainly because (a) the pattern isn't taught, and (b) there are some minor speed bumps in current Swift (like the inability to shadow `self` in a closure parameter list).

If users actually need a strong reference there are ways to handle that. First, if it is likely that a user might want self to be captured strong the API might choose to not use `@selfsafe`.

This doesn't work. The API designer doesn't know what's at the call site, and the user can't modify the API to suit the use.

If they *do* choose to use it the could also add an overload that does not use it and is disambiguated using some other means (base name or argument label).

That seems like an ugly way to design APIs.

Finally, users can always add an extra closure wrapper {{ self.myMethod() }} to bypass the API's weak callback semantics. Here, `self` is captured by the inner closure rather than the the outer closure and the API only converts strong `self` references in the outer closure.

That wouldn't work. The outer closure captures `self` from the context, and the inner closure captures `self` from the outer closure. The outer closure's capture would still be weak.

I really think this focus on `self` is counterproductive. I'm thinking we might be able to address this in a different way.

Let's look at your code sample:

  addAction(takesInt)

With the implicit `self`s made explicit, that would instead be:

  self.addAction(
    self.takesInt
  )

Now let's look at mine:

  alert.addAction(UIAlertAction(title: "Delete", style: .destructive) { _ in
    …
    viewController.performSegue(withIdentifier: "Cancel", sender: self)
  })
  …
    
  viewController.present(alert)

These have a similar pattern: The closure ends up being passed to a method on one of the variables it captures. Your example:

  self.addAction(
  ^ Closure gets passed to method on `self`
    self.takesInt
    ^ Closure captures `self`
  )

Mine:

  alert.addAction(UIAlertAction(title: "Delete", style: .destructive) { _ in
  ^ Closure passed to method on `alert`
    …
    viewController.performSegue(withIdentifier: "Cancel", sender: self)
    ^ Closure captures `viewController`
  })
  …
    
  viewController.present(alert)
  ^ `alert` passed to `viewController`

This seems like something the compiler—or perhaps a static analyzer?—could warn about. And it wouldn't target `self` specifically, or silently change the meaning of your code.

···

On Feb 19, 2017, at 6:45 AM, Matthew Johnson <matthew@anandabits.com> wrote:

--
Brent Royal-Gordon
Architechies


(David Hart) #11

Thanks for reply Mathew. Here are some more comments:

Sent from my iPad

I must admit I find this proposal very weird.

I am going to remove the `withWeakSelf` property. This conversion can be handled behind the scenes as an implementation detail of passing an argument to a parameter that includes the `selfsafe` annotation. Hopefully it feels less weird after I do that.

• it only handles strong references to self. While the APIs we work with mostly direct us towards accidental references cycles including self in closures, I would want a compiler feature to be much more generic.

How would such a feature work? We already have special case handling for strong self capture by escaping closures in the language. I don't know of a way to do it more generally in a way that a library could meaningfully take advantage of. That said, I'm interested in hearing ideas if you have them.

Are you talking about the fact that the language forces us to explicitly prefix member access in escaped closures by 'self.'? If that is the case, it's true that it is a special case handling, but only a syntax feature to make the strong reference stand out, not a semantic difference. And it's the semantic difference that you propose to introduce with 'self' and not with other strong captures that worry me.

• I'm not a fan of "magically" fixing reference cycles in certain APIs. It creates special cases, inconsistencies this will create will make it that much worse for newcomers to understand reference cycles and for advanced users to reasons about reference cycles (always having to check the API to check if it handles this special magic).

How do you feel about the Apple frameworks moving to weak callback semantics for many of their APIs? Everyone I've talked to *likes* this. Let's enable them to be implemented and used in a way that is more at home in Swift.

Would you feel better if we used a sigil at call sites like we do with `inout`? This would make it clear in the calling code that this semantic is in effect and would also make it clear when it is not in effect. If users tried to use an API incorrectly they would get a compiler error.

It might help, but I would still have reserve on the balance between the the usefulness this provides on one side against the cognitive weigh of the feature by introducing a semantic inconsistency between references to self and strong references to other variables which can also create reference cycles.

···

On 19 Feb 2017, at 15:51, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 19, 2017, at 7:41 AM, David Hart <david@hartbit.com> wrote:

On 19 Feb 2017, at 09:15, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 18, 2017, at 5:24 PM, Matthew Johnson via swift-evolution <swift-evolution@swift.org> wrote:

This proposal introduces the `@selfsafe` function argument attribute which together with a `withWeakSelf` property on values of function type. Together these features enable library authors to create APIs can be statically verified to never extend the lifetime of the `self` a function they take may have captured.

Both of these mechanisms are weirdly ad hoc. They involve the callee assuming things about the caller that are not necessarily correct—in particular, that the caller's `self` is going to, directly or indirectly, hold a strong reference to the callee's `self`.

For instance, suppose you've read too many design pattern books, and you're using the Command Pattern:

class DeleteRecordCommand: Command {
     let record: Record

     func execute(with viewController: RecordViewController) {
         let alert = UIAlertController(title: "Really delete?", preferredStyle: .alert)

         alert.addAction(UIAlertAction(title: "Delete", style: .destructive) { _ in
             self.record.delete()
             viewController.performSegue(withIdentifier: "Cancel", sender: self)
         })
         alert.addAction(UIAlertAction(title: "Cancel", style: .cancel))

         viewController.present(alert)
     }
}

Now, imagine that the `UIAlertAction` initializer used `@selfsafe`, hoping to prevent incorrect use of `self`. Well, that would be totally wrong in this case—it would weaken `self`, which *needs* to be strong, and leave `viewController` strong, when it's creating a retain cycle. `@selfsafe` didn't prevent a bug—it introduced another one, without any visible sign of it in the code.

*The problem is not `self`.* The problem is that you need to pay attention to memory management in Swift. And that's a tradeoff that's baked into the language.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Matthew Johnson) #12

1. Swift *already* acknowledges that it is far easier to create a reference cycle through captured strong references to `self` than any other way. This is why you have to explicitly say `self.` in escaping closures.

The reason you have to explicitly say `self` is that `self` is the only variable you can implicitly reference. That's disabled in closures, so it's on the same footing as other variables.

It's only disabled in escaping closures, not all closures. It is disabled there specifically because capturing it can produce a reference cycle. But I agree, while it is by far the most frequent source of cycles it is not the only way to create them and a more general approach is better.

Other than that, the only reason `self` is more likely to cause a loop is that, in common Cocoa patterns, you're more likely to be handing the closure to something owned by `self`. But that's not *necessarily* the case, it just happens to be true in many cases.

Often you hand it to something owned by self, but it's also the case that you often hand it to something not owned by self, but that should not extend the lifetime of self. In that case you don't necessarily create a cycle but you do create what is effectively a leak. In both cases getting a callback after self should have been released can lead to very bad things happening.

The reason self is somewhat unique is that usually when you pass a callback to something the implementation of the callback either modified instance state or calls methods on instance properties. Occasionally you also capture an argument to the function that registers the callback or connect a method on some other object, but those are far less frequent than starting with your own instance state. The majority of the time you do need to capture self and much of the time you don't need to capture any other references. This is (IMO) why self is somewhat special and why API designs like target / action (which only allow "capture" of a single object) work very well.

2. There are *already* APIs which are designed to capture an object weak and then call a method on it if it's still around when an event occurs. In fact this has been a trend in Apple's newer APIs. Users are able to learn the semantics of these APIs without a problem. In fact, users like the, because they solve a real problem by ensuring that3. object lifetime is not extended when using them.

3. Swift libraries don't tend to design APIs with weak callback semantics, probably because they are currently syntactically heavy for both users and libraries. This is true even when weak callback semantics would be beneficial for users and help prevent leaks (which can lead to crashes and other badly behaved apps).

4. There have been several ideas proposed to make weak capture easier to do on the call side but they haven't gone anywhere. The syntactic savings aren't that significant for callers and the burden is still on callers to get it right.

I definitely agree this is a problem, but again, that doesn't mean `self` is the issue here. I think this is mainly because (a) the pattern isn't taught, and (b) there are some minor speed bumps in current Swift (like the inability to shadow `self` in a closure parameter list).

I agree that self isn't the problem. The biggest problem is that callback APIs that feel Swifty (i.e. just take a function) place the burden of not extending the lifetime of object references on the callee. It is pretty rare to actually want a callback API to extend the lifetime of an object that you need to use when you are callback.

In other words, for this class of API there is a significant limitation in the language that makes correct usage much more difficult than it should be.

I feel pretty good about the design I came up with for guarded closures. This allows us to flip the default capture in a closure from strong to guarded (a new kind of capture) by using the `?` sigil (while still allowing explicit strong capture in the capture list). APIs are allowed to use the `@guarded` argument annotation to require users to provide a guarded closure. This means that they must be used with defaults that make sense for their use case while still giving users an easy way to change those defaults if necessary.

If users actually need a strong reference there are ways to handle that. First, if it is likely that a user might want self to be captured strong the API might choose to not use `@selfsafe`.

This doesn't work. The API designer doesn't know what's at the call site, and the user can't modify the API to suit the use.

If they *do* choose to use it the could also add an overload that does not use it and is disambiguated using some other means (base name or argument label).

That seems like an ugly way to design APIs.

I agree. It was a sign that I did not have the right design yet.

Finally, users can always add an extra closure wrapper {{ self.myMethod() }} to bypass the API's weak callback semantics. Here, `self` is captured by the inner closure rather than the the outer closure and the API only converts strong `self` references in the outer closure.

That wouldn't work. The outer closure captures `self` from the context, and the inner closure captures `self` from the outer closure. The outer closure's capture would still be weak.

I really think this focus on `self` is counterproductive. I'm thinking we might be able to address this in a different way.

I agree. I'm interested in your thoughts on my second stab at this when you have a chance. I started a new thread for it called "guarded closures".

Let's look at your code sample:

   addAction(takesInt)

With the implicit `self`s made explicit, that would instead be:

   self.addAction(
       self.takesInt
   )

No, `addAction` was supposed to be a top level function in a library that was called from user code. I wrote it as a top level function to try and keep the example concise but I see how it was misleading because of that.

···

Sent from my iPad
On Feb 19, 2017, at 6:30 PM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Feb 19, 2017, at 6:45 AM, Matthew Johnson <matthew@anandabits.com> wrote:

Now let's look at mine:

   alert.addAction(UIAlertAction(title: "Delete", style: .destructive) { _ in
       …
       viewController.performSegue(withIdentifier: "Cancel", sender: self)
   })
   …
       
   viewController.present(alert)

These have a similar pattern: The closure ends up being passed to a method on one of the variables it captures. Your example:

   self.addAction(
   ^ Closure gets passed to method on `self`
       self.takesInt
       ^ Closure captures `self`
   )

Mine:

   alert.addAction(UIAlertAction(title: "Delete", style: .destructive) { _ in
   ^ Closure passed to method on `alert`
       …
       viewController.performSegue(withIdentifier: "Cancel", sender: self)
       ^ Closure captures `viewController`
   })
   …
       
   viewController.present(alert)
   ^ `alert` passed to `viewController`

This seems like something the compiler—or perhaps a static analyzer?—could warn about. And it wouldn't target `self` specifically, or silently change the meaning of your code.

--
Brent Royal-Gordon
Architechies


(Brent Royal-Gordon) #13

I don't agree that it shouldn't extend the lifetime of `self`. By default, Swift assumes that if you capture an object in a closure, you want that object to stay alive as long as the closure does.

I see absolutely no reason that this assumption should be different for `self` than it is for any other variable, and I see no reason to believe the caller would have a particularly good idea about this.

···

On Feb 19, 2017, at 7:06 PM, Matthew Johnson <matthew@anandabits.com> wrote:

Often you hand it to something owned by self, but it's also the case that you often hand it to something not owned by self, but that should not extend the lifetime of self.

--
Brent Royal-Gordon
Architechies


(David Hart) #14

Often you hand it to something owned by self, but it's also the case that you often hand it to something not owned by self, but that should not extend the lifetime of self.

I don't agree that it shouldn't extend the lifetime of `self`. By default, Swift assumes that if you capture an object in a closure, you want that object to stay alive as long as the closure does.

I see absolutely no reason that this assumption should be different for `self` than it is for any other variable, and I see no reason to believe the caller would have a particularly good idea about this.

Totally agree. The proposal would complicate reasoning about reference cycles and lifetime of objects by creating special cases which depend on what variable is concerned (self or other) and what the API does.

···

On 20 Feb 2017, at 06:05, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 19, 2017, at 7:06 PM, Matthew Johnson <matthew@anandabits.com> wrote:

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Matthew Johnson) #15

Often you hand it to something owned by self, but it's also the case that you often hand it to something not owned by self, but that should not extend the lifetime of self.

I don't agree that it shouldn't extend the lifetime of `self`. By default, Swift assumes that if you capture an object in a closure, you want that object to stay alive as long as the closure does.

I was not being clear. What I meant is that when you register a method as a callback with something like NSNotificationCenter you usually do not want self to be retained by that callback registration. I agree that this proposal didn’t adequately address making the semantics visible at the call site and it was also too focused explicitly on self. But the basic statement is still true: today we use weak capture manually. We don’t *intend* for the lifetime of self to be extended and registering a callback that *does* extend the lifetime of self (or any other object) is often an error.

I see absolutely no reason that this assumption should be different for `self` than it is for any other variable, and I see no reason to believe the caller would have a particularly good idea about this.

Totally agree. The proposal would complicate reasoning about reference cycles and lifetime of objects by creating special cases which depend on what variable is concerned (self or other) and what the API does.

I agree that this proposal had flaws. It was a step in the thought process which led to the new guarded closures proposal which is much stronger.

···

On Feb 20, 2017, at 12:20 AM, David Hart <david@hartbit.com> wrote:
On 20 Feb 2017, at 06:05, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 19, 2017, at 7:06 PM, Matthew Johnson <matthew@anandabits.com> wrote:

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(David Hart) #16

Often you hand it to something owned by self, but it's also the case that you often hand it to something not owned by self, but that should not extend the lifetime of self.

I don't agree that it shouldn't extend the lifetime of `self`. By default, Swift assumes that if you capture an object in a closure, you want that object to stay alive as long as the closure does.

I was not being clear. What I meant is that when you register a method as a callback with something like NSNotificationCenter you usually do not want self to be retained by that callback registration. I agree that this proposal didn’t adequately address making the semantics visible at the call site and it was also too focused explicitly on self. But the basic statement is still true: today we use weak capture manually. We don’t *intend* for the lifetime of self to be extended and registering a callback that *does* extend the lifetime of self (or any other object) is often an error.

I see absolutely no reason that this assumption should be different for `self` than it is for any other variable, and I see no reason to believe the caller would have a particularly good idea about this.

Totally agree. The proposal would complicate reasoning about reference cycles and lifetime of objects by creating special cases which depend on what variable is concerned (self or other) and what the API does.

I agree that this proposal had flaws. It was a step in the thought process which led to the new guarded closures proposal which is much stronger.

Will definitely give it a read :slight_smile: Thanks!

···

On 20 Feb 2017, at 16:02, Matthew Johnson <matthew@anandabits.com> wrote:

On Feb 20, 2017, at 12:20 AM, David Hart <david@hartbit.com> wrote:
On 20 Feb 2017, at 06:05, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Feb 19, 2017, at 7:06 PM, Matthew Johnson <matthew@anandabits.com> wrote:

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution