Pitch: Weak method storage modifiers (aka weak references)

[eneko (Eneko Alonso)]

Swift made functions first-class citizens. This means we can get references to functions and assign them to variables or pass them around as parameters to other functions. This is great.

The downside is that, as with any other reference, function references can cause retain cycles. Specially class method references which can easily create strong references to self.

This is a common problem that many Swift developers have had to learn the hard way [1, 2].

Existing solutions
Swift provides solutions to prevent retain cycles for both class properties (members) and closures:

• Properties can be declared as weak optionals.
• Closures can use [weak self] (also unowned)
• Static class methods prevent references to self

However, Swift does not provide any support for weak method references when assigned to strong properties or when passed as arguments.

I would like to submit a proposal to add this feature to the language, but first I thought of pitching it here to see if there was any support or no support at all.

Examples
The following is a basic example of a retain cycle caused by using a method reference which turns into a strong reference to self:

import Foundation

class MyClass {
    var foo: () -> Void = {}

    init() {
        foo = defaultFoo
    }

    private func defaultFoo() {
        print("Foo Bar")
    }

    deinit {
        print("Released 🎉")
    }
}

MyClass().foo()
// prints "Foo Bar" but instance is never released

Proposed (pitched) solution
Add to the language support for using the weak keyword for method references:

    init() {
        foo = weak defaultFoo
    }

and

    setFoo(foo: @escaping () -> Void) {
        self.foo = weak foo
    }

or

    setFoo(weak foo: @escaping () -> Void) {
        self.foo = foo
    }

What are your thoughts?

I have no idea how feasible this is, but I believe in most cases this could be implemented syntax sugar for a closure with [weak self].

Thank you for your time.
Eneko

[zoul (Tomáš Znamenáček)]

@eneko

I have no idea how feasible this is, but I believe in most cases this could be implemented syntax sugar for a closure with [weak self].

That changes the type of self, though, from T to T?. So how would defaultFoo behave? For -> Void functions you could exit early if self is nil, but what if the method returns a value?

I would like to have a decent solution for this, as I have a similar problem. Also, there’s a related pitch about requiring an explicit self when passing methods to escaping closures. So there appears to be some demand for improvements in this part of the language.

[Slava_Pestov]

I won't try to answer two language design questions here because honestly I don't have any good answers:

• Whether introducing weak (and presumably unowned and unowned(unsafe)) in this location in the grammar creates any ambiguities for parsing and semantic analysis
• Whether this feature is desirable

However from an implementation standpoint I don't forsee any difficulties with adding something like this. I would assume that the formal type of such a weak-self closure would return an Optional of the method's original return type, with the call evaluating to nil if the weak reference has already expired. With an unowned self capture, it could be an IUO (or force self inside the closure body?)

Also another minor suggestion, I think this would apply not only to classes, but to the self inside protocol extensions where the protocol is class-constrained.

Also, we allow method references to be formed with 'super.foo' as well, or in fact any non-self value. So I would generalize this pitch to allow creating such a closure with any class-like base expression, not just self.

[DevAndArtist (Adrian Meister)]

This is not an entirely new pitch, because I already pitched it a while ago but never pursued it that much after @anandabits pointed me at his proposal that needs some revisiting.

I initially asked about that on Twitter:

https://twitter.com/DevAndArtist/status/873241946483040256

Then there is this thread converted from the mailing list:

[Pitch] Introduction of `weak/unowned` closures

Hello Evolution, I’d like to pitch a new idea and see where it would go. Recently I tapped into a small trap and just now realized that even that non-escaping should have been the default for closures (SE–0103) there is an exception for that. Apparently generics don’t follow that rule and a closure like Optional<() -> Void> or simply (() -> Void)? is still escaping by default. But that was the half of the story yet. As we all know and “love” reference lists inside closures, methods don’t have…

---

I've been bitten by this issue too so I know how it feels like, and this is exactly why I don't like the implicit @escaping in generic context (Optional is just one of them). My pitch was all about invalidating a weak closure when the captured reference are no longer available or use the unowned closure on own risk with an assumption that it will not fail. At the same time it would avoid retain-cycles.

[hartbit (David Hart)]

Another idea which was already proposed is to start by requiring references to member functions to be prefixed by self. when stored or passed as espcaing arguments.
import Foundation

foo = defaultFoo // error: implicit captures of `self` must be prefixed by `self.`

foo = self.defaultFoo

Edit: It's actually being discussed right now ^^ Revisiting requiring explicit `self.` when passing a method as an escaping closure

[Chris_Lattner3 (Chris Lattner)]

This specific design doesn't make sense to me, but it seems perfectly reasonable to extend weak/unowned to work with function types. You would write your example as:

weak var foo: (() -> Void)?

[tkrajacic (Thomas Krajacic)]

So could we also write this then inside a type?

public weak func someMethod() {
    …
}

[QuinceyMorris]

No. AFAICT, the semantics of strong/weak in Swift are currently the same as the semantics of strong/weak in Objective-C. In Obj-C-land (and in this thread too, I believe) there is a persistent — almost irresistible, apparently — misconception that there are things known as strong and weak references. This isn't true, there are only strong and weak storage modifiers (i.e. for variables and stored properties).

A "strong" variable holds an owning reference to some other object. A "weak" variable holds a non-owning reference. But there is no difference in the references themselves. You can't tell which "kind" it is by looking only at the reference. Rather, you need to know what kind of variable it is stored in.

Hence Chris's suggestion, that weak should become an allowable modifier for declarations of function type. That's all that's needed to resolve the reference cycle problem. The alternative solution was something along the lines of a "strong" variable referencing a closure that captures self weakly, and that's the design that "doesn't make sense".

[anon9791410]

This seems to address the popular case of capturing self, but doesn't address what happens with anything else you capture. Would the weak/unowned cascade to all the captured reference types?

I think this would be better solved by extending the syntactic shortcut we have for methods, so that this:

try myClass?.foo(myClass)()

could become this:

try myClass.foo()

// Standardize this and the rest so we don't need to write it ourselves anymore:
struct DeallocatedInstanceErrorOrSomethingLikeThat: Error {}

typealias WeakInstanceMethod<Instance, Method> = (Instance?) throws -> Method

func makeWeakInstanceMethod<Instance, Method>(
  _ getMethod: @escaping (Instance) -> Method
) -> WeakInstanceMethod<Instance, Method> {
  return {instance in
    guard let instance = instance
    else {throw DeallocatedInstanceErrorOrSomethingLikeThat()}
    
    return getMethod(instance)
  }
}

class MyClass {
  let foo = makeWeakInstanceMethod(MyClass.defaultFoo)
 
  private func defaultFoo() {}
}

[eneko (Eneko Alonso)]

In this concrete example, yes, the property could be set to weak and that should prevent the retain cycle. However, the strong retain cycle can also happen without storing the reference on a property. Hence the pitch to pass references as weak.

[eneko (Eneko Alonso)]

@QuinceyMorris

there is a persistent — almost irresistible, apparently — misconception that there are things known as strong and weak references. This isn’t true, there are only strong and weak storage modifiers (i.e. for variables and stored properties).

That is a great point, thank you for bringing that up. I always refer to strong/weak references, my bad.

To that point, how does that apply to captured variables when closure blocks are used? Those properties are captured and stored transparently, and unless [weak self] is specified in the closure capture list, the reference is stored as strong modifier. Correct?

The goal for this pitch/proposal is to allow references to self to be stored as weak modifiers when class or protocol methods are being passed as an argument to another escaping/capturing method.

Some syntax suggestions:

setFoo(foo: weak foo, bar: bar)
setFoo(foo: [weak self] foo, bar: bar)
setFoo(foo: foo, bar: weak bar)
setFoo(foo: foo, bar: [weak self] bar)

Thanks,
Eneko

[anandabits (Matthew Johnson)]

@Chris_Lattner3

This specific design doesn’t make sense to me, but it seems perfectly reasonable to extend weak/unowned to work with function types. You would write your example as:

weak var foo: (() -> Void)?

That looks to me like it would create a weak reference to the closure itself, not a weak reference to the object that declared the method coupled with a strong reference to the method. What semantics did you have in mind for this?

[QuinceyMorris]

@eneko

However, the strong retain cycle can also happen without storing the reference on a property.

How can that happen? Do you have an example that we can consider?

[Chris_Lattner3 (Chris Lattner)]

@anandabits

This specific design doesn’t make sense to me, but it seems perfectly reasonable to extend weak/unowned to work with function types. You would write your example as:

weak var foo: (() -> Void)?

That looks to me like it would create a weak reference to the closure itself, not a weak reference to the object that declared the method coupled with a strong reference to the method. What semantics did you have in mind for this?

I haven't thought deeply about this, but my off-the-cuff behavior is that a weak reference to a closure leads to weak references to any closed-over state (e.g. the self argument of a method). The closure would be deallocated if any of the closed over state is deallocated.

-Chris

[anandabits (Matthew Johnson)]

@Chris_Lattner3

I haven’t thought deeply about this, but my off-the-cuff behavior is that a weak reference to a closure leads to weak references to any closed-over state (e.g. the self argument of a method). The closure would be deallocated if any of the closed over state is deallocated.

Interesting. It seems to me like this approach might lead to confusion about the behavior of capture specifiers because it allows a library to effectively turn every capture into a weak capture. For example, somebody might write a type that exposes an addObserver(_ observer: @escaping (Event) -> Void) method and stores the closures with weak references with the idea that users won't need to specify weak self.

If this style of API became common people may begin to think less about how they capture objects. It also introduces the need for libraries to document how they store closures and for users to be aware of the semantics the library uses. It would probably lead to requests to introduce a reallyStrong capture specifier (name intentionally awkward) that is strong even in the presence of a weak reference to the closure.

I gave some thought to this problem space last year and was heading in the direction of introducing guarded closures. I was working on a second draft incorporating some of the feedback from that thread when I decided to set it aside. This design makes capture semantics clear at the declaration site of the closure but supports an "inverted" default that is sugar for closures that (mostly) capture weak with a guard on the weak captures at the beginning of the closure. It also allows an API to "invert" the default by using @guarded instead of @escaping, requiring users to provide a guarded closure. Users would still be able to use explicit strong captures where necessary.

I recently started a new thread for guarded closures when the topic came up in The Future Of [weak self] Rebinding. It's clear that there is a lot of interest in doing something in this area. It's less clear what the right balance is.

[eneko (Eneko Alonso)]

I realized I never got back to you on this one. Sorry.

I was wrong on the way I expressed myself, so my assertion was incorrect. The point I was trying to make is that while retain cycles can happen with in a single class (like the example from my original post), they can al so happen with other classes: A retains B and B retains A.

Back to my original pitch, we currently prevent this with closures by using [weak self] or [unowned self], but we don't have a way to achieve the same with direct method references, which is a bummer.

Thanks

[John_McCall]

In the long term, I would like to push these unapplied/partially-applied method references into the same basic syntax that key paths currently use, so that e.g. you can write \self.foo(x:y:z:) and get a function back whose parameters depend on how much you fill in. That is a syntax which gives us a lot more room to incorporate things like weak or unowned modifiers, as opposed to having to contort the general expression grammar to support them and then retroactively forbid them when an expression is fully-applied.

But the semantic question of what the function actually does when the weak captures have gone away is still quite open. I don't think it's a good idea to add a general ability to ask whether any of the captures of an arbitrary function value have been destroyed.