Pitch: Syntactic sugar for circumventing closure capture lists

Syntactic sugar for circumventing closure capture lists

Currently, if you wish to pass an instance method of self as an escaping closure within Swift it is necessary to wrap that call in a closure with a capture list to prevent a retain cycle:

{ [unowned self] in self.myInstanceMethod($0) }

I'm proposing the introduction of some syntactic sugar to circumvent this necessity.

Motivation

Lately, I've been enjoying playing with RxSwift. This library and others expose declarative APIs that make liberal use of escaping closures. In fact, many declarative APIs encourage the chaining together of dozens of escaping calls.

With this kind of syntax comes the frequent need to make use of closure capture lists to avoid retain cycles.

When working in code bases which don't make such frequent use of escaping closures, this works well enough and feels satisfactory, but with these declarative APIs the density and repetitiveness of these calls highlight an area that I feel Swift could be improved.

I'm sure many will have worked with a closure with a capture list containing self that might be a little more complex than usual. The code can begin to feel dense when self appears too frequently, extending the length of lines and negatively affecting the legibility of expressions.

At this point many will probably do what I do and factor out the routine into its own instance method, replacing the call to the instance method with the far more terse:

.reduce(0) { [unowned self] in self.myFactoredOutReduceBodyMethod($0, $1) } // it works, but could be better

In fact, I feel this pattern is so common, and (in my opinion) wanting for elegance that I believe it may have earned the right to some sugared syntax.

Consider also that the intuitive thing to do (and exactly what I mistakenly did until I saw the leaks) is actually pass the instance method directly without wrapping it in a closure. Swift offers no warning, and there is no mention of self to nudge users that perhaps they may be capturing self in scope.

.reduce(0, myFactoredOutReduceBodyMethod) // beautiful, but dangerous

Proposal

Some syntactic sugar to make explicit the desired capture behaviour of an instance method in regards to self without the need to construct a full closure with capture list.

In terms of language design and implementation – to say I'm no expert would be an understatement – but it seems the naive approach would be the following:

Prefix the instance method with a special keyword for designating that the function that follows should be wrapped in a closure that explicitly places self in an unowned/weak capture list.

.reduce(0, @disowned myFactoredOutReduceBodyMethod)

Thanks for reading

Interested to hear others thoughts and if it's a pain point people share.

3 Likes

I’d like to see support for so-called “pure” functions, which do not mutate state.

If one could declare a pure func myFactoredOutReduceBodyMethod(...) -> Result and the compiler would make sure such functions could not mutate self one could reference it by function reference without having to capture self at all.

In the meantime you can define your functions as static and refer to them using Self.

2 Likes

@sveinhal unfortunately, I come across the case fairly frequently where I really do need access to self within the body of the closure (hence the capture lists).

I should should clarify, that this would still 'capture' self, it's simply a shorthand way of defining a closure which refers to an instance method of any arity, whilst capturing self as 'unowned'.

Essentially a macro for { [unowned self] in self.instanceMethod($0...,$n) }.

Aha! I thought you only captured self because you needed to call a function on it. Your example led me to believe that your function was really pure, and that you were only keeping it as an instance func to avoid having long inline closures.

I certainly agree that it’s useful to be able to reference a function on self, without having to capture it strongly.

I wrote some loose thoughts on this three years ago: How to avoid implicit retain cycles when using function references - sveinhal on swift

About two thirds down you can see an unown function you can create for a few arities

2 Likes

@sveinhal This is great! Thank you. I'd forgotten about the curried static methods. Re-factoring for Swift 5 we have:

public func unown<Target: AnyObject, T, U>(_ target: Target, _ classFunction: @escaping (Target) -> (T) -> U) -> (T) -> U {
    return { [unowned target] args in classFunction(target)(args) }
}

Which, when we're able to use Self should make for a significantly tidier call site.

I'd still love to see this built into the language with just a keyword, but in the meantime, for my purposes – this is great.

1 Like

I always felt that this was an oversight. Could we get a warning here at least?

I think it would be worth exploring a solution for this, it happens to often. Right now a common workaround I do is to call a method than returns a closure with the unowned inside. Makes the call site somewhat nicer but the implementation still ugly.

2 Likes

Totally. It’s the intuitive thing to do, and as the compiler doesn’t warn – as it does for a regular closure wrapping an instance member of self – it’s all too easy to think that the compiler has given its nod of approval.

I have to say that I think this is moving in the wrong direction. Using self.foo to mean { self.foo($0, $1, ...) } has the problems you mentioned, but I also think it's less clear in practice, and it doesn't support a bunch of things that regular closures do. It also makes source compatibility harder (you can't add a parameter with a default today, though maybe that should work) and doesn't interact too well with overloading (it'd be better if the labels were required: self.foo(_:bar:)).

To counter my abstract points, can you link to some of your real-world uses of this, so that we can see it in context?

3 Likes

Right now Functions and Objects don't really play nice together in these scenarios.

This happens any time you work between UI components and don't use the Selector or Delegate pattern.

I would love to see something like

list.selectionAction = @weakself indexWasSelected

and

list.set(rows, with: @weakself indexWasSelected)

since it's a heck of a lot cleaner than

list.selectionAction = { [weak self] in self?.indexWasSelected(with: $0) }

and

list.set(rows)`{ [weak self] in 
    self?.indexWasSelected(with: $0) 
}

And IMO it's just as clear, or possibly more clear since there's so much less syntax getting in the way.

I can't tell you how many times I've had to add an additional management function like this as an in-between step to what I'm actually calling.

3 Likes

Hi @jrose, thanks for your feedback, it's appreciated! Let me attempt to articulate my feeling on the matter:

I think @GetSwifty is right with their observation that the particular area of friction revolves around the interplay of objects and their own instance methods, or the instance methods of their owned members.

Also, it seems to me that by far the most common capture lists will be either [unowned self] or alternatively [weak self]. I certainly don't have the stats, but let's assume that the odds of a capture list looking like the above is > 60% – then I would suggest an elegant shorthand for the syntax might be something adopters of the language would appreciate.

In terms of use-cases, in the world of UIKit and Cocoa, the first API that comes to mind is the Animation APIs.

Frequently, these calls require the setting of a dozen or more properties with the various values often coming from the existing state of the instance. Lots of expressions containing self – often multiple times in one line – these closures can be particularly dense when we're talking about a weakly captured self.

A short example adapated from the current project I'm working on:

private func finishNavigationTransition() {

    transitionCoordinator?.animateAlongsideTransition(alongsideTransition: { [unowned self] context in
        self.updateContentAreaSubject(withViewController: self.topViewController) // double call to `self`
        self.baseView.layoutOverlayIfNeeded()
        self.baseView.backgroundColor = self.style.backgroundColor
        self.baseView.transform = CGAffineTransform(scaleX: self.style.outScale, y: self.style.outScale)  // triple call to `self`
        self.baseView.alpha = 0
        ...
    }, completion: { [weak self] context in
        guard let self = self else { return } // guard for self required
        self.bindNavigationHeaderViewToChildViewController(self.topViewController) // double call to `self`
        ...
    })
}

This is the simplest of animation examples.

Consider a more complex animation requirement, with nested keyframe blocks, many calls to self. If it's a weakly held closure maybe some optional coalescing gets thrown in – at least until you need a parameter from an instance member - at which point you'll need to rewrite with a guard... As any Cocoa dev knows, animation closures can be far more complex, many scores of lines long.

I think it's fair to say at this point that the sensible thing to do is break out a complex closure like this into its own instance method:

func finishNavigationTransition() {
    transitionCoordinator?.animateAlongsideTransition(
        alongsideTransition: { [unowned self] in self.transitionAnimations(context: $0) },
        completion:  { [weak self] in self?.transitionAnimationCompletion(context: $0) }
    )
}
    
func transitionAnimations(_ context: UIViewControllerTransitionCoordinatorContext) {
    updateContentAreaSubject(withViewController: topViewController)
    baseView.layoutOverlayIfNeeded()
    baseView.backgroundColor = style.backgroundColor
    baseView.transform = CGAffineTransform(scaleX: style.outScale, y: style.outScale)
    baseView.alpha = 0
    ...
}

func transitionAnimationCompletion(_ context: UIViewControllerTransitionCoordinatorContext) {
    bindNavigationHeaderViewToChildViewController(topViewController)
    ...
}

It is indeed a few more lines vertically, but with a complex and lengthy routine the space you add vertically is made up for by the space you save horizontally – which in my opinion also makes it far clearer.

But now we see the side-effect of these common refactors:

{ [unowned self] in self.transitionAnimations(context: $0) } and { [weak self] in self?.transitionAnimationCompletion(context: $0) }.

These frequent single line closures – simple trampolines to their associated instance methods – seem pervasive in most Swift code bases.

Most of the syntax here feels redundant – especially when you consider the seductively elegant but retain-cycle creating alternative:

func finishNavigationTransition() {
    transitionCoordinator?.animateAlongsideTransition(
        alongsideTransition: transitionAnimations, completion: transitionAnimationCompletion) // Whoops. Reference cycle! 
}

In fact, I'd argue that until you're well versed with swift, this is exactly what you will end up doing – especially as the compiler remains silently complicit.

In addition, I would say precisely because this idiom exists we must add in the non strongly held counterparts – to mirror the syntactic expressiveness available when a strongly held reference to self isn't an issue.

private func finishNavigationTransition() {
    transitionCoordinator?.animateAlongsideTransition(
        alongsideTransition: @unowned(transitionAnimations), completion: @weak(transitionAnimationCompletion))
}

The technical reasons, I wouldn't dare to comment on, but from an ergonomics perspective I hear the argument about we should really be using full label lists anyway (that's a -1 from me, only to disambiguate) but seeing as the syntactic precedent has already been set, my own feeling is we should be consistent in its support.

There's a real elegance to being able to pass a function in this way – it would be a shame to lose it.

So in summary, I think the point is this:

  • There are two abundantly common capture lists within closures; [weak self] and [unowned self]. (My guess would be at least 60% of capture lists are one or the other.)
  • Often times, said closures contain a call to a single instance method that has been factored out to aid the readability/line-length of a routine due to numerous calls to self – often multiple times per line. Or to a avoid a 'pyramid of doom' of nested closures.
  • The most intuitive and ergonomic syntax (passing the instance method directly) causes 1) a retain cycle and 2) offers no warnings from the compiler that this might be an issue.
  • There exists a syntactic precedent of allowing functions without labels to be passed as closures almost everywhere else in the language – it seems unintuitive to prevent that here.

So, in the interests of mirroring existing capabilities and as a nod to the frequency in which this pattern in encountered, I would say – if not via a prefixed keyword – there is a strong argument to resolve to come up with a way to elegantly pass instance methods as closures with unowned or weakly held references to self.

I hope this clarifies my thinking somewhat and thanks again for feedback so far.

7 Likes

Thank you for this thorough write-up! It did indeed connect some missing pieces for me / remind me of some important idioms, mainly that it really is common for callbacks to call a bunch of other methods on self. Which in turn means factoring the callback out to an instance method, rather than a local or top-level function, makes sense.

I'm still concerned about proliferating the self.foo syntax as shorthand for self.foo(bar:), but I suppose in this use case it won't be a problem, because foo(bar:) is very unlikely to be overloaded or to change its signature, since its only purpose is to be used in a callback.

3 Likes

Yes, this is really the crux of it – thank you for distilling it so succinctly!

On this point, if we're enforcing this rule elsewhere, I would agree that it makes sense here also – consistency over brevity in this case. (I do like the brevity however!)

Also, reading you reply made me wonder if there is perhaps another solution: as this idiom does seem specific to callbacks on instance methods of self.

If this is indeed the specificity of the issue, perhaps we don't have to mark it as such at the call site at all, but at the method definition:

callback func myCallback(_ blah: Blah) {
    //--
}

This keyword would be specific to Classes and would do two things:

  1. Mark the instance method to always be called from a weakly held instance of self (from what I understand, the performance benefit of using unowned vs weak is almost negligible now?), i.e a macro for { [weak self] in self?. myCallback($0) } and,
  2. Behave in the exact same way as the private modifier. This way its distinct retention behaviour is restricted to the context a developer has visibility over and prevents nasty surprises elsewhere.

We can then use:

myAsyncMethod(myCallback(_:))

Just as we might intuitively expect.

After 3+ year of using Swift regularly (and Obj-C before that) I still sometimes fall into this trap. Part of it is moving between structs/static and classes, part of it is due to moving towards a more functional approach to programming in general. Now, for me, passing a function reference both looks and feels better than a trailing closure.

Given I've seen multiple posts wanting to improve [weak self] and/or self-in-closures, I'm hoping the core team is looking at it seriously. I'd really like something syntactically similar to the proposed PropertyDelegates,

The whole thing feels very out of place in Swift with it's lack of brevity and almost user-hostile behavior.

  • It's syntax-heavy - Square brackets? Is it calling an Objective-C message? :grimacing:
  • Unfamiliar - AFAIK there isn't anything else in Swift that uses that Pattern, and no hints to indicate what the [] means. You either have to already know, or look it up.
  • 0 compile time checks / warnings - Maybe this could be done with a linter? But it's hard because it's not wrong, it's just probably wrong
  • Violates progressive disclosure - The simplest solution causes retain cycles, and doing it "right" presents a significant burden of knowledge to new programmers that delves into complex memory management that's easier to be hand-waved than explain.

This also has me wondering if we really have the wrong defaults in this case. Maybe in a class the default should be [weak/unowned self]. I'm not sure if I've ever written an @escaping closure/callback in a class where I actually wanted to retain self. And if I did, it would fail more gracefully than it does with the current defaults.

10 Likes

That would also be more „fair“ in terms of typing:
For strong captures, you also don’t have to perform any guard let stunts - weak captures are much more tedious.

The defaults should guide swift programmers to write safe code, that’s how it is in so many other areas. The dangerous stuff should be more verbose. In this situation, it’s backwards.

I wholly agree that default should be weak.

EDIT: on second thought, unowned is not a good default, it's even more dangerous than strong.

2 Likes

Sorry if it is obvious, but why wouldn't we prefer to use self.foo(bar:)? It makes much more sense to me as we're referring to that specific function (in case there were other overloads). Specially since you seem to have a concerns for " proliferating the self.foo syntax" I would be happy to stop that proliferation.

I wonder if that should be a separate pitch from the other desired changes. It could be done without any changes to the current APIs, and would also apply if/when a new API is added. No idea if it would be easy to implement or not...would also be very much a breaking change so may not get approved.

Side note: I'm curious how often capture lists are used for anything other than self. I think the only time I've seen it is in examples.

Very rarely for me, but it does happen.

If the operations within my closure are restricted to one or two members on self, I'll forgo putting self in the capture list and put those members there instead.

It's cleaner than capturing self as now those members don't need to be prefixed with self so the code is just as terse as it would be in a non-escaping closure – with the exception of the capture list, of course.

This is exceedingly rare though, 95% of the time it's a [weak self] or an [unowned self].

And if i'm capturing self, very often it's one of these 'trampoline' closures that just forwards to a factored out local instance method.

1 Like

Huh...I hadn't thought of that. Good idea! I'm really hoping for a future version of Swift where we can have something like an @UnwrappingWeakSelf prefix where self isn't even required in the closure body and kill 2 birds with one stone.