An odd error: "Escaping closure captures mutating 'self'"

rayx · 2021-06-30T05:42:59.917Z

Hi, I have read SE-0035 and discussions on the net about the error. I think I understand the general cause for the error well. But in my case, the behavior is odd. It may occur or not occur in very similar code. See below.

First, the Counter class is just a utility to reproduce the issue. The key setup is that it has a method which takes a escaping closure.

class Counter {
    var n: Int = 0

    // optional closure is @escaping
    func increase(action: (() -> Void)? = nil) {
        n += 1
        action?()
    }

    func reset() {
        n = 0
    }
}

The following code works fine. It doesn't cause "Escaping closure captures mutating 'self'" error. This is as expected.

// Example 1

struct WorkingTest {
    var counter = Counter()

    func test1() {
        counter.increase {
            print(counter.n)
        }
    }

    func test2() {
        counter.increase {
            counter.reset()
        }
    }
}

let wt = WorkingTest()
wt.test1()
wt.test2()

But if I move the code in test1() or test2() to init(), the compiler reports "Escaping closure captures mutating 'self'" error. Why? And even odder, the code in the closure below doesn't modify self at all.

// Example 2

struct FailedTest {
    var counter = Counter()

    init() {
        counter.increase {
            print(counter.n)
        }
    }
}

While investigating this, I find if I wrap the above problematic code in a function, it works.

// Example 3

struct WorkingTest2 {
    var counter = Counter()

    func test1() {
        counter.increase {
            print(counter.n)
        }
    }

    func test2() {
        counter.increase {
            counter.reset()
        }
    }

    init() {
        test1()
        test2()
    }
}

let wt2 = WorkingTest2()
wt2.test1()
wt2.test2()

I'm completely at lost. Why example 2 fails and example 3 works? Thanks for any help!

Note: I find a thread about this error in init(), but I suspect it's not the same issue.

tem · 2021-06-30T07:19:34.552Z

Hmm

I think it's because init is mutating and mutating functions try to ensure exclusive memory access. You get the same error with a mutating func f(). One way to fix it would be to add [self] in to the closure.

It does still seem a bit strange to me. I would like to know why the compiler won't do this simple change automatically for us.

I think the capture list [self] makes a copy of self. Maybe that's why it needs to be explicit.

rayx · 2021-06-30T09:38:48.063Z

tem:

I think it's because init is mutating and mutating functions try to ensure exclusive memory access. You get the same error with a mutating func f() .

That's an interesting point and it makes sense to me.

tem:

One way to fix it would be to add [self] in to the closure.

Your suggestion works indeed. But I wonder why it works? In my understanding, adding an explicit [Self] doesn't change the nature of the issue: an escaping closure captures mutating self (if the self isn't mutating, example 2 wouldn't fail in the first place).

tem:

I think the capture list [self] makes a copy of self. Maybe that's why it needs to be explicit.

If I understand it correctly, it's a copy even if one doesn't specify [self] explicitly. See the explanation in SE-0035 (search for "shadow copy" in Motivation section). And that's why it doesn't work with escaping closure.

So I think the question we both have now is that 1) how an explicit [self] declaration makes the difference, and 2) why it works with escaping closure?

Thanks for the discussion.

rayx · 2021-06-30T09:54:59.374Z

@tem I find the answer. I didn't read through SE-0035 carefully. It says the following (highlighting is added by me):

I propose we make it so that implicitly capturing an inout parameter into an escapable closure is an error.

Capturing an inout parameter, including self in a mutating method, becomes an error in an escapable closure literal, unless the capture is made explicit (and thereby immutable)

That explains it!

tem · 2021-06-30T10:20:21.643Z

Nice! I ~~am still~~ was pretty confused.

Edit: thanks for the clarification!

Lantua:

That is ill-formed and seems to (correctly) fail to compile in Swift Playground 3.4.1 and Xcode 12.5.1. You can't capture inout variables in escaping function for the same reason you can't have a partial application of mutating methods.

So mutating self is inout? I would think the following let f and func f are equivalent but I get an error for the partial application:
-- Edit: as well as for the f(inout S), so it's consistent after all, thanks --

struct S {
    var x: Int
    mutating func f() { x += 1 }
    func g() { print(x) }
}

let g: (S) -> () -> Void = S.g // OK
let f = S.f  // 🛑 Partial application of 'mutating' method is not allowed
// equivalent
func f(_ self: inout S) -> () -> Void {
    return { // 🛑 Escaping closure captures 'inout' parameter 'self'
        self.x += 1
    }
}

Also, implicit capture is basically a reference even for value-types. At least this bit from the document gives me that impression:

As an implementation detail, this eliminates the need for a shadow copy to be emitted for inout parameters in case they are referenced by closures.

I still don't like that optional closures are @escaping. Can't the compiler see that the optional is a value-type parameter and, given that (non-inout) parameters are always immutable, and value-types are copied, there's no chance for the closure to escape? Maybe that would be too complicated.

Thanks for the discussion.

rayx · 2021-06-30T10:36:49.320Z

tem:

So for me the main takeway is that mutating self is inout

Yes, that's my understanding. I remember I read a comment in this forum that mutating is implemented as inout, but I forget what thread it was.

tem:

And implicit capture is basically a reference even for value-types.

Yes, that's an implementation detail. Before SE-0035 was implemented, it's implemented in a different way: the caller passed a copy to callee, callee modified it and returned it back, and the caller overwrote the original copy. That's how I understand it.

tem:

I still don't like that optional closures are @escaping . Can't the compiler see that the optional is a value-type parameter and, given that (non- inout ) parameters are always immutable, and value-types are copied, there's no chance for the closure to escape? Maybe that would be too complicated.

I think the rule that optional closure must be escaping is irrelevant to the issue in this post. The way how I understand it is that, if you save a closure somewhere, it has to be escaping. Given that optional type is implemented by using enum and value is saved as its member, an optional closure is escaping by definition.

Hope that helps.

tem · 2021-06-30T10:41:31.252Z

Okay, thanks for elaborating.

rayx · 2021-06-30T12:39:50.166Z

rayx:

The way how I understand it is that, if you save a closure somewhere, it has to be escaping.

@tem I thought a bit more about the above statement after I replied to you. I think it's both correct and incorrect.

It's incorrect in theory. According to the Swift language book, a closure is said to escape a function when the closure is passed as an argument to the function, but is called after the function returns. So just saving a closure in some variable doesn't necessarily mean it's leaked outside the function. For example, that variable may be a local variable. For the same reason, an optional closure doesn't have to be escaping. I guess this was probably what you meant above? There was discussion and a proposal to change this behavior.

It's correct because that's the behavior in current implementation. It seems that, to save a closure in any variable, the closure must be escaping. See this example:

struct Struct {
    var callback: () -> Void
}

func test(_ callback:() -> Void) { //  Not working. Must be @escaping
    let x = Struct(callback: callback)
}

It is true for non-struct variable also:

func test(_ callback: () -> Void) {  //  Not working. Must be @escaping
    let x: () -> Void = callback 
    x()
}

tem · 2021-06-30T13:48:24.006Z

Good find! I'll need some time to digest it all.

rayx:

It is true for non-struct variable also:

I didn't even realize. Hidden in plain sight!

It's been a pleasure learning from/with you. Thanks!

Jens · 2021-06-30T16:30:21.414Z

rayx:

func test(_ callback: () -> Void) {  //  Not working. Must be @escaping
    let x: () -> Void = callback 
    x()
}

Removing the explicit type makes it compile:

func test(_ callback: () -> Void) { // Compiles, no need for it to be @escaping
    let x = callback
    x()
}

Is this a compiler bug?

Jens · 2021-06-30T17:01:17.283Z

And this crashes the compiler (default toolchain of Xcode 12.5):

func test(_ a: () -> Void) {
  let b = { print("b") }
  let x = Bool.random() ? a : b
  x()
}

rintaro · 2021-06-30T17:14:24.117Z

I don't think this is an intentional behavior. @Jens Could you file a bug in https://bugs.swift.org with the crashing case?

Jens · 2021-06-30T17:28:59.349Z

Filed SR-14859

Lantua · 2021-07-02T07:54:01.032Z

tem:

func f(_ self: inout S) -> () -> Void {
    return {
        self.x += 1
    }
}

That is ill-formed and seems to (correctly) fail to compile in Swift Playground 3.4.1 and Xcode 12.5.1. You can't capture inout variables in escaping function for the same reason you can't have a partial application of mutating methods.

rayx:

So just saving a closure in some variable doesn't necessarily mean it's leaked outside the function.

That is true in theory. However, Swift type-checks each expression separately. It doesn't look around, so it wouldn't know if the variable is escaping or not, assuming the most restrictive scenario, which is escaping.

Jens:

Removing the explicit type makes it compile:

That is weird. AFAIK, Swift never introduces non-escaping local variables, implicitly or explicitly. That should be ill-formed . FWIW, it does crash in Swift Playground 3.4.1 (Swift 5.3)

Peter-Schorn · 2021-07-03T08:21:20.988Z

Jens:

func test(_ a: () -> Void) {
  let b = { print("b") }
  let x = Bool.random() ? a : b
  x()
}

What a bizzare bug. Interestingly, if you mark the closure as escaping—even though it doesn't escape—then it compiles again. I suggest adding this to the report.

Jens · 2021-07-03T11:40:11.069Z

Done.