How is this a reference to a var in concurrently executing code?

Using Swift
1 
func simple() {
    var failures = [Int]()

    // error (line below):
    // Reference to captured var 'failures' in concurrently-executing code
    lock.withLock { print(failures.count) }
}

I am using OSAllocatedUnfairLock(). In what way could the purely local variable failures be captured in concurrently executing code?

Note: I have abstracted this down from a more complicated example, don't read anything into the code. I just want to know if this is truly illegal or not. I could make a copy of failures with a let variable to avoid the error, but I fail to see just what entity could mutate failures out from under me while I'm in the lock...

2

I can't explain why, but the error disappears if you capture the variable:

import os

@main
enum ScratchPad {
    static func main () {
        simple ()
    }
    
    static var lock = OSAllocatedUnfairLock ()

    static func simple() {
        var failures = [Int]()
        // lock.withLock {print (failures.count) }
        // Error: ^^^ Reference to captured var 'failures' in concurrently-executing code
        
        lock.withLock { [failures] in print (failures.count) }
        // Okay
    }
}

Or if you do this:

import os

@main
enum ScratchPad {

    static func main () {
        simple ()
    }
    
    static var lock = OSAllocatedUnfairLock ()

    class State {
        var failures = [Int]()
    }

    static func simple() {
        let state = State ()
        lock.withLock {print (state.failures.count) }
    }
}
3

With that formulation, the lock doesn't statically protect the state. It's merely a runtime thing and hence you get the error. There's nothing to stop another entity coming along and mutating failures outside of the withLock closure; you could put a queue.async { failures.shuffle() } at the start of that same function body for instance). I believe there's a way to workaround this with @unchecked Sendable or something, but I'm not wholly familiar so I'll let someone else answer that.

For your use case, I believe the go-to recommendation is to use the OSAllocatedUnfairLock init that takes the state directly and then you can do lock.withLock { failures in print(failures.count) }.

4

For any closure in general (not just the async and escaping ones), these two forms are not equivalent:

foo { print(failures.count) }
foo { [failures] in print(failures.count) }

The former refers to the local variable, the latter holds a copy of it as of the time of constructing the closure (and thus prints always zero); the former is not allowed in concurrently-executing code, the latter is perfectly ok but probably not what you intended.

5

I wouldn't trust concurrency warnings too much at this point, they are half baked. For example this results in no concurrency warnings, as if everything is alright:

let queue1 = DispatchQueue(label: "", attributes: .concurrent)
let queue2 = DispatchQueue(label: "", attributes: .concurrent)

func simple() {
    var failures = [Int]()
    queue1.async {
        failures.shuffle()
        print(failures)
    }
    queue1.async {
        failures.shuffle()
        print(failures)
    }
    print(failures)
}

Indeed. The later is a syntax optimised version of:

let copy = failures
foo { print(copy.count) }
1 Like
6

I understand now. I was thinking that because failures was local there couldn’t possibly be concurrent execution touching it, (i.e. someone else calling the function at the same time produces a separate copy of failures), but, yes, I could easily add new code to the function in the future that might induce concurrent modification of failures.

So the warning, in this case, is really “if someone changes this function and adds concurrency, or if I [the compiler] didn’t notice existing concurrency, you’re screwed.”

Good to know.

7

Compiling this code with -warn-concurrency produces the following errors:

test.swift:9:9: error: mutation of captured var 'failures' in concurrently-executing code
        failures.shuffle()
        ^
test.swift:10:15: error: reference to captured var 'failures' in concurrently-executing code
        print(failures)
              ^
test.swift:13:9: error: mutation of captured var 'failures' in concurrently-executing code
        failures.shuffle()
        ^
test.swift:14:15: error: reference to captured var 'failures' in concurrently-executing code
        print(failures)
              ^

There's be a deliberate choice not to enable all concurrency checking by default just yet, since it will be massively source breaking.

8

Yes that’s what it looks like. And I think that’s why a lot of effort was put to allow us to initialize the lock with protected state so that we can house the state that the lock is working over. Reading between the lines that’s how it looks at least. And I agree the warning there is a bit confusing as I wouldn’t expect something concurrent to have access to that local stack var either. Seems it is possible as you mentioned, but in a way that makes the warning more of a preemptive suggestion rather than strictly illegal situation.

I am Curious if it is actually possible for the compiler to look at the entire function and conclude that the array is not at risk of being mutated? Probably not?

Even if it could, might that become more confusing/less performant than just altogether banning this?

9

In my case, the lock is not meant to protect the local variable “failures.” It is protecting something else (omitted in my stripped down example which is why it seems silly).

So in this case, to silence the compiler, I would be forced to copy failures (implicitly or by capture) even though there is in fact no other async code in that function that could modify failures. Not a big deal, but an annoyance.

1 Like