Region isolation of array elements

Using Swift
1

Hello! I have another question about concurrency isolation checks that I would like to clarify.

Given a non-isolated type as follows:

class File {
    func read() -> String { "" }
}

That is part of an old library not ported yet to Swift 6. So it's used in our codebase with a @preconcurrency import.

When trying to use task groups to spawn a child task for each individual file in an array we get a sendaiblity error:

func someFunction(
    files: [File]
) async throws {
    await withTaskGroup(of: Void.self) { group in
        for file in files {
            group.addTask { // Task-isolated value of type '() async -> Void' passed as a strongly transferred parameter; later accesses could race
                let c = file.read()
                print(c)
            }
        }
    }
}

I understand that file is not senadble, but since is not used after being captured by addTask I would assume this is safe?

Our existing workaround is to create a closure where we can use sending to give more information to the compiler:

func someFunctionWorkaround(
    files: [File]
) async throws {
    let workaroundAddTask: (
        _ file: sending File, // NOTE FILE IS SENDING
        _ group: inout TaskGroup<Void>
    ) -> Void = { file, group in
        group.addTask {
            let c = file.read()
            print(c)
        }
    }
    
    await withTaskGroup(of: Void.self) { group in
        for file in files {
            workaroundAddTask(file, &group) // COMPILES
        }
    }
}

With this workaround we get a successful compilation. But should it be necessary?

After further investigaiton I have more doubts. If instead of having the non-senadble File class in another module that is imported via preconcurrency, I have it on the same module, then the first example fails with the same errors but now the workaround doesn't work.

// non-sendable, but now in the same module
class File {
    func read() -> String { "" }
}

func someFunction(
    files: [File]
) async throws {
    await withTaskGroup(of: Void.self) { group in
        for file in files {
            group.addTask { // same error as before
                let c = file.read()
                print(c)
            }
        }
    }
}

func someFunctionWorkaround(
    files: [File]
) async throws {
    let workaroundAddTask: (
        _ file: sending File,
        _ group: inout TaskGroup<Void>
    ) -> Void = { file, group in
        group.addTask {
            let c = file.read()
            print(c)
        }
    }
    
    await withTaskGroup(of: Void.self) { group in
        for file in files {
            workaroundAddTask(file, &group) // but now this errors too! 
// Sending 'file' risks causing data races
// Task-isolated 'file' is passed as a 'sending' parameter; Uses
// in callee may race with later task-isolated uses
        }
    }
}

I imagine this is due to the difference with @preconcurrency but it was surprising. The new error also seems a bit weird since there are no uses after is sent. (at least from my code, hence my question... I thought maybe the for loop could have something to do so I changed to iterating indices but the issue is the same)

  1. How do arrays and its elements play a role in region isolation? Is passing an element of an array actually try to transfer the entire array to the new region?
  2. If so, is there any workaround?
  3. I assume the difference on behavior with @preconcurrency is expected?

Tried with Xcode Version 16.1 beta 3 (16B5029d)
swift-driver version: 1.115 Apple Swift version 6.0.2 (swiftlang-6.0.2.1.2 clang-1600.0.26.4)

EDIT: Repo with the code above in a SPM setup with modules to easy replicate this.

5 Likes
2

Iterating over an array (today) is “copying” its elements (in the Copyable) sense. sending the File to another task does not mean it’s been removed from the Array, and indeed, since you got that Array from outside the function, there could be another 50 references to the same Files in the caller (not necessarily in an array, maybe the caller also has a Dictionary of all files). So the diagnostic is correct: you are trying to send a class instance of unknown provenance to another task even though this task might still be using it.

To my knowledge, there’s not currently a way to express the thing you want, which is something like [sending File], an array of Files where you promise no File is referenced anymore after this call. I don’t know if Swift’s type system is strong enough to ever support that.

I don’t have any general recommendations for you, but in this case in specific you may be able to make things work by putting paths/URLs in your array instead of Files, or perhaps closures that produce a fresh sending File when they are called; in either case the compiler would then be able to see that the File can’t be used anywhere outside of the loop.

8 Likes
3

The same problem also arises when not dealing with Array:

class File {
    func read() -> String { "" }
}

func someFunction(
    file: File
) async throws {
    await withTaskGroup(of: Void.self) { group in
        group.addTask { // same error as before
            let c = file.read()
            print(c)
        }
    }
}

Neither will the following trick work:

func someFunction(
    file: sending File
) async throws {
    func helper(_ file: sending File, _ group: inout TaskGroup<Void>) {
        group.addTask {
            let c = file.read()
            print(c)
        }
    }

    await withTaskGroup(of: Void.self) { group in
        helper(file, &group)  // error: Sending 'file' risks causing data races
    }
}

There's some related discussions about this in the forum (e.g. New task-creation APIs - #5 by philipp.gabriel) but no elegant solution can be found. I agree with jrose that withTaskGroup is nowadays best suited with Sendable inputs.

2 Likes
4

In my opinion this...

class File {
    func read() -> String { "" }
}

func someFunction(
    file: sending File
) async throws {
    await withTaskGroup(of: Void.self) { group in
          group.addTask {
              let c = file.read()
              print(c)
          }
    }
}

... should definitely work. I am pretty sure there is something wrong with the isolation checker here. That's why I filed a bug. And to further expand on this, Task does work how you would expect it:

class File {
    func read() -> String { "" }
}

func someFunction(
    file: sending File
) async throws {
    await Task {
        let c = file.read()
        print(c)
    }.value
}

Edit: the error with the array however, is definitely valid, as @jrose already mentioned, just to clear up my statement.

3 Likes
5

Thank you all, that makes complete sense indeed. In the original code File is a struct so in my mind the copy in the loop could have made a transfer for sendability, but now that I think about it doesn't make sense because if File is not sendable even if it's a struct that copy is not safe for isolation. Gotcha.

I do wonder though, if the array was not passed from the outside but created in the same function, doesn't that make it safe? Something like the following still gives the same error:

func inSameFunction() async {
    let files: [File] = []
    await withTaskGroup(of: Void.self) { group in
        for file in files {
            group.addTask {
                let c = file.read()
                print(c)
            }
        }
    }
}

Yes good suggestions. And don't worry, I'm in the process of striping out this old dependency cause I prefer to work with Foundation directly anyway. This post was more for me to fully understand what's going on rather than a blocker. Thanks for the suggestions!

Yes I agree, using sending for a single value should work. Thanks for filing the bug. :+1:

6

I think there are two parts to this (hopefully I am using the correct terminology):

  1. I think it is impossible for the region-based isolation checker to know the implementation of your sequence. I am not sure if this is the best example to demonstrate this, but assume you apply this to your array:
struct DoubleSequence<Base>: Sequence where Base: Sequence {
    
    let base: Base
    
    func makeIterator() -> Iterator {
        return Iterator(base: base.makeIterator())
    }
    
    struct Iterator: IteratorProtocol {
        
        var base: Base.Iterator
        var previous: Base.Element?
        
        mutating func next() -> Base.Element? {
            if let previous {
                defer { self.previous = nil }
                return previous
            } else if let next = base.next() {
                previous = next
                return next
            } else {
                return nil
            }
        }
    }
}

func inSameFunction() async {
    let files: [File] = []
    await withTaskGroup(of: Void.self) { group in
        for file in DoubleSequence(base: files) {
            group.addTask {
                let c = file.read()
                print(c)
            }
        }
    }
}

You now pass the same element to a different isolated region twice.

  1. The bug I reported, prevents this as well. You have to move it on closure deeper. Again the example with a simple File not an array:
func inSameFunction() async {
    await withTaskGroup(of: Void.self) { group in
        let file: File = .init()
         group.addTask {
             let c = file.read()
             print(c)
         }
    }
}
2 Likes