How can I safely use a non-sendable object in an async context?

Using Swift
1

I am trying to append a CVPixelBuffer to an AVAssetWriter on a specific queue, but I get errors because CVPixelBuffer is not Sendable. As I understand it, the error is valid, because another context could be changing the underlying data concurrently with this access. What is the right way to do this safely?

(@preconcurrency import CoreVideo doesn't silence the warning.)

I could create an @unchecked Sendable wrapper struct to hide the problem, but I'm wondering what is the intended way to do this in line with Swift concurrency?

func writeFrame(_ frame: CVPixelBuffer, at timeStamp: CMTimeStamp) async {
   await withCheckedContinuation { continuation in
        queue.async { [frame] in
            pixelBufferAdaptor.append(frame, withPresentationTime: timeStamp) // Capture of 'pixelBuffer' with non-sendable type 'CVPixelBuffer' (aka 'CVBuffer') in a `@Sendable` closure; this is an error in the Swift 6 language mode
            continuation.resume()
        }
    }
}

I tried creating a ~Copyable wrapper struct and having the function consume it as a way of communicating that it wasn't allowed to be used anywhere else at the same time, like this:

private struct VideoFrame: ~Copyable {
    var timeStamp: CMTime
    var pixelBuffer: CVPixelBuffer
}

func writeFrame(_ frame: consuming VideoFrame) async {
    await withCheckedContinuation { continuation in
        queue.async { [frame] in
            pixelBufferAdaptor.append(frame.pixelBuffer, withPresentationTime: frame.timeStamp)
            continuation.resume()
        }
    }
}

But I got Missing reinitialization of closure capture 'frame' after consume. I think that's because it's trying to borrow frame into the escaping closure? But I'm not sure.

2

The best option here is actor with custom executor (SE-0392). In cases when some bridging to an pre-Swift concurrency code is needed, use assumeIsolated (e.g. in delegates with are known to be called on the same queue).

3

Yeah, I’m a big fan of this technique, but there’s one gotcha with assumeIsolated(…). It works fine on the latest systems but traps if you back deploy. In my case that meant that my code worked on macOS 15 but failed on macOS 14 )-:

I filed a bug about this (r. 139354130) but I don’t expect to see a fix because the older systems are missing key infrastructure required to make this work.

I ended up using non-isolated delegate methods that start a task to get back to the actor. That’s less than ideal, but I needed macOS 14 support.

Share and Enjoy

Quinn “The Eskimo!” @ DTS @ Apple

4 Likes
4

Thanks for sharing. It helps me a lot.

5

Thanks for the suggestion. I'm trying to use this approach. Here is what I've done (not the same as my original question, but similar):

actor AudioSampleCopier {
    private var iterator: VideoReader.Stream<CMSampleBuffer>.Iterator
    private let input: AVAssetWriterInput
    private let executor: Executor

    nonisolated var unownedExecutor: UnownedSerialExecutor {
        executor.asUnownedSerialExecutor()
    }

    final class Executor: SerialExecutor {
        let queue: DispatchQueue

        init(queue: DispatchQueue) {
            self.queue = queue
        }

        func enqueue(_ job: consuming ExecutorJob) {
            let unownedJob = UnownedJob(job)
            let unownedExecutor = asUnownedSerialExecutor()
            queue.async {
                unownedJob.runSynchronously(on: unownedExecutor)
            }
        }
    }

    init(_ stream: VideoReader.Stream<CMSampleBuffer>, input: AVAssetWriterInput, queue: DispatchQueue) {
        iterator = stream.makeIterator()
        self.input = input
        executor = Executor(queue: queue)
    }

    func copyNext() -> Bool {
        if let sampleBuffer = iterator.next()?.1 {
            input.append(sampleBuffer)
            return true
        } else {
            return false
        }
    }
}

let audioSampleCopier = AudioSampleCopier(audioStream, input: audioInput, queue: audioCopyQueue)
audioInput.requestMediaDataWhenReady(on: audioCopyQueue) {
    while audioInput.isReadyForMoreMediaData {
        if !audioSampleCopier.assumeIsolated({ $0.copyNext() }) {
            audioInput.markAsFinished()
            break
        }
    }
}

But I'm getting: Fatal error: Unexpected isolation context, expected to be executing on Executor

Any idea what I'm doing wrong?

6

OK I was able to fix that by replacing my DispatchQueue with a DispatchSerialQueue and using queue.asUnownedSerialExecutor() instead of creating a custom Executor. But I'm still worried about iOS 17 support given @eskimo 's comment. So I'll try to find a workaround. There are a few mentioned here: Actor `assumeIsolated` erroneously crashes when using a dispatch queue as the underlying executor.

7

Is calling a nonisolated(unsafe) method the solution for older OS versions?

8

But I'm still worried about iOS 17 support given @eskimo’s comment.

The nice thing about the problem I mentioned is that there’s no ambiguity: You just end up trapping with a Incorrect actor executor assumption error.

That thread you referenced has a lot of complex discussion but my workaround was trivial. Instead of doing this:

actor Foo {
    func myCallback() {
        … isolated by magic of custom executor …
    }
}

I did this:

actor Foo {
    nonisolated func myCallback() {
        Task {
            await self.myRealCallback()
        }
    }
    
    func myRealCallback() {
        … isolated the hard way …
    }
}

It’s not efficient, but it’s easy (-:

Share and Enjoy

Quinn “The Eskimo!” @ DTS @ Apple

1 Like
9

That seems to be a correct implementation at first sight, let me check, I have been relying on out-of-the-box conformance for DispatchSerialQueue mostly, trying to avoid this in project that can’t support this yet.

Note that this conformance also is only available from 17.0 IIRC, so if you need to support earlier versions, the initial custom implementation is the way to go.

As for implementing assumeIsolated, I’m on the same page with @eskimo to simply wrap this in a Task — for most of the purposes the efficiency is not an issue if we’re talking about basic stuff in the app. Yet if you need some effective sound processing, you want to try suggestions from the topic. If nonisolated(unsafe) works, that’s good — what it does is simply turns off compiler checks and makes you keep an eye on this part. If won’t trap at runtime if you get isolation wrong somewhere, but that can be mitigated by structuring the code in defensive way.

10

For my use case, I need to wait for the copy to complete before continuing, otherwise, the queue will continue the while loop and my Task will never have a chance to execute.

11

My minimum deployment target is 17.0 so that's OK. I wasn't able to get nonisolated(unsafe) to work—it seems that's only used for annotating properties, not functions? I used the assumeIsolatedHack shared by @Karl . I will remove it when I drop iOS 17 support next year.

12

If you’re OK with 17 as minimum target, you shouldn’t need workarounds for assumeIsolated. If it keeps crashing, something probably off in your code then.

13

It's not crashing, but according to that thread, assumeIsolated will crash on pre-iOS-18 (Swift 6) devices. I don't actually have such a device handy so I haven't been able to verify yet. assumeIsolated is working fine on my iOS 18 device.

1 Like
14

My bad, got lost in versions and releases.