AVCaptureSession and concurrency

Using Swift
,
1

I'm having trouble using AVCaptureSession with SwiftUI and complete concurrency checks enabled.

My Camera class is MainActor isolated for simplicity, this alone eliminates a lot of concurrency warnings in the code. However, I'm getting run-time warnings on AVCaptureSession.startRunning() that it should be called on a background thread. Previously I would use DispatchQueue with a background priority queue but with Swift 5.10 that gives a "non-sendable type AVCaptureSession in asynchronous access" warning:

@MainActor
class Camera {
	private let captureSession = AVCaptureSession()
	// ...

	func start() async {
		// ...
		DispatchQueue.global(qos: .background).async {
			self.captureSession.startRunning()
		}
	}
}

On the other hand, the following compiles but still gives a run-time warning:

@MainActor
class Camera {
	private let captureSession = AVCaptureSession()
	// ...

	func start() async {
		// ...
		Task(priority: .background) {
			captureSession.startRunning()
		}
	}
}

What is the proper way of resolving this?

Also, shouldn't there be a more modern camera API already?

1 Like
2

I suggest use actor with custom executor to wrap camera behaviors, and executor would be a dispatch queue you create for camera. In that way you can set session delegate queue to this background queue as well, so everything is executed on that queue. That’s probably the most convenient way as for now to connect old-style dispatch queue APIs like camera has to new concurrency.

3

I'm not entirely sure that your solution would eliminate the "non-sendable AVCaptureSession" warnings, because inevitably my captureSession property will end up in a non-MainActor context. Why MainActor: most of the camera callbacks and delegate methods are required to be on MainActor since the API is quite old.

Could you show a piece of code?

4

It actually required to be on the queue you set delegate to. It can be main queue/actor, though it’s not recommended practice.

By capture session and all other related parties being isolated to an actor, your accesses to it would be ensured to happen only on that actor and never leave it, which will eliminate sendability warnings and give you compile time safety for concurrency.

Roughly, that would be

actor Camera { 
    // don’t remember exactly naming for a custom executor, so that can be a little incorrect naming
    nonisolated var unownedExecutor: UnownedSerialExecutor { cameraQueue }
    // queue conformance to executor protocol available from iOS 17 IIRC, before that you need to implement it by yourself, but that’s easy
    private var cameraQueue = DispatchSerialQueue(…)
    
    // isolated to actor and ensure being called from cameraQueue 
    private let session = AVCaptureSession()
}
4 Likes
5

While @vns's solution seems to make more sense, I've found a quicker fix for now, by adding preconcurrency attribute and using Task.detached:

@preconcurrency import AVFoundation

@MainActor
class Camera {
	private let captureSession = AVCaptureSession()
	// ...

	func start() async {
		// ...
		Task.detached {
			captureSession.startRunning()
		}
	}
}

Seems to work OK without any compile-time or run-time warnings.

1 Like
6

I found this approach fragile: startRunning also prohibits being called after beginConfiguration has been called until commitConfiguration has been called, and if those calls are issued in a different thread/queue/actor the unfortunate timing of events could happen:

    Task.detached {
        captureSession.startRunning()
    }
    ...
    // some actor like main actor or a dedicated actor or a queue, etc
    session.beginConfuguration()
    ...
    <--- the above detached `startRunning` call could unfortunately happen while we are here
    ...
    session.commitConfiguration()

In other words startRunning must be synchronised with beginConfuguration and commitConfiguration. But other calls that you use to configure session should be synchnorised with beginConfuguration and commitConfiguration either (otherwise it makes little sense using beginConfuguration / commitConfiguration brackets to begin with), so effectively all these calls must be synchronised with each other, and because one of them can't be called on the main actor (startRunning) - neither of them could or should... The easiest would be to isolate all that functionality to a serial queue or a non-main actor. And to avoid extra thread hopping we could indeed make a dedicated serial queue, specify that queue when setting session output's delegate and make actor live on that queue.

1 Like