Why can't I send an NSImage across actor boundaries?

Can anyone explain the logic of this to me?

I have a @MainActor class that handles downloading some stuff.

I want to be able to call it, let it make an image for me (if it has downloaded the appropriate resources) and send the image over.

The source class isn't going to change it after creation.

So - why is it an error for me to receive that as the result of an async function in a different isolation ?

I get that NSImage is not Sendable - but I don't get why I should care.

Is there some risk here that I have misunderstood?

2 Likes

NSImage is mutable (unlike UIImage). So if you send it across actor boundaries, but the original actor keeps a reference to it, then one side could mutate it while the other side is trying to display it.

Dealing with this is what sending is for: “even though NSImage can’t always be sent across actor boundaries, this NSImage is safe because I’m giving up all references to it”. I confess I haven’t tried it in practice to see how well it works, though.

7 Likes

I hate that the language acts as nanny here.

I know that NSImage is mutable.
How very dare you tell me that I can't be trusted with a mutable object!

anyway...

sending fixes things in Swift 6 language mode - thank you.

However I'm not going near swift 6 mode until I have no other option.

Any way to make the warning go away in Swift 5?
Or do I just declare NSImage Sendable and YOLO my way out of it...

Kinda doubly absurd that Swift 5 gives me a warning about something that will be an error in Swift 6 - but it actually isn't an error in Swift 6!

1 Like

When faced with this problem myself, I've sometimes used CGImage which is Sendable. Unsure if that'll help here or not...

The Sink type does look tricky to use though. It is non-Sendable with an init method that can only be called on MainActor. This means it will be difficult to invoke doStuff. Have you considered making the whole Sink type MainActor? This would fix that issue and also avoid needing to transfer the NSImage across isolation boundaries.

3 Likes

I think turning on region-based isolation feature in Swift 5 mode should help. This is the one that helps compiler to reason better about isolations and resolves some warnings/errors that might be felt out of place. You still would need sending though.

1 Like

@mattie Sink is just a minimal example to demonstrate the issue. Even to demonstrate something so trivial - it has to be contorted to create the Source in the init :man_facepalming:

@vns region-based isolation???

I'm sure I'll find a clear explanation of how to enable it, and what it does in the swift language guide :rofl:

Swift is such a mess. How has it got to a point where returning an object from a function requires two undocumented language features?

2 Likes

I think the problems you've run into with NSImage is a special-case of a more general problem stemming from how you've structured Sink's isolation. Types like this will always be hard, and sometimes impossible, to use.

1 Like

To my mind, a language where creating a class, then passing it to be used in a background thread is 'hard and sometimes impossible' is a very badly designed language.

The practical impact is that it just pushes devs to do everything on the main thread - which is obviously a fail.

Considering NSImage a special case chimes well with how I percieve the development of Swift.
It's obviously a central class for anyone developing anything real on MacOS. But the 'science project' that is modern swift evolution doesn't seem to consider real usage important.

To be fair - using sending and enabling RBI has fixed this issue for me.
Two undocumented swift features...

3 Likes

You just use feature flag in compiler configuration with RegionBasedIsolation value. There is a detailed instruction how to use it depending on whether it is SPM or Xcode: Swift.org - Using Upcoming Feature Flags

Initial proposal is here: swift-evolution/proposals/0414-region-based-isolation.md at main · swiftlang/swift-evolution · GitHub

And migration guide mentioning this: Documentation

It is unfortunate that it is missing from documentation straight away. I think this worth probably filling an issue. However, this is not undocumented completely, you can find this in a migration guide that is made to help with transitioning to Swift 6.

Also, you are not simply returning object from a function, you pass it across concurrency domains, which could easily led to issues if missed. Yes, you here know what you do, but (1) not everybody can be so swift with concurrency logic; (2) we all can make mistake – and with concurrency that's quite easy – so this is a tradeoff to make code safer in long term.


Have to add that I extemelly agree with @mattie on the fact that this minimal example already seems to be hard to use (for example, Sink has to be created from the main actor, but cannot be passed anywhere since it is non-Sendable). We have had a long thread IIRC regarding this some time ago, so this is just one note :slight_smile:

2 Likes

Here's some documentation on how Region-Based Isolation and sending works:

https://www.swift.org/migration/documentation/swift-6-concurrency-migration-guide/dataracesafety#Flow-Sensitive-Isolation-Analysis

3 Likes

No no, you misunderstand what I meant! Your example using NSImage here is just a particular example of a very common issue. You are using an isolation pattern that is known to be a problem. That is making it hard for you to work with the non-Sendable types involved (Sink and NSImage). But they are not special at all. Using sending or relying on region-based isolation is a workaround for the underlying issue.

Any time you use this isolation pattern (which I've come to call "split isolation") you will run into problems.

1 Like

re documentation - the Language Reference doesn't even have the word sendable.

I get that this stuff is in evolution proposals. But I don't think those can be called documentation.

I was incorrect earlier - RBI + sending seems to only partially fix this.
It works in my test project - but not in my real one. Possibly because the result is crossing a package boundary???

to the specific case. This is the problem I'm trying to solve:

My app generates videos for a dropzone.
That dropzone has a bunch of branding (images/videos).
The source of truth is the web server - so I have a @MainActor class which fetches updates from the web, and handles downloading the latest assets.

This could be some other defined Actor - but it needs to be something and MainActor is fine.

So - I can call my AssetManager and ask for an image asynchronously.
It loads the correct image from the disk, or if necessary, uses an async semaphore to run a single download, save and return the asset.

Now I want to use that image in the rendering process which I want to happen on a background thread.

So - how should I avoid 'split isolation' ?

The general idea is: don't isolate members of a type, isolate the entire thing.

I'm not sure I'd go so far as to call this a rule, because I'm sure there are valid uses for types that have mixed isolation, especially if the type is Sendable.

But, people often arrive at this arrangement while trying to get rid of compiler warnings, especially around protocol conformances. If that was the case here, take a look at that migration guide too, because there's quite a lot of discussion around that topic.

Something I've done in the past is work with CGImage instances in the background because they are Sendable, but do all my storage/caching on the MainActor. The doesn't sound all that far off from what you currently have and might be worth taking a look at.

2 Likes

The general idea is: don't isolate members of a type, isolate the entire thing.

I think I'm following that here.
My AssetManager is entirely on @MainActor

One of the things it produces is an NSImage which I want to use in different Type in another isolation. (to do my expensive rendering work)

The RenderingType gets the image by an async call
let image = await assetManager.latestLogo()

I assume you're not suggesting that any type which interacts with AssetManager should itself be MainActor too?

Or are you?

  • Use Xcode 16, so you get the "swift 6 compiler", which you'll use in "Swift 5 mode" (please don't shoot me, I'm just the messenger). This will give you access to sending.
// package A
@MainActor
final class ImageVendor {
    func gimmeAnImage() -> sending NSImage {
        NSImage()
    }
}

// package B
nonisolated func processAnImage(vendor: ImageVendor) async {
    let image = await vendor.gimmeAnImage()
    process(image)
}

This works fine. Leave off sending though, and you'll get:

error: non-sendable type 'NSImage' returned by call to main actor-isolated function cannot cross actor boundary
error: non-sendable type 'NSImage' returned by call to main actor-isolated function cannot cross actor boundary

You can still create problems for yourself, though; if you try to return an existing image, you'll end up in trouble:

@MainActor
final class ImageVendor {

    var image = NSImage()

    func gimmeAnImage() -> sending NSImage {
        image
    }
}

error: sending 'self.image' risks causing data races
note: main actor-isolated 'self.image' cannot be a 'sending' result. main actor-isolated uses may race with caller uses

The right way to fix this should be with NSCopying, but unfortunately that also doesn't work, this time with a very unhelpful error:

@MainActor
final class ImageVendor {

    var image = NSImage()

    func gimmeAnImage() -> sending NSImage {
        image.copy() as! NSImage
    }
}

error: task or actor isolated value cannot be sent

This is because Swift has to treat the return value of copy as being in the same isolation zone as the receiver. I'd consider it a bug that Swift doesn't understand NSCopying here, but it is what it is. So we need to actually find a way to create a new NSImage instance that doesn't refer to the old one.

This is also where we run out of good solutions. This will work for some kinds of image:

@MainActor
final class ImageVendor {

    var image = NSImage()

    func gimmeAnImage() -> sending NSImage {
        let cgImage = image.cgImage(
            forProposedRect: nil,
            context: nil,
            hints: nil
        )!
        return NSImage(cgImage: cgImage, size: image.size)
    }
}

But now that we're trafficking through CGImage anyway, I'd suggest using CGImage instead of NSImage in the first place — it's Sendable, and you can just write what you wanted to all along:

// package A
@MainActor
final class ImageVendor {

    var image = someCGImage()

    func gimmeAnImage() -> CGImage {
        return image
    }
}

// package B
nonisolated func processAnImage(vendor: ImageVendor) async {
    let image = await vendor.gimmeAnImage()
    process(image)
}
4 Likes

Hi Keith,

Thanks for the detailed response.

I'm already on 16.2 in Swift 5 mode, so that isn't the issue.

Looking more closely - perhaps the error is that I'm returning an NSImage? rather than an NSImage

Obviously my mistake for expecting esoteric language features like Optional would be handled sanely :man_facepalming:

At what point is the answer to nuke from orbit and start again ???

Re your suggestion to move to CGImage - I'm going to keep fighting the fight of believing that the compiler works for me and not the other way around.

NSImage is the natural object for me to return - I don't want to start contorting my code to to make the compiler happy when fundamentally it is the compiler that is wrong.

Um... HSImage? Is that a typo in the compiler?

it has long puzzled me why a feature (RBI) that has had so much work invested into it and is so critical to the success of Swift 6 has such sparse documentation…

3 Likes

Just a crossplatform typealias. On MacOS, it is NSImage, on iOS, UIImage
The two are so similar that I can mostly use them interchangeably.

1 Like

Optional is irrelevant; Optional<T> is Sendable iff T is Sendable. Region-based isolation doesn't care about the specifics of the types, only if they're Sendable.

This works fine:

// package A
@MainActor
final class ImageVendor {

    var image = NSImage()

    func gimmeAnImage() -> sending NSImage? {
        let cgImage = image.cgImage(
            forProposedRect: nil,
            context: nil,
            hints: nil
        )!
        return NSImage(cgImage: cgImage, size: image.size)
    }
}

// package B
nonisolated func processAnImage(vendor: ImageVendor) async {
    guard let image = await vendor.gimmeAnImage() else { return }
    process(image)
}
3 Likes