Codable, SwiftConcurrency, Swift 6 - Fundamentally incompatible?

How would you update it though?

Thinking only about Encodable, assume that it was possible to write this without issues:

class Doc: Encodable {
  var foo: String
  @MainActor var title: String

  @MainActor
  func encode(to encoder: any Encoder) throws { ... }
}

You're telling Swift that your class is Encodable, so passing an instance of it to, say, a function that takes some Encodable for argument should be valid, right? Something like this:

func getEncodedData(from encodable: some Encodable, using jsonEncoder: JSONEncoder) throws -> Data {
  try jsonEncoder.encode(encodable)
}

But this is clearly wrong, because getEncodedData is not isolated to MainActor itself.

It seems to me that this should be based on a different protocol, something like MainActorEncodable or AsyncEncodable (Encodable itself cannot be changed, of course, for backwards compatibility), because the protocol requirement is stronger than the basic Encodable.

A different set of protocols would then require different tooling, like async versions of the encoder functions (like JSONEncoder.encode function). I think it's technically possible, but it would be a lot of work.

1 Like

Unless I'm mistaken, it was agreed by language maintainers, above, that the topic of this thread is worth addressing. The current state of the language is known to be lacking, here. We're at the stage of "future directions", though. We need a pitch, a review, etc.

7 Likes

Exactly. And the compiler would need to reason about isolation regions in order to synthesise conformance.

Absolutely. Just as Swift 6 thus far, and Swift generally has been an enormous amount of work.

Still, even with ability to conform protocols on isolated types (which is already partially possible with SE-0434), not all usages — especially ones that mix isolation – would be possible. For example, I have no idea how Codable would be able ever support something like

class Doc: Codable {
    @MainActor
    var title: String
    @SomeOtherActor
    var body: String
}

as I don't see how's that is possible to implement non-async methods here, respecting isolation of every property.

So it will inevitable come with restrictions on usage, that might be not much different from solutions (in terms of additional complexity to deal with them) suggested here to address current issue. We definitely need this, but I don't think we should treat it as a silver bullet.

1 Like

I think we can look to SE-0421 as road map for how to migrate a protocol to one that plays more nicely with actor-isolated types.

Off the top of my head:

protocol Encodable {
    func encode(to: any Encoder, isolation: isolated (any Actor?)) throws
}

I predict that this pattern is going to show up a lot, because it gives usage sites a source-compatible API via #isolation. Though it does still require changes to conforming types to opt into the capability, which is definitely a pain.

Should other foundational standard library protocols (like Codable) investigate doing what AsyncSequence did here? I have no idea. Particularly because of SE-0434, which even futher relaxed the constraints in this area.

Here's my take, for whatever it's worth:

  • The language still does need an easy way for global actor isolated types to conform to non-isolated protocols
  • I have never seen a type with split isolation work out well (yet anyways)
  • This problem in this specific thread can almost certainly be solved by removing isolation. But I would not be surprised if doing so turns up other problems.

Edit: just to be really clear. Even if the language does eventually get a capability kinda like an @preconcurrency conformance to allow global actors to use non-isolated protocols, it still won't solve the issues in this thread because of the isolation pattern being used.

2 Likes

Why would that be the goal of an ergonomic serialisation system?

It seems perfectly reasonable that an upgraded Codable (or equivalent) would have async methods.

Putting aside that this is too confusing (I clearly don't see a situation where such mix of isolations on single type, that play role in protocol conformance, could've make sense), how do you see this in the language?

Swift has distinction of sync and async functions, and while you can satisfy async requirement with non-async, it is not possible the other way. What that means then, is that we cannot neither update Codable (as it would be a huge source breaking change, that would introduce async behaviour to many places where it is not needed), nor bridge Codable & AsyncCodable, but have two parallel serialization mechanisms, that cannot be easily substituted, creating a lot of complications in codebases and dependencies.

We can go even further, and put aside Swift Concurrency model. Consider we in old GCD world. How this intermixed conformance, that respects values update, would work in order to conform Codable? I cannot see any robust solution, that wouldn't suffer from concurrency bugs. You need to ensure (at the cost of performance, of course), that each property is synchronously initialized on a desired queue. I'm not sure if Swift would allow this during init at all...

AsyncCodable would work for me.
Same methods as Codable - but async.

Same support for automatic conformance (where possible) -and I'm a happy bunny.

If I want to opt a pre-existing type into AsyncCodable, then I can write my own extension to do it.

Have you tried removing the isolation from the properties to see what happens? I'm very curious, because if you have found a case where this isolation pattern is useful I'd like to study it more closely.

I'm a long way from turning on the Swift 6 options that would make this safe. (Running Swift5 with minimal checking right now)

Even if I was to rely on non-sendability to force my class to stay in the @MainActor context where it was created though - I'd want to explicitly annotate that class @MainActor. (Because I think intent should be explicit in code)

And I'd be back to wanting a serialisation system which was able to handle an isolated class.

(and a way to do all the other things SwiftUI relies on: Identifiable, Hashable, Equatable, etc)

Ah ha! I think this is the key. You are worried that the lack of isolation implies lack of intent. But that is absolutely not the case!

The lack of a Sendable conformance is a very clear statement to the compiler. But I think it might not feel that way right now because you do not have any checks turned on. So the only warnings you will ever see are being surfaced by the explicit MainActor annotations you add.

You are second-guessing the compiler's ability to guarantee safety for good reason! Without warnings, there is no safety.

1 Like

I use @MainActor to be explicit about isolating properties that drive UI.

That could be on the class, or (as now) on the properties - but I can't see it ever going away.

If I wasn't explicit - then perhaps someday the compiler would infer it from Sendability or lack thereof, and the fact that @MainActor View accesses the data. Even then - if it has to be true, why not annotate it to be clear?

But even if the compiler has figured out that my class is on MainActor - and is enforcing that (without my explicit annotation), then it still is on @MainActor, and I still have the issue of needing the basic language plumbing (serialisation, Identifiable, Equatable, etc) which works with that isolation...

Or am I misunderstanding you?

Because a) the compiler makes the guarantee this is impossible to get wrong and b) doing so will cause you problems like you are experiencing right now.

You are correct that non-isolated protocols + isolated types don't mix well. But, that's actually not the root cause of your issue here. What is tripping you up is how you are making use of isolation. This is why it feels like all the protocols are wrong and need changing.

Unforutnately, I think what is happening is you are using concurrency with the warnings off. This is an extremely dangerous place to be. It caused me to build up a totally incorrect mental model of how Swift concurrency works. I built lots of stuff that was just totally wrong. There's a lot of pain down this path.

I'd really like to expand the migration guide to help people navigate this situation better. Have you had a look at it? Specifically, the intention of the "Migration Strategy" section was to help people ease their way into progressively more-aggressive checking in a way that doesn't require an all-or-nothing approach.

8 Likes

Adding global actor isolation isn’t just annotation. It effectively changes how the isolated party behaves. Type, isolated to a global actor, becomes Sendable implicitly, and can be passed to any isolation domain, but will always be ensured to called on its isolation.

Non-isolated and non-Sendable type on the other hand is bounded to only one isolation and compiler ensures that it will never leaves it. That’s not the same as explicit isolation.

Non-Sendable that cannot leave isolation and therefore always accessed from on isolation domain, is completely not the same as isolating type on @MainActor. And since type itself isn’t isolated, you can adopt protocols freely.

UPD. Code

Non-isolated case

class NonSendableDoc {
    var title: String?
    var foo: String?
}

@MainActor
@Observable
final class ViewModel {
    private let doc: NonSendableDoc

    init(doc: NonSendableDoc) {
        self.doc = doc
    }
}

let doc = NonSendableDoc()
// doc is transferred to main actor isolated type,
// as it non-`Sendable`, Swift will prevent it from leaving 
// this isolation boundary after this point
let vm = ViewModel(doc: doc)

Global-actor isolated case

@MainActor
final class SendableDoc {
    var title: String?
    var foo: String?
}

@MainActor
@Observable
final class ViewModel {
    private let doc: SendableDoc

    init(doc: SendableDoc) {
        self.doc = doc
    }
}

let doc = NonSendableDoc()
// doc is `Sendable` and guards its own isolation to main actor,
// so even though we pass it as parameter here
let vm = ViewModel(doc: doc)

// nothing prevents us from using it later somewhere else, like

nonisolated func otherIsolation(doc: NonSendableDoc) async {
    // type itself ensures that it is accessed from correct executor
    let title = await doc.title
    print(title)
}

otherIsolation(doc)
1 Like

I think I'm following now.

I can see how the deliberately Non-Sendable gets its isolation implicitly by where it is created/passed.

I can see how the compiler then enforces that nothing else can touch it except on that same isolation.

And - it can conform to protocols.

Downsides I see:

  1. This creates a cascade of @MainActor annotated methods. Every function that indirectly acts on the class has to be @MainActor.
    (as opposed to only the lines of code that read/write properties)
    This seems clumsy - though it's only a preference
final class Updater: Sendable {

    func updateSendable(doc:SendableDoc) async {
        let newTitle = await getNewTitle()
        
        //Other work
        
        //Only direct interactions have to be on MainActor
        await MainActor.run {
            doc.title = newTitle
        }
    }
    
    @MainActor
    func updateNonSendable(doc:NonSendableDoc) async {
        //whole fn is on @MainActor
        
        //Other work
        
        let newTitle = await getNewTitle()
    
        doc.title = newTitle
    }
    
    private func getNewTitle() async -> String {
        return "\(Date())"
    }
}
  1. You're giving up on warnings about thread safety. Those protocols still potentially break thread safety. You're just not getting the warnings any more.
1 Like

As @mattie pointed out, that’s not the case. The issue why you aren’t seeing any warnings right now is due to them being turned off. If you start gradually turning features on, remaining in Swift 5 mode (migration guide pointed earlier suggests this flow), you’ll be able to still compile the project, while having all concurrency warnings from Swift 6. And Swift evaluates the same for protocols, keeping everything safe (if protocols isn’t Sendable, and that’s probably true for wast majority of them, then still the same restrictions will apply as for non-Sendable type).

I have switched to Swift6, and turned on all warnings.

This code:

@objc protocol SetsTitle {
    func setTitle(newValue:String)
}

@objc class NonSendableDoc:NSObject,ObservableObject, SetsTitle {
    
    @Published var title:String = ""
    
    func setTitle(newValue:String) {
        title = newValue
    }
}

No warnings here - but I can absolutely update the title from a background thread through the protocol - and get a thread sanitizer warning at runtime.

Obviously I'm being deliberately psychotic here - but plenty of Foundation frameworks call back to protocols on background threads.

+ (void)updateProtocolDoc:(NSObject *)doc {
    // Check if the NSObject conforms to the SetsTitle protocol
    if ([doc conformsToProtocol:@protocol(SetsTitle)]) {
        dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
            NSDateFormatter *dateFormatter = [[NSDateFormatter alloc] init];
            [dateFormatter setDateFormat:@"yyyy-MM-dd HH:mm:ss"];
            NSString *currentDate = [dateFormatter stringFromDate:[NSDate date]];
            
            // Cast NSObject to id<SetsTitle> and call setTitleWithNewValue
            id<SetsTitle> setTitleObject = (id<SetsTitle>)doc;
            [setTitleObject setTitleWithNewValue:currentDate];
        });
    } else {
        NSLog(@"The object does not conform to the SetsTitle protocol");
    }
}

I can't do the same with my SendableDoc because it won't conform to the protocol

Main actor-isolated instance method 'setTitle(newValue:)' cannot be used to satisfy nonisolated protocol requirement

The reason this fails is because this signature is lying to the Swift compiler about its internal behavior. This function is not possible to write in Swift without warnings.

You have to keep in mind, all Apple's frameworks have been carefully audited to add the needed compiler annotations. Just have a look at dispatch_async .

I believe you can fix this particular one with:

+ (void)updateProtocolDoc:(NSObject * NS_SWIFT_SENDABLE)doc {
}

You may also be able to use sending here, but that will break compatibility with a Swift 5.10 compiler, and I'm not sure there is a NS_SWIFT_SENDING defined yet. There was not the last time I looked.

3 Likes

This will become relevant when all framework / third party code is guaranteed to be written in Swift...

Is that now a guarantee?

all of them??

If so, then I wasn't aware of it. Certainly that would help at the point where I jump all the way to Swift 6 and sufficient third party packages etc are similarly upgraded.

I'm interested in how the framework updating works though. For example CBCentralManagerDelegate certainly used to get notifications in the background.
I don't see any restrictions on adopting it in my NonSendable class (compiler doesn't complain)

Does that mean that the framework itself has been somehow updated to only call back on the 'right' thread? How would it know what 'right' was?

What's the implication if you run your code on an older OS - do you lose the guarantees then???

Yup - preconcurrency code is a serious hole, especially non-Swift code. This is something you have to be acutely aware of when adopting concurrency.

No choice. The alternative would mean Apple's own APIs would not work with Swift. But the API surface area is vast, and nothing's perfect...

Annotations are all encoded in the SDKs, and they apply at compile time. The one exception (that I know of anyways) is some dynamic isolation features (like @preconcurrency strict enforcement) will require newer OSes to catch at runtime.