Some thoughts on the proliferation of Sendable

[taylorswift]

bottom line up front: the default inference behavior for Sendable is wrong; it should be inferred for public types too.

for those keeping score, i’ve been transitioning to the experimental strict concurrency checking mode, and i am taken aback my the sheer volume of types that need Sendable annotations.

there are some extremely common culprits:

extension Foo
{
    public
-   enum CodingKey:String
+   enum CodingKey:String, Sendable
    {
        ...
    }
}

extension Bar:Identifiable
{
    @frozen public
-   struct ID:Hashable
+   struct ID:Hashable, Sendable
    {
        ...
    }
}

these are tedious, mechanical changes that i’ve applied hundreds of times so far.

what benefit do we gain from spelling out the word Sendable over and over again for these types?

[Jumhyn (Frederick Kellison-Linn)]

Sendable is a contract, and switching from Sendable to not Sendable is a breaking change. So for vendors of public types, the benefit (as originally envisioned) is that you will not accidentally promise something as Sendable just because it happens to be Sendable at the moment based on the recursive rules.

[taylorswift]

yeah, that’s what i thought too while i was using the limited concurrency checking mode. but real world experience with the strict concurrency mode is starting to convince me that this is just not a sensible approach. the number of types that need this is just too high, and the only reason i wasn’t noticing this problem before was because the limited concurrency checking mode was letting these types get away with not having explicit Sendable conformances.

much of what you say about Sendable is just as valid for Copyable. but we are not required to spray Copyable everywhere, because there is recognition that adding this conformance to every public type in a project is counterproductive.

[Slava_Pestov]

@taylorswift

much of what you say about Sendable is just as valid for Copyable. but we are not required to spray Copyable everywhere, because there is recognition that adding this conformance to every public type in a project is counterproductive.

But the compiler doesn’t infer Copyable conformances for types, it just makes everything Copyable by default with an opt-out.

I believe we already infer Sendable from field types for non-public types, and it causes request cycles during type checking, so I’m generally not a fan of this sort of thing from an implementation perspective.

[taylorswift]

@Slava_Pestov

But the compiler doesn’t infer Copyable conformances for types, it just makes everything Copyable by default with an opt-out.

would doing the same for Sendable really be the worst thing in the world?

when i think of things that are genuinely ~Sendable, they generally fall into one of two categories:

1. class types with mutable stored properties, or things that wrap such types.
2. structs/enums that wrap unsafe constructs that are nominally Sendable, but for which we want to suppress the Sendable conformance for semantic reasons.

#1 can be handled automatically. every time i’ve had a #2, i’ve been so aware of the non-Sendable characteristic that i went so far as to add a

@available(*, unavailable, message: "not Sendable because ...")
extension FooHandle:Sendable
{
}

so that i don’t mindlessly attach a Sendable to it later. and this is not a particularly common situation either. and mindlessly attaching Sendable to things that should not have it becomes a real problem when you are constantly attaching Sendable to things to resolve compiler warnings.

it feels to me like the defaults are not right.

[jrose (Jordan Rose)]

[Pitch] Region Based Isolation has a chance to make things a lot better too, not in changing Sendable inference but in not requiring it nearly as often.

[tgoyne (Thomas Goyne)]

I think RBI will mostly make it so that app developers (or more generally leaf code) won't need to mark things as Sendable very often because genuinely shared values should be pretty uncommon compared to values which move between isolation domains, but I don't think it changes anything for library code.

[Jumhyn (Frederick Kellison-Linn)]

Increasing the usability of non-sendable types should mean that the consequences of a library author forgetting to mark something as Sendable on a first pass are drastically reduced.

[tgoyne (Thomas Goyne)]

RBI will increase the frequency with which users will have to open bug reports asking for a type to be made Sendable because it made the library author not notice that they forgot. I don't consider this a meaningful problem with RBI, but as a library author things which make me notice problems later rather than sooner certainly aren't a benefit. RBI won't actually eliminate the need to mark anything as Sendable which could be because inevitably users will want to use every type in a sendable context (even if it makes no sense to do so).

[taylorswift]

@tgoyne

even if it makes no sense to do so

to be fair, in the linked thread @Jumhyn and i were disagreeing on precisely that: i felt that constraining

(JSONDecodable & RawRepresentable).RawValue:Sendable

made no sense, but he noted that the RawValue could easily end up in the payload of an error type thrown from a task inside a task group.

there is also the de-facto reality that in the absence of typed throws, Sendable “makes sense” almost everywhere.

[jrose (Jordan Rose)]

“Almost everywhere” is a really strong statement for a language with first-class classes, Any, and UnsafePointer. It’s certainly possible to make a different trade-off here (Rust did, though it was over a decade ago when it made that decision!), but it’d be better to do it without hyperbole.

[taylorswift]

@jrose

“Almost everywhere” is a really strong statement for a language with first-class classes, Any, and UnsafePointer.

let’s go through these exceptions one by one.

UnsafePointer

UnsafePointer is a textbook example of an unsafe construct that gets wrapped by some “safe” abstraction which is ostensibly Sendable, but only because the lax concurrency mode doesn’t require the wrapper type to only contain Sendable stored properties.

under the strict concurrency mode, it raises warnings because the wrapped pointer is not Sendable (SE-0331).

but such a type would usually still carry a

@available(*, unavailable, message: "not Sendable because ...")
extension PointerWrapper:Sendable

to document why it is not Sendable, and to force the warning to appear even with the lax concurrency mode.

this isn’t a very common case, it’s just an example of a place where being able to write PointerWrapper:~Sendable would be helpful.

Any

Any is already a problem when using the strict concurrency mode, it inevitably needs to be refactored into any Sendable in code that uses concurrency. Any without any Sendable qualifications just isn’t that useful in a concurrent context.

this is a weird inverse of the problem we have with ~Copyable and Any, and you could imagine an alternate reality where Any didn’t imply Copyable and we had a similar problem with underconstrained existentials that cannot be copied.

Classes

i personally think classes with mutable state are going to become an endangered species as the strict concurrency mode becomes more popular, they just don’t compose very well with concurrency.

for a concrete case study, you can browse through some of the discussions about SwiftNIO’s ClientBootstrap class, which was conceived before the language had first-class concurrency support, and is now widely viewed as ill-adapted for modern concurrency features.

@sliemeobn

I just stumbled over the very same annoyance of the ClientBootrap not being sendable - including the same thought process about using an actor being overkill.

@lukasa

We've got tentative plans to deprecate-and-replace the current bootstraps, because they aren't really serving their purpose well anymore. In particular the builder pattern has aged poorly. The next versions will likely be value types, which makes this entire problem go away.

@dnadoba

You are right, ClientBootstrap should have been a struct. Sadly we can't change this without breaking API. Note that this would need some more redesign in the current implementation but AFAIK nothing fundamentally impossible in a new major version.

i personally think a lot (not all, but a lot) of these types will have to evolve into actors if they need shared state, and structs/enums otherwise.

conclusion

i don’t think these exceptions really contradict my argument that Sendable is too fundamental a conformance to have to be sprayed everywhere preemptively, or reported as an issue when it is missing. the types that are ~Sendable are the special cases, and i think as the warnings pile up they are going to become less common and the ones that remain will become more-clearly documented as such.

[bbrk24]

@taylorswift

i personally think classes with mutable state are going to become an endangered species

They aren't going anywhere as long as UIViewController/etc still exist. SwiftUI is increasingly popular but UIKit is still very much in use.

[cmonsour (Christopher Monsour)]

@taylorswift

personally think classes with mutable state are going to become an endangered species as the strict concurrency mode becomes more popular, they just don’t compose very well with concurrency.

Observation only works with classes (for now, but seems significant language changes would be needed to change that) and we’re being encouraged to use Observation for data model types with SwiftUI.

[tgoyne (Thomas Goyne)]

@taylorswift

i personally think classes with mutable state are going to become an endangered species as the strict concurrency mode becomes more popular, they just don’t compose very well with concurrency.

RBI will hopefully make mutable classes usable, but in Swift 5.9 I do increasingly think that if you're using concurrency you have Sendable types, @MainActor types, and actors. Anything else is just kind of unusable. Strictly isolation-context isolated types just don't work very well with lots of suspension points; with non-threadsafe types you want much more corse-grained interactions.

[JuneBash]

@bbrk24

They aren't going anywhere as long as UIViewController/etc still exist. SwiftUI is increasingly popular but UIKit is still very much in use.

I don't think these types really "count" within the discussion above, as they are @MainActor-isolated. Their state can only be mutated from within a single, documented actor context.