SE-0302 (second review): Sendable and sendable closures

Instead of having implicit conformance, when you pass a non-Sendable value to where one is expected (say to an actor), could Xcode offer a fix-it to add it for you?

1 Like

I'm mostly happy with the implicit conformance for non-public types. For public types, though, the potential harm is that authors may be vending API that they do not realize, which could constraint future changes in unexpected ways.

If I, as a library author, write a struct

public struct S {
  var x: Int
}

I may not intend for this type to be usable across concurrency domains, but the definition written here (if implicit conformances were permitted for public types) would preclude me from ever making this type concurrency-unsafe unless I was willing to have a potential source break. At the very least we'd need a way for library authors to spell "do not make this implicitly Sendable, it may become un-Sendable in the future."

IMO, this is similar to the reason we disallow the synthesis of a public memberwise initializer—it's not that exposing the memberwise initializer is potentially incorrect, it's that it potentially causes library authors to expose public API without even realizing it.

3 Likes

It’s not clear to me how this issue would apply to frozen types. The proposal explicitly excludes either public or frozen types, but if you’ve ruled out ever changing the stored members, then...

This is true, but I actually think we need such a spelling anyway to disable retroactive conformances via @unchecked Sendable regardless of how we decide the implicit conformance issue.

For instance, if we decide that we don’t want raw pointers to be sendable (for reasons I enumerate above and in line with Rust’s rationale for the same decision), then it would be rather useless if any third-party library could vend extension UnsafeMutableRawPointer: @unchecked Sendable { } for all unwitting users of that library. The same line of reasoning for other standard types we’ve already decided should not conform to Sendable.

(The brain-dead but I think sufficient solution here would be a delightful marker protocol named Unsendable. This trichotomy would allow an optimal expression of user intent: Can this type be sent across concurrency domains without manually implemented internal synchronization? (A) Yes, always. (B) No, never. (C) Let the compiler decide.)

3 Likes

Yeah, I can’t really think of an objection here off the top of my head. Maybe there’s a situation where a type would want to start using it’s existing members in a concurrency-unsafe way? Say, initializing an Int value from a pointer and then converting it back internally. That’s pretty contrived, though.

Eh, I don’t know. We don’t have a general facility for preventing retroactive conformances that a type author has determined can never be valid, so I don’t really see why we’d need it for Sendable.

I suppose there’s an argument that an improper Sendable conformance is more ‘fundamentally unsafe’ than, say, an improper Equatable conformance, but for virtually any semantic guarantee that a protocol makes you could easily write a program that exhibits unsafe behavior in the face of a conformance that violates the guarantee.

Regardless, I still think we’re better off forcing library authors to make API promises explicitly than hoping that they remember to opt-out of implicit API promises.

I can sort of understand structs being implicitly Sendable in a world where actors and structured concurrency were built into Swift 1, then it would probably make sense for vanilla structs to be completely capable of being sent across concurrency domains out of the box. You, the developer, think naturally of what structures you want to have, and it-just-works with concurrency.

I do wonder in that world if things like this would be necessary:

struct X<T> {
  var value: T
}
extension X: Sendable where T: Sendable { }

If structs just-work, why don’t generic structs? Are generics an advanced feature of Swift, or are they fundamental, or somewhere in the middle?

To get the implicit behaviour, I also have to add this Sendable concept:

struct X<T: Sendable> {  // implicitly conforms to Sendable
  var value: T
}

That feels weird in what I imagine an intro-to-this-imaginary-Swift-1 tutorial looks like. To introduce the concept of generics, I soon also have to introduce the concept of Sendable? But all my tutorial was trying to do was basic UI concepts and it send something to a background queue so suddenly I have to dive in much deeper on how Swift treats types in a concurrent world. Pretty sure getting things off the main queue is a regularly recommended best-practice by Apple, so I feel like this wouldn’t be an advanced tutorial.

A barely-thought-through alternative:

struct X<T: valuetype> {  // implicitly conforms to Sendable
  var value: T
}

struct Y<T> {  // does NOT implicitly conform to Sendable
  var value: T
}

Why? This valuetype concept is probably easier to explain than Sendable.

My point is to ask — how long into a pretty standard Swift tutorial would you have to introduce this concept of Sendable due to talking about generics or similar?

I agree that @concurrent is stronger, and I also think it is generally more understandable and meaningful: a @concurrent function can be executed concurrently, whether with itself or with respect to the place it was formed. That implies that it conforms to Sendable.

As much as I would want the Sendable/@sendablenaming to line up, I think@concurrent` remains the better name for the attribute.

I agree that this makes a lot of sense, and am happy to ban such extensions.

I agree that the latter is the clearest option, and the only one we should support.

I tend to agree with you. As I've noted before and has been mentioned here, I am not at all concerned about types that start out being Sendable and then add some reference-type-ish field that makes them not Sendable. We have tools that catch it, and one has almost surely also broken value semantics with such a change, which will be the bigger surprise.

This is a good point. The implicit Sendable makes it much more likely that we'll end up with a surprise with unsafe pointers.

It's an important optimization, because the vast majority of types will conform to Sendable one way or another. It also makes it possible to introduce Sendable conformances in a manner that can be backward-deployed.

I think this is a good feature to make public at some point in the future, because there are undoubtedly other semantic tags we might want later (e.g., the ValueSemantics protocol idea that came up in earlier pitches).

The language does not have per-conformance annotations yet, although the idea has come up for a couple of other reasons:

  • @available on conformances is a recent addition (Swift 5.4), and is currently inferred from the availability of the extension. We could narrow this to an annotation on the specific conformance that has availability.
  • @retroactive on a conformance could acknowledge that a conformance is in neither the original type's module nor the protocol's module. Retroactive conformances can cause of some confusion, so there's a possible future where folks want to know about them and mark them as such explicitly.
  • Actors pitch #4 has a "future direction" for isolated conformances.
  • Scoped conformances could put access control on the specific conformance.

The compiler can generate Fix-Its to help migrate code, and IDEs or other tools can help you apply those Fix-Its.

Doug

2 Likes

If anything this might be a strong argument to go down this road: having something that somewhat implicitly signals "hey you broke value semantics here" might be useful. It won't catch all cases, but it will catch many.

2 Likes

Sendable brings more questions than explanations to me: what are the sources and the destinations of the sendable values? I cannot find rationales behind why the original ValueSemantic is dropped (IMO this name is much better to understand and conform to than both Sendable and ConcurrentValue).

1 Like

Hmm, I don't know that I'm so compelled by this. My personal concern is less about the ability of library authors to catch the source break, and more about the ability to constrain themselves in unexpected or surprising ways. Yeah, I suppose it's better to realize you have a potential source break so you can do a major semver bump, or whatever, but that doesn't help the library author who simply doesn't want to break source by introducing a property. Allowing the implicit conformance would put the onus on the author to always think "hm, I might possibly want to make this concurrency-unsafe in the future, better opt-out."

I'm not as troubled by the break implied by loss of value semantics since IMO clients relying on the value semantics of a type which makes no such explicit guarantee are more clearly buggy than clients which rely on conformance to a protocol.

I agree that this might be independently useful, but not at the cost of, essentially, forcing library authors to guarantee value semantics for all value-semantics-composed types by default, with explicit opt-out required. We could have a tool that alerts authors to violations of value semantics without imparting implicit API promises on library authors.

I believe the reason is that there is no agreement on what, exactly, value semantics are, nor on how to map those semantics to the requirements for safely sending a value across concurrency domains.

Long story short, value semantics are not what we're after. It is some notion of "this is safe to send/share across actor/execution/thread contexts", which is why Send(able) works well. And "ValueSemantics" is just a small subset of it -- there exist plenty of non-value semantics types which are totally fine and expected to be sent between actors.

// I need to finish my review / writeup for this thread, will post soon... but it's shaping up very good :+1:

4 Likes

Hmmm, if the name split that way, I would worry that @concurrent implies that a closure will execute concurrently, not merely that it can.

IOW, on a naive reading,

let longRunningTask: @concurrent () -> Void

…wrongly seems to imply that a longRunningTask() call will not block.

I also grant, however that @sendable () -> Void doesn’t communicate much on the naive reading, and the @asyncSendable () -> Void that I proposed upthread is a bit unwieldy.

Maybe @concurrencySafe?

And I take it the concern is just the sheer size of all of those witness tables…?

That does make sense.

I do wonder if there isn’t some ingenious way to finagle runtime support for memberless protocols that requires neither witness tables nor ABI breakage…but I assume you all understand this many orders of magnitude better than me, and have already thought it through! Given that, I can let go my wish for is Sendable.

That’s an exciting one! Thanks for the context. It does make the conformance annotation syntax in this proposal look more comfortable.

1 Like

Maybe ConcurrentAccessible is a good candidate? Sendable drops too much information IMO.

Mulling it over, I do rather like the name ConcurrencySafe / @concurrencySafe. It reads well in context:

struct Person : ConcurrencySafe {
  var name: String
  var age: Int
}

@propertyWrapper
struct UnsafeTransfer<Wrapper> : @unchecked ConcurrencySafe {
  ...
}

actor MyContactList {
  func filteredElements(_ fn: @concurrencySafe (ContactElement) -> Bool) async -> [ContactElement] { … }
}

Pros:

  • Unlike Sendable, it correctly communicates that we’re talking about concurrency — not networking or IPC or some other kind of “sending.”
  • Unlike Concurrent, it does not imply that the data structure / closure is inherently concurrent, merely that it is suitable for use in concurrent contexts.
  • It’s nice to maintain the alignment between the annotation and the protocol.
  • I particularly like the way @unchecked ConcurrencySafe reads: unlike the alternatives, it makes clear that it is the safety that is unchecked.

Cons:

  • Per the concern in my OP, it might imply that a variable of ConcurrencySafe type is safe to access concurrently across threads; it removes the focus on value transferring being what is safe rather than value sharing. Though I realize now that the compiler would catch most mistakes resulting from this misunderstanding, e.g. a developer assuming a ConcurrencySafe global variable requires no locking. (The compiler would catch that under this proposal, correct?)

As much I’m loath to inflict yet more bikeshedding on this long-suffering forum, this particular naming decision does seem like one where discoverability, fluency, naive reading, and name-implied heuristics all matter a lot.

4 Likes

On the other hand, I think it'd be good fight back against the urge to wordsmith and keep the names aligned. Other unnecessarily different wordings in the language for the same concept, like mutating/inout and consuming/owned for self vs. other arguments, or class-in-protocols-and-classes-but-static-everywhere-else for type-level methods, create barriers for learning the language.
(Calling any of this "concurrent" also wrinkles my concurrency-vs-parallelism pedant nose; I know it's a lost cause, but it's not concurrency we're making safe, but parallel execution. A single-core concurrent runtime wouldn't need most of this, after all.)

7 Likes

I'm normally the kind of person who pushes hard against the language making assumptions on my behalf in my API, but in this case I think we're fairly safe.

Firstly, I'll note that there is already prior art for implicit conformance in the language making API promises on your behalf, such as the implicit Equatable conformance on enums without associated types. This is not a strong reason to add more implicit conformances, but it is some prior art.

More broadly, I think I have the exact opposite concern to you. You say this:

My perspective is the other way. It is really annoying to have to add : Sendable every time I write something that is trivially value semantic by construction, which is the only time the implicit conformance would kick in.

Types that are simple aggregates of : Sendable types are, definitionally, Sendable. I see no particular reason to remove the capability. We analyse the thread-safety of types today when writing code, and we do it in the exact way that the compiler will do for this implicit conformance. Developers will break people if a type that was previously clearly thread-safe by construction stops being so, even though they never promised it was thread-safe.

For my part, I just don't see this being a big deal. It's in-line with what Rust does as well, which is arguably another point in its favour.

3 Likes

For what it's worth, I think your perspective is the better one. For a type a library author wishes to guarantee as Sendable, he can add an explicit conformance, and the compiler will happily produce an error when the constraint is no longer satisfied.

Yes, this will mean careful consideration on the part of library authors, especially those who have code to migrate, but I think it's a manifestation of the 80-20 rule: 80% of the time, implicit conformance will help all developers, and 20% of the time it will annoy library maintainers.

As I read the proposal I can't help but draw parallels with how movable types might look like -- both features solve what appears to be the same problem, with movable types being a more generic approach. Have the authors considered going full on towards specifying movable types?

I see now it's briefly mentioned in Alternatives Considered, apologies. I do wonder however how large a leap is to rename Sendable to Movable and see how it applies to Swift; seems to me we're almost there.

Move semantics and "sendability" in terms of this proposal are different things: the former is about ensuring that implicit copies are never created, the latter one ensures that if they are, they're safe to be accessed in parallel. In fact, these are strict subsets of each other: movable is sendable by default (no copy — no worries), but not otherwise. Movable is much more restrictive though and generally defeats the purpose of checking for concurrency safety (and even talking about concurrency). Correct me if I'm thinking about some wrong kind of "movability". (EDIT: My intuition was wrong, of course there could have been other references to some memory beyond the one stored in the moved value, as John corrects me later).

But it is indeed confusing that Sendable is also talking about some form of locomotion; I wonder if that won't become a larger issue later. Which makes me further believe that Sendable is too broad of a name.

3 Likes
Terms of Service

Privacy Policy

Cookie Policy