SE-0302: ConcurrentValue and @concurrent closures

Yes, @Karl is right, these shouldn't conform to ConcurrentValue. The implementation is correct (none of the Any* types is marked as ConcurrentValue), but I forgot to mention these in the writeup. We'll add them.

As for actual existential types, you can state ConcurrentValue as part of an existential type (e.g., Codable & ConcurrentValue), but you cannot have ConcurrentValue on the right-hand side of is or as?.

Doug

4 Likes

Would it make sense to add the restriction that a protocol cannot refine a marker protocol?

Otherwise the protocol would β€˜inherit’ properties of marker protocols like the fact that the conformance can only be added in the same source file as the definition of the conforming type.

UPDATE: I just noticed that the conformance in the file where the type is defined only applies to ConcurrentValue and not to marker protocols in general. Is this restriction expressed as arguments to the @_marker attribute somehow, or is it hard coded as an attribute of ConcurrentValue?

It is definitely reasonable. My concern is that this is going to become de-facto required on basically every type you declare, or else you will suffer significant usability problems, even if you're not making use of concurrency.

For instance, what if you're writing some kind of lazy collection wrapper which captures a closure, like the ones in swift-algorithms? Personally, I tend to write a lot of these kinds of types - maybe because I love generics so much and want to exploit the unique characteristics of whatever collection I'm using as much as possible, and closures are so useful to inject little bits of customisation. Now, if I want those lazy wrappers to be usable with concurrency (which of course I do), I'll have to make the closure unconditionally @concurrent (just having a collection which conforms to ConcurrentValue is not enough). That then propagates to all of the types I might want to capture, even in non-concurrent code.

I also won't be able to throw the type as an error, or capture it in a keypath. So non-ConcurrentValue types become much more difficult to use, and the new standard will be that instead of writing struct MyType { to declare a new type, you should write struct MyType: ConcurrentValue { - just by default, unless there's some reason not to.

I'm concerned about that. I think it makes the language harder to learn and use.

I'm a bit concerned about that as well. Again, it means your function that would have taken a SomeProtocol existential now needs to take a SomeProtocol & ConcurrentValue, or you need to add it to SomeProtocol itself. Otherwise you can suffer significant usability challenges down in the bowels of your code.

I certainly appreciate that there will be some impact on non-concurrent code, but the impact of this seems rather high.

In the New @concurrent attribute for functions:

// Capturing a 'searchName' string is ok, because String conforms
// to ConcurrentValue. searchName is captured by value implicitly.
list = await contactList.filteredElements { $0.firstName==searchName }

Don't we need to put searchName in the capture list, or have it be a let variable? Neither is obvious in the context.

As most of the existing and new created types were/will be ConcurrentValue by default unless some exceptions such as ManagedBuffer Lazy* and Any* erasure types; why not use NonConcurrent instead of Concurrent, we just need to add NonConcurrent to the exception list types to tag them Non-Concurrent by default, that's it.

The compiler will treat every type as Concurrent-able Values, only if when programmer add NonConcurrent or UnsafeConcurrent to make them as an exception.

It's a characteristic of ConcurrentValue, which has to check whether the type carries non-concurrent component values for correctness.

For what it's worth, I believe the precedent this proposal sets in the standard libraries is that lazy collection types are not concurrent values, primarily to avoid needing to impose this restriction on the argument closures.

1 Like

It would be nice if there was a way to parameterise this one day...

struct MyLazyWrapper<C: Collection, @concurrent?> {
  var base: C
  var someTransform: <@concurrent?> (C.Element) -> String
}

extension MyLazyWrapper: ConcurrentValue 
  where C: ConcurrentValue, @concurrent == true {}

Probably not with this syntax, but you get the idea - the closure wouldn't have to be unconditionally @concurrent. I think there have been similar discussions for throws (and async?).

Still, I think we do want as many things as possible to be ConcurrentValue. It's just going to be boilerplaty to write it out every time, and possibly confusing/annoying for new users just trying to get a grip on the language.

Yes, making ConcurrentValue and UnsafeConcurrentValue both be attributes is initially attractive. The problem with this is that we'd need to reinvent something like conditional conformances to allow us to state that "arrays are CVs when their elements are". Directly leveraging the protocol design in Swift for this is convenient and avoids introducing a new form of complexity.

Yes, I think you're right. After pondering on this more, I tend to think this is a good way to go - better than a separate UnsafeConcurrentValue protocol.

Yep, this got revised in the proposal for clarity, thanks!

-Chris

Thank you for the detailed feedback Xiaodi, I consistently appreciate your perspective and insights.

I also sometimes hear people complain about bikeshedding in conversations about naming, and have also lived through times where people bickered about small differences that didn't matter. However, I am also a huge believer that naming drives conceptual clarity, and in this case naming will cast a large shadow (or illumination) across the future of Swift code, so I think that this is really really important to get right. I appreciate your detailed thought here and agree with your observations and framing.

Yes. I also have an optimistic hope that many uses of the the ConcurrentValue marker protocol will one day be subsumed with a "ValueSemantical" sort of protocol, at least for integers and related types. Thinking about how that would ultimately fit in is also warranted, even if we don't have time to nail it down in this round of proposal.

Cherry picking some bullets:

I love how this lines these terms up so the function attribute has the same name as the protocol. I also think that something like Transferable or Sendable (further nod to Rust) are a better direction than ConcurrentValue. I am personally +1 on this proposal.

This is less clear to me, because final classes with a bunch of immutable data in them don't need any synchronization. Do you have any specific concerns with just using Transferable or Sendable for these cases as well? The fact that one may be implemented with mutex, another with fancy RCU techniques, another by being immutable seems like an implementation detail that doesn't need to be captured in the protocol name.

I'd also love to know what people who have experience with this think about it!

-Chris

2 Likes

Seems fine to me if we go down the @unchecked Transferrable/Sendable route; I was more thinking that if there is a desire to stick to the current design of having two protocols, the unchecked one could play up the distinction of why it's unchecked or unsafe by going with a name such as Synchronized.

1 Like

I also prefer the Sendable protocol and @sendable attribute (especially now that actors have a "mailbox" instead of a queue).

Could the same @sendable β€” or @sendable(unsafe) β€” attribute be used:

  • on function types as proposed.
  • on "unsafe" function parameters, instead of an @UnsafeTransfer wrapper?
  • on "unsafe" stored properties, instead of an UnsafeSendable protocol?
  • on "non-trivial" stored properties, in the standard library types?
  • on "retroactive" Sendable conformances?
struct MyPerson: Sendable {
  var name: String
  var age: Int
}

struct MyNSPerson: Sendable {
  @sendable(unsafe) var name: NSMutableString
  @sendable(unsafe) var age: NSNumber
}
4 Likes

I like this. It also works very well with protocols inheriting from Sendable:

protocol MySendable: Sendable {...}

struct MyNSPerson: MySendable {
  @sendable(unsafe) var name: NSMutableString
  @sendable(unsafe) var age: NSNumber
}

I feel that there's a difference between Codable and Transferrable/Sendable: the former isn't trying to answer the question "codable how?" because the user is free to choose between encoders and decoders.

The latter, however, wants to imply specifically "transferrable/sendable across concurrency domains" β€” but this is arguably not even the most common way to "send" or "transfer" something in programming (sockets or notifications come to mind). Again, as a user, when looking at a declaration and seeing Sendable, my immediate thought/question is "sendable to where...?" or "sendable how exactly...?". Especially that such values have very precise and strict requirements, it might leave me wondering why I myself can't declare something arbitrary as Transferrable or Sendable.

This might be getting too nit-picky, but one thing I really don't like about Rust is that some names or syntax to me seem to be chosen not because of their usefulness or clarity, but because they look "cool" and "programming-y" (though this is, of course, subjective), and I'd like Swift to do better β€” especially that clarity is one of the stated goals for Swift.

So, something like Sendable as-is feels to me not clear enough, but I agree that ConcurrencySendable might be too bulky. Perhaps something like ConcurrencySafe or AsyncSendable would be better; I don't have many suggestions off top of my head, but I believe that not mentioning some Async or Conc at all would occupy a too generic name for a very specific concept.

3 Likes

If this is workable, I think this is a nicer direction too. A blanket type-level declaration like the proposed UnsafeConcurrentValue seems to me like it'd be too unspecific to help with auditing very large types, whereas a property-level annotation like this makes it more clear what specific parts of a type need special handling. Maybe @sendable(unsafe) could even be a property wrapper, if we have a single magic UnsafeTransfer type. However, In order to accommodate some of the use cases the proposal suggests for retroactively conforming types to UnsafeConcurrentValue, particularly to be able to use types outside of your control in known-safe ways that have not been updated for language-level concurrency support, I think we'd at least need the attribute here to be applicable to parameters and returns in function declarations, and arguments to call expressions, as well.

Also, although I understand the desire to have no ABI impact for ConcurrentValue for backward-deployment reasons, I think it'd be unfortunate if we had no runtime metadata at all for indicating when types or closures are safe to share across threads. I can imagine SwiftUI or other reflection-based frameworks wanting to use that information in order to use more efficient or more parallel processing for known-safe types while still being able to accommodate traditional classes and other reference types. That runtime support could also mean that we could make conditional conformance, dynamic casting, and other runtime-dependent features conditionally available on newer deployment targets, rather than completely unavailable at all.

2 Likes

Agree, I like this approach a lot for value types receiving a conformance in their base declaration. It's less clear to me how this would work for classes though. Would @sendable(unsafe) have to be applied to all mutable (and non-sendable constant) properties? One consequence of using that approach would be that immutable classes whose properties are all sendable would not require any unsafe annotation.

Some of this would be covered by SE-0293 (second review): Extend Property Wrappers to Function and Closure Parameters wouldn't it?

See, on the contrary, to me "coding" is actually more vacuous a term than "transferring." What does it mean, outside of Swift, that a value can be "coded"? All basic protocols have very precise requirements: look at the detail about Equatable in its documentation. None of this is expressible in a reasonable name, and doing so is a non-goal.

2 Likes

That would make sense to me, yeah, because any mutable variable in a class is not inherently safe to share among threads.

Yeah, I was thinking of that as I wrote my post. In order to match the expressivity of UnsafeConcurrentValue, though, I think you'd need users to also be able to annotate return types in actor method/sharable closure declarations, as well as to annotate arguments in calls to functions that would normally require ConcurrentValue.

Something like ContainerSerializable would have come pretty close to describing what Codable does in general terms. But that would also have been a worse name than Codable. To me at least, that’s a good argument that Transferrable might be a better name here than ConcurrentValue.

(Sendable is a bit too generic in my opinion.)

I've heard various folks argue that Codable should be redesigned now that we have more experience with it. One nice thing about the ":fish:able" name is that it left Serializable open for future use if/when we want to roll out a new model.

-Chris

3 Likes