SE-0302: ConcurrentValue and @concurrent closures

Evolution Proposal Reviews
10

Hmm, this is a very interesting proposal.

My opinion is that it's pretty ugly, but it's necessary ugliness in order to safely support concurrency. What I mean by that is the business of adding this marker to almost every standard library type, as well as most types in the libraries you publish, and then likely having a file in your projects where you patch up a bunch of types from libraries whose authors didn't go around marking almost every type they expose (and hoping that they stay accurate, even across semver minor updates which add private data members).

But I understand why it's needed - we've never really defined value semantics as a language-level concept, so no code today is annotated to support safe concurrency, and lots of code will continue to be written without paying mind to concurrency so these holes will likely persist. Therefore we need a way to add those annotations/conformances, but retroactive conformances are bad, hence we limit them to compile-time and call them "markers". Still, it's a bit yucky.

  1. What about resilient types? By definition, they are updated independently of your application, and are free to evolve and add new data members. Any assumptions you make about a type on iOS 14 could become invalid and unsafe when your users update to iOS 15. IMO, we should ban any attempt to claim that a resilient type is a ConcurrentValue (even via UnsafeConcurrentValue). Since our only resilient types come from Apple's SDKs, this won't be a problem so long as Apple correctly annotates everything they won't evolve to be concurrency-unsafe.

  2. Except for the cases listed below, all struct, enum, and class types in the standard library conform to the ConcurrentValue protocol.
    ...

    • Lazy algorithm adapter types: the types returned by lazy algorithms (e.g., as the result of array.lazy.map { … }) never conform to ConcurrentValue . Many of these algorithms (like the lazy map ) take non- @concurrent closure values, and therefore cannot safely conform to ConcurrentValue .

    What about AnyCollection and friends (also AnyHashable)? Surely they also can't conform to ConcurrentValue - otherwise I could just erase any of the lazy algorithm types and pass them around freely.

    In fact, existentials in general are quite problematic, since they erase compile-time knowledge of a value's type and allow recovering that knowledge at runtime - but ConcurrentValue is a compile-time only thing. You couldn't, say, write something which hides concurrent processing as an implementation detail like this:

    protocol MyValueProtocol { /* ... */ }

func process(_ value: MyValueProtocol) {
  if let conc_value = value as? MyValueProtocol & ConcurrentValue {
    _processConcurrently(conc_value)
  } else {
    _processSerially(value)
  }
}

    The only way to use ConcurrentValue with existentials is for that constraint to bubble up, even if it has nothing to do with the semantics of the process function or the MyValueProtocol protocol.

    This is what the standard library has chosen to do for Error, and to be honest it is a bit disappointing that we're going to be restricting the kinds of types which can be thrown solely to support concurrency, even for code which has no interest in concurrency. Is there any way to narrow this? For example by only requiring that async functions throw errors which are concurrency-safe?

    • The change to keypath literals subscripts will break exotic keypaths that are indexed with non-standard types.

    I disagree with the idea that using non-standard-library types in a subscript is somehow "exotic". It may be that most values captured in keypaths can be marked as concurrency-safe, but again, it's still disappointing that we're potentially disallowing code that doesn't care about concurrency.

    While the captured values are erased from the keypath's type, I wonder if there is some way we could make a kind of ConcurrentKeyPath with a failable initializer that inspects a keypath's actual captured values to determine whether it is concurrency-safe. I suppose that would require ConcurrentValue to have some kind of runtime representation, though.

  4. Nested functions are also an important consideration, because they can also capture values just like a closure expression. We propose requiring the @concurrent attribute on nested function declarations:

    I think this should say "allowing" rather than "requiring", since non-concurrent nested functions will still be allowed (won't they?). Hopefully this sets a precedent which would allow us to add @noescape nested functions one day; sometimes the compiler fails to determine that a nested function does not escape, which can result in massive performance penalties.

2 Likes