Two thoughts on concurrency

Hi everyone,

(the standard disclaimer of “I’m neither a compiler engineer nor a language design expert” applies)

I’ve been trying to keep up with all the discussion around concurrency going on, and I’m admittedly not very familiar with async/await or the actor pattern.

However, a couple of things worry me about the direction the conversation seems to be going:

Keyword Explosion

During the Great Access Control Wars of Swift 4, one of the points that kept coming up was the reluctance to introduce a bazillion new keywords to address all the cases that were being brought up. The impression I got is that adding new keywords was essentially an anti-pattern. And so when I’m reading through this onslaught of emails, I’m troubled by how everything is seeming to require new keywords. There’s the obvious async/await, but there’s also been discussion of actor, reliable, distributed, behavior, message, and signal (and I’ve probably missed others).

I’m not opposed to adding new keywords by any means, but can we get some clarification on some limits of reasonableness? What restraint (if any) should we be exercising as we consider this feature?

Language vs Library Feature

Related to the explosion of keywords is the question of whether the concurrency model is going to be a language feature or a library feature. Allow me to explain…

We currently have language support for errors with throws and try (and friends):

func doSomething() throws → Value { … }

let value = try doSomething()

However, this could be viewed as sugar syntax for a hypothetical library feature involving a Result<T, Error> type:

func doSomething() → Result<Value, Error> { … }

let value = doSomething().value! // or however you get the value of a Result

In other words, throws and try are the language support for silently hiding a Result<T, Error> type.

I would be really happy if whatever concurrency model we end up with ends up being sugar syntax for a library feature, such that async and await (or whatever we decide on) become sugar for dealing with a Future<T> type or whatever. Implementing concurrency in this manner would free app developers to handle concurrency in the manner in which they’re familiar. If you wanted, you call the function without await and get back the underlying Future<T>. async becomes sugar for simplifying the return type, like in the throws example above. try await becomes sugar for fulfilling the promise or dealing with a cancellation (or other) error, etc.

In other words:

async func doSomething() → Value { … }

Let value = await doSomething()

Becomes sugar for this pseudocode:

func doSomething() → Future<Value> { … }

let value = doSomething().value // or however you “wait” for the value

(Incidentally, I would love to see this pattern retroactively applied for throws and errors)

Please don’t all break out your pitchforks at once. :smile:

Cheers,

Dave

I generally agree that async/await could be more valuable built on top of library support. I did have one nitpick about this:

In other words:

async func doSomething() → Value { … }

Let value = await doSomething()

Becomes sugar for this pseudocode:

func doSomething() → Future<Value> { … }

let value = doSomething().value // or however you “wait” for the value

These two examples do fundamentally different things. The await version doesn’t block the thread (it would return to the caller instead of blocking and pick up again later). The .value version would have to block. In C# this is equivalent to using the .Result property of the Task<T> type. However, that is strongly discouraged because it easily leads to deadlocks.

A closer equivalent would be something like this:

func doSomething() -> Future<Value> { … }

doSomething().continueWith { (future: Future<Value>) in
    let value = try? future.value
}

Now it’s asynchronous either way. The continuation block takes a Future<Value> so that you could do error handling (more complex error handling not shown).

···

On Aug 24, 2017, at 1:59 PM, Dave DeLong via swift-evolution <swift-evolution@swift.org> wrote:

I generally agree that async/await could be more valuable built on top of library support. I did have one nitpick about this:

In other words:

async func doSomething() → Value { … }

Let value = await doSomething()

Becomes sugar for this pseudocode:

func doSomething() → Future<Value> { … }

let value = doSomething().value // or however you “wait” for the value

These two examples do fundamentally different things. The await version doesn’t block the thread (it would return to the caller instead of blocking and pick up again later). The .value version would have to block. In C# this is equivalent to using the .Result property of the Task<T> type. However, that is strongly discouraged because it easily leads to deadlocks.

A closer equivalent would be something like this:

func doSomething() -> Future<Value> { … }

doSomething().continueWith { (future: Future<Value>) in
    let value = try? future.value
}

Now it’s asynchronous either way. The continuation block takes a Future<Value> so that you could do error handling (more complex error handling not shown).

That’s fine. I assumed it was blocking because it has a very prominent “wait” in the name. :man_shrugging:

I’m admittedly not very familiar with async/await or the actor pattern.

:)

Dave

···

On Aug 24, 2017, at 3:59 PM, Adam Kemp <adam.kemp@apple.com> wrote:

On Aug 24, 2017, at 1:59 PM, Dave DeLong via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Keyword Explosion

During the Great Access Control Wars of Swift 4, one of the points that kept coming up was the reluctance to introduce a bazillion new keywords to address all the cases that were being brought up. The impression I got is that adding new keywords was essentially an anti-pattern. And so when I’m reading through this onslaught of emails, I’m troubled by how everything is seeming to require new keywords. There’s the obvious async/await, but there’s also been discussion of actor, reliable, distributed, behavior, message, and signal (and I’ve probably missed others).

I can’t speak for message/signal, but you need to understand a bit more about how Swift works. There is a distinction between an actual keyword (which ‘async’ would be, and ‘class’ currently is) and “modifiers”. Modifiers occur with attributes ahead of a real keyword, but they are not themselves keywords. They are things like weak, mutating, reliable, distributed, etc. If we go with the “actor class” and “actor func” approach, then actor would not be a keyword.

I’m not opposed to adding new keywords by any means, but can we get some clarification on some limits of reasonableness? What restraint (if any) should we be exercising as we consider this feature?

There is no general rule that can be applied. Details matter.

Language vs Library Feature

Related to the explosion of keywords is the question of whether the concurrency model is going to be a language feature or a library feature. Allow me to explain…

We currently have language support for errors with throws and try (and friends):

func doSomething() throws → Value { … }

let value = try doSomething()

However, this could be viewed as sugar syntax for a hypothetical library feature involving a Result<T, Error> type:

func doSomething() → Result<Value, Error> { … }

let value = doSomething().value! // or however you get the value of a Result

In other words, throws and try are the language support for silently hiding a Result<T, Error> type.

Sort of, but I see what you’re saying.

I would be really happy if whatever concurrency model we end up with ends up being sugar syntax for a library feature, such that async and await (or whatever we decide on) become sugar for dealing with a Future<T> type or whatever. Implementing concurrency in this manner would free app developers to handle concurrency in the manner in which they’re familiar. If you wanted, you call the function without await and get back the underlying Future<T>. async becomes sugar for simplifying the return type, like in the throws example above. try await becomes sugar for fulfilling the promise or dealing with a cancellation (or other) error, etc.

I understand the conceptual ideal of something being "pure sugar”, but that it is also limiting. The design as proposed is exactly the opposite of what you suggest: since there are many interesting user facing libraries (future is just one of them) that can be built on top of the primitive language features.

You can still think of the primitives as being built in terms of futures if that makes sense to you, but it isn’t any more correct than it is to say that throws is implemented in terms of a hidden result type.

-Chris

···

On Aug 24, 2017, at 1:59 PM, Dave DeLong via swift-evolution <swift-evolution@swift.org> wrote:

Hi everyone,

(the standard disclaimer of “I’m neither a compiler engineer nor a language design expert” applies)

I’ve been trying to keep up with all the discussion around concurrency going on, and I’m admittedly not very familiar with async/await or the actor pattern.

However, a couple of things worry me about the direction the conversation seems to be going:

Keyword Explosion

During the Great Access Control Wars of Swift 4, one of the points that kept coming up was the reluctance to introduce a bazillion new keywords to address all the cases that were being brought up. The impression I got is that adding new keywords was essentially an anti-pattern. And so when I’m reading through this onslaught of emails, I’m troubled by how everything is seeming to require new keywords. There’s the obvious async/await, but there’s also been discussion of actor, reliable, distributed, behavior, message, and signal (and I’ve probably missed others).

I’m not opposed to adding new keywords by any means, but can we get some clarification on some limits of reasonableness? What restraint (if any) should we be exercising as we consider this feature?

Language vs Library Feature

Related to the explosion of keywords is the question of whether the concurrency model is going to be a language feature or a library feature. Allow me to explain…

We currently have language support for errors with throws and try (and friends):

func doSomething() throws → Value { … }

let value = try doSomething()

However, this could be viewed as sugar syntax for a hypothetical library feature involving a Result<T, Error> type:

func doSomething() → Result<Value, Error> { … }

let value = doSomething().value! // or however you get the value of a Result

In other words, throws and try are the language support for silently hiding a Result<T, Error> type.

It not only hide a Result type, it also allow the compiler to perform some optimization that would not be possible with a pure library implementation, like using a custom calling convention.

The same probably apply to coroutines.

I would be really happy if whatever concurrency model we end up with ends up being sugar syntax for a library feature, such that async and await (or whatever we decide on) become sugar for dealing with a Future<T> type or whatever. Implementing concurrency in this manner would free app developers to handle concurrency in the manner in which they’re familiar. If you wanted, you call the function without await and get back the underlying Future<T>. async becomes sugar for simplifying the return type, like in the throws example above. try await becomes sugar for fulfilling the promise or dealing with a cancellation (or other) error, etc.

I don’t think Future should even exists. As stated in the manifest, it add some cost to all calls with no benefit for most coroutine users.

···

Le 24 août 2017 à 22:59, Dave DeLong via swift-evolution <swift-evolution@swift.org> a écrit :

Hi,

Although `throws` and `async` are similar to return a `Result` and a
`Future` respectively as you say, However, `try` and `await` are
corresponding to `flatMap` theoretically.

// `throws/try`
func foo() throws -> Int { ... }

func bar() throws -> Int {
  let a: Int = try foo()
  let b: Int = try foo()
  return a + b
}

// `Result`
func foo() -> Result<Int> { ... }

func bar() -> Result<Int> {
  return foo().flatMap { a: Int in
    foo().flatMap { b: Int in
      return a + b
    }
  }
}

// `async/await`
func foo() async -> Int { ... }

func bar() async -> Int {
  let a: Int = await foo()
  let b: Int = await foo()
  return a + b
}

// `Future`
// `flatMap` works like `then` of JS's `Promise`
// I have an implementation of such `Promise` in Swift
// https://github.com/koher/PromiseK/tree/dev-3.0
func foo() -> Future<Int> { ... }

func bar() -> Future<Int> {
  return foo().flatMap { a: Int in
    foo().flatMap { b: Int in
      return a + b
    }
  }
}

Also, thinking about something like `do/catch` for `async/await`, I
think `do` for blocking is consistent because it should be possible to
return a value from inside `do {}` as well as `do/catch`. For example:

// `throws/try`
func bar() -> Int {
  do {
    return try foo()
  } catch {
    ...
  }
}

// `async/await`
func bar() -> Int {
  do {
    return await foo()
  } block
}

···

--
Yuta

Concrete example, this is (weird but) valid code:

var weak = 42
weak += 2
print(weak+weak)

This is a consequence of weak not being a keyword.

-Chris

···

On Aug 24, 2017, at 8:57 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Aug 24, 2017, at 1:59 PM, Dave DeLong via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Keyword Explosion

During the Great Access Control Wars of Swift 4, one of the points that kept coming up was the reluctance to introduce a bazillion new keywords to address all the cases that were being brought up. The impression I got is that adding new keywords was essentially an anti-pattern. And so when I’m reading through this onslaught of emails, I’m troubled by how everything is seeming to require new keywords. There’s the obvious async/await, but there’s also been discussion of actor, reliable, distributed, behavior, message, and signal (and I’ve probably missed others).

I can’t speak for message/signal, but you need to understand a bit more about how Swift works. There is a distinction between an actual keyword (which ‘async’ would be, and ‘class’ currently is) and “modifiers”. Modifiers occur with attributes ahead of a real keyword, but they are not themselves keywords. They are things like weak, mutating, reliable, distributed, etc. If we go with the “actor class” and “actor func” approach, then actor would not be a keyword.

I feel that it's important to point out that this example feels weird because even though the compiler doesn't treat "weak" as a reserved term, most developers perceive it as one. I don't think that David is worried that we're taking away all the cool words from the realm of identifiers; the problem is that "technically not a keyword" is a qualifier mostly distinct from "not perceived as a keyword". Even if attributes don't live in the same token namespace as identifiers as far as the compiler is perceived, I'd argue that they add about the same complexity (or more) to the mental model that developers have to have about the language.

Félix

···

Le 24 août 2017 à 20:58, Chris Lattner via swift-evolution <swift-evolution@swift.org> a écrit :

On Aug 24, 2017, at 8:57 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Aug 24, 2017, at 1:59 PM, Dave DeLong via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Keyword Explosion

During the Great Access Control Wars of Swift 4, one of the points that kept coming up was the reluctance to introduce a bazillion new keywords to address all the cases that were being brought up. The impression I got is that adding new keywords was essentially an anti-pattern. And so when I’m reading through this onslaught of emails, I’m troubled by how everything is seeming to require new keywords. There’s the obvious async/await, but there’s also been discussion of actor, reliable, distributed, behavior, message, and signal (and I’ve probably missed others).

I can’t speak for message/signal, but you need to understand a bit more about how Swift works. There is a distinction between an actual keyword (which ‘async’ would be, and ‘class’ currently is) and “modifiers”. Modifiers occur with attributes ahead of a real keyword, but they are not themselves keywords. They are things like weak, mutating, reliable, distributed, etc. If we go with the “actor class” and “actor func” approach, then actor would not be a keyword.

Concrete example, this is (weird but) valid code:

var weak = 42
weak += 2
print(weak+weak)

This is a consequence of weak not being a keyword.

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I feel that it's important to point out that this example feels weird because even though the compiler doesn't treat "weak" as a reserved term, most developers perceive it as one. I don't think that David is worried that we're taking away all the cool words from the realm of identifiers; the problem is that "technically not a keyword" is a qualifier mostly distinct from "not perceived as a keyword". Even if attributes don't live in the same token namespace as identifiers as far as the compiler is perceived, I'd argue that they add about the same complexity (or more) to the mental model that developers have to have about the language.

Obviously details matter here, but there is an important time and place to introduce a “conceptual” keyword: it is when there is an important and distinct concept that needs to be thought about by the humans that interact with the code.

In the proposal “actor” vs “distributed actor” is one of those really important distinctions, which is why (IMO) it deserves the “complexity” of a new identifier. It isn’t the identifier that adds the complexity, it is the expansion of the language surface that the identifier designates.

-Chris

···

On Aug 24, 2017, at 9:17 PM, Félix Cloutier <felixcloutier@icloud.com> wrote:

Félix

Le 24 août 2017 à 20:58, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

On Aug 24, 2017, at 8:57 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Aug 24, 2017, at 1:59 PM, Dave DeLong via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Keyword Explosion

During the Great Access Control Wars of Swift 4, one of the points that kept coming up was the reluctance to introduce a bazillion new keywords to address all the cases that were being brought up. The impression I got is that adding new keywords was essentially an anti-pattern. And so when I’m reading through this onslaught of emails, I’m troubled by how everything is seeming to require new keywords. There’s the obvious async/await, but there’s also been discussion of actor, reliable, distributed, behavior, message, and signal (and I’ve probably missed others).

I can’t speak for message/signal, but you need to understand a bit more about how Swift works. There is a distinction between an actual keyword (which ‘async’ would be, and ‘class’ currently is) and “modifiers”. Modifiers occur with attributes ahead of a real keyword, but they are not themselves keywords. They are things like weak, mutating, reliable, distributed, etc. If we go with the “actor class” and “actor func” approach, then actor would not be a keyword.

Concrete example, this is (weird but) valid code:

var weak = 42
weak += 2
print(weak+weak)

This is a consequence of weak not being a keyword.

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

I don’t think that there are any simple rules that define “Complexity”. I hope that is not too off-topic...

<An aside:
Whenever anyone tells me that a UI must only have 6 (or some other arbitrary number) of elements, because of some limits on short term memory or some such, I always bring-up the fact that even simple maps (the physical things) can have thousands of names and symbols and most people (admittedly after some training as children) use them without too much effort. Of, course source files are similarly complex (well, at least iOS view controllers :). On the flip side, just a few poorly chosen options in a UI can make it incomprehensible to most users.

In short, presenting complex information is as much art as science. These books used to be very popular and are a great illustration of this: Edward Tufte: Books - The Visual Display of Quantitative Information

It makes sense to make a distinction between adding keywords/modifiers to implement a niche feature and implementing fairly well established abstractions like "async/await" and “actor”. There should be a lower bar to using well established terms if the implementation is faithful to standard (and “modern”) usage. Swift does not exist in isolation, but in world where developers commonly use multiple languages. The notion of progressive disclosure is mentioned as being very important to the design to Swift and a lot of code can be written in Swift without ever needing to see "async/await”, so it this feature "meets the standard" for progressive disclosure (which probably can be more formally defined).

There should be a very high bar to adding keywords to support special cases and using common terms in non-standard ways. Of course, there will always be some differences in the exact usage of a term in different languages.

It might make sense to create a manifesto-like document of guidelines for evolving Swift that would serve as a first test or filter on proposals. The danger of creating such a document is that great proposals might fail the test “on paper” and be effectively killed.

A small nit: Swift exceptions are (thankfully) kind-of sugar for : `Result<T>` not `Result<T, Error>` meaning that Error is a protocol, not a generic parameter in Swift standard usage. As I have mentioned (to often already, probably) that I believe that whatever benefit `Result<T, Error>` brings is not worth the hassle and mismatch with Swift's standard usage of errors.

···

On Aug 24, 2017, at 10:29 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org> wrote:

On Aug 24, 2017, at 9:17 PM, Félix Cloutier <felixcloutier@icloud.com <mailto:felixcloutier@icloud.com>> wrote:

I feel that it's important to point out that this example feels weird because even though the compiler doesn't treat "weak" as a reserved term, most developers perceive it as one. I don't think that David is worried that we're taking away all the cool words from the realm of identifiers; the problem is that "technically not a keyword" is a qualifier mostly distinct from "not perceived as a keyword". Even if attributes don't live in the same token namespace as identifiers as far as the compiler is perceived, I'd argue that they add about the same complexity (or more) to the mental model that developers have to have about the language.

Obviously details matter here, but there is an important time and place to introduce a “conceptual” keyword: it is when there is an important and distinct concept that needs to be thought about by the humans that interact with the code.

In the proposal “actor” vs “distributed actor” is one of those really important distinctions, which is why (IMO) it deserves the “complexity” of a new identifier. It isn’t the identifier that adds the complexity, it is the expansion of the language surface that the identifier designates.

-Chris

Félix

Le 24 août 2017 à 20:58, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> a écrit :

On Aug 24, 2017, at 8:57 PM, Chris Lattner via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

On Aug 24, 2017, at 1:59 PM, Dave DeLong via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Keyword Explosion

During the Great Access Control Wars of Swift 4, one of the points that kept coming up was the reluctance to introduce a bazillion new keywords to address all the cases that were being brought up. The impression I got is that adding new keywords was essentially an anti-pattern. And so when I’m reading through this onslaught of emails, I’m troubled by how everything is seeming to require new keywords. There’s the obvious async/await, but there’s also been discussion of actor, reliable, distributed, behavior, message, and signal (and I’ve probably missed others).

I can’t speak for message/signal, but you need to understand a bit more about how Swift works. There is a distinction between an actual keyword (which ‘async’ would be, and ‘class’ currently is) and “modifiers”. Modifiers occur with attributes ahead of a real keyword, but they are not themselves keywords. They are things like weak, mutating, reliable, distributed, etc. If we go with the “actor class” and “actor func” approach, then actor would not be a keyword.

Concrete example, this is (weird but) valid code:

var weak = 42
weak += 2
print(weak+weak)

This is a consequence of weak not being a keyword.

-Chris

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution