Contextualizing async coroutines

The coroutine proposal as it stands essentially exposes raw delimited continuations. While this is a flexible and expressive feature in the abstract, for the concrete purpose of representing asynchronous coroutines, it provides weak user-level guarantees about where their code might be running after being resumed from suspension, and puts a lot of pressure on APIs to be well-behaved in this respect. And if we're building toward actors, where async actor methods should be guaranteed to run "in the actor", I think we'll *need* something more than the bare-bones delimited continuation approach to get there. I think the proposal's desire to keep coroutines independent of a specific runtime model is a good idea, but I also think there are a couple possible modifications we could add to the design to make it easier to reason about what context things run in for any runtime model that benefits from async/await:

# Coroutine context

Associating a context value with a coroutine would let us thread useful information through the execution of the coroutine. This is particularly useful for GCD, so you could attach a queue, QoS, and other attributes to the coroutine, since these aren't reliably available from the global environment. It could be a performance improvement even for things like per-pthread queues, since coroutine context should be cheaper to access than pthread_self.

For example, a coroutine-aware `dispatch_async` could spawn a coroutine with the queue object and other interesting attributes as its context:

extension DispatchQueue {
func `async`(_ body: () async -> ()) {
   dispatch_async(self, {
     beginAsync(context: self) { await body() }
   })
}
}

and well-behaved dispatch-aware async APIs could use the context to decide how they should schedule completion:

func asyncOverlay() async -> T {
// If the coroutine is associated with a queue, schedule completion on that queue
if let currentQueue = getCoroutineContext() as? DispatchQueue {
   if #available(iOS 23, macOS 10.24, *) {
     // Maybe Apple frameworks add APIs that let you control completion dispatch up front…
     suspendAsync { continuation in
       originalAPI(on: currentQueue, completion: continuation)
     }
   } else {
     // …but we still need to queue-hop explicitly for backward deployment
     suspendAsync { continuation in
       originalAPI(completion: { dispatch_async(currentQueue, continuation) })
     }
   }
} else {
   // If the coroutine isn't associated with a queue, leave it up to the API to schedule continuation
   suspendAsync { continuation in
     originalAPI(completion: continuation)
   }
}
}

Similarly, if you've built your own framework on another platform with per-thread event loops and want to maintain thread affinity for coroutines through your APIs, you could similarly provide APIs that beginAsync with a thread ID as context and use that context to figure out where to schedule the continuation when you do something that suspends.

# `onResume` hooks

Relying on coroutine context alone still leaves responsibility wholly on suspending APIs to pay attention to the coroutine context and schedule the continuation correctly. You'd still have the expression problem when coroutine-spawning APIs from one framework interact with suspending APIs from another framework that doesn't understand the spawning framework's desired scheduling policy. We could provide some defense against this by letting the coroutine control its own resumption with an "onResume" hook, which would run when a suspended continuation is invoked instead of immediately resuming the coroutine. That would let the coroutine-aware dispatch_async example from above do something like this, to ensure the continuation always ends up back on the correct queue:

extension DispatchQueue {
func `async`(_ body: () async -> ()) {
   dispatch_async(self, {
     beginAsync(
       context: self,
       body: { await body() },
       onResume: { continuation in
         // Defensively hop to the right queue
         dispatch_async(self, continuation)
       }
     )
   })
}
}

This would let spawning APIs provide a stronger guarantee that the spawned coroutine is always executing as if scheduled by a specific queue/actor/event loop/HWND/etc., even if later suspended by an async API working in a different paradigm. This would also let you more strongly associate a coroutine with a future object representing its completion:

class CoroutineFuture<T> {
enum State {
   case busy // currently running
   case suspended(() -> ()) // suspended
   case success(T) // completed with success
   case failure(Error) // completed with error
}

var state: State = .busy

init(_ body: () async -> T) {

   beginAsync(
     body: {
       do {
         self.state = .success(await body())
       } catch {
         self.state = .failure(error)
       }
     },
     onResume: { continuation in
       assert(self.state == .busy, "already running?!")
       self.state = .suspended(continuation)
     }
   }
}

// Return the result of the future, or try to make progress computing it
func poll() throws -> T? {
   switch state {
   case .busy:
     return nil
   case .suspended(let cont):
     cont()
     switch state {
     case .success(let value):
       return value
     case .failure(let error):
       throw error
     case .busy, .suspended:
       return nil
     }
   case .success(let value):
     return value
   case .error(let error):
     throw error
}
}

A downside of this design is that it incurs some cost from defensive rescheduling on the continuation side, and also prevents writing APIs that intentionally change context across an `await`, like a theoretical "goToMainThread()" function (though you could do that by spawning a semantically-independent coroutine associated with the main thread, which might be a better design anyway).

-Joe

The coroutine proposal as it stands essentially exposes raw delimited continuations. While this is a flexible and expressive feature in the abstract, for the concrete purpose of representing asynchronous coroutines, it provides weak user-level guarantees about where their code might be running after being resumed from suspension, and puts a lot of pressure on APIs to be well-behaved in this respect. And if we're building toward actors, where async actor methods should be guaranteed to run "in the actor", I think we'll *need* something more than the bare-bones delimited continuation approach to get there. I think the proposal's desire to keep coroutines independent of a specific runtime model is a good idea, but I also think there are a couple possible modifications we could add to the design to make it easier to reason about what context things run in for any runtime model that benefits from async/await:

# Coroutine context

Associating a context value with a coroutine would let us thread useful information through the execution of the coroutine. This is particularly useful for GCD, so you could attach a queue, QoS, and other attributes to the coroutine, since these aren't reliably available from the global environment. It could be a performance improvement even for things like per-pthread queues, since coroutine context should be cheaper to access than pthread_self.

[...]

YES!

We need that. You're very focused on performance and affinity and whatnot here, but knowing where the completion will run upfront is critical for priority inheritance purposes.

This is exactly the spirit of the mail I just wrote in reply to Chris a bit earlier tonight. Execution context matters to the OS, a lot.

The OS needs to know two things:
- where is the precursor of this coroutine (which work is preventing the coroutine to execute)
- where will the coroutine go (which for GCD is critical because the OS lazily attributes threads, so any typical OS primitive to raise an existing thread priority doesn't work)

In other words, a coroutine needs:
- various tags (QoS, logging context, ...)
- precursors / reverse dependencies
- where it will execute (whether it's a dispatch queue or a runloop is completely irrelevant though).

And then if you do it that way when the precursor fires and allows for your coroutine to be scheduled, then it can actually schedule it right away on the right execution context and minimize context switches (which are way worse than shared mutable state for your performance).

# `onResume` hooks

Relying on coroutine context alone still leaves responsibility wholly on suspending APIs to pay attention to the coroutine context and schedule the continuation correctly. You'd still have the expression problem when coroutine-spawning APIs from one framework interact with suspending APIs from another framework that doesn't understand the spawning framework's desired scheduling policy. We could provide some defense against this by letting the coroutine control its own resumption with an "onResume" hook, which would run when a suspended continuation is invoked instead of immediately resuming the coroutine. That would let the coroutine-aware dispatch_async example from above do something like this, to ensure the continuation always ends up back on the correct queue:

extension DispatchQueue {
func `async`(_ body: () async -> ()) {
  dispatch_async(self, {
    beginAsync(
      context: self,
      body: { await body() },
      onResume: { continuation in
        // Defensively hop to the right queue
        dispatch_async(self, continuation)
      }
    )
  })
}
}

This would let spawning APIs provide a stronger guarantee that the spawned coroutine is always executing as if scheduled by a specific queue/actor/event loop/HWND/etc., even if later suspended by an async API working in a different paradigm. This would also let you more strongly associate a coroutine with a future object representing its completion:

class CoroutineFuture<T> {
enum State {
  case busy // currently running
  case suspended(() -> ()) // suspended
  case success(T) // completed with success
  case failure(Error) // completed with error
}

var state: State = .busy

init(_ body: () async -> T) {

  beginAsync(
    body: {
      do {
        self.state = .success(await body())
      } catch {
        self.state = .failure(error)
      }
    },
    onResume: { continuation in
      assert(self.state == .busy, "already running?!")
      self.state = .suspended(continuation)
    }
  }
}

// Return the result of the future, or try to make progress computing it
func poll() throws -> T? {
  switch state {
  case .busy:
    return nil
  case .suspended(let cont):
    cont()
    switch state {
    case .success(let value):
      return value
    case .failure(let error):
      throw error
    case .busy, .suspended:
      return nil
    }
  case .success(let value):
    return value
  case .error(let error):
    throw error
}
}

A downside of this design is that it incurs some cost from defensive rescheduling on the continuation side, and also prevents writing APIs that intentionally change context across an `await`, like a theoretical "goToMainThread()" function (though you could do that by spawning a semantically-independent coroutine associated with the main thread, which might be a better design anyway).

Given the limitations, I'm very skeptical. Also in general suspending/resuming work is very difficult to handle for a runtime (implementation wise), has large memory costs, and breaks priority inversion avoidance. dispatch_suspend()/dispatch_resume() is one of the banes of my existence when it comes to dispatch API surface. It only makes sense for dispatch source "I don't want to receive these events anymore for a while" is a perfectly valid thing to say or do. But suspending a queue or work is ripping the carpet from under the feet of the OS as you just basically make all work that is depending on the suspended one invisible and impossible to reason about.

The proper way to do something akin to suspension is really to "fail" your operation with a "You need to redrive me later", or implement an event monitoring system inside the subsystem providing the Actor that wants suspension to have the client handle the redrive/monitoring, this way the priority relationship is established and the OS can reason about it. Said another way, the Actor should fail with an error that gives you some kind of "resume token" that the requestor can hold and redrive according to his own rules and in a way that it is clear he's the waiter. Most of the time suspension() is a waiting-on-behalf-of relationship and this is a bad thing to build (except in priority homogenous environments, which iOS/macOS are *not*).

Also implementing the state you described requires more synchronization than you want to be useful: if you want to take action after observing a state, then you really really really don't want that state to change while you perform the consequence. the "on$Event" hook approach (which dispatch uses for dispatch sources e.g.) is much better because the ordering and serialization is provided by the actor itself. The only states that are valid to expose as a getter are states that you cannot go back from: succes, failure, error, canceled are all perfectly fine states to expose as getters because they only change state once. .suspended/.busy is not such a thing.

FWIW dispatch sources, and more importantly dispatch mach channels (which is the private interface that is used to implement XPC Connections) have a design that try really really really hard to not fall into any these pitfalls, are priority inheritance friendly, execute on *distributed* execution contexts, and have a state machine exposed through "on$Event" callbacks. We should benefit from the many years of experience that are condensed in these implementations when thinking about Actors and the primitives they provide.

-Pierre

···

On Aug 31, 2017, at 11:35 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

# Coroutine context

# `onResume` hooks

`onResume` hooks seem like a really good way to, essentially, allow arbitrary concurrency primitives to be passed into `async` functions. My main question is, if we have it, why do we need coroutine contexts? It seems to me that anything the coroutine might do with the context could either be passed into its parameters, or encapsulated in its `onResume` hook.

and also prevents writing APIs that intentionally change context across an `await`, like a theoretical "goToMainThread()" function

You say that like it's a bad thing. :^)

(Seriously, I feel like that makes it *way* too easy to jump to another thread in the middle of a function using what looks like an ordinary line of code. A `let result = await someQueue.async { … }` syntax is a lot clearer that you're hopping to another thread and about what code will run on it, but can get around our current "pyramid of doom" problem. If the async syntax guarantees that the code before and after the `await` runs in the same "environment"—whatever environment is set up by the onResume hook—then that makes async functions much safer and easier for programmers to reason about, while only restricting their power slightly.)

···

On Aug 31, 2017, at 11:35 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

--
Brent Royal-Gordon
Architechies

The coroutine proposal as it stands essentially exposes raw delimited continuations. While this is a flexible and expressive feature in the abstract, for the concrete purpose of representing asynchronous coroutines, it provides weak user-level guarantees about where their code might be running after being resumed from suspension, and puts a lot of pressure on APIs to be well-behaved in this respect. And if we're building toward actors, where async actor methods should be guaranteed to run "in the actor", I think we'll *need* something more than the bare-bones delimited continuation approach to get there. I think the proposal's desire to keep coroutines independent of a specific runtime model is a good idea, but I also think there are a couple possible modifications we could add to the design to make it easier to reason about what context things run in for any runtime model that benefits from async/await:

# Coroutine context

Associating a context value with a coroutine would let us thread useful information through the execution of the coroutine. This is particularly useful for GCD, so you could attach a queue, QoS, and other attributes to the coroutine, since these aren't reliably available from the global environment. It could be a performance improvement even for things like per-pthread queues, since coroutine context should be cheaper to access than pthread_self.

For example, a coroutine-aware `dispatch_async` could spawn a coroutine with the queue object and other interesting attributes as its context:

extension DispatchQueue {
func `async`(_ body: () async -> ()) {
  dispatch_async(self, {
    beginAsync(context: self) { await body() }
  })
}
}

I think it makes perfect sense to add a magically available context to async functions, and something like the above is a good way to populate it. Because is is a magic value that is only available in async functions, giving it a keyword like asyncContext might make sense. That said, I don’t understand how (by itself) this helps the queue hopping problem.

# `onResume` hooks

Relying on coroutine context alone still leaves responsibility wholly on suspending APIs to pay attention to the coroutine context and schedule the continuation correctly. You'd still have the expression problem when coroutine-spawning APIs from one framework interact with suspending APIs from another framework that doesn't understand the spawning framework's desired scheduling policy. We could provide some defense against this by letting the coroutine control its own resumption with an "onResume" hook, which would run when a suspended continuation is invoked instead of immediately resuming the coroutine. That would let the coroutine-aware dispatch_async example from above do something like this, to ensure the continuation always ends up back on the correct queue:

Yes, we need something like this, though I’m not sure how your proposal works:

extension DispatchQueue {
func `async`(_ body: () async -> ()) {
  dispatch_async(self, {
    beginAsync(
      context: self,
      body: { await body() },
      onResume: { continuation in
        // Defensively hop to the right queue
        dispatch_async(self, continuation)

If I’m running on a pthread, and use "someQueue.async {…}”, I don’t see how DispatchQueue.async can know how to take me back to a pthread. If I understand your example code above, it looks like the call will run the continuation on someQueue instead.

That said, I think that your idea of context pretty much covers it: a non-async function cannot have any idea whether it is run on a queue or thread, but there is also no language way to call an async function from a non-async function. I think this means that beginAsync and DispatchQueue.async will have to define some policy: for example, the implementation of DispatchQueue.async could use its own internal data structures to decide if the current task is being run on some dispatch queue (using the maligned “give me the current queue” operation), and ensure that it returns to the originating queue if it can find one.

Chains of async functions calling each other would maintain their context, so the other transitions we have to worry about are when an async function calls a non-async function (this just drops the context) or when you get to the bottom of the pile of async :turtle:’s and want to actually do something on another context. This implies you’d actually want an async form of DispatchQueue.async, something like this:

extension DispatchQueue {
func `async`(_ body: () async -> ()) async {
  dispatch_async(self) {
    beginAsync(
      context: self,
      body: {
         await body()
         asyncContext.restore()
      })
  }}

Going back to the silly example, if you call DispatchQueue.async from an async function on a pthread, the asyncContext would be the pthread’s, and asyncContext.restore() would take you back to it.

Another nice thing about this is that it gets you back to a single context per async thing.

-Chris

···

On Aug 31, 2017, at 11:35 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

Hello,

···

On 31. Aug 2017, at 20:35, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

# `onResume` hooks

Relying on coroutine context alone still leaves responsibility wholly on suspending APIs to pay attention to the coroutine context and schedule the continuation correctly. You'd still have the expression problem when coroutine-spawning APIs from one framework interact with suspending APIs from another framework that doesn't understand the spawning framework's desired scheduling policy. We could provide some defense against this by letting the coroutine control its own resumption with an "onResume" hook, which would run when a suspended continuation is invoked instead of immediately resuming the coroutine. That would let the coroutine-aware dispatch_async example from above do something like this, to ensure the continuation always ends up back on the correct queue:

I do feel the “correct” scheduling should be done by the actual scheduler / run-loop system, so we’re not paying the context switch cost twice. Of course this means that the run-time / OS needs enough information to do this correctly.

To be honest, the enqueuing of messages to actors seems quite straight-forward, but I’m having trouble envisioning the draining / processing of the messages.

  Daniel.

The coroutine proposal as it stands essentially exposes raw delimited continuations. While this is a flexible and expressive feature in the abstract, for the concrete purpose of representing asynchronous coroutines, it provides weak user-level guarantees about where their code might be running after being resumed from suspension, and puts a lot of pressure on APIs to be well-behaved in this respect. And if we're building toward actors, where async actor methods should be guaranteed to run "in the actor", I think we'll *need* something more than the bare-bones delimited continuation approach to get there. I think the proposal's desire to keep coroutines independent of a specific runtime model is a good idea, but I also think there are a couple possible modifications we could add to the design to make it easier to reason about what context things run in for any runtime model that benefits from async/await:

# Coroutine context

Associating a context value with a coroutine would let us thread useful information through the execution of the coroutine. This is particularly useful for GCD, so you could attach a queue, QoS, and other attributes to the coroutine, since these aren't reliably available from the global environment. It could be a performance improvement even for things like per-pthread queues, since coroutine context should be cheaper to access than pthread_self.

[...]

YES!

We need that. You're very focused on performance and affinity and whatnot here, but knowing where the completion will run upfront is critical for priority inheritance purposes.

This is exactly the spirit of the mail I just wrote in reply to Chris a bit earlier tonight. Execution context matters to the OS, a lot.

The OS needs to know two things:
- where is the precursor of this coroutine (which work is preventing the coroutine to execute)
- where will the coroutine go (which for GCD is critical because the OS lazily attributes threads, so any typical OS primitive to raise an existing thread priority doesn't work)

In other words, a coroutine needs:
- various tags (QoS, logging context, ...)
- precursors / reverse dependencies
- where it will execute (whether it's a dispatch queue or a runloop is completely irrelevant though).

And then if you do it that way when the precursor fires and allows for your coroutine to be scheduled, then it can actually schedule it right away on the right execution context and minimize context switches (which are way worse than shared mutable state for your performance).

# `onResume` hooks

Relying on coroutine context alone still leaves responsibility wholly on suspending APIs to pay attention to the coroutine context and schedule the continuation correctly. You'd still have the expression problem when coroutine-spawning APIs from one framework interact with suspending APIs from another framework that doesn't understand the spawning framework's desired scheduling policy. We could provide some defense against this by letting the coroutine control its own resumption with an "onResume" hook, which would run when a suspended continuation is invoked instead of immediately resuming the coroutine. That would let the coroutine-aware dispatch_async example from above do something like this, to ensure the continuation always ends up back on the correct queue:

extension DispatchQueue {
func `async`(_ body: () async -> ()) {
  dispatch_async(self, {
    beginAsync(
      context: self,
      body: { await body() },
      onResume: { continuation in
        // Defensively hop to the right queue
        dispatch_async(self, continuation)
      }
    )
  })
}
}

This would let spawning APIs provide a stronger guarantee that the spawned coroutine is always executing as if scheduled by a specific queue/actor/event loop/HWND/etc., even if later suspended by an async API working in a different paradigm. This would also let you more strongly associate a coroutine with a future object representing its completion:

class CoroutineFuture<T> {
enum State {
  case busy // currently running
  case suspended(() -> ()) // suspended
  case success(T) // completed with success
  case failure(Error) // completed with error
}

var state: State = .busy

init(_ body: () async -> T) {

  beginAsync(
    body: {
      do {
        self.state = .success(await body())
      } catch {
        self.state = .failure(error)
      }
    },
    onResume: { continuation in
      assert(self.state == .busy, "already running?!")
      self.state = .suspended(continuation)
    }
  }
}

// Return the result of the future, or try to make progress computing it
func poll() throws -> T? {
  switch state {
  case .busy:
    return nil
  case .suspended(let cont):
    cont()
    switch state {
    case .success(let value):
      return value
    case .failure(let error):
      throw error
    case .busy, .suspended:
      return nil
    }
  case .success(let value):
    return value
  case .error(let error):
    throw error
}
}

A downside of this design is that it incurs some cost from defensive rescheduling on the continuation side, and also prevents writing APIs that intentionally change context across an `await`, like a theoretical "goToMainThread()" function (though you could do that by spawning a semantically-independent coroutine associated with the main thread, which might be a better design anyway).

Given the limitations, I'm very skeptical. Also in general suspending/resuming work is very difficult to handle for a runtime (implementation wise), has large memory costs, and breaks priority inversion avoidance. dispatch_suspend()/dispatch_resume() is one of the banes of my existence when it comes to dispatch API surface. It only makes sense for dispatch source "I don't want to receive these events anymore for a while" is a perfectly valid thing to say or do. But suspending a queue or work is ripping the carpet from under the feet of the OS as you just basically make all work that is depending on the suspended one invisible and impossible to reason about.

Sorry, I was using the term 'suspend' somewhat imprecisely. I was specifically referring to an operation that semantically pauses the coroutine and gives you its continuation closure, to be handed off as a completion handler or something of that sort, not something that would block the thread or suspend the queue. Execution would return back up the non-async layer at the point this happens.

-Joe

···

On Aug 31, 2017, at 7:50 PM, Pierre Habouzit <phabouzit@apple.com> wrote:

On Aug 31, 2017, at 11:35 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

The proper way to do something akin to suspension is really to "fail" your operation with a "You need to redrive me later", or implement an event monitoring system inside the subsystem providing the Actor that wants suspension to have the client handle the redrive/monitoring, this way the priority relationship is established and the OS can reason about it. Said another way, the Actor should fail with an error that gives you some kind of "resume token" that the requestor can hold and redrive according to his own rules and in a way that it is clear he's the waiter. Most of the time suspension() is a waiting-on-behalf-of relationship and this is a bad thing to build (except in priority homogenous environments, which iOS/macOS are *not*).

Also implementing the state you described requires more synchronization than you want to be useful: if you want to take action after observing a state, then you really really really don't want that state to change while you perform the consequence. the "on$Event" hook approach (which dispatch uses for dispatch sources e.g.) is much better because the ordering and serialization is provided by the actor itself. The only states that are valid to expose as a getter are states that you cannot go back from: succes, failure, error, canceled are all perfectly fine states to expose as getters because they only change state once. .suspended/.busy is not such a thing.

FWIW dispatch sources, and more importantly dispatch mach channels (which is the private interface that is used to implement XPC Connections) have a design that try really really really hard to not fall into any these pitfalls, are priority inheritance friendly, execute on *distributed* execution contexts, and have a state machine exposed through "on$Event" callbacks. We should benefit from the many years of experience that are condensed in these implementations when thinking about Actors and the primitives they provide.

-Pierre

I think you are raising very important concerns. As the proposal stood I was concerned that it would lead people to write poor quality code, in particular hard to debug deadlock prone code.

I see the proposal as it stands as exposing the equivalent of a pointer in concurrent programming and everyone using it like they do in C with disastrous consequences; because it is built in and therefore considered, naively, to be the ‘proper’ way to do it. I think that Swift gets it right with UnsafePointer etc., perhaps we also need UnsafeAsync etc. to remind people they are playing with fire.

I would also add into the mix:

  1. Timeouts, since these are required to break deadlocks or at least detect them.
  2. Cancel; users change there mind.

Glad the proposal is been expanded.

-- Howard.

···

On 1 Sep 2017, at 12:50 pm, Pierre Habouzit via swift-evolution <swift-evolution@swift.org> wrote:

On Aug 31, 2017, at 11:35 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

The coroutine proposal as it stands essentially exposes raw delimited continuations. While this is a flexible and expressive feature in the abstract, for the concrete purpose of representing asynchronous coroutines, it provides weak user-level guarantees about where their code might be running after being resumed from suspension, and puts a lot of pressure on APIs to be well-behaved in this respect. And if we're building toward actors, where async actor methods should be guaranteed to run "in the actor", I think we'll *need* something more than the bare-bones delimited continuation approach to get there. I think the proposal's desire to keep coroutines independent of a specific runtime model is a good idea, but I also think there are a couple possible modifications we could add to the design to make it easier to reason about what context things run in for any runtime model that benefits from async/await:

# Coroutine context

Associating a context value with a coroutine would let us thread useful information through the execution of the coroutine. This is particularly useful for GCD, so you could attach a queue, QoS, and other attributes to the coroutine, since these aren't reliably available from the global environment. It could be a performance improvement even for things like per-pthread queues, since coroutine context should be cheaper to access than pthread_self.

[...]

YES!

We need that. You're very focused on performance and affinity and whatnot here, but knowing where the completion will run upfront is critical for priority inheritance purposes.

This is exactly the spirit of the mail I just wrote in reply to Chris a bit earlier tonight. Execution context matters to the OS, a lot.

The OS needs to know two things:
- where is the precursor of this coroutine (which work is preventing the coroutine to execute)
- where will the coroutine go (which for GCD is critical because the OS lazily attributes threads, so any typical OS primitive to raise an existing thread priority doesn't work)

In other words, a coroutine needs:
- various tags (QoS, logging context, ...)
- precursors / reverse dependencies
- where it will execute (whether it's a dispatch queue or a runloop is completely irrelevant though).

And then if you do it that way when the precursor fires and allows for your coroutine to be scheduled, then it can actually schedule it right away on the right execution context and minimize context switches (which are way worse than shared mutable state for your performance).

# `onResume` hooks

Relying on coroutine context alone still leaves responsibility wholly on suspending APIs to pay attention to the coroutine context and schedule the continuation correctly. You'd still have the expression problem when coroutine-spawning APIs from one framework interact with suspending APIs from another framework that doesn't understand the spawning framework's desired scheduling policy. We could provide some defense against this by letting the coroutine control its own resumption with an "onResume" hook, which would run when a suspended continuation is invoked instead of immediately resuming the coroutine. That would let the coroutine-aware dispatch_async example from above do something like this, to ensure the continuation always ends up back on the correct queue:

extension DispatchQueue {
func `async`(_ body: () async -> ()) {
  dispatch_async(self, {
    beginAsync(
      context: self,
      body: { await body() },
      onResume: { continuation in
        // Defensively hop to the right queue
        dispatch_async(self, continuation)
      }
    )
  })
}
}

This would let spawning APIs provide a stronger guarantee that the spawned coroutine is always executing as if scheduled by a specific queue/actor/event loop/HWND/etc., even if later suspended by an async API working in a different paradigm. This would also let you more strongly associate a coroutine with a future object representing its completion:

class CoroutineFuture<T> {
enum State {
  case busy // currently running
  case suspended(() -> ()) // suspended
  case success(T) // completed with success
  case failure(Error) // completed with error
}

var state: State = .busy

init(_ body: () async -> T) {

  beginAsync(
    body: {
      do {
        self.state = .success(await body())
      } catch {
        self.state = .failure(error)
      }
    },
    onResume: { continuation in
      assert(self.state == .busy, "already running?!")
      self.state = .suspended(continuation)
    }
  }
}

// Return the result of the future, or try to make progress computing it
func poll() throws -> T? {
  switch state {
  case .busy:
    return nil
  case .suspended(let cont):
    cont()
    switch state {
    case .success(let value):
      return value
    case .failure(let error):
      throw error
    case .busy, .suspended:
      return nil
    }
  case .success(let value):
    return value
  case .error(let error):
    throw error
}
}

A downside of this design is that it incurs some cost from defensive rescheduling on the continuation side, and also prevents writing APIs that intentionally change context across an `await`, like a theoretical "goToMainThread()" function (though you could do that by spawning a semantically-independent coroutine associated with the main thread, which might be a better design anyway).

Given the limitations, I'm very skeptical. Also in general suspending/resuming work is very difficult to handle for a runtime (implementation wise), has large memory costs, and breaks priority inversion avoidance. dispatch_suspend()/dispatch_resume() is one of the banes of my existence when it comes to dispatch API surface. It only makes sense for dispatch source "I don't want to receive these events anymore for a while" is a perfectly valid thing to say or do. But suspending a queue or work is ripping the carpet from under the feet of the OS as you just basically make all work that is depending on the suspended one invisible and impossible to reason about.

The proper way to do something akin to suspension is really to "fail" your operation with a "You need to redrive me later", or implement an event monitoring system inside the subsystem providing the Actor that wants suspension to have the client handle the redrive/monitoring, this way the priority relationship is established and the OS can reason about it. Said another way, the Actor should fail with an error that gives you some kind of "resume token" that the requestor can hold and redrive according to his own rules and in a way that it is clear he's the waiter. Most of the time suspension() is a waiting-on-behalf-of relationship and this is a bad thing to build (except in priority homogenous environments, which iOS/macOS are *not*).

Also implementing the state you described requires more synchronization than you want to be useful: if you want to take action after observing a state, then you really really really don't want that state to change while you perform the consequence. the "on$Event" hook approach (which dispatch uses for dispatch sources e.g.) is much better because the ordering and serialization is provided by the actor itself. The only states that are valid to expose as a getter are states that you cannot go back from: succes, failure, error, canceled are all perfectly fine states to expose as getters because they only change state once. .suspended/.busy is not such a thing.

FWIW dispatch sources, and more importantly dispatch mach channels (which is the private interface that is used to implement XPC Connections) have a design that try really really really hard to not fall into any these pitfalls, are priority inheritance friendly, execute on *distributed* execution contexts, and have a state machine exposed through "on$Event" callbacks. We should benefit from the many years of experience that are condensed in these implementations when thinking about Actors and the primitives they provide.

-Pierre
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

-Pierre

# Coroutine context

# `onResume` hooks

`onResume` hooks seem like a really good way to, essentially, allow arbitrary concurrency primitives to be passed into `async` functions. My main question is, if we have it, why do we need coroutine contexts? It seems to me that anything the coroutine might do with the context could either be passed into its parameters, or encapsulated in its `onResume` hook.

No it's not the same. Arbitrary code is this: arbitrary code and data.

Please read the few mails I sent recently about this, but to recap here quickly:

It is needed for the runtime (in a broad sense, from language to the operating system) to be able to introspect these:
- priority attributes
- dependencies
- execution contexts (thread/queue/runloop/...)
- ownership

Without this info, the scheduling of these coroutines will essentially be random, subject to priority inversions and other similar issues.

and also prevents writing APIs that intentionally change context across an `await`, like a theoretical "goToMainThread()" function

You say that like it's a bad thing. :^)

(Seriously, I feel like that makes it *way* too easy to jump to another thread in the middle of a function using what looks like an ordinary line of code. A `let result = await someQueue.async { … }` syntax is a lot clearer that you're hopping to another thread and about what code will run on it, but can get around our current "pyramid of doom" problem. If the async syntax guarantees that the code before and after the `await` runs in the same "environment"—whatever environment is set up by the onResume hook—then that makes async functions much safer and easier for programmers to reason about, while only restricting their power slightly.)

I do agree that an explicit syntax that spells out the context when it changes is really preferable, and if no context is provided it should mean "the current one".
The context doesn't necessarily need to be a dispatch queue.

Ideally API surface that provide similar things could be annotated for Swift to reason about, e.g. to take a realatively obvious example:

  void xpc_connection_send_message_with_reply(xpc_connection_t conn, xpc_object_t msg, dispatch_queue_t queue, void (^handler)(xpc_object_t reply))

Could provided sufficient overlay, could be understood by Swift and provide something like this (completely random syntax, I barely do any Swift, so bear with me):

  let reply = await(queue) connection.sendQuery(message: msg)

Because the API could be annotated with `queue` in that function is the execution context for the coroutine, and `handler` is the said coroutine.

-Pierre

···

On Sep 2, 2017, at 1:58 AM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Aug 31, 2017, at 11:35 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

# Coroutine context

# `onResume` hooks

`onResume` hooks seem like a really good way to, essentially, allow arbitrary concurrency primitives to be passed into `async` functions. My main question is, if we have it, why do we need coroutine contexts? It seems to me that anything the coroutine might do with the context could either be passed into its parameters, or encapsulated in its `onResume` hook.

To be clear, I'm *not* claiming we necessarily need both. I'm exploring two options here to the problem of "how can I ensure a coroutine's execution is consistently associated with a specific context", from different directions. Context values allow information about the desired execution context to be pushed through the actual execution, which is the most efficient thing if all the code is cooperating, although it's still up to the yielding operation to use that information. 'onResume' can pull a resumed coroutine back into whatever context it expects to run in, which imposes some fundamental overhead in the form of "queue-hopping" in the GCD model or other rescheduling, but on the other hand can provide stronger guarantees about where execution is happening even without perfect cooperation across all code.

-Joe

···

On Sep 2, 2017, at 1:57 AM, Brent Royal-Gordon <brent@architechies.com> wrote:

On Aug 31, 2017, at 11:35 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

and also prevents writing APIs that intentionally change context across an `await`, like a theoretical "goToMainThread()" function

You say that like it's a bad thing. :^)

(Seriously, I feel like that makes it *way* too easy to jump to another thread in the middle of a function using what looks like an ordinary line of code. A `let result = await someQueue.async { … }` syntax is a lot clearer that you're hopping to another thread and about what code will run on it, but can get around our current "pyramid of doom" problem. If the async syntax guarantees that the code before and after the `await` runs in the same "environment"—whatever environment is set up by the onResume hook—then that makes async functions much safer and easier for programmers to reason about, while only restricting their power slightly.)

--
Brent Royal-Gordon
Architechies

The coroutine proposal as it stands essentially exposes raw delimited continuations. While this is a flexible and expressive feature in the abstract, for the concrete purpose of representing asynchronous coroutines, it provides weak user-level guarantees about where their code might be running after being resumed from suspension, and puts a lot of pressure on APIs to be well-behaved in this respect. And if we're building toward actors, where async actor methods should be guaranteed to run "in the actor", I think we'll *need* something more than the bare-bones delimited continuation approach to get there. I think the proposal's desire to keep coroutines independent of a specific runtime model is a good idea, but I also think there are a couple possible modifications we could add to the design to make it easier to reason about what context things run in for any runtime model that benefits from async/await:

# Coroutine context

Associating a context value with a coroutine would let us thread useful information through the execution of the coroutine. This is particularly useful for GCD, so you could attach a queue, QoS, and other attributes to the coroutine, since these aren't reliably available from the global environment. It could be a performance improvement even for things like per-pthread queues, since coroutine context should be cheaper to access than pthread_self.

For example, a coroutine-aware `dispatch_async` could spawn a coroutine with the queue object and other interesting attributes as its context:

extension DispatchQueue {
func `async`(_ body: () async -> ()) {
dispatch_async(self, {
   beginAsync(context: self) { await body() }
})
}
}

I think it makes perfect sense to add a magically available context to async functions, and something like the above is a good way to populate it. Because is is a magic value that is only available in async functions, giving it a keyword like asyncContext might make sense. That said, I don’t understand how (by itself) this helps the queue hopping problem.

By itself, it doesn't, but the context can carry information from when the coroutine was spawned to functions that yield control from the coroutine, so that there's an opportunity to use the context information to schedule the continuation properly up-front, avoiding the need

# `onResume` hooks

Relying on coroutine context alone still leaves responsibility wholly on suspending APIs to pay attention to the coroutine context and schedule the continuation correctly. You'd still have the expression problem when coroutine-spawning APIs from one framework interact with suspending APIs from another framework that doesn't understand the spawning framework's desired scheduling policy. We could provide some defense against this by letting the coroutine control its own resumption with an "onResume" hook, which would run when a suspended continuation is invoked instead of immediately resuming the coroutine. That would let the coroutine-aware dispatch_async example from above do something like this, to ensure the continuation always ends up back on the correct queue:

Yes, we need something like this, though I’m not sure how your proposal works:

extension DispatchQueue {
func `async`(_ body: () async -> ()) {
dispatch_async(self, {
   beginAsync(
     context: self,
     body: { await body() },
     onResume: { continuation in
       // Defensively hop to the right queue
       dispatch_async(self, continuation)

If I’m running on a pthread, and use "someQueue.async {…}”, I don’t see how DispatchQueue.async can know how to take me back to a pthread. If I understand your example code above, it looks like the call will run the continuation on someQueue instead.

That's the intended behavior. `someQueue.async` never "runs on a pthread", it spawns a new coroutine that always runs on `someQueue`, just like the closure-based form always runs a synchronous block of code on `someQueue` today. You would need some other API to spawn a coroutine that schedules onto a specific thread using something other than dispatch.

That said, I think that your idea of context pretty much covers it: a non-async function cannot have any idea whether it is run on a queue or thread, but there is also no language way to call an async function from a non-async function. I think this means that beginAsync and DispatchQueue.async will have to define some policy: for example, the implementation of DispatchQueue.async could use its own internal data structures to decide if the current task is being run on some dispatch queue (using the maligned “give me the current queue” operation), and ensure that it returns to the originating queue if it can find one.

That's what I was going for.

-Joe

···

On Sep 2, 2017, at 12:07 PM, Chris Lattner <clattner@nondot.org> wrote:
On Aug 31, 2017, at 11:35 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

-Pierre

The coroutine proposal as it stands essentially exposes raw delimited continuations. While this is a flexible and expressive feature in the abstract, for the concrete purpose of representing asynchronous coroutines, it provides weak user-level guarantees about where their code might be running after being resumed from suspension, and puts a lot of pressure on APIs to be well-behaved in this respect. And if we're building toward actors, where async actor methods should be guaranteed to run "in the actor", I think we'll *need* something more than the bare-bones delimited continuation approach to get there. I think the proposal's desire to keep coroutines independent of a specific runtime model is a good idea, but I also think there are a couple possible modifications we could add to the design to make it easier to reason about what context things run in for any runtime model that benefits from async/await:

# Coroutine context

Associating a context value with a coroutine would let us thread useful information through the execution of the coroutine. This is particularly useful for GCD, so you could attach a queue, QoS, and other attributes to the coroutine, since these aren't reliably available from the global environment. It could be a performance improvement even for things like per-pthread queues, since coroutine context should be cheaper to access than pthread_self.

[...]

YES!

We need that. You're very focused on performance and affinity and whatnot here, but knowing where the completion will run upfront is critical for priority inheritance purposes.

This is exactly the spirit of the mail I just wrote in reply to Chris a bit earlier tonight. Execution context matters to the OS, a lot.

The OS needs to know two things:
- where is the precursor of this coroutine (which work is preventing the coroutine to execute)
- where will the coroutine go (which for GCD is critical because the OS lazily attributes threads, so any typical OS primitive to raise an existing thread priority doesn't work)

In other words, a coroutine needs:
- various tags (QoS, logging context, ...)
- precursors / reverse dependencies
- where it will execute (whether it's a dispatch queue or a runloop is completely irrelevant though).

And then if you do it that way when the precursor fires and allows for your coroutine to be scheduled, then it can actually schedule it right away on the right execution context and minimize context switches (which are way worse than shared mutable state for your performance).

# `onResume` hooks

Relying on coroutine context alone still leaves responsibility wholly on suspending APIs to pay attention to the coroutine context and schedule the continuation correctly. You'd still have the expression problem when coroutine-spawning APIs from one framework interact with suspending APIs from another framework that doesn't understand the spawning framework's desired scheduling policy. We could provide some defense against this by letting the coroutine control its own resumption with an "onResume" hook, which would run when a suspended continuation is invoked instead of immediately resuming the coroutine. That would let the coroutine-aware dispatch_async example from above do something like this, to ensure the continuation always ends up back on the correct queue:

extension DispatchQueue {
func `async`(_ body: () async -> ()) {
  dispatch_async(self, {
    beginAsync(
      context: self,
      body: { await body() },
      onResume: { continuation in
        // Defensively hop to the right queue
        dispatch_async(self, continuation)
      }
    )
  })
}
}

This would let spawning APIs provide a stronger guarantee that the spawned coroutine is always executing as if scheduled by a specific queue/actor/event loop/HWND/etc., even if later suspended by an async API working in a different paradigm. This would also let you more strongly associate a coroutine with a future object representing its completion:

class CoroutineFuture<T> {
enum State {
  case busy // currently running
  case suspended(() -> ()) // suspended
  case success(T) // completed with success
  case failure(Error) // completed with error
}

var state: State = .busy

init(_ body: () async -> T) {

  beginAsync(
    body: {
      do {
        self.state = .success(await body())
      } catch {
        self.state = .failure(error)
      }
    },
    onResume: { continuation in
      assert(self.state == .busy, "already running?!")
      self.state = .suspended(continuation)
    }
  }
}

// Return the result of the future, or try to make progress computing it
func poll() throws -> T? {
  switch state {
  case .busy:
    return nil
  case .suspended(let cont):
    cont()
    switch state {
    case .success(let value):
      return value
    case .failure(let error):
      throw error
    case .busy, .suspended:
      return nil
    }
  case .success(let value):
    return value
  case .error(let error):
    throw error
}
}

A downside of this design is that it incurs some cost from defensive rescheduling on the continuation side, and also prevents writing APIs that intentionally change context across an `await`, like a theoretical "goToMainThread()" function (though you could do that by spawning a semantically-independent coroutine associated with the main thread, which might be a better design anyway).

Given the limitations, I'm very skeptical. Also in general suspending/resuming work is very difficult to handle for a runtime (implementation wise), has large memory costs, and breaks priority inversion avoidance. dispatch_suspend()/dispatch_resume() is one of the banes of my existence when it comes to dispatch API surface. It only makes sense for dispatch source "I don't want to receive these events anymore for a while" is a perfectly valid thing to say or do. But suspending a queue or work is ripping the carpet from under the feet of the OS as you just basically make all work that is depending on the suspended one invisible and impossible to reason about.

Sorry, I was using the term 'suspend' somewhat imprecisely. I was specifically referring to an operation that semantically pauses the coroutine and gives you its continuation closure, to be handed off as a completion handler or something of that sort, not something that would block the thread or suspend the queue. Execution would return back up the non-async layer at the point this happens.

I wasn't worried about the "blocking the thread" but worried about an anonymous suspend/resume like dispatch has where you can't predict who will do the "resume".
I'm fine with a token that has a unique owner that will call it, because tracking who reference is will tell you who will resume and who needs to receive an override in a priority aware world if needed.

I think I'd rather have this operation called yield than suspend, because it's obvious you can't yield twice, and that it is a decision the actor does from its own code, and not an external entity that has the power to suspend you from the outside (like dispatch queues). That's clearly what threw me off here :wink:

···

On Aug 31, 2017, at 9:12 PM, Joe Groff <jgroff@apple.com> wrote:
On Aug 31, 2017, at 7:50 PM, Pierre Habouzit <phabouzit@apple.com <mailto:phabouzit@apple.com>> wrote:

On Aug 31, 2017, at 11:35 AM, Joe Groff via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

-Joe

The proper way to do something akin to suspension is really to "fail" your operation with a "You need to redrive me later", or implement an event monitoring system inside the subsystem providing the Actor that wants suspension to have the client handle the redrive/monitoring, this way the priority relationship is established and the OS can reason about it. Said another way, the Actor should fail with an error that gives you some kind of "resume token" that the requestor can hold and redrive according to his own rules and in a way that it is clear he's the waiter. Most of the time suspension() is a waiting-on-behalf-of relationship and this is a bad thing to build (except in priority homogenous environments, which iOS/macOS are *not*).

Also implementing the state you described requires more synchronization than you want to be useful: if you want to take action after observing a state, then you really really really don't want that state to change while you perform the consequence. the "on$Event" hook approach (which dispatch uses for dispatch sources e.g.) is much better because the ordering and serialization is provided by the actor itself. The only states that are valid to expose as a getter are states that you cannot go back from: succes, failure, error, canceled are all perfectly fine states to expose as getters because they only change state once. .suspended/.busy is not such a thing.

FWIW dispatch sources, and more importantly dispatch mach channels (which is the private interface that is used to implement XPC Connections) have a design that try really really really hard to not fall into any these pitfalls, are priority inheritance friendly, execute on *distributed* execution contexts, and have a state machine exposed through "on$Event" callbacks. We should benefit from the many years of experience that are condensed in these implementations when thinking about Actors and the primitives they provide.

-Pierre

I will freely admit that I don't understand all of these details, so in lieu of rebutting this, I will simply state what I'm saying more explicitly and ask you to explain why I'm wrong in smaller words. :^)

Basically, what I'm saying is: Why do the context details need to be available *within the async function*, rather than being available only to the resume hook?

For a GCD example, suppose the normal, C-based `dispatch_async` function is exposed to Swift as `__dispatch_async`, and `beginAsync` has a signature like this:

  // I don't think `rethrows` works this way, but pretend it did.
  //
  /// Starts running an asynchronous function which is started and restarted by `resumeHook`.
  ///
  /// - Parameter body: The async function to run.
  /// - Parameter resumeHook: A function called once for each time `body` starts or resumes running.
  /// It is passed a `continuation` function representing the next synchronous chunk of
  /// `body`, which it should run (or schedule to run). If the `continuation` throws or returns
  /// a non-`nil` value, the function has terminated, and the result should be handled
  /// appropriately. If the `continuation` returns `nil`, then it has not finished executing.
  func beginAsync<Return>(do body: () async throws -> Return, startingWith resumeHook: @escaping (_ continuation: @escaping () rethrows -> Return?) -> Void) { … }

You can then write async-function-handling versions of `async` like:

  extension DispatchQueue {
    // This version runs a nullary, non-throwing, Void async function, and can be called from non-async code.
    func async(qos: DispatchQoS = .default, flags: DispatchWorkItemFlags = [], execute body: () async -> Void) {
      beginAsync(do: body, startingWith: { continuation in
        let workItem = DispatchWorkItem(qos: qos, flags: flags) {
          _ = continuation()
        }
        __dispatch_async(self, workItem)
      })
    }
    
    // This version runs any (nullary) async function, and can be called from async code.
    func async<Return>(qos: DispatchQoS = .default, flags: DispatchWorkItemFlags = [], execute body: () async throws -> Return) async rethrows -> Return {
      return try await suspendAsync { successHandler, failureHandler in
        beginAsync(do: body, startingWith: { continuation in
          let workItem = DispatchWorkItem(qos: qos, flags: flags) {
            do {
              if let returnValue = try continuation() {
                successHandler(returnValue)
              }
            }
            catch {
              failureHandler(returnValue)
            }
          }
          __dispatch_async(self, workItem)
        })
      }
    }
  }

Information like the QoS is encapsulated by the closure, so that each time it enqueues another chunk of work, it attaches that information to it. Is that good enough? Or do you need more?

  * * *

I *think* you might be saying that, when GCD wants to run a given async block, it wants to be able to look ahead to where the `successHandler` will want to run so it can schedule the first block on a thread that will be able to immediately run the `successHandler`. But if so, that still only requires `suspendAsync` to extract the context and pass it to its parameter—it doesn't require arbitrary code running when the function is *not* suspended to access the context.

You could perhaps imagine the standard library providing these declarations:

  protocol AsyncContext {
    func resumeAsync(_ resumePoint: @escaping () -> Void)
  }
  struct AsyncContinuation<Returning, Throwing: Error> {
    // These are notionally packaged up here, although it might actually be implemented differently.
    private let successHandler: (Returning) -> Void
    private let failureHandler: (Throwing) -> Void
    
    func resumeAsync(in context: AsyncContext, returning value: Returning) {
      context.resumeAsync { successHandler(value) }
    }
    func resumeAsync(in context: AsyncContext, throwing error: Throwing) {
      context.resumeAsync { failureHandler(error) }
    }
  }
  func beginAsync(in context: AsyncContext, do body: () async -> Void) { … }
  func suspendAsync<Returning, Throwing>(_ handler: (AsyncContext, AsyncContinuation<Returning, Throwing>) -> Void) async throws<Throwing> -> Returning { … }

Then GCD could do something like:

  extension DispatchQueue {
    struct Context: AsyncContext {
      let queue: DispatchQueue
      let qos: DispatchQoS
      let flags: DispatchFlags
      let nextContext: Context?

      func resumeAsync(_ resumePoint: @escaping () -> Void) {
        let workItem = DispatchWorkItem(qos: qos, flags: flags, block: resumePoint)
        __dispatch_async_with_next_queue(queue, workItem, nextContext?.queue)
      }
    }
    
    // This version runs a nullary, non-throwing, Void async function, and can be called from non-async code.
    func async(qos: DispatchQoS = .default, flags: DispatchWorkItemFlags = [], execute body: () async -> Void) {
      beginAsync(in: Context(queue: self, qos: qos, flags: flags, nextContext: nil), do: body)
    }
    
    // This version runs any (nullary) async function, and can be called from async code.
    func async<Return>(qos: DispatchQoS = .default, flags: DispatchWorkItemFlags = [], execute body: () async throws -> Return) async rethrows -> Return {
      return try await suspendAsync { context, continuation in
        let newContext = Context(queue: self, qos: qos, flags: flags, nextContext: context as? DispatchQueue.Context)
        
        beginAsync(in: newContext) {
          do {
            continuation.resumeAsync(in: context, returning: try await body())
          }
          catch {
            continuation.resumeAsync(in: context, throwing: error)
          }
        }
      }
    }
  }

This allows GCD to look arbitrarily deep into the future, but the context can only be inspected at suspension points; it's otherwise encapsulated. The context is also now in control of execution, rather than being some passive data that may or may not be present and may or may not have any particular meaning.

  * * *

Actually, looking at this, it seems to me that `beginAsync(in:do:)` internally just creates a continuation for the beginning of an async function and resumes it. With a small language feature addition, we can have this in the standard library:

  protocol AsyncContext {
    // …as before…
  }
  struct AsyncContinuation<Returning, Throwing: Error> {
    // …as before…
  }
  extension AsyncContinuation where Throwing == Never {
    init(starting function: (#splat(Returning)) async -> Void) { … }
  }
  func suspendAsync<Returning, Throwing>(_ handler: (AsyncContext, AsyncContinuation<Returning, Throwing>) -> Void) async throws<Throwing> -> Returning { … }

And this in GCD:

  extension DispatchQueue {
    struct Context: AsyncContext {
      let queue: DispatchQueue
      let qos: DispatchQoS
      let flags: DispatchFlags
      let nextContext: Context?

      func resumeAsync(_ resumePoint: @escaping () -> Void) {
        let workItem = DispatchWorkItem(qos: qos, flags: flags, block: resumePoint)
        __dispatch_async_with_next_queue(queue, workItem, nextContext?.queue)
      }
    }
    
    // This version runs a nullary, non-throwing, Void async function, and can be called from non-async code.
    func async(qos: DispatchQoS = .default, flags: DispatchWorkItemFlags = [], execute body: () async -> Void) {
      let context = Context(queue: self, qos: qos, flags: flags, nextContext: nil)
      let starter = AsyncContinuation(starting: body)
      starter.resumeAsync(in: context, returning: ())
    }
    
    // This version runs any (nullary) async function, and can be called from async code.
    func async<Return>(qos: DispatchQoS = .default, flags: DispatchWorkItemFlags = [], execute body: () async throws -> Return) async rethrows -> Return {
      return try await suspendAsync { context, continuation in
        let newContext = Context(queue: self, qos: qos, flags: flags, nextContext: context as? DispatchQueue.Context)
        let starter = AsyncContinuation(starting: {
          do {
            continuation.resumeAsync(in: context, returning: try await body())
          }
          catch {
            continuation.resumeAsync(in: context, throwing: error)
          }
        })
        starter.resumeAsync(in: newContext, returning: ())
      }
    }
  }

We could even encapsulate the second version's chaining logic in `AsyncContinuation`:

  extension AsyncContinuation where Throwing == Never {
    init<StarterReturning, StarterThrowing>(starting starter: (#splat(Returning)) async throws<StarterThrowing> -> StarterReturning, returningTo continuation: AsyncContinuation<StarterReturning, StarterThrowing>, in returningContext: AsyncContext) {
      self.init(starting: {
        do {
          continuation.resumeAsync(in: returningContext, returning: try await starter())
        }
        catch {
          continuation.resumeAsync(in: returningContext, throwing: error)
        }
      })
    }
  }
  
  extension DispatchQueue {
    // …as before…
    
    // This version runs any (nullary) async function, and can be called from async code.
    func async<Return>(qos: DispatchQoS = .default, flags: DispatchWorkItemFlags = [], execute body: () async throws -> Return) async rethrows -> Return {
      return try await suspendAsync { context, continuation in
        let newContext = Context(queue: self, qos: qos, flags: flags, nextContext: context as? DispatchQueue.Context)
        let starter = AsyncContinuation(starting: body, returningTo: continuation, in: context)
        starter.resumeAsync(in: newContext, returning: ())
      }
    }
  }

Make `suspendAsync` a class method on `AsyncContinuation` and we've pretty much walled off all these low-level guts in a single type!

(P.S. Should the `AsyncContext` be a public property of the continuation? Maybe—that would make it harder to accidentally run continuations in the wrong context.)

···

On Sep 2, 2017, at 2:56 AM, Pierre Habouzit <phabouzit@apple.com> wrote:

`onResume` hooks seem like a really good way to, essentially, allow arbitrary concurrency primitives to be passed into `async` functions. My main question is, if we have it, why do we need coroutine contexts? It seems to me that anything the coroutine might do with the context could either be passed into its parameters, or encapsulated in its `onResume` hook.

No it's not the same. Arbitrary code is this: arbitrary code and data.

Please read the few mails I sent recently about this, but to recap here quickly:

It is needed for the runtime (in a broad sense, from language to the operating system) to be able to introspect these:
- priority attributes
- dependencies
- execution contexts (thread/queue/runloop/...)
- ownership

Without this info, the scheduling of these coroutines will essentially be random, subject to priority inversions and other similar issues.

--
Brent Royal-Gordon
Architechies

-Pierre

`onResume` hooks seem like a really good way to, essentially, allow arbitrary concurrency primitives to be passed into `async` functions. My main question is, if we have it, why do we need coroutine contexts? It seems to me that anything the coroutine might do with the context could either be passed into its parameters, or encapsulated in its `onResume` hook.

No it's not the same. Arbitrary code is this: arbitrary code and data.

Please read the few mails I sent recently about this, but to recap here quickly:

It is needed for the runtime (in a broad sense, from language to the operating system) to be able to introspect these:
- priority attributes
- dependencies
- execution contexts (thread/queue/runloop/...)
- ownership

Without this info, the scheduling of these coroutines will essentially be random, subject to priority inversions and other similar issues.

I will freely admit that I don't understand all of these details, so in lieu of rebutting this, I will simply state what I'm saying more explicitly and ask you to explain why I'm wrong in smaller words. :^)

Basically, what I'm saying is: Why do the context details need to be available *within the async function*, rather than being available only to the resume hook?

For a GCD example, suppose the normal, C-based `dispatch_async` function is exposed to Swift as `__dispatch_async`, and `beginAsync` has a signature like this:

  // I don't think `rethrows` works this way, but pretend it did.
  //
  /// Starts running an asynchronous function which is started and restarted by `resumeHook`.
  ///
  /// - Parameter body: The async function to run.
  /// - Parameter resumeHook: A function called once for each time `body` starts or resumes running.
  /// It is passed a `continuation` function representing the next synchronous chunk of
  /// `body`, which it should run (or schedule to run). If the `continuation` throws or returns
  /// a non-`nil` value, the function has terminated, and the result should be handled
  /// appropriately. If the `continuation` returns `nil`, then it has not finished executing.
  func beginAsync<Return>(do body: () async throws -> Return, startingWith resumeHook: @escaping (_ continuation: @escaping () rethrows -> Return?) -> Void) { … }

You can then write async-function-handling versions of `async` like:

  extension DispatchQueue {
    // This version runs a nullary, non-throwing, Void async function, and can be called from non-async code.
    func async(qos: DispatchQoS = .default, flags: DispatchWorkItemFlags = [], execute body: () async -> Void) {
      beginAsync(do: body, startingWith: { continuation in
        let workItem = DispatchWorkItem(qos: qos, flags: flags) {
          _ = continuation()
        }
        __dispatch_async(self, workItem)
      })
    }
    
    // This version runs any (nullary) async function, and can be called from async code.
    func async<Return>(qos: DispatchQoS = .default, flags: DispatchWorkItemFlags = [], execute body: () async throws -> Return) async rethrows -> Return {
      return try await suspendAsync { successHandler, failureHandler in
        beginAsync(do: body, startingWith: { continuation in
          let workItem = DispatchWorkItem(qos: qos, flags: flags) {
            do {
              if let returnValue = try continuation() {
                successHandler(returnValue)
              }
            }
            catch {
              failureHandler(returnValue)
            }
          }
          __dispatch_async(self, workItem)
        })
      }
    }
  }

Information like the QoS is encapsulated by the closure, so that each time it enqueues another chunk of work, it attaches that information to it. Is that good enough? Or do you need more?

I think it's good enough.

  * * *

I *think* you might be saying that, when GCD wants to run a given async block,

Not only when it runs it, at the time it's enqueued, it needs to introspect it then, to make the context it's enqueued onto inherit the right properties.
Because we don't want (nor can) look into the enqueued work onto a queue, so we need to coalesce the info at the queue level. And for real the only thing GCD really cares about here is the queue that is at the bottom of the queue hierarchy graph, we do coalesce these info down until we find the bottom queue, and for the sake of scheduling, all enqueued closures are collapsed to that bottom queue which is the owner of all the work recursively enqueued onto it.

What I'd like is for the closure to have some amount of space that the runtime can mutate so that we can explain to the object in which state it is. This requires for these closures to be a different object than regular lean closures. The latter are shared and can be called several times, async blocks should be called at most once and never reused. IOW GCD would want to augment and maintain this context.

DispatchWorkItem today tries to be that but without support from the langage can't, so we do this annotation (where you were enqueued) the once, which allows for DispatchWorkItem.wait() (or dispatch_block_wait() in C) to work *once* exactly. This is done with atomics because we have no exclusion guarantee.

But yeah, you're right that basically we (GCD) would like for coroutines in the async/await world to have at least the annotations DispatchWorkItem has today:

https://github.com/apple/swift-corelibs-libdispatch/blob/0e35ed9cf9c6de27f4ba8ba606a78d1da551602b/src/block.cpp#L50

which really is this:

https://github.com/apple/swift-corelibs-libdispatch/blob/666df60ce98b9f38814b544f9ff1ca74b7bbf70d/src/queue_internal.h#L986-L994

And for real most of these fields don't need to be here if we have the guarantee the coroutine is single-use (or really one-use-at-a-time-then-reset-state, this is fine too).

We don't need significantly *more* for coroutines than that from the GCD perspective.

···

On Sep 2, 2017, at 7:51 PM, Brent Royal-Gordon via swift-evolution <swift-evolution@swift.org> wrote:

On Sep 2, 2017, at 2:56 AM, Pierre Habouzit <phabouzit@apple.com> wrote:

it wants to be able to look ahead to where the `successHandler` will want to run so it can schedule the first block on a thread that will be able to immediately run the `successHandler`. But if so, that still only requires `suspendAsync` to extract the context and pass it to its parameter—it doesn't require arbitrary code running when the function is *not* suspended to access the context.

You could perhaps imagine the standard library providing these declarations:

  protocol AsyncContext {
    func resumeAsync(_ resumePoint: @escaping () -> Void)
  }
  struct AsyncContinuation<Returning, Throwing: Error> {
    // These are notionally packaged up here, although it might actually be implemented differently.
    private let successHandler: (Returning) -> Void
    private let failureHandler: (Throwing) -> Void
    
    func resumeAsync(in context: AsyncContext, returning value: Returning) {
      context.resumeAsync { successHandler(value) }
    }
    func resumeAsync(in context: AsyncContext, throwing error: Throwing) {
      context.resumeAsync { failureHandler(error) }
    }
  }
  func beginAsync(in context: AsyncContext, do body: () async -> Void) { … }
  func suspendAsync<Returning, Throwing>(_ handler: (AsyncContext, AsyncContinuation<Returning, Throwing>) -> Void) async throws<Throwing> -> Returning { … }

Then GCD could do something like:

  extension DispatchQueue {
    struct Context: AsyncContext {
      let queue: DispatchQueue
      let qos: DispatchQoS
      let flags: DispatchFlags
      let nextContext: Context?

      func resumeAsync(_ resumePoint: @escaping () -> Void) {
        let workItem = DispatchWorkItem(qos: qos, flags: flags, block: resumePoint)
        __dispatch_async_with_next_queue(queue, workItem, nextContext?.queue)
      }
    }
    
    // This version runs a nullary, non-throwing, Void async function, and can be called from non-async code.
    func async(qos: DispatchQoS = .default, flags: DispatchWorkItemFlags = [], execute body: () async -> Void) {
      beginAsync(in: Context(queue: self, qos: qos, flags: flags, nextContext: nil), do: body)
    }
    
    // This version runs any (nullary) async function, and can be called from async code.
    func async<Return>(qos: DispatchQoS = .default, flags: DispatchWorkItemFlags = [], execute body: () async throws -> Return) async rethrows -> Return {
      return try await suspendAsync { context, continuation in
        let newContext = Context(queue: self, qos: qos, flags: flags, nextContext: context as? DispatchQueue.Context)
        
        beginAsync(in: newContext) {
          do {
            continuation.resumeAsync(in: context, returning: try await body())
          }
          catch {
            continuation.resumeAsync(in: context, throwing: error)
          }
        }
      }
    }
  }

This allows GCD to look arbitrarily deep into the future, but the context can only be inspected at suspension points; it's otherwise encapsulated. The context is also now in control of execution, rather than being some passive data that may or may not be present and may or may not have any particular meaning.

  * * *

Actually, looking at this, it seems to me that `beginAsync(in:do:)` internally just creates a continuation for the beginning of an async function and resumes it. With a small language feature addition, we can have this in the standard library:

  protocol AsyncContext {
    // …as before…
  }
  struct AsyncContinuation<Returning, Throwing: Error> {
    // …as before…
  }
  extension AsyncContinuation where Throwing == Never {
    init(starting function: (#splat(Returning)) async -> Void) { … }
  }
  func suspendAsync<Returning, Throwing>(_ handler: (AsyncContext, AsyncContinuation<Returning, Throwing>) -> Void) async throws<Throwing> -> Returning { … }

And this in GCD:

  extension DispatchQueue {
    struct Context: AsyncContext {
      let queue: DispatchQueue
      let qos: DispatchQoS
      let flags: DispatchFlags
      let nextContext: Context?

      func resumeAsync(_ resumePoint: @escaping () -> Void) {
        let workItem = DispatchWorkItem(qos: qos, flags: flags, block: resumePoint)
        __dispatch_async_with_next_queue(queue, workItem, nextContext?.queue)
      }
    }
    
    // This version runs a nullary, non-throwing, Void async function, and can be called from non-async code.
    func async(qos: DispatchQoS = .default, flags: DispatchWorkItemFlags = [], execute body: () async -> Void) {
      let context = Context(queue: self, qos: qos, flags: flags, nextContext: nil)
      let starter = AsyncContinuation(starting: body)
      starter.resumeAsync(in: context, returning: ())
    }
    
    // This version runs any (nullary) async function, and can be called from async code.
    func async<Return>(qos: DispatchQoS = .default, flags: DispatchWorkItemFlags = [], execute body: () async throws -> Return) async rethrows -> Return {
      return try await suspendAsync { context, continuation in
        let newContext = Context(queue: self, qos: qos, flags: flags, nextContext: context as? DispatchQueue.Context)
        let starter = AsyncContinuation(starting: {
          do {
            continuation.resumeAsync(in: context, returning: try await body())
          }
          catch {
            continuation.resumeAsync(in: context, throwing: error)
          }
        })
        starter.resumeAsync(in: newContext, returning: ())
      }
    }
  }

We could even encapsulate the second version's chaining logic in `AsyncContinuation`:

  extension AsyncContinuation where Throwing == Never {
    init<StarterReturning, StarterThrowing>(starting starter: (#splat(Returning)) async throws<StarterThrowing> -> StarterReturning, returningTo continuation: AsyncContinuation<StarterReturning, StarterThrowing>, in returningContext: AsyncContext) {
      self.init(starting: {
        do {
          continuation.resumeAsync(in: returningContext, returning: try await starter())
        }
        catch {
          continuation.resumeAsync(in: returningContext, throwing: error)
        }
      })
    }
  }
  
  extension DispatchQueue {
    // …as before…
    
    // This version runs any (nullary) async function, and can be called from async code.
    func async<Return>(qos: DispatchQoS = .default, flags: DispatchWorkItemFlags = [], execute body: () async throws -> Return) async rethrows -> Return {
      return try await suspendAsync { context, continuation in
        let newContext = Context(queue: self, qos: qos, flags: flags, nextContext: context as? DispatchQueue.Context)
        let starter = AsyncContinuation(starting: body, returningTo: continuation, in: context)
        starter.resumeAsync(in: newContext, returning: ())
      }
    }
  }

Make `suspendAsync` a class method on `AsyncContinuation` and we've pretty much walled off all these low-level guts in a single type!

(P.S. Should the `AsyncContext` be a public property of the continuation? Maybe—that would make it harder to accidentally run continuations in the wrong context.)

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution