SE-0417: Task Executor Preference

Evolution Proposal Reviews
1

Hello Swift community,

The review of SE-0417 "Task Executor Preference" begins now and runs through December 26th, 2023.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to me as the review manager via the forum messaging feature or email. When contacting the review manager directly, please put "SE-0417" in the subject line.

Try it out

Toolchains with this feature implemented (where each API name has a leading underscore) are available:

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available at

https://github.com/apple/swift-evolution/blob/main/process.md

Thank you,
Doug Gregor
Review Manager

22 Likes
3

Shouldn't it be 0416 not 0417 :thinking:

4

One question, in the async sequence example, if the code is modified with an additional print, like so:


actor Looper { 
  func consumeSequence() async {
    withTaskExecutor(self) {
      for await value in getAsyncSequence() {
        // 1.a. 'next()' can execute on Looper's executor
        // 1.b. if next() needs to call some other isolated code, we would hop there
        //      but only when necessary.
        // 2.a. Following the fast path where next() executed directly on Looper.executor,
        //      the "hop back to actor" is efficient because it is a hop to the same executor which is a no-op.
        // 2.b. Following the slow path where next() had to call some `isolated` code,
        //      the hop back to the Looper is the same as it would be normally.
        print("got: \(value)")
      }
    }
    print(“exit”)
  }
}

Would the stated ‘fast path’ (one element immediately available) print:

  1. got ‘first’
  2. exit

Or:

  1. exit
  2. got ‘first’

The proposal mentions that the new Task initialisers execute immediately:

Tasks created this way are immediately enqueued on given executor.

I’m trying to work out how immediately that might be and if it applies to withTaskExecutor.

5

I'm super excited to see this in the pipeline, we'll really have use for this! :fire: :fire: :fire:

I've done a read-through and will revert later with additional feedback (if any), but one thing that I'd want to clarify with the proposal is the follow semantics:

The preferred executor also may influence where actor-isolated code may execute, specifically:

  • if task preference is set:
  • (new) default actors will use the task's(***) preferred executor
  • actors with a custom executor execute on that specified executor (i.e. "preference" has no effect), and are not influenced by the task's preference

(***) What task would that be? The one that created the Actor instance? It can't be the task that is calling into an actor as that would fail the serialisation requirement.

It would be great if that could be clarified perhaps, the latter section outlines it nicely how to set the preference for tasks etc, so that is clear, but the above is not (to me).

6

A default actor's executor does not conform to TaskExecutor so you can't write the sample as written.

In whichever "fast path" or "slow path" you wrote out the order is always: "got first", "exit".

There is no concurrency introduced by your code snippet. The snippet is also missing an await which would make this more clear: AWAIT withTaskExecutor(...) { }.

The "immediately" doesn't matter to withTaskExecutor in a way because there is no new task created at all. We simply hop the current task to the provided executor -- in that sense yeah it is "immediately" but no different than any other async function.

This does apply to Task(on:) because that "creates a task". And the difference is that normally Task { await actor.hopSomewhere() } ALWAYS first gets executed on the global concurrency pool and then will hop to wherever the async function needs it to go.

The Task(on: taskExecutor) immediately enqueues on taskExecutor rather than first enqueueing on the global pool.

7

"Default actors" are explained in the proposal here:

A default actor (so an actor which does not use a custom executor), has a "default" serial executor that is created by the Swift runtime and uses the actor is the executor's identity.

This is a "default" actor (because it does not declare a custom executor):

actor Greeter { func hello() {} } 

await withTaskExecutor(something) { 
  // current task now has a "preference" to run on "something"
  await Greeter().hello() 
}
// current task no longer has any "preference" where to run

The hello() will execute on "something" task executor, while remaining isolated by the actor's serial executor i.e. "the actor", such that assertIsolated and friends still work as expected.

1 Like
8

Here is a toolchain to try out the proposed feature, prefix all APIs with _ (e.g. Task(_on:) and _withTaskExecutor, _TaskExecutor), to be able to try them out:

The feature will also appear in nightly builds with the underscores soon so it'll be easier to try out.

By the way, we benchmarked the changes with @FranzBusch with swift-nio and @hassila's great benchmark plugin and we see a significant reduction in context switching in a real benchmark NIO cares about:

TCPEchoAsyncChannel no executor
│ Metric                │      p0 │     p25 │     p50 │     p75 │     p90 │     p99 │    p100 │ Samples │
╞═══════════════════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╪═════════╡
│ Context switches      │    1272 │    1451 │    1500 │    1544 │    1583 │    1661 │    1712 │    1000  |

TCPEchoAsyncChannel task executor
│ Context switches      │       7 │      15 │      19 │      30 │      55 │      96 │     150 │    1000 │

The throughput numbers in this benchmark are actually skewed since we end up comparing multi-threaded vs single threaded suddenly, but this is exactly the change in runtime behavior this feature is designed for :+1:

7 Likes
9

I lifted that code sample directly from the proposal. The only addition is the second print statement.

OK, if the order is 'got first', 'exit' for the fast path (described in the proposal) that means that not only is the hop to the global concurrency pool eliminated, but the same-actor hop is also eliminated. Is that what you're saying?

The reason I ask for clarification is that it seems to me that the 'slow path' (where the async sequence's next() had to call some isolated code) would necessarily suspend the Task, and I'd expect the order of the prints to be inverted – "exit", "got first".

EDIT: Yes, adding the await makes this much clearer. The proposal should be amended to include the await.

When you say 'immediately enqueues', if the taskExecutor happens to be the current executor, are you saying it eliminates both the hop to the global concurrent actor AND the same-actor/executor hop to the current executor, or just the hop to the global concurrency pool?

For example:

@MainActor final class Thing {

  private var count = 0

  init(sequence: SomeAsyncIntSequence) {
    Task(on: MainActor.shared) {
        // A SomeAsyncIntSequence immediately produces '5', then exits
      for await value in sequence {
        self.count = value
      }
    }
    print("count: \(count)")
  }
}

Would we see 'count: 0' or 'count: 5'?

10

Ah I see what happened here, I'm sorry about the confusion with the AsyncSequence example, I was convinced we had removed this entirely from the proposal already.

I have removed the two last sections that wrongly suggested an actor could act as a task executor.

Actors are not being proposed to function as task executors, sorry again that there was a section remaining which still suggested that.

This was an early idea but quickly we realized this would not work, and is not being proposed. In the being proposed version of these APIs they are not, because we think this would be conflating responsibilities of isolation and "where we take a thread from".

I can answer the "immediately enqueues" parts of the question though:

I know you're specifically asking about the main queue/actor/thread so let's bring up an example that can simulate this, even while actors are not task executors.

NOTE: I don't think the below is the right way to achieve ordering semantics with actors, we're still missing better APIs for this specifically, but we can "kind of" showcase what you're asking about so for illustration purposes let's experiment with it.

You could imagine a task executor implementation that runs the jobs on the main queue, since that's effectively what you're asking about, so it'd be something like:

final class MyMainQueueTaskExecutor: TaskExecutor { 
  static var shared: MyMainQueueTaskExecutor { ... }

  public func enqueue(_ job: consuming Job) {
    let unownedJob = UnownedJob(job)
    DispatchQueue.main.async { 
      unownedJob.runSynchronously(on: self.asUnownedTaskExecutor())
    }
  }
  // ... 
}

Okey so now write your snippet:

Task(on: MyMainQueueTaskExecutor.shared) {
  print("task")
}
print("count: \(count)")

I believe this shape of the snippet should explain the "enqueues immediately" point:

  • as we execute Task(on: executor)
  • we call executor.enqueue(job), immediately, without first running on the global pool
  • the executor should ensure this job will be run synchronously at a future point in time, e.g. here we used .async {}

So, in this sense -- no, you are not going to be guaranteed seeing the other unexpected order.

However, task executors are not guaranteeing either serial or single-threaded at all execution - they may be backed using a thread-pool and the exact order of submitted tasks is not guaranteed in general -- though specific task executors may guarantee some specific semantics here.

1 Like
11

No worries! :slight_smile:

So by enqueuing via DispatchQueue.main.async I would expect this to output:

  1. count: 0
  2. task

Personally, for UI stuff, I'd love an executor with the capability of executing immediately (it seems immediacy is relative in this context) so that this order is inverted. Only returning to the outer scope when the Task first suspends (or exits). It would be interesting to know how feasible that is.

This is a common pattern in UI code that uses reactive frameworks, it prevents animation glitches caused by state changing across event loop cycle iterations caused by an async dispatch. You can use Combine in this way for example. It would be great to have the ability to use async sequences in the same way.

4 Likes
12

Thanks @ktoso et al, I very much support this pitch and have been involved in discussions (providing use cases etc). So it's probably not surprising that I believe this solves the performance issues that Swift Concurrency and any I/O system (such but not restricted to SwiftNIO) has today.

Today, most async functions get pulled onto global default executor which forces thread switches as you can't sensibly do I/O there. So if you have a high-performance system that needs to avoid the thread hops, the only option you have today is to take over the entire global default executor and make it run on a more capable and powerful system. As an example, this complete takeover can be done with SwiftNIO as outlined in this PR . That of course works but feels heavy-handed. It would be much nicer to retain the regular Swift Concurrency thread pool, hop once to perform I/O and then stay on the I/O system's executor until some other actor forces us to leave it.

I believe that with the implementation of this pitch the need to completely take over the global default executor is pretty much gone (at least for the vast, vast, vast majority of use cases that I can think of). That's wonderful, let's do it!

5 Likes
13

Disclaimer: I worked on this proposal together with @ktoso and provided feedback during the implementation with our learnings from SwiftNIO.

Is the problem being addressed significant enough to warrant a change to Swift?

This problem is 100% important enough to address. I/O frameworks often want to provide developers the possibility to stay on the I/O executor so that work can be handled synchronously. This is important for both throughput and latency. With the current aggressive hop-off semantics of Swift Concurrency there is no way to achieve this. This proposal gives developers the tools to control where their code is executing and they can make a conscious choice what fits their application best. Moreover, I don't see this proposal as being something that only server developers are using but also something very useful for UI developers.

Does this proposal fit well with the feel and direction of Swift?

Yes, it does. Initially Swift Concurrency abstracted away over where code is running since Swift wanted to get the semantics of async and Structured Concurrency right. Now that more code has been written using Swift Concurrency it became clear that we need to some of the control back.

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

Kotlin does provide very similar capabilities. This proposal follows Kotlin's prior art closely while incorporating it nicely into Swift Concurrency.

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

I reviewed multiple iterations of this proposal, tested the toolchains with our NIO benchmarks and provided feedback during the implementation period.

Open suggestions

Having said all that, there is one thing that stands out to me and I would like to discuss here. The proposed semantics for setting the global executor preference is by passing nil to the various methods/initializers. This feels wrong to me and the implied semantics of nil meaning global executor are not obvious. Furthermore, I think we can make the various methods take a generic TaskExecutor type instead of an existential.

Something along the lines of the below enum might work nicely (name bikeshedding aside)

enum TaskExecutorPreference<Executor: TaskExecutor) {
    case inheritsExecutor
    case globalExecutor
    case preferredExecutor(Executor)
}
10 Likes
14

As an aside: It is not possible to name a case of a generic enum in Swift, without supplying the generic parameters somehow, even if the given case has no dependencies on any of the generic parameters.

How can I pass in .inheritsExecutor to a function that expects a TaskExecutorPreference value without somehow making the type explicit?

15

Thanks for chiming in @johannesweiss and @FranzBusch !

I would like to introduce such "explicit enum" and the reason I didn't in the initial proposal was mostly because there did not seem to be much prior art of such API style in the standard library.

Personally I'm quite fond of such explicit "spell out what it does" rather than just riding on optionals for everything, so I'd like to see if we can find a way to do it...

Things get complicate if we want to have both some TaskExecutor and some enumeration...

enum approach (further amended below, skip this)

Summary

The enum as proposed does hit the issue @sveinhal mentions, the following issue exists:

enum Preference<Ex: TaskExecutor> {
  case some(Ex)
  case _inheritsExecutor
}

// TODO: uhm... what would this be... just a placeholder type seems weird
final class PreferredExecutor: TaskExecutor {
  func enqueue(_ job: consuming ExecutorJob) {}
}

extension Preference where Ex == PreferredExecutor {
  static var inheritsExecutor: Preference<PreferredExecutor> {
    ._inheritsExecutor
  }
}

func take<Ex: TaskExecutor>(_: Preference<Ex>) {}

func test(ex: some Executor) {
  take(.some(ex))
  take(.inheritsExecutor)
  take(._inheritsExecutor) // error, can't just spell this
}

Sadly Never is not a real bottom type in Swift, so we can't do this:

// no type for 'Ex' can satisfy both 'Ex == Never' and 'Ex : Executor'
// type 'Never' does not conform to protocol 'Executor'
extension Preference where Ex == Never {
  static var none: Preference<Never> {
    ._none
  }
}

which would have been a nice solution...

One simple solution is to wrap an any TaskExecutor -- this then works as expected, the same way as the current Optional<any TaskExecutor> works. So it feels the some is at odds with the shape of the API otherwise.

Potential solution 1: struct with fake executor types for the generic

Introducing some form of Preference struct that has a number of static preferences is something we could do here.

struct TaskExecutorPreference<Ex: TaskExecutor> {
  static func taskExecutor(_: Ex) -> Self 
  static var defaultExecutor: TaskExecutorPreference <Never>
}

we can make this work, by conforming the Never type to TaskExecutor (and it'd just crash if actually used). Thanks @FranzBusch for reminding me of the Never trick.

I'd be okey with this I think.

Alternative solution: more overloads

We could side step this issue by doing more overloads, like this...

Task(on: taskExecutor) // on: TaskExecutor
Task(on: .preferredExecutor) // on: TaskExecutorPreference // no generic
Task(on: .globalExecutor) // on: TaskExecutorPreference // no generic

API wise we get another overload (we have a lot already...) but it'd be doable :thinking:

We would be able to avoid the weird fake executor types.

16

That's a good point but we can just use the usual struct + static var/func workaround here. The following should do the trick:

struct TaskExecutorPreference<Executor: TaskExecutor> {
    static var globalExecutor: ExecutorPreference<Never> { ... }
    static func preferred(_ executor: Executor) -> Self { ... }
}

extension Never: TaskExecutor { ... }
1 Like
17

Hah, I forgot to conform Never to the executor huh -- thanks for the reminder @FranzBusch :slight_smile:

Yeah so that's viable then:

struct UnstructuredTaskExecutorPreference<Ex: TaskExecutor>: Hashable {
  enum _Storage {
    case taskExecutor(Ex)
    case defaultExecutor
  }
  var storage: _Storage

  private init(storage: _Storage) {
    self.storage = storage
  }
  init(_ ex: Ex) {
    self.storage = .taskExecutor(ex)
  }
  static func taskExecutor(_ ex: Ex) -> Self {
    .init(storage: .taskExecutor(ex))
  }
  static var defaultExecutor: Preference<Never> {
    .init(storage: .defaultExecutor)
  }
}

class Never: TaskExecutor {
  func enqueue(_ job: consuming ExecutorJob) {}
}

func take<Ex: TaskExecutor>(_: Preference<Ex>) {}

func test(ex: some Executor) {
  take(.taskExecutor(ex))
  take(.defaultExecutor)
}

I think this would be okey. Would we still want a plain Task(on: taskExecutor) overload or how would we want these APIs to be spelled..

Task(on: .taskExecutor())
Task(on: .defaultExecutor)

:warning: On the explicit omission of Task(preferred executor) :warning:

I should mention though that we casually just added the

Task(on: .preferredExecutor) // this is actually the dangerous API

during this discussion here but this API is rather dangerous and NOT present in the proposal at this point (!). The reason is the lifetime of executors.

:warning: We on purpose do not offer this API to automatically inherit the preferred executor into an unstructured task in this proposal.

The reason is executor lifetime is NOT guaranteed by tasks that run on it. So the following code is safe because we only inherit the preferred executor in structured tasks:

let executor: MyTaskExecutor = ...
defer { executor.shutdown() } 

await withTaskExecutor(executor) { 

  async let x = ... // uses the task executor
  
  Task {} // does NOT use the task executor
}

// executor is deinitialized here (!!!)

this is by design in order to allow the executor to be lifecycle managed using structured programming: we know that if we have finished this block of code, we will release this executor.

If we just inherited it into unstructured concurrency magically, we lost this guarantee.

You can still use the UnsafeCurrentTask to compare your expected task executor with the current one, and hop if you want to. But this is an advanced pattern, accessible only via the unsafe APIs.

:bulb: enum-like executor preference style API

So... the API would have to be split between UnstructuredTaskExecutorPreference and ChildTaskExecutorPreference, and would have to take the following shape:


// used in `Task(on:)`
struct UnstructuredTaskExecutorPreference<Ex: TaskExecutor>: Hashable {
  static func taskExecutor(_ ex: Ex) -> Self
  static var defaultExecutor: Preference<Never> // aka. "ignore preferred"; same as not passing a value to `Task()`
}

/// used in `withTaskExecutor` and `group.addTask`
struct ChildTaskExecutorPreference<Ex: TaskExecutor>: Hashable {
  static func taskExecutor(_ ex: Ex) -> Self
  static var defaultExecutor: Preference<Never> // aka. "ignore preferred"
  static var preferredExecutor: Preference<Never> // same as not passing a value to `addTask()`
}

I do think this clearly communicates the capabilities and does not lead to unsafe code, so this would be okey I believe.

18

I think that's a fair point and I agree we should not offer a preferredExecutor case. I don't think we need the two types you proposed we can just treat preferredExecutor as passing nothing to the addTask method.

Yeah we probably want that overload just for brevity sake.

2 Likes
19

doesn’t TaskExecutor have the layout constraint AnyObject?

20

This seems straightforward, but I question the use of Preference as the noun, as it makes it seem like it may not be executed on that executor in certain situations. At first glance I would imagine a "task executor preference" to be an optimization thing where it would be nice if it ran on that executor but not required. Maybe TaskExecutorRequirement is a better name?