[Pitch] Task Executor Preference

Evolution Pitches
,
21

We drew the line at "structured concurrency inherits execution context". Task{} inheriting the actor's execution context really is somewhat of a special case and I'd hope we would generalize such thing by being able to explicitly spell (on: actor) IF we made default actor executors task executors as well... but at present this isn't being proposed.

If we think this would be very good we should probably gather use cases and think about it, but it'd take the form of passing a self of an actor to the on: of a task I think.

22

Inheriting the current isolation is also specifically static, based on the formal static isolation of the context calling Task{}. Task executors are an inherently dynamic technology.

2 Likes
23

A huge +1 from me. Thanks @ktoso et. all for pushing this forward.

This pitch solves the much discussed problem that io can currently not be done without thread hops using Swift structured concurrency.

I really like the way the pitch is structured and it answered all questions.

However I do have a feature request: UnsafeCurrentTask as provided by withUnsafeCurrentTask should gain a new property that allows access to the underlying preferred executor.

This feature is needed to build ConnectionPools that are executor aware. Using the proposed above API, a ConnectionPool could query a connection lease request for its underlying executor and lend a connection that is running the same executor, if one is available. In the case where multiple connections are available the connection that is on the same executor should be chosen.

Without this feature it will be hard for client library authors to match existing connections to client requests, which would finally result in thread hops once again.

For this reason an API addition, that allows users to query for the underlying preferred executor is required to unlock the full potential of Swift on server applications, especially in proxy like applications.

6 Likes
24

So would it be correct to say that formal actor isolation is for correctness and avoiding data races, while the executor control in this proposal is only for performance, and does not affect the formal isolation?

The proposal does seem to suggest that in several places, but I think it could be a bit clearer about it.

It also raises a question about assumeIsolated. Here's what the proposal says:

It is possible however to write a custom SerialExecutor and conform ot to the TaskExecutor protocol at the same time, if indeed one intended to use it for both purposes. The serial executor conformance can be used for purposes of isolation (including the asserting and "assuming" of isolation), and the task executor conformance allows using a type to provide a hint where tasks should execute although cannot be used to fulfil isolation requirements.

I imagine we wouldn't want people to use executor preference APIs in order to ensure isolation, and would prefer they used a more formal method for that (e.g. annotating their code with @SomeGlobalActor and await-ing where hops may occur -- even if those hops are dynamically determined to be unnecessary).

So should the assumeIsolation APIs perhaps ignore SerialExecutor conformances that arise solely due to task preference?

And one more thing, from the "Future Directions" section:

Task executor preference and global actors

It is more efficient to write Task(on: MainActor.shared) {} than it is to Task { @MainActor in } because the latter will first launch the task on the inferred context (either enclosing actor, or global concurrent executor), and then hop to the main actor. The on MainActor spelling allows Swift to immediately enqueue on the actor itself.

Is there any reason the compiler is prohibited from optimising the latter spelling to be just as efficient as the former?

AFAICT, if you write Task { @MainActor in }, it is not possible for any of the enclosing code to run on the inferred initial context (whatever it happens to be), because it immediately hops to the specified global actor. I thought the compiler was allowed to remove redundant hopping, and that in cases such as this, it is not obliged to enqueue the hop on the initial context.

2 Likes
25

That's certainly the core idea in my opinion. An executor is an abstract provider of execution resources, and an actor is an abstract provider of isolation. Actors can provide isolation in any number of ways, and while we consider asynchronous enqueuing onto a serial executor to be a baseline that any actor should be able to support[1], we don't consider that to be a hard restriction, and in fact our actors by default are cleverer than that and can establish isolation without changing the underlying executor. All else being equal, we really want that to continue to work in the presence of a preferred executor for a task.

Now, our actor system is currently designed around the idea that a task can only be dynamically isolated to a single actor at once. This restriction greatly simplifies a number of things around e.g. priority inversion avoidance, custom actor executors, etc. Maybe in time we'll have to relax it, e.g. if we ever want to support actors that stay blocked while they're calling out to other actors; but we'll definitely need to have satisfactory solutions to these problems in hand.

The immediate problem there is that there's nothing stopping you from using actor A's serial executor as the preferred executor for some task. How does that interact with actor B's ability to provide isolation? We basically have three choices as I see it:

  • We honor that A is isolated but maintain the rule that only one actor can be isolated at once. This means we have to suspend and enqueue the task when switching on and off B, even if B is a default actor that could normally achieve isolation on any executor. This is likely to increase overheads in core routines in the actor runtime because we have to consider the task executor as part of the isolation.
  • We honor that A is isolated and allow multiple actors to be isolated at once. This means we have to solve a bunch of thorny design problems.
  • We tell people not to do this with some level of stress, and then we just ignore it in the runtime, dynamically suppressing our knowledge that A is isolated.

We're currently leaning towards the third option.

  1. as well as the only customization hook currently supported by the language ↩︎

5 Likes
26

It's not prohibited from doing it, no; it just needs to propagate the information from one place to another. We can do that either statically or dynamically. Doing it statically would only work for this specific API, and only when we're passing a function that we know the static isolation of. My preference would be to do it dynamically, which would require some sort of new function type that would carry its formal isolation in a way that can be recovered at runtime, which would obviously enable this to be used in many other ways. I think we're likely to have that feature sooner rather than later, but I can't put an exact timeframe on it, of course.

3 Likes
27

Interesting and good point! This also seems to conveniently line up with my own feature request:

To say it more concisely: We need access to the current preferred executor (if set -- nil if not) and we need to be able to compare it to a stored one.

28

We can probably expose it on the UnsafeCurrentTask specifically only -- because lifetime gets tricky with executors as they're only stored Unowned so "someone" has to be keeping them alive. I think it may be ok to expose through the UnsafeCurrentTask only, but rather not on Task itself.

1 Like
29

That makes sense, however to compare executors we may be able to provide some form of ID for an executor (pointer value may or may not work because of ABA and we'd still need to look it up from somewhere). And creating a Task(on: .currentPreference) we don't actually have to provide the actual value and lifetime wouldn't be an issue.

Do you think this could work?

30

I am a bit concerned with that spelling because it is super easy for users to then escape the executor into unstructuredness and you cannot reliably clean it up anymore. In detail, I expect the following pattern to work but exposing currentPreference would make this highly risky.

try await withTaskGroup(of: Void.self) { group in
    let executor = MyAwesomeExecutor()
    group.addTask(on: executor) { ... }
    await group.next()
    // This is safe to do since we know the executor can not be escaped out of the structured child task
    executor.shutdown()
}

That's why I like @fabianfett suggestion to put it behind some unsafe operation with a big disclaimer that one should not escape the executor beyond the lifetime of the current task.

1 Like
31

This wasn't meant to actually return the executor as a usable value. This was meant to pass an opaque flag "please use current".

The technical details on how to do that aren't particularly interesting but we could use an overload or we could use a different parameter name Task(inheritCurrentPreference: true) or a lot of other options. Let's not complicate the lifetime of executors, I just need to be able to say "spawn Task on current pref" and Fabian needs "lookup by executor ID". Both seem feasible without issues.

32

Are they without issues? For Fabian’s request yes probably but for your request to inherit the executor preference to an unstructured task I am not sure. Nothing is guaranteeing that the unstructured task is indeed running as a hidden structured task like you describe above. In the case where it isn’t we will escape the executor.
In the NIO world we know how painful this is since users have been trying to shutdown EventLoops and have consistently run into the problem where something still holds a reference to it and tries to enqueue work on an already shutdown executor. That’s one of the reasons why we (you :stuck_out_tongue:) have introduced the singleton pattern for EventLoops recently so that people can just treat them as immortal objects.

1 Like
33

I see, that's a different angle that I hadn't considered. The problem of the executor actually going away. Fair game. Two solutions come up:

  1. Treat the executor preference as a hint that may not be fulfilled (it kinda is anyway as you can't enforce it for a whole hierarchy).
  2. Allow this only for immediately-awaited Tasks (keeping them structured). That would need a different spelling ofc.
  3. Probably the best solution to my immediate issue but not as general: Instead have a withUncancelledTask { ... } or similar which would inherit the preference and otherwise does what it says on the tin.

Separately from this: What keeps executors alive in a structured task hierarchy? Who says they can't shut down.

This might be debatable I can tell you that this was not the reason (on my part) to introduce the singletons. The reason was purely:

Apple's SDKs are full of singletons (Dispatch, URLSession, ...) and with Swift Concurrency it feels completely unreasonable that an implementation detail like SwiftNIO would necessarily leak into your API (unless you tolerate substandard performance). It shouldn't. Given that we don't have a generally accepted dependency injection solution apart from singletons I thought singletons are the best way out of this dilemma.

The same argument is actually the reason for #700 & #701: HTTPClient.shared a globally shared singleton & .browserLike configuration by weissi · Pull Request #705 · swift-server/async-http-client · GitHub . URLSession doesn't need to be dependency-injected for best performance because it's a global singleton. AHC currently does (unless every library creates its own singleton -- which is not ideal). I'm arguing to change that.

And again: I don't think the singletons are great but without dependency injection being an accepted part of an ecosystem, there's little else that can be done. For advanced and experienced users there is still the option to inject whatever you need.

34

The main thing is that you're setting the preferred task executor within a scoped operation (i.e. withTaskExecutor(e) { ... }), and the executor isn't used after that scope exits, and keeping it alive for that time is a predictable and easily-understood contract you're presumably willing to uphold.

1 Like
35

Okay, so the idea is: It's technically on me but scoped resource management is dead obvious so I would need to be a fool to shut it down before the scope exits. If so: I buy that, makes sense, thanks.

1 Like
36

Would it make sense to allow non-copyable TaskExecutors?

The unowned reference is essentially a borrow, isn't it?

37

As John said the contract is relatively straight forward with the scoped access. For the above problem we should really look at uncancellable scopes akin to Trio. For the time being providing an unsafe mechanism to get the executor should be enough to implement your structured unstructured task.

38

Right. You’re offering the executor to this API explicitly, and the API is giving you the strongest guarantee you could reasonably ask for about when the program will be done with the executor.