[Pitch] Custom Actor Executors

[ktoso (Konrad 'ktoso' Malawski 🐟🏴‍☠️)]

@QuinceyMorris

I'm a bit confused because the following code description is for Job not UnownedJob, and the definition of UnownedJob doesn't actually come until later.

I'll update the doc to stick more to using Job more in the proposal text, but for what it's worth: they're the same, except for the ownership/safety. The Job is move-only, consumed by running it and therefore generally safe, and UnownedJob whose lifetime is not managed, and is unsafe to access after it was "consumed" (e.g. by runJobSynchronously(theJob)).

@QuinceyMorris

The description of Executor.enqueue() makes it sound like UnownedJob is something deprecated, but the later discussion seems to suggest that it's not. It's sort of odd if UnownedJob isn't deprecated, but the way that you use it (enqueue) is.

That is what is happening here. We have an existing API today that wasn't documented, but exists, that was accepting an UnownedJob: Executor.enqueue(UnownedJob). This API exists and is becoming deprecated in favor of the move-only Job accepting version Executor.enqueue(Job).

At the same time, current limitations of the early version of move-only types (which are being pitched, but have not reached a formal review yet) can be limiting enough that users may need to resort to using the UnownedJob. The type is therefore not deprecated, and should be treated as an escape hatch, that you may need to use when job is used in a generic context (e.g. like the proposal mentions: you cannot store Job in an array, but you can store UnownedJob). Therefore, entry-point is deprecated - we lead you towards the safe API, however the escape hatch is not deprecated.

Hope this clarifies why the enqueue method is deprecated, but the type not.

@QuinceyMorris

At first I thought this meant that runJobSynchronously would run a previously enqueued job, but I'm confused by the fact that both enqueue and runJobSynchronously have a job parameter, so they apparently refer to different jobs. The code description of SerialExecutor.runJobSynchronously(_:) doesn't seem to have anything to do with a previously enqueued job.

This is the method to "run a job", synchronously, immediately when and where this method is invoked:

// in an SerialExecutor:
self.runJobSynchronously(job)

The happens-before and happens-after typical terminology in concurrent systems to express ordering guarantees, here we just mean that this order of an enqueue's effects must be visible to the run; if there is any synchronization necessary to make this happen, the executor must take care of that.

@QuinceyMorris

What is the significance of "happen-after" then, in the above quote? Or is enqueue really just runJobAsynchronouslyAtAPossiblyLaterStartTime?

No; it is as the name implies "run this job now, synchronously, on the current thread".

@jrose

(Alternately, if it can be backwards-deployed at all, then “neat” and also “how will that work for the introduction of new protocols and requirements?”)

The proposal includes several existing (!) types that are @available(5.1) because they've been already back deployed ever since the Swift concurrency was back deployed. We never formalized those types through an SE review, so we're doing this now, and while doing so, adding the new APIs.

AFAICS backdeployed things will continue to work as they have until today, and some of the APIs probably we can back deploy -- for example, there never was an official API to "run a job" before this proposal, but the runtime function to do so obviously existed already back then -- it is what Swift itself uses to run jobs -- so I think we can try a bit and backdeploy the runJobSynchronously method perhaps. So old implementations, using UnownedJob only, could at least run a job using an official API rather than an underscored one.

I don't have a full picture of what we can and cannot do though with this.

@Joannis_Orlandos

What happens if I adopt the asme SerialExecutor in two actors, would Swift be able to check it's running on the same SerialExecutor? Or would I still need to mark my calls between these two pieces of code with await?

@DevAndArtist is right that @John_McCall just explained this elsewhere actually: `nonisolated lazy let` on an actor - #13 by John_McCall

You'd have to write await but it wouldn't actually suspend if they shared the same serial executor. To get this guarantee into the type-system, we'd need what is discussed in Future Directions: DelegateActor property of the proposal

The previous pitch of custom executors included a concept of a delegateActor which allowed an actor to declare a var delegateActor: Actor { get } property which would allow given actor to execute on the same executor as another actor instance. At the same time, this would provide enough information to the compiler at compile time, that both actors can be assumed to be within the same isolation domain, and await s between those actors could be skipped (!). A property (as in, "runtime behavior") that with custom executors holds dynamically, would this way be reinforced statically by the compiler and type-system.