[Pitch] Task Priority Escalation APIs

Evolution Pitches
21

The UnsafeCurrentTask overload is really just there for completeness.

You COULD use it in some weird situations and we try to not randomly leave off APIs from the various accessors to task just because, so it's there...

Is it a great idea to use? Probably not.

Technically you could use it to boost the priority of a task from the inside of the task (?!) which is pretty unusual but could be done:

// VERY weird, don't do this please...?
Task { 
  if rememberedThisIsVeryImportant() {
    withUnsafeCurrentTask { 
      Task.escalatePriority(of: $0, to: .high)
    }
  }
}

It's pretty weird and I would recommend against such random self escalations... but I could imagine that maybe you'd get some callback somewhere and may need to escalate the task you originally entered a function with or something...

In any case, it is possible to use correctly if very careful, so I think it belongs in this proposal.

1 Like
22

What came to my mind, after trying this out:

Minor remark, but I agree with this. Could be more aligned here.

I think the isolation parameter placement is a bit unfortunate but so is it with withTaskCancellationHandler. If you want to pass along a specific isolation, you lose the trailing closure sugar.

Nevertheless, I think this is a great addition. With the combination of the mentioned child task handlers and being able to escalate those as well in the future, I think it should finally be possible to implement AsyncQueue/AsyncLimiter/... without priority inversion.

2 Likes
23

Am I the only one who feels this is just a huge XY problem?

The example issue is merge from swift-async-algorithms in which you can't create a structured task which has a lifetime which matches the merge operation.

Using current tools, this could be solved by having merge receive a TaskGroup parameter, which would determine the lifetime of the contained tasks (and hence, the merged stream). If you read a bit about structured tasks/nurseries, you'd realize that's how it's supposed to be used. However, this originates from garbage-collected languages, in which lifetimes must be made explicit.

A correct solution to the problem would be structured tasks of which lifetime is associated with a parent structure/class, relying on ARC. This would require asynchronous deinitializers.

In other words, this entire pitch is basically trying to get structured concurrency without using structured concurrency. In use cases where it's entirely appropriate to use structured concurrency. Instead of trying to figure out how to make structured concurrency work.

24

The proposal is now in review, so it'd be best to provide feedback in the official review thread @SlugFiller: SE-0462: Task Priority Escalation APIs

I can answer here though: our structured concurrency primitives are not powerful enough yet, e.g. we'd like to share a task group into child tasks and be able to add tasks into it from them -- we don't have a model for this yet where it would not trigger concurrency safety errors, and it'll require much more work in the typesystem and library to support these. We'd also be able to get rid of the with... methods for task groups along this work... but for now we need to support escalation with existing tasks, and this functionality has existed but was internal to Swift, we're making it available to low-level concurrency library authors, which does not preclude continued improvements in the structured concurrency story.

25

As the author of the current merge implementation, I do agree that the unstructured task here is not what I want to use but it is the only possibility as Konrad noted. We can't currently add a child task to the caller. Furthermore, we would need that child task to be awaited after the iteration to make sure everything is cleaned up.

Using ARC to manage lifetimes is not a correct solution either for two reasons:

  • First, the clean up of async sequences might be async itself e.g. closing a file descriptor. This can't be done in deinits since those are sync.
  • Even if we had async deinits using them to clean up resources is incredibly brittle and can lead to security problems since deinits on classes run at arbitrary times. That's why in general, we prefer to use with-style methods in the ecosystem to have scoped lifetime of resources
4 Likes
26

Hence the need for a new language feature.

This I have to disagree with. One of the main advantages of ARC over GC is that it's predictable. It is not arbitrary, not brittle, and would not lead to security problems. RIAA is a tried and true method for secure and predictable resource management and scoping, and ARC is an efficient and easy to use RIAA implementation.

IMO with-style methods are more of a disease of the current ecosystem, and not something to strive for. It mimics the approach of garbage-collected languages, despite the fact that not being garbage-collected is one of Swift's greatest strengths. And it carries the clear disadvantage that the scope is limited to function/code scoping, and cannot be attached to other lifetimes, like structs, nor returned to a parent scope. With the extra disadvantage of using closures to mimic blocks, thereby losing access to block controls like break or continue inside the closure.

1 Like
27

I have experience converting codebases that rely on object lifetime to instead use manual invalidation. I can certainly attest that under ARC it is unpredictable when any escapable reference type will be deinitialized. This becomes even more of an issue when the reference can escape to another thread.

3 Likes
28

Hi y'all, can discussion about this proposal be directed to the active review thread?

2 Likes