Ideas for some concurrency syntax

Evolution Discussion
1

A recent thread inspired me to try to put some more work into ideas that have been bouncing around in my head for a while. These are all, at best, half-baked. This is not a pitch for a single, unified proposal. I guess maybe it could turn into that, or maybe just some parts. But I think the things here are at least worth some discussion.

The whole motivation is around reducing syntactic complexity. This can put a considerable burden on users, and not just when first learning. I've found there can be quite a lot of cognitive load here even for people that have experience working with the concurrency system.

Here are some of the ideas.

1 - implicit @escaping

A synchronous function that accepts asynchronous or isolated function arguments cannot do meaningful work unless they are permitted to escape. This is <waves hands> kinda sorta similar to how optional function types no longer require @escaping. I know it actually isn't the same. But, is the required @escaping annotation here actually helping with clarity?

// these would be implicitly @escaping
func example1(_ work: () async-> Void) 
func example2(_ work: @MainActor () -> Void) 
func example3(_ work: (isolated any Actor) -> Void) 
func example4(_ work: @isolated(any) () -> Void)

Maybe there are situations where non-escaping closures could be useful?

2 - A sending/@Sendable function parameter of an @isolated(any) type implies @_inheritActorContext

Users become really familiar with the semantics of Task, but then get surprised when they have a hard time matching it. When a function accepts a sending/@Sendable function argument of arbitrary isolation, capturing the current actor context seems like a very reasonable default.

func inherits(_ work: sending @isolated(any) () -> Void) {}

@MainActor
func example() {
    inherits {
        // this would be @MainActor
    }
}

3 - parameter attribute of @detached to disable actor context inheritance

There is an existing proposal to formalize @_inheritActorContext. But I think this is too biased towards the detached semantics. There are only two APIs I have found that do not want to inherit actor context by default: Task.detached and TaskGroup.addTask. Yet I have encountered many real situations where I wanted to inherit context. I also like that there is a special thing there in the signature to signal something unusual is going on.

extension Task {
    @discardableResult
    static func detached(
        priority: TaskPriority? = nil,
        @detached operation: sending @isolated(any) () async throws -> Success
    ) -> Task<Success, Failure>
}

However, I bet there are situations where capturing actor context so aggressively will result in problems I have not thought of.

4 - All async or isolated functions get @isolated(any)

I haven't been able to figure out a reason why all functions which could have isolation do not carry it dynamically. The @isolated(any) attribute is a common source of confusion, probably because while it is rarely needed it is frequently encountered. It is handy in a number of situations, and if it has no downsides why not? Maybe ABI compatibility?

// these would be implicitly @isolated(any)
func example1(_ work: () async-> Void) 
func example2(_ work: @MainActor () -> Void) 
func example3(_ work: (isolated any Actor) -> Void)

5 - Stricter parsing rules for attributes that apply to the type of a function parameter

The following both compile, but I don't think they should. This is basically harmless if you understand what's going on, but when you do not, it is quite confusing. It really obscures the difference between the type of a function and the behavior of a parameter.

func problem1(_ work: @Sendable @escaping () -> Void)  {}
func problem2(_ work: @isolated(any) @escaping () -> Void) {}

This is source-breaking, but a fix-it seems doable.

Most of these introduce more implicit behavior around some pretty complex functionality. But overall, I think changes along these lines could be very helpful for progressive disclosure. You'd only get introduced to the extra complexity if you actually need to interact with it. And the most negatively-affected users are probably those that already understand the nuances of the language deeply.

I want to leave you with a theoretical before-and-after for two Task APIs, which would both be heavily impacted by these:

@discardableResult
init(
    priority: TaskPriority? = nil,
    @_inheritActorContext operation: sending @escaping @isolated(any) () async -> Success
)

@discardableResult
init(
    priority: TaskPriority? = nil,
    operation: sending () async -> Success
)

@discardableResult
static func detached(
    priority: TaskPriority? = nil,
    operation: sending @escaping @isolated(any) () async throws -> Success
) -> Task<Success, Failure>

@discardableResult
static func detached(
    priority: TaskPriority? = nil,
    @detached operation: sending () async throws -> Success
) -> Task<Success, Failure>

And with that, where have I gone wrong?

12 Likes
2

I'd be concerned about the subtlety that making any of these function signatures async would then make the function parameters nonescaping again. If we ever have a "block on async work" operation in the future, then it would be theoretically possible (albeit probably discouraged) to use nonescaping closures with these signatures from a synchronous context.

3 Likes
3

It's a bit unfortunate that the type resolution rules for function types settled on these semantics, actually -- an optional non-escaping function type is a reasonable thing to support but the current syntax forecloses the possibility. Perhaps some day we can revisit how non-escaping function types work, and integrate them better with the more recent language features for ~Escaping and lifetimes, to address this limitation.

13 Likes
4

Would @detached be limited to Task.detached and TaskGroup.addTask? What if users add @detached anywhere without understanding the implications? Could adding it to Task as an API likeTask.withDetachedContext be more explicit?

1 Like
5

Yeah that's a good question. You can do this with @_inheritActorContext today, so the absence of that attribute is the default now. Though the effects aren't the same as what I'm suggesting here.

A with function is definitely interesting idea! How do you see that communicating the needed behavior to the compiler?

Your question has also just made me realize that Task.detached removes all context, while TaskGroup.addTask removes only isolation inheritance. Now, I guess you could say that because addTask is structured that could still make sense with a single attribute, but the case definitely has gotten weaker.

1 Like
6

Let me chime in on the "detached" part a bit. I don't think that one is quite right: "Detached" means a very very specific thing: detaching from the task tree. It is not the same as an async let nor task group. The word "detached" should not be used anywhere with child task context.

And other than that, it's not that we're expressing some "one off" thing specific to those APIs here, but it is just a way to specify "where" the execution shall happen.

Instead, I think this will fall out of the changing the default execution isolation of async functions, which used to be pitched as "@.concurrent" but instead I believe is converging on @execution(caller | concurrent) and therefore @execution(concurrent) might be exactly what we're talking about here.

For what it's worth this also relates to the closure isolation control improvements which have been pitched a while ago, all these make sense:

func param(act: any Actor) async {
  Task { [isolated act] in ... }
  Task.detached { [isolated act] in ... } // OK
  await withTaskGroup { g in 
    g.addTask { [isolated act] in ... }
    g.addTask { nonisolated in ... } // @execution(concurrent)
    // we can imagine other add task versions
    g.addTaskOnCaller { /*execution(caller)*/ ??? }
  }
}

Either way, the key point here is that this is about execution not about "detaching".

3 Likes
7

This is great, and you are you absolutely right. The term "detached" isn't appropriate.

But do you see where I'm going with this concept? Right now, function arguments that could have any isolation default to not inheriting their enclosing isolation. You have to opt into that behavior with @_inheritActorContext. I think it's possible that @_inheritActorContext is a better default, and instead there should be a mechanism to turn it off. And while I'm 100% into the changes you brought up, I don't think they will address this, will they?

8

This would be great instead of isolation: isolated (any Actor) = #isolation.
Overall, I find that during Swift Concurrency-related evolutions the keyword and annotations are inconsistent. If all of it could be simplified to lightened to e.g. @execution like:

Task { @execution(caller | concurrent) in }
Task { @execution(MainActor) in }

@execution(caller)
func foo {}

func foo(bar: @execution(..) -> Void) { ... }

even for the feature purposes

                                    or @execution(actor) -> Void  ...
func foo<T: any Actor>(actor: T, bar:  @execution(T) -> Void) -> { ... }
1 Like