SE-0304 (4th review): Structured Concurrency

Sure, it has many facets. For example, it is also a struct, but I don't think we'd consider that to be the primary nature of its purpose. Less abstractly, people won't use a TaskGroup as a Collection, they'll use it because they're want to create subtasks within its scope.

This is why I see its primary purpose as being an "orchestrator of concurrent tasks which launches and tracks them (in an internal collection)". It "has a collection" as part of its implementation, but its primary purpose is not to be a collection.

Right, technically it conforms to AsyncSequence.

I respect your opinion, but I pretty strongly disagree with you here. The primary purpose of this method is to launch a new concurrent task. The entire reason you're using this method is to introduce concurrency (as expressed eloquently in the introduction). The verb "add" does not convey that at all.

As other's have pointed out, Cocoa APIs are full of multistep objects where you "create the thing" then you "initiate the thing". As I mentioned, it is completely unprecedented in the standard library to have a discardableResult initializer - and for good reason. Swift's focus on value semantics makes it important to be able to reason about side effects, and this undermines/reduces that.

I appreciate declarative APIs, but that makes sense for something like a function builder where the entire method is declarative. Dropping one line of "declarative" code into an imperative context doesn't make the code declarative, it is likely to cause confusion.

One really important aspect of API design that is it is important to deploy consistent practices across APIs in the system. You're explaining very detailed rationale that the typical user will not know, and without that, this will just be confusing.

As I said, this "can be learned" -- but why does that make it a better API design?

As other's have pointed out, that makes sense for side effect free operations like sorted. That doesn't make as much sense when the primary operation in question is to create a new unit of concurrency.

You also ignored many of my points here: all the static members on Task refer to the current task except this one. In addition to being inconsistent with general API design principles, this method is inconsistent with the rest of the API /in this proposal/.

Again, I respect your (and everyone else on this thread's) opinion. However, I tried to make an argument based on strong principles of API design. You generally ignored all those principles and made blanket statements without rationale other than "you consider it to be unnecessary verbosity".

Can you please explain why you think the points I made are unimportant? For example, why should this be the first initializer with a discardable result? Why should detached be inconsistent with the rest of the task API? Why shouldn't active actions like launching tasks include verbs?

-Chris

7 Likes

Does it happen? Yes. Is it so common that it's worth putting self.'s into every task body? I don't think so, because putting self.'s everywhere devalues their usefulness. Again, this precedent set by SE-0269: captures structs and enums can create reference cycles, too, but it's rare enough that there isn't value in requiring self..

Yes, those APIs exist, but they aren't necessarily a good match for Swift's concurrency model. NSURLSessionTask predates Swift (!) and is mostly subsumed by an async function now.

So, we've had a bit of experience with Swift's task APIs so far, and I have never once needed to create a task that I didn't want to run immediately. I haven't received a bug report that either asked for the addition of a "create a suspended task" operation or indicated that anyone was confused about task creation running the task. Are you suggesting that we change the most fundamental API to not run the task?

It captures an isolated self. This isn't a new notion at all:

actor A {
  func f() { }
  func g() {
    let closure = { // `self` is captured as isolated
      self.f() // okay to call this synchronously, `self` is isolated
    }
  }
}

Yes. Your globalfunc doesn't have an isolated parameter of type A to capture. If your ... stuff ... involved an operation that operated on an instance of A (e.g., the isolated self in A.f()) I suspect the distinction would be more clear.

For the subtask to be within the actor, it needs an isolated parameter of the actor's type. Isolation guarantees are statically determined by isolated parameters; they're not dynamically determined or else we wouldn't be able to check them statically.

Because it's statically checked. That's fundamental to the design of actor isolation, that a given piece of code knows what actor it must run in (whether an isolated parameter or a global actor) and the system ensures that it is running there.

Hi! We're discussing it, and I've given technical arguments that haven't been addressed. I do not consider this a "huge change". It is backed both by direct experience with the feature and by precedent in SE-0269. By your argument, we should never have accepted SE-0302 because Sendable is a "marker protocol" and we didn't pull out that mini-feature as a separate proposal.

The purpose of this method is to add a new task into the group. The purpose of the group is to execute those tasks and provide the results.

To be clear, a task is not a value-semantic entity. They are very much reference types that refer to a resource where identity is key. To reiterate, the point of creating a task is to run it. Should we go ahead and add a way to create a task without executing it, that's easy to do within this API without loss of clarity for either.

The purpose of defining tasks is to perform some operation concurrently. It is important that we emphasize that you are creating a new entity---the task---because it is a new conceptual task that carries with it priority, task locals, etc. You can use that new entity to manipulate or check on the running operation if you'd like. I think there is an important distinction here between this and, say, DispatchQueue.async, which is a one-off hop over to a different queue to go do one thing. Tasks are longer-lived and more semantic in nature, because they are meant to capture a complete operation.

It's a static method that produces a new Task instance, which is not an uncommon pattern.

Doug

3 Likes

I still think we should forego at least one of these three:

  • withThrowingTaskGroup cancels its outstanding child tasks upon receiving throws (and only in that case),
  • async let cancels its task if the result is not awaited,
  • treating async let as sugar of task group.

The difference between the two is far subtle to be called sugar. IMO, it should be reconciled.

3 Likes

As they are both creating Tasks, maybe they should both be initializers. Indeed, could they not be consolidated to a single init(detached: Boolean = false, priority: TaskPriority? = nil, operation: @Sendable @escaping () async throws -> Success)?

No, because the non-detached APIs have @_inheritActorContext and @_implicitSelfCapture attributes, on the trailing closure argument.

3 Likes

I am personally a big fan of Task's discardableResult closure initializer ā€“ it declaratively shows that you are creating a new Task. It carries much more cognitive weight than a simple free function, which I personally find awkward to use. Although this may be my functional programming mind speaking, I actually prefer that it's in a closure initializer that side effects are executed, and not in a free function which I usually prefer making pure (just like min, max, cos, etc.).

Although this is unfortunately undermined by other free functions such as withTaskGroup or withTaskCancellationHandler, there is still another important "declarative" API with side effects: async let.

Task { ... } and async let will undoubtedly be the most widely used structured concurrency APIs, so I actually feel that the declarativeness introduces some consistency to the API.

Furthermore, just because an API design in unprecedented in the Standard Library doesn't make it de facto undesirable / a bad thing. API design evolves with the new features and paradigms Swift gains. Declarative APIs are gaining increasing popularity in Swift thanks to features such as Result Builders and libraries such as SwiftUI, Combine, Argument Parser, etc., and the standard library can (doesn't always mean that it necessarily should) evolve with that.

1 Like

I'm not sure Swift users have had enough time to play with the concurrency features and the new APIs to determine whether it is common or rare yet. That it's not intended to be common by the designers of the feature doesn't mean it'll actually be the case when put in the hands of users.

In particular, observing notifications seems like something tempting to do with a task, but the API for that has just become available (and only in beta) so we don't know about adoption yet. I wouldn't be surprised if this becomes common, but mainly I think it's too soon to have conclusive data on this.

SE-0269 was built on the experience that it's rare for value types to create cycles. I'm not sure this necessarily translates to "reference types aren't going to create long-lived cycles when used in tasks".

8 Likes

I'm not talking about anything concurrent here. I'm referring to the fact that we have existing APIs which separate object creation from object usage clearly. There is consistent naming (and semantics ) style cross these interfaces. If we want to provide an API that combines both object creation and object usage, I hope we could make the semantics explicit at least.

I think you totally missed my point here. I am NOT suggesting that we need an API to create a task that doesn't run immediately (actually I agree with you at this point). What I'm suggesting is that we should put the running semantic into the API name explicitly:

runTask { ... }
// or
Task.run { ... }

It is confusing to me whether or not the task will be launched after calling Task { ... }. This gives me an impression that we are doing some declarative programming here. But the other API this proposal introduces is imperative instead of declarative. There seems to be a big inconsistency of naming here.

6 Likes

Can I also argue that the purpose of Task { ... } is to add a new task into the global executor? And is the purpose of the global executor to execute those tasks (while not provide the results like task groups)?

1 Like

Right. You've said this before, so there must be some confusion. I have not seen anyone argue in favor of a public initializer that allows one to create a task, and then launch it separately. We agree with the design approach.

We are pointing out that initializers have never been used this way in the Swift standard library (and I gave rationale for that) and pointed out that there are multiple simple alternatives that fit better with the existing language/library design principles.

Furthermore, while several arguments have been made about why alternative approaches would be better, your only argument for why those alternatives are worse are "I don't feel like we need an extra word here" and things like 'I haven't received a bug report that either asked for the addition of a "create a suspended task" operation'. These responses don't address the core design points that people are raising.

Fundamentally, you are arguing for significant deviance from the API design standards in the Swift standard library. I feel like we should have extremely strong justification for such deviance, or we should do the obvious thing and consider other approaches (e.g. discuss their tradeoffs).

In terms of approaches, in addition to the global function approach, I think that @Zhu_Shengqi 's approach also has a lot of promise:

  1. group.launchTask {}
  2. Task.launch {}
  3. Task.launchDetached {}

-Chris

11 Likes

I feel like this is an overt attempt to be dismissive about my concern here. This proposal has gone through four revisions and many pitch phases. This is a significant change to the core language, and I haven't seen anyone in the community discuss this - this core language change is buried in a massive library proposal.

Furthermore, your analogy to SE-0302 seems to miss the point. I'm specifically asking that we turn this language feature into something like SE-0302: a small proposal specifically focused on the language change proposal, separate from the huge library proposal that is SE-0304.

Just to be clear, I'm not concerned about the underbar'd attribute implementation details - I agree this is a reasonable way to go if this is the right thing for the language. My concern is that taking this for Task means that:

  1. we are opening a new door for API design in Swift: taking this means we (newly!) consider "explicit self" harmful enough for specific use cases that it needs to be disable-able.
  2. thus we are saying that Swift should have a way for API authors to opt out of it.
  3. thus we are saying that API authors should think about this (but we'd presumably keep the safe default of requiring self if they don't)
  4. thus we need principles for "when" explicit self is required and when it "should" be turned off.
  5. and we need to capture those in the API design guides.

I can see the argument that we don't want to do all this work in the push to get Swift 5.5 out the door. However, it seems pretty important to answer the question of "do we ever want to do this work?" which is another form of asking the question of "is this soft of control good for Swift?".

Task is merely one API in a whole sea of APIs that a developer interacts with. Language design is forever and crosscuts everything. We should be careful to look at the big picture of the ecosystem, and avoid temptations to accidentally make global decisions just because they are locally beneficial for the "hot API of the day".

-Chris

30 Likes

Wouldn't names similar to the existing GCD methods be more understandable for developers ?

Task.async { .. } instead of just Task { .. } would be clearer indication that the task is to be executed asynchronously on the current execution context.

Task.global.async { .. } would seem clearer to me than Task.detatched { .. }

One question, is it correct that

fun main() {
  print("One")
  Task {
    print("Three")
  }
  print("Two")
}

will always result in One, Two, Three, as would a GCD DispatchQueue.main.async {}, or could the Task creation result in a suspension and early execution of the Task ?

I believe async has been proposed in a previous review, and was replaced later because in Swift concurrency domain async means "function may suspend".

In the current implementation of Task.init there is no async modifier, so you can think of it as analogous to DispatchQueue.async (not precisely though).

Thanks ShengQi,
I've just tried this out in a Monterey playground, and realise I have completely misunderstood and am more confused.

print("One \(Thread.isMainThread)")
Task {
    print("Three \(Thread.isMainThread)")
}
print("Two \(Thread.isMainThread)")

I was very surprised to find produces:

One true
Three false
Two true

And if the task is created with Task.detached { .. } instead of Task { ... } I see the same result.

I had read in one of the 'under the hood' articles going around that the Task initialiser would create the Task in the current execution context, and experiments with that in a SwiftUI app resulted in a Task { ... } being called on the main queue. Now I am lost. Why is a Task { } in a playground run on a background queue but a Task { } executed from a swiftui modifier like onAppear() run on the main queue ?
Here is the SwiftUI test:

struct ContentView: View {
    var body: some View {
        Text("Hello, world!")
            .padding()
            .onAppear {
                print("One \(Thread.isMainThread)")
                Task {
                    print("Three \(Thread.isMainThread)")
                }
                print("Two \(Thread.isMainThread)")

            }
    }
}

which produces

One true
Two true
Three true

which is what I expected from the online explanations. Replacing Task with Task.detached results in

One true
Two true
Three false

Again in line with my perhaps errornous understanding. So what have I misunderstood which results in very different behaviour in a macOS playground ?

On other thing I tried was Task(priority: .userInitiated) { ... } in the playground but that also resulted in the same result with Three being printed off the main thread.

1 Like

Check out this comment for a more detailed explanation about how the Task initialiser Task { } inherits its execution context from the local actor context.

On the macOS playground, you're not calling Task { } on a specific actor such as MainActor, so it will be run on a random background thread. However, one can presume that view modifiers in SwiftUI are run on the main thread, therefore on the MainActor. And since Task { } inherits the local actor context, its closure is also run on the main thread / MainActor.

The issue I find with Taskā€˜s @discardableResults is that it is non-obvious that cancellation must be handled.

Whatā€™s the rationale for discarding the handle when thereā€˜s no other way of preventing a leak short of completion? I would be more comfortable with having a forget free function should one truly want to discard an unstructured task than it being the default.

On the beauty of a @discardableResult trailing closure init like Task { } Iā€˜ll say this: while, like Chris said, Task is merely one API, async is part of the language, and until Task stays the currency of unstructured concurrency, a default symmetry of non-@discardableResult attach and the now-deprecated detach for unstructured tasks would have conveyed meaning and preserved those aesthetics while pairing well with their equivalents within the Task namespace, still there for ease of discovery through completion, among others. Task.init would evolve into a non-executing/deferred initializer, given a @discardableResult initializer does not have any semantic affordance to indicate that a Task is executed and that its lifetime is unspecified, i.e. does not conform to standard scope/parent-based destruction, and therefore is solely a programmerā€˜s responsibility to handle.

1 Like

As the two print statements either side of the Task show, they are running on the main thread, so following the logic you describe, why are they also not therefore on the main actor ? Is this just an anomaly of the current XCode 13 beta 3 playground perhaps ?

Iā€™m also intrigued by the change in ordering of execution . In the playground case, the Task is executed immediately, as stated in SE-0304, however in the SwiftUI context, it behaves like a DispatchQueue async and is executed after the last print statement, which seems to contradict SE-0304.

So we're agreeing that the API that creates a task launches the task.

As @nikitamounier points out, we shouldn't exclude a pattern just because the standard library doesn't already have it.

One of our principles is "omit needless words." We don't need both "launch" and "task", because we've established that creating and running a task are the same API. I contend that the creation is the more dominant aspect of this relationship, which supports use of the Task initializer.

I've noted repeatedly why group.addTask is the right name, and going in circles here doesn't help. Task.launch and Task.launchDetached are slight improvements over the free functions you've proposed, because we shouldn't pollute the module-scope namespace with free functions unless they really need to be free functions.

That said, I'm surprised that you'd prefer it given your complaints about Task.detached earlier:

Either this consistency argument equally dismisses Task.detached and Task.launchDetached/Task.launch, or it's not actually a problem. Personally, I don't consider it to be a problem (as I've stated before) for either API.

Doug

2 Likes

It's not really "new". As I've said, SE-0269 reconsidered when it's useful to require explicit self. and laid out the low-likelihood-of-causing-actual-cycles criteria.

Doug

Agreed. I believe that creating an "attached" task ā€“ one which keeps task-local values, inherits the actor context ā€“ should be considerably easier than creating a "detached" task, since the "attached" task is almost always what the programmer needs.

That's why I appreciate the Task { } initialiser. It's cognitively more lightweight than Task.detached(_:), yet still shows the tight relationship the two have. They both create and launch new tasks, after all. And an API which conveys different levels of importance / "defaultness", while still showing tight relationships between concepts ā€“ that's what I call good API design, with accessible "best defaults" being one of the core pillars of Swift.