Expressing "must be called from a non-nil isolation"

Evolution Discussion
1

I wanted to discuss concurrency ergonomics to see if anyone else thinks this is a gap worth putting effort into.

First, this isn't a functionality gap, I can express what I want to in swift today, which looks a bit like this:

final class Model { var count = 0 }

func doWork(with model: Model, on isolation: isolated any Actor = #isolation) { 
	Task {
		_ = isolation
		model.count += 1
	} 
}

This is an actor-isolated function that:

  • Takes a 'Non-Sendable' Model type
  • Launches a Task (strongly capturing the isolation)
  • Mutates the 'Non-Sendable' from the Task on the same isolation

Note: Calling the above on Swift 6.4 from a nonisolated context eliding the isolation parameter via the macro results in: "Error: Call to actor-isolated global function 'doWork(with:on:)' in a synchronous nonisolated context" which is behaviour I want

The example is in essence the actor generalised version of the following:

@MainActor func doWork(with model: Model) {
    Task { model.count += 1 }
}

The above code is a lot nicer to read. Because the isolation is statically known, the function does not need to be parameterised and the Task can also inherit that same isolation automatically.

You may be wondering why I'm even launching a Task like this, which is a fair question. Staying in structured concurrency longer can certainly make things easier. The above examples are simplified stand-ins for more complex code I'm working on. Which is more akin to an entry point that takes events and launches/cancels Tasks, where the lifetimes of those Tasks can also be bound to app events (like sign out). This is shared app facing code targeting iOS and Android so .task {} isn't available on both platforms.

Generalising the code to be runnable on the caller's actor, whatever it is, is helpful to allow code to have the ability to mutate non sendable objects and to unlock parallel testing (e.g. not forcing tests to serialise through MainActor but also to be runnable in parallel via a TestActor).

For my purposes it'd be helpful to see an addition or sugar added to the language that allows specifying something as: isolated(inherited) (following the @_inheritActorContext name) or isolated(caller) (I understand this was the rejected name for nonisolated(nonsending)). Where in this case we're saying "We're on the isolation of the caller and child Tasks should also inherit it by default, also disallow calling this function from a nonisolated context".

I know there is work being done to allow explicit isolation capture (which looks useful) but what I'm looking at here hopefully allows automatic inferred / inherited isolation capture. In a way that is static and determined by the call site in contrast to dynamic (and optional) runtime isolation.

As an aside: I'd also argue most app-facing code usually runs "on an actor". I imagine this is why @MainActor was chosen as the sensible default isolation for 'Approachable Concurrency'. But as many find that can be quite restrictive and I feel like "isolated to a non-nil Actor" is a decent middle ground between @MainActor and nonisolated. This is probably well out of scope for this discussion but on class types it could allow inferring [isolated self] for Task creation too.

For this discussion in the simple case the final example would look something like this:

isolated(inherited) func doWork(with model: Model) {
    Task { model.count += 1 }
}

There's a semantic difference here too, not just nicer syntax. Today the parameter lets an async caller override the isolation:

@MainActor let model = Model()

func nonisolatedFunc() async {
    await doWork(with: model, on: MainActor.shared)
}

That works because the await hops onto the actor you passed in. Whereas I'm looking for "run on the caller's isolation, without substitution." You can still do a similar thing but it now requires the same syntax as @MainActor func doWork variant:

func nonisolatedFunc() async {
    await MainActor.run { doWork(with: model) } // Hop to match the isolation of `model`
}

I assume this is non-trivial to implement like this, but I wanted to raise it for discussion because while passing isolation as a parameter does work it seems like something that could be more semantic for the language. Being able to express "having an isolation" generally, at least the function level seem useful. I'm not attached to any particular naming here, just the idea of expressing "we have a non nil isolation".

2 Likes
2

I'm delighted to see this topic come up, because it's something that I've been thinking about for a very long time.

First, I wanted to point out how interesting the shape of problem is. Your doWork is a free function. But a very similar kind of problem can come up with methods as well. I think is a pretty important detail. All of the non-Sendable arguments of the function, including an implicit self, are in the same isolation domain as the parameter.

final class Model {
  var count = 0
  func doWork(on isolation: isolated any Actor = #isolation) { 
    // self must also be in `isolation` here
    Task {
      _ = isolation
      self.count += 1
    } 
  }
}

Just to spell it out, the reason the Task initializer gives you trouble here are the isolation inheritance rules for closures. A closure does not inherit isolation for actor types or isolated parameters unless the actor value is strongly captured. Further, the capture list does not count for this purpose. Global actors, on the other hand, are implicitly inherited.

These rule are largely governed by the @_inheritActorContext parameter attribute used by the Task.init functions. I think there are some other closure inheritance rules that may matter in the general case, but I'm less familiar with those.

Now, I think (hope) it is uncontentious that fixing the discrepancy between body captures and capture lists is something we should just do. It isn't a source-compatible change, but I think it should be equivalent.

func doWork(with model: Model, on isolation: isolated any Actor = #isolation) { 
  // this does not mean the same thing today, but it should
  Task { [isolation] in
    model.count += 1
  } 
}

Another really interesting thing that you've observed is the relationship with nonisolated(nonsending). When you use a non-optional isolated parameter in this way, passing in anything other than the caller's isolation is almost certainly invalid. The use of the #isolation macro helps somewhat, but it is still syntactically possible.

(I think it could be the case that there are actual, valid uses where the isolated param does not match the caller but I haven't thought hard enough about it to be sure.)

With a nonsending function, the caller's isolation must be used. However, that form also allows that isolation to be optional. I sometimes wonder if we got the polarity right there. Is using non-Sendables from a nonisolated context the common case? Or, could it have made sense to restrict nonsending callers to non-optional isolation, and force the isolated parameter route when you really do want to support both isolated and nonisolated uses?

I'm pretty sure that what we have now makes the right trade-offs, so while it might be fun to consider, it's a moot point anyways.

We do have to consider whether the pattern of unstructured Task creation is something that should be easier. In these contexts, it's a huge pain. But I think there's definitely a case to be made that some friction here is a good thing. Encouraging programmers to remain in the structured world has a large number of advantages. Further, as you said, this is not an expressivity issue. It's possible to do what you want, but it's just really annoying.

Now all that said, I think this is an area that definitely deserves attention.

The distinction between body captures and capture lists just need to be fixed so we can stop writing the _ = isolation incantation.

I also believe that the semantics of @_inheritActorContext should be changed. Isolated parameters should be unconditionally inherited. Actor types are slightly trickier, because I think the risk for retain cycles is pretty strong. However! Task also uses implicit self capture. I find it very difficult to believe that the conditional isolation inheritance rules here are worth preventing reference cycles that the implicit self capture doesn't already create.

(In fact, there is a variant, @_inheritActorContext(always) that behaves this way today!)

These are still, relatively-speaking, modest improvements. The inheritance behavior would be much more intuitive. But, you'd still need the isolated parameter, and still face the problem of callers being able to supply an invalid isolation argument. The isolated(inherited) concept you include here is along similar lines to something else I've thought of before. I'm very open to exploring options here.

4 Likes
3

I believe under the current linked isolation capture proposal draft the explicit capture would end up looking more like:

Task { [isolated isolation] in ... }

But as you say, I too would like to see the @_inheritActorContext rules extended to auto capture the parent isolation.

If the primary way forward for actor-isolated functions is isolation as a parameter plus explicit captures for Tasks that is also fine.

To me the existence of the macro #isolation does imply that auto-propagation of isolation is desirable. And the diagnostic that you can't call an actor-isolated function without an isolation also speaks to required isolation being a considered case.

You're right this is all incremental. I think I'll eventually bond with parameter as isolation. But as per your linked discussion, a solution with a keyword preceding the func could be desirable, it would fall in line with the other isolation keyword positioning.

nonisolated func myFunc() {} // Has no isolation available
@MainActor func myFunc() {} // Has MainActor isolation

// theoretical syntax options for "having non-optional inherited isolation"
isolated func myFunc() {} // As a peer to the nonisolated form to describe requires isolation
@InheritedActor func myFunc() {} // As a peer to the GlobalActor form to say must inherit an actor
2 Likes
4

Sorry, I wasn't clear when talking about captures/capture lists. Today, these two things are semantically-distinct, but I believe that is an error.

func usesParam(isolation: isolated any Actor) {
  Task {
    _ = isolation // this will cause static isolation inheritance
  }

  // but this will not?
  Task { [isolation] in
  }
}

The isolation control facility proposed allows you to influence closure isolation independent of any isolated parameter. It should work with any actor value. It looks very close syntactically, but it is actually not the same.

This is more of a personal annoyance of mine though, and not really a major issue since changes around @_inheritActorContext would probably just make this irrelevant.

Anyways sorry about the distraction. Getting back to Task creation.

I suggested isolated func myFunc() {}. But what we are really talking about, if we are extremely explicit, is a variant of nonisolated(nonsending). Is it still nonisolated? It's still nonsending (in the async case). But to solve this problem, the function signature must be statically known to be incompatible with a nonisolated caller.

nonisolated(nonoptional) func a() {}

nonisolated func b() {
  a() // <- this must be an error
}

nonisolated(nonsending) func b() async {
  a() // <- this must be an error as well, because the isolation could be nil
}

@MainActor func c() {
  a() // this is ok
}

actor MyActor {
  func d() {
    a() // this is also ok
  }
}

nonisolated(nonoptional) func e() {
  a() // and so is this
}

I'm struggling to come up with a succinct means of expressing this.

I categorically reject nonisolated(isolated), even if it does make sense in a bizarre way.

nonisolated(caller) is also strange. Because all nonisolated functions run on the caller and that's not the important quality of this function.

@InheritedActor also gives me pause, because this is already how nonisolated functions work.

@CallerIsolated is perhaps accurate. But for purely aesthetic reasons, I prefer to avoid attributes unless there is no other option.

isolated(caller) was suggested and it does make a fair bit of sense here... this function cannot actually be nonisolated, since it is effectively sugar around an isolated param.

5

I'm not 100% sure about it but I could see this being an optimisation quirk where if the variable isn't actually used in the closure it may not be captured. Agree that it should be fixed or cleaned up somehow.

It isn't nonisolated, we can confirm this by trying to mark one of these "parameterised actor isolated" functions as nonisolated:

nonisolated func example(isolation: isolated any Actor) /*async*/ {}

➜ swiftc -swift-version 6 example.swift
actor-isolated-func.swift:1:18: error: global function with 'isolated' parameter cannot be 'nonisolated'
1 | nonisolated func example(isolation: isolated any Actor) {}
  |                  `- error: global function with 'isolated' parameter cannot be 'nonisolated'

But yes, as you say the relationship of a theoretical isolated(caller) func to a nonisolated(nonsending) func is the same desire to run on the caller's actor and not send (in the async variant).

And yes, in this instance we're disallowing calling the function from nonisolated or @concurrent contexts as we want to make access to isolation in the body of the function required.

Your examples do provide errors in the right spots today with the parameterised version + the macro allowing you to elide the param:

 1 | func a(isolation: isolated any Actor = #isolation) {}
   |      `- note: calls to global function 'a(isolation:)' from outside of its actor context are implicitly asynchronous
 2 |
 3 | nonisolated func b() {
 4 |   a() // <- this must be an error
   |   `- error: call to actor-isolated global function 'a(isolation:)' in a synchronous nonisolated context [#ActorIsolatedCall]
 5 | }

 7 | nonisolated(nonsending) func b() async {
 8 |   a() // <- this must be an error as well, because the isolation could be nil
   |   `- error: actor-isolated global function 'a(isolation:)' cannot be called from outside of the actor
 9 | }

But we also get the macro expansion errors which aren't that intuitive. They do however, further clarify the situation, it is not "isolation could be nil" here. It is actually isolation within the body of a nonisolated func is always nil. Even though the caller may be isolated:

 7 | nonisolated(nonsending) func b() async {
 8 |   a() // <- this must be an error as well, because the isolation could be nil
   +--- actor-isolated-func.swift --------------------------------------
   |6 |
   |7 |
   |8 |    #isolation
   |  |    `- note: in expansion of macro 'isolation' here
   |  +--- macro expansion #isolation ----------------------------------
   |  |1 | nil
   |  |  | `- error: 'nil' is not compatible with expected argument type 'any Actor'
   |  +-----------------------------------------------------------------
   +--------------------------------------------------------------------
 9 | }
1 Like
6

That was a useful exercise and really does clarify things. I think I'm pretty convinced in the utility of an isolated(caller) concept.

The only use-case that I can currently aware of it addressing is internal task creation. And that's something that should be done with great care. I might even go so far as to say it should be avoided unless there is truly no alternative.

So I guess the big question is, now, is such a thing worth pursuing?

1 Like
7

Isolation inheritance from an isolated parameter captured in a capture list was never implemented; IIRC, there are also no plans to do so. The same applies to some other edge cases, like optional binding. Generally, isolation and aliasing don't work together.

P.S. I'd like to see improvement in this area, but I'm doubtful.

1 Like
8

I am a little bit unsatisfied with this answer. At least depending on how I interpret 'internal task creation'. If I take this to mean: 'Tasks should be created from SwiftUI Views', at one point I think I would have agreed with you since I had my sights set on only code targeting SwiftUI.

However there are always transition points between synchronous and asynchronous code in apps. For something as benign as a button that makes a network request. Even from SwiftUI if you want to control the cancellation of this Task and not have it be bound to a SwiftUI View tree it can get awkward. More so if you're doing things in a framework agnostic way.

However if I take your 'internal task creation' to mean actor-isolated tasks shouldn't be created from places where the isolation isn't statically known. Then I do see a way forward, and in fact I am getting traction with the sample pattern below:

@MainActor func doWork(with model: Model) {
    doWork(with: model) { work in Task { await work() } }
}

typealias SpawnTask = (@escaping () async -> Void) -> Task<Void, Never>

nonisolated func doWork(with model: Model, task: SpawnTask) {
    _ = task { model.count += 1 }
}

Here we can leverage injecting a factory function to create task handles that can mutate our non sendable type. The compiler can determine we aren't crossing any isolation boundaries as all the closure types involved are non sendable as well.

This way the shared logic can stay nonisolated (nonsending too with the work closure) and the isolation is injected externally by way of @_inheritActorContext on Task.

This looks a lot nicer as you don't need any isolation parameters or explicit captures. But I'm not sure having to indirect through non sendable closures like is as clean at conveying the intent some code should be isolated to its caller.

Instead we now have something that is "correct by construction" and is impossible to construct anywhere without isolation. Which is pretty cool! But it is perhaps non obvious this is the case until you attempt to construct it without isolation available.

15 | nonisolated func nonisolatedContext() {
16 |     let model = Model()
17 |     doWork(model: model) { work in Task { await work() } }
   |                                    |                 `- note: closure captures 'work' which is accessible to code in the current task
   |                                    `- error: passing closure as a 'sending' parameter risks causing data races between code in the current task and concurrent execution of the closure [#SendingClosureRisksDataRace]
18 | }
1 Like
9

Sorry, I wasn't very clear. Yes, this is what I meant.

Ha, well that's a clever approach!

The question in my mind is really if the language needs changes in this area or not. Because as we have explored, there are a number of ways to do this. You just found another!

I think that, specifically, isolated parameters are too difficult to use. Thankfully, there's been attention in that area, and I remain hopefully it will be improved.

I want to stress, I'm not opposed to language evolution in this area. I just go back and forth sometimes. You are right, of course, that there will always need to be sync->async transitions and providing tools to make those transitions easier and more correct makes sense.

10

You're right to challenge me on it, I was challenged by some others as well when discussing it.

There are a lot of ways to go about it. In fact, I ended up pivoting again.

One of the things that was pushing me towards creating Tasks was preserving synchronous execution, followed by queuing an async action if required. However I just discovered Swift 6.2's Task.immediate.

Using this it is possible to transition to structured concurrency immediately and have my main entry point to logic be nonisolated(nonsending).

Now mutations to my 'Non-Sendable' can be simplified to a nonisolated(nonsending) context, before and after an await.

I still need to use Tasks internally to get the cancellation behaviors that I want outside of SwiftUI. But now I can re-scope them to work on Sendable things. Like downstream services and their tasks return Sendable data.

1 Like
11

I really do not want you to interpret this as a challenge at all! More like a call-for-use-cases.

I think the complexity around internal task creation is bad and I'd like to see improvement. But I also want to make sure to carefully consider the trade-offs and don't get too caught up in whims.