[Pitch] Isolated default value expressions

Evolution Pitches
1

Hello, Swift evolution!

I've been working on a proposal for unifying the actor isolation rules for default value expressions across the language. There are several issues with the current actor isolation rules for default value expressions: the rules for stored properties admit data races, the rules for default argument values are overly restrictive, and the rules between the different places you can use default value expressions are inconsistent with each other, making the actor isolation model harder to understand. I'd like to unify the actor isolation rules for default value expressions to eliminate data races, and improve expressivity by safely allowing isolation for default values.

The basic idea is to allow default value expressions to require a specific isolation that the caller must satisfy in order to use the default value, which would make the following code valid:

@MainActor class C { ... }

@MainActor func f(c: C = C()) { ... } // the default argument value for 'c' requires the caller to be @MainActor-isolated

@MainActor func useFromMainActor() {
  f() // okay, caller is @MainActor-isolated
}

You can view the full proposal draft on GitHub here.

I welcome your questions, thoughts, and other constructive feedback!

-Holly

28 Likes
2

This seems like it's fixing bugs - is a proposal technically necessary? I don't see any downsides or caveats…?

(not that I object to the well-written proposal; it's educational and documents the changes very well!)

What about async inits? The caller is already using await to call them, so couldn't default arguments tied to different actor contexts be implicitly awaited as well?

3

I think a proposal is useful. I've been fixing a variety of other Sendable and actor isolation checking bugs without proposals, but this one felt like a big enough pile of semantic changes that it'd benefit from a conceptual review outside of code review. This is also proposing shrinking SE-0327 -- which is accepted but not implemented because of these issues -- to remove the old rule that's in the proposal now: https://github.com/apple/swift-evolution/blob/main/proposals/0327-actor-initializers.md#global-actor-isolation-and-instance-members

It's feasible but I'm personally against introducing implicit suspension points for the purpose of evaluating default initializer expressions, especially because there could be many of them, resulting in many hops between executors that would be invisible in the initializer body. The same reasoning applies to asynchronous calls to functions with default arguments that require asynchronous evaluation. Even if the await is written explicitly at the call-site, I feel like hiding the actual asynchronous parts of the expression would make the behavior difficult to reason about.

8 Likes
4

Sorry, but how is that different than what we have now where we can use a single await to call a function that takes multiple async parameters, regardless of how many suspensions there may be overall?

2 Likes
5

It's not different with respect to the suspension point being marked with await. I'm talking about the fact that all asynchronous function calls within the await'd expression are explicitly written in the code, and you can interact with them at the point of the call. Allowing default arguments to be evaluated asynchronously also begs the question of whether you should be allowed to write asynchronous function calls in the default expression itself. I prefer the simpler rule that I'm proposing here, at the expense of having to write out the default argument if you want to evaluate it from across isolation boundaries.

1 Like
6

I think it's reasonable to get everything working correctly first, at least, as @hborla is inclined to do. Afterwards, it can be considered whether the async initialiser case should get enhanced behaviour.

I don't have any qualms with the idea of default arguments executing (safely) in other actor contexts as part of my overall await init, but it doesn't seem like anyone could be really relying on this right now since - if I understand correctly - that situation is just as broken today as the sync case…?

7

The approach of enforcing constraints at the callsite and banning use of default values in non-isolated contexts seems consistent with the implementation of default arguments Swift uses, but I think it will be surprising to some API authors.

When I'm using default arguments, my goal is usually to spare the user from knowing that they need to provide a value until they actually need to set something other than the default.

But if users call my function from the wrong context, they suddenly become painfully aware of all the default arguments, and potentially have to look up the values so they can await them. If the defaults change in my library, the consumer's code won't change with it.

Looking at one of the examples in the pitch, I wonder if it's technically feasible to implement awaiting default arguments such that

useDefault(value: await requiresMainActor())

could become

await useDefault()

The error message if the user simply wrote useDefault() in that context could be something along the lines of "This function has default arguments that require isolation to MainActor, await the function itself to isolate them."

6 Likes
8

Would this proposal also apply to isolation domains that aren't global actors? For example, currently the below code generates a warning, because NSMutableString is not Sendable but is sent across an actor isolation boundary.

actor SomeActor {
  func doThing(string: NSMutableString = NSMutableString()) { ... }
}

let a = SomeActor()
await a.doThing()

We could make it so that the default value expression is executed within the isolation domain of a. Going further, we could do that for arguments that are written out at the call site. So this would be legal too:

let a = SomeActor()
await a.doThing(string: NSMutableString())
// okay, the `NSMutableString()` expression
// is implicitly isolated to `a`
1 Like
9

It's not about the syntax - it's definitely feasible to consider the await on the call as covering the default argument evaluation. The concern that I stated above is about the fundamental implications of evaluating an isolated default argument at a call-site that is across isolation boundaries. Argument evaluation semantics are a little complicated, but I believe the order of argument evaluation is:

  1. Explicit rvalue arguments
  2. Default arguments
  3. Formal variable access

Each of these evaluations happen in the caller's context. This means that even if we're able to hop to the required isolation only once to evaluate all default arguments (instead of hopping back and forth between the required isolation and the caller's isolation for each default argument), we still have to hop back to the caller's isolation for formal access evaluation, before hopping to the callee's isolation.

I think what people would expect for a @MainActor-isolated function with @MainActor-isolated default arguments is for the hop to the MainActor to happen once to evaluate the default arguments and then immediately enter the function, but I do not think we can make that work. Of course, writing out the arguments explicitly doesn't help with this at all. A better alternative is to write your code to get on the MainActor first before making the call:

let result = await MainActor.run { useDefault() }

The compiler could even emit a fix-it to do this in the case where the programmer attempts to use isolated default arguments from across isolation boundaries if all default arguments have the same isolation (or nonisolated).

3 Likes
10

No, I don't think we can change the fundamental isolation rules for call arguments. I agree that it should be valid to transfer non-Sendable values across isolation boundaries when they are not protected by the isolation domain they were created in (e.g. because it's just a local variable or a temporary), and that's the goal of a different pitch:

2 Likes
11

I think what people would expect for a @MainActor -isolated function with @MainActor -isolated default arguments is for the hop to the MainActor to happen once to evaluate the default arguments and then immediately enter the function, but I do not think we can make that work

Thanks for explaining this! I just want to make sure I understand: because of the existing argument evaluation order, the ways to "make it work" are

  • change the argument evaluation order for this special case, so that default args come last, after Formal Variable Access. And this is out of the question
  • have some potentially surprising and slow code get generated with an extra actor hop to preserve this evaluation order

And therefore it's better to let users decide exactly which set of actor hops make sense for their use case explicitly?

I think the only drawback to the explicit hop is that if you have a nonisolated function with a MainActor isolated argument like

@MainActor func requiresIsolation()

func useDefault(theDefault: Int = requiresIsolation()) // idk what the purpose of code like this is, but it sounds like it will be legal after the pitch is implemented, based on your prior comment?

And then you change the definition of useDefault to switch to a new, non-isolated default, the compiler can't tell the author of the consuming code that their main actor hop is no longer necessary. Probably not a big deal in practice, unless there is a use case for this kind of code that I'm missing.

A better alternative is to write your code to get on the MainActor first before making the call... The compiler could even emit a fix-it to do this

Strong +1 to the fixit, though I understand fixing the bug is probably the first priority.

2 Likes
12

It still feels a bit weird to me that the function signature would essentially change depending on the caller's isolation context. It seems like the compiler should be able to communicate this clearly, which is a big mitigator, but still it feels a bit weird in principle.

Maybe it's just unfamiliarity; maybe in practice it'll work well enough and we'll get used to it.

Perhaps one of the FixIts offered should be to wrap the whole call [to init] in a movement to the necessary isolation context (await MainActor.run { … } or similar), where applicable (I guess only for global actors?).

13

Yes, that's exactly my thinking. I'll add this to the Alternatives Considered section so it's documented in the proposal.

Maybe the compiler can communicate this! There's an existing general problem where people write @MainActor (or @MainActor is inferred) on functions that really should be nonisolated because nothing in the function body actually requires running on the @MainActor. Part of my implementation for this proposal is adding a mode to the actor isolation checker that computes the required isolation of an expression or function body. I think we can use this mechanism to implement a warning for code that's running on the MainActor but doesn't need to be. If the required isolation of a function is nonisolated, there's no reason for it to have any specific isolation.

This is already the case for synchronous functions that are isolated to an actor, because calling such a function from another isolation domain will make the call implicitly asynchronous.

Yeah, default arguments can only ever be isolated to global actors, because there's no way to capture an actor instance in a default initializer expression to make it implicitly isolated to that instance. If there were ever a case where the compiler needed to emit a fix-it to isolate an expression to an actor instance, we'd need better control over closure actor isolation (not yet pitched, still undergoing early design iteration) in order to kick off a Task { [isolated actorInstance] in ... } that is explicitly isolated to the actor instance.

I think it'd be good to implement the fix-it we're discussing, because it seems like it'll important for understanding how best to address errors when you attempt to use a default argument from across isolation boundaries. I'll work on that, and hopefully we'll get some feedback from folks who try out the experimental implementation if/when this goes into review.

3 Likes
14

From a user's perspective I have a bit of concern about the fix-it. If I'm calling a function:

@MainActor
func usesMainActor(...) { ... }

func nonisolatedFunc() async {
  await usesMainActor()
}

and I get an error because usesMainActor has an isolated default expression somewhere in the signature, after applying the fix-it I have:

func nonisolatedFunc() async {
  await MainActor.run { usesMainActor() }
}

But to a subsequent reader of this code, I don't think the end state is particularly understandable... like, it's clear we're trying to move to the main actor, but part of the promise of Swift concurrency is that it's the callee which decides where it executes, and callers shouldn't need to care—they just have to suspend. IOW, I think your analysis here is totally right:

At the very least I think it may be worth privileging the above case. To me, using MainActor.run very much looks like an escape hatch used by someone who was trying to brute-force migrate old DispatchQueue.main.async code. If we had something to signify "move the evaluation of this entire expression to a global actor" e.g. await(MainActor) I think it would be wise to recommend that use here.

Again, though, I worry this slides back towards "callers need tell their callees where to run," counter to Swift concurrency's current philosophy.

This feels like an pretty unfortunate downside—the cost is not only additional code to write out, but we lose the behavior that default argument expressions may be updated transparently to the caller. Authors who wanted to maintain that behavior would essentially need to reinvent the current resilience regime where they define their own function for emitting each default argument they care about.

What sort of interactivity are you looking for here? In terms of debugging it seems like we should perfectly well be able to 'step to' the default argument expression in the interface to show when it's being evaluated. And if the concern is that there are potentially many suspension points at that line of source being hidden by an await, is that really meaningfully different than the fact that calling into an arbitrary async function may admit arbitrarily many suspension points internally?

I guess what I'm getting at is that I can forsee these isolation rules pushing authors of APIs like:

@MainActor class C { ... }
@MainActor func f(c: C = C()) { ... }

towards something like:

@MainActor func f(c: C? = nil) {
  let c = c ?? C()
  ...
}

just to spare their users of the additional annoyance of using the default argument expression directly. Do we really think this is better?

6 Likes
15

Sorry I should have been more specific here; I meant that the programmer can tweak the call / break up the expression to explicitly control the isolation domain where different subexpressions are evaluated.

1 Like
16

+1 to the point about nil being a better default after this change lands. I can think of a few alternatives to this though:

  • use a property wrapper. Unfortunately you can't have a mutating get on a parameter property wrapper though, so the wrapper in question has to be a class. Probably not the best solution from a performance standpoint
  • a macro (@LateDefaults?) that rewrites all the default arguments of a function to be nil with the aliasing from your last code example, just before the rest of the function body

I think that the macro option is pretty reasonable, it lets the author of the API explicitly say "please break the order of argument evaluation for me, in this specific context."

17

Your points about the caller not needing to care about the callee's isolation are really interesting, and make me wonder whether we're going down the wrong path for how to solve this. If an API wants a default argument to always be evaluated in the callee's isolation domain -- which I think is the case for isolated functions with isolated default arguments -- it should probably use an @autoclosure. Maybe this proposal should only cover computing the required isolation for expressions that can be evaluated synchronously from anywhere, i.e. closures. If an API author tries to use a default argument expression that's isolated, the compiler can suggest wrapping it in an @autoclosure. This would mean that we cannot make memberwise initializers nonisolated, because the initializer would need to accept @autoclosures for each of the isolated stored properties and evaluate those inside the init body. Hmmm.

4 Likes
18

I agree with the people above me in this thread :slight_smile:

  • I'm happy with the proposal — it seems to cover the safety well, and gives clear enough errors that you'll be able to work your way through the issues involved with the interaction of default arguments and actor isolation.
  • I think that this direction doesn't address the common "intent" of default arguments — to effectively provide a suite of related functions with a single implementation — where the default arguments require isolation. In effect, it becomes necessary to use optional argument types and nil defaults and manually move the default arguments into the body of the function.

(@autoclosure arguments have other current problems with isolation: Autoclosures and global actors don't work together · Issue #67688 · apple/swift · GitHub)

19

It may not be fixable now, but I think in effect the original decision to evaluate default arguments in the caller rather than the callee was wrong (it was done this way for #file, etc., I'm sure, and by analogy to many other languages).

Could we instead consider something like:

  • two kinds of default argument, @callerEvaluated and @calleeEvaluated;
  • In swift 5 the default remains @callerEvaluated. In Swift 6 the default becomes @calleeEvaluated, with the source migrator suggesting adding @callerEvaluated where default arguments are macros, or where ABI stability is required;
  • In swift 5 where the argument has an isolation requirement, a fixit to suggest adding @calleeEvaluated

I guess the implementation of @calleeEvaluated would effectively need to create functions with each of the possible effective signatures (combinations of arguments omitted) each of which evaluates the default arguments in the correct isolation domain & passes them synchronously to the user-written function with all possible arguments? So the ABI would effectively be "as if I'd manually written out each of these signatures" (might be better anyway?)

1 Like
20

I don't see any evidence in this thread that the fundamental evaluation semantics of default arguments are harmful in general and warrant a widespread source and semantic breaking change. Magic literals are not the only benefit to caller-side defaults arguments. The alternative:

has a combinatorial code size impact and effectively makes default arguments ABI by default. If you're also talking about introducing these overloads into the type system, that will seriously impact overload resolution performance, and the argument matching behavior will differ between caller-side and callee-side default arguments.

As you point out, there are tradeoffs between caller-side and callee-side default arguments, but having both creates bigger problems than it solves, and we have other options for closing the data race safety hole with isolated stored properties.

We already have a tool to delay the evaluation of arguments to the callee. That tool is @autoclosure. If we decide that's the right tool for the job here, we can fix the existing problems with actor-isolated autoclosures.

3 Likes