SE-0431: `@isolated(any)` Function Types

Dmitriy_Ignatyev · 2024-03-28T10:19:38.099Z

ktoso:

I'd argue against this because it yet again is "special" spelling IMHO.

I thought it would be legal because current @MainActor doesn't specify .self or .shared.
Now I get it, thanks for claryfying this:

ktoso:

Generally in Swift you can not just spell "TheType " and you'd normally spell TheType.self

ktoso:

I'm not sure we should be focusing on this piece and focus on the @isolated(any) [1] being proposed perhaps?

Shure. I'm totally happy with the proposal and @isolated(any), just wanted to discuss one moment from future direction. You've provided enough explanations .

juliand665 · 2024-04-01T22:53:27.940Z

If only G specifies @isolated(any), then the behavior depends on the specified isolation of F:

If F has an isolated parameter, the conversion is invalid.

What's the rationale behind this limitation? My intuition is that isolation to a parameter is just another form of isolation to a specific actor, which this proposal already supports for captures. Is there some consequence that I'm not thinking of?

John_McCall · 2024-04-01T23:33:43.729Z

Well, consider an example. Suppose I have a @Sendable (isolated MyActor) -> (), and I convert that to @isolated(any) (MyActor) -> (). What is the isolation of that function? It's not isolated to some specific actor — it's isolated to whatever actor you pass in as its argument. So what does isolation return?

Now, I can imagine a feature that's like "here's an algorithm for dynamically deciding the isolation of this function given all its arguments". I think that's a different and more complicated feature, though. Moreover, in practice I think it probably wouldn't be very useful, because I don't think it's common to have a function that takes an actor argument like this but not know whether the function will be isolated to it; and if you're "partially applying" the function to a specific actor, you can express that with an isolated capture:

let fn: @Sendable (isolated MyActor) -> () = ...
let myActor = ...

let convertedFn: @isolated(any) () -> () =
  { [isolated myActor] in fn(myActor) }

juliand665 · 2024-04-02T03:59:42.632Z

Thank you, that clears it up! I think I glossed over the fact that the specific actor is chosen by the caller instead of the callee in this case. Maybe this could be quickly mentioned in the proposal, but it's not that important.

rexmas · 2024-04-08T02:33:09.120Z

Overall I think the proposal & design looks great, but I'd like to echo a similar sentiment as Dmitriy.

Most of Swift has the property of gracefully introducing the user to new concepts when they're needed, often in fixits or error messages, and/or by making common enough harder concepts (like concurrency) syntactically simple. I'm not getting the sense however that it will be obvious to the average dev team member, even one working on a library, to utilize @isolated(any) in the cases where it should be included here unless they have read this proposal (or future documentation), so this seems like this could be a common performance miss. Will there be mechanisms (or tooling) for communicating this more directly or will this need to just be one of those culturally encoded "best practices" in Swift?

dnadoba · 2024-04-08T16:32:36.774Z

Great proposal that adds a missing feature!

I'm wondering though how we could extend this to other types than just functions.
Swift has recently ported features that were once only available to functions/closures to other places e.g.:

Typed throws have made rethrows possible in protocols and generics. Also replaces @rethrows for protocols.
~Escapable is being discussed on the forums as well that brings the power of @nonescapble closures to protocols and generics.

On the other hand, protocols/structs/enums/classes have features that aren't available to closures. Closures can not properly participate in the generics system because a function signature is not a valid generic constraint. Protocols are sometimes needed to workaround this limitation.

This has come up recently in another thread and I have written down some of the use cases where functions/closures currently can't be used and we need to fallback to protocols: FunctionalProtocols for Swift - #15 by dnadoba

I can imagine that we will eventually need to bring this and other isolation features to other types in general, potential make it part of the generics system so this information can be propagate through chained operations.

This is likely out of scope for this proposal but I would love to see something in the Future directions section.

John_McCall · 2024-04-08T18:36:45.832Z

Allowing generics to talk explicitly about abstract isolations in the same as they reason about, say, typed errors would require value-dependent types, which are already discussed in Future Directions; the quick summary is that it's inherently conceptually difficult and not likely to be something we'd pursue.

Some kind of generic function constraint is something we could think about, but I don't think it's really something that needs to be developed in this proposal. It would have to consider isolation, but that's not new.

jamieQ · 2024-04-23T12:54:08.449Z

i realize that the nominal review period for this is over, so i apologize if this feedback would be better placed elsewhere, but i had a few thoughts/questions on this and some related proposals.

in particular, i'm wondering how the proposal is intended to interact with the recently-approved 'task executor preference' API (SE-0417).

consider the following example:

let op: @Sendable @MainActor () async -> Void = {
    MainActor.assertIsolated()
    print("main actor")
}

Task(
    executorPreference: CustomTaskExecutor.shared,
    operation: op
)

based on my reading of the two documents, i would expect the formal isolation of the operation to be resolved to the main actor, and thus would expect the Task to be directly enqued there, effectively bypassing the explicitly-specified TaskExecutor preference. is this interpretation accurate? perhaps it is expected given the current implementation status of things, but this did not appear to be how some of the recent development toolchain snapshots behave.

an additional thought, which admittedly is a bit vague, is in regards to potential 'macro-scale' behavioral changes that this feature may induce in existing code. consider another (somewhat contrived) example where we enqueue a number of tasks of 2 different priorities to be run on the main actor:

func test_priorities() async {
    let iterationCount = 100
    var done: (() async -> Void)?
    for i in 1...iterationCount {
        let pri: TaskPriority = (i > iterationCount / 2) ? .high : .low
        
        // aside: an explicit type annotation is needed for some reason.
        // @MainActor isolation appears to be dropped without it.
        let op: @Sendable @MainActor () async -> Void = {
            let qos = qos_class_self()
            print("running task: \(i), pri: \(pri), qos: \(qos)")
        }

        let t = Task(
            priority: pri,
            operation: op
        )
        done = { await t.value }
    }

    await done?()
}

today, due to the initial 'hop' to the generic executor, this form of code will often effectively behave somewhat like a DispatchWorkloop. that is, the Tasks will (often) be run in decreasing priority order. with the enqueuing logic change proposed here, this will no longer be the case, and in this example, all the low priority Tasks will be executed before the high priority ones.

now, i think in this example this is a non-issue (the effective priority of the Tasks is escalated since they run on the main thread), but it at least raises the question of whether this change will potentially lead to priority inversions that did not used to exist, or more generally will have observable and undesirable changes on existing code.

overall this seems like a good addition to the language, and personally i think making initial execution order match Task initialization order where that is possible and reasonable would be a huge win, but do wonder about potential impacts to existing code.

John_McCall · 2024-04-23T18:13:54.804Z

jamieQ:

based on my reading of the two documents, i would expect the formal isolation of the operation to be resolved to the main actor, and thus would expect the Task to be directly enqued there, effectively bypassing the explicitly-specified TaskExecutor preference. is this interpretation accurate?

That’s right. Function isolation should always “win” over the task executor when they’re in conflict. They’re not always in conflict, though — actors will by default run on task executors. If you’re not seeing that, we’ll look into it.

John_McCall · 2024-04-23T23:56:26.208Z

jamieQ:

now, i think in this example this is a non-issue (the effective priority of the Tasks is escalated since they run on the main thread), but it at least raises the question of whether this change will potentially lead to priority inversions that did not used to exist, or more generally will have observable and undesirable changes on existing code.

Almost any scheduling change can theoretically have undesirable changes on existing code. It's not something that's worth keeping oneself up all night over.

John_McCall · 2024-04-24T00:28:48.415Z

I'd like to open up the conversation about the spelling here.

The spelling in the current proposal, @isolated(any), intentionally carves out @isolated as a namespace for similar attributes for controlling isolation. For example, we could use @isolated(caller) as a shorthand for SE-0420's param: isolated (any Actor)? = #isolation, or @isolated(to: value) in the future direction of value-dependent function isolation. However, we have existing uses of isolated and nonisolated that don't use an @, and while it's not grammatically problematic to distinguish features by @-vs.-no-@, it is likely to be confusing to people using the language. (This is why I include nonisolated — formally, yes, it's a totally different keyword, but people would be surprised for it to have different rules from isolated.) We probably don't want to create this artificial question of whether isolated starts with an @.

At the same time, I'm very reluctant to just remove the @. Originally, I added the @ in order to solve a thorny parsing problem; I think that problem can probably be overcome, and I don't think we should worry too much about it. However, I think it's wrong in a much deeper way for isolated and isolated(any) to both exist but apply in very different ways in the language:

// `isolated` is a parameter modifier.  It can only be used in this
// way on a parameter and isn't reflected in the type of a reference
// to `param`.
func foo(param: isolated MyActor)

// `isolated(any)` is part of the type.  It can be used anywhere you
// can write a type, and of course it is reflected in the type of a
// reference to `param`.
func bar(param: isolated(any) () -> ())

We should not muddy the waters between type attributes and parameter modifiers this way.

If we want to avoid both of these problems, I think the best approach is to use a different spelling entirely. We can still have isolated in the name in order to make the attribute more self-documenting, but we should try to avoid making it look like the existing modifier. That would suggest something like @dynamicallyIsolated. If we introduce other isolation-controlling attributes in the future, we can follow that same pattern with e.g. @callerIsolated.

Jon_Shier · 2024-04-24T02:46:42.170Z

I like @isolated exactly because it takes a parameter. If Xcode could autocomplete in there I could see it being easier to learn, since you only need to remember isolated, which is used in other ways, and not all the types of isolation. Swift already has tons of keywords, so parameterizing one seems much nicer than creating slightly different variants. And I think your point in the example is well taken, dropping the @ would be confusing (at least, to anyone who knows the difference between parameter and type modifiers).

Douglas_Gregor · 2024-04-30T20:02:12.359Z

John_McCall:

If we want to avoid both of these problems, I think the best approach is to use a different spelling entirely. We can still have isolated in the name in order to make the attribute more self-documenting, but we should try to avoid making it look like the existing modifier. That would suggest something like @dynamicallyIsolated. If we introduce other isolation-controlling attributes in the future, we can follow that same pattern with e.g. @callerIsolated.

Hrm. Having this whole "zoo" of Isolated-suffixed new attributes is probably worse than the @-or-not confusion with @isolated. I suppose we could address this with tooling to help you add or remove the @ as appropriate.

Doug

Nickolas_Pohilets · 2024-04-30T21:32:28.131Z

I do like @isolated(any) exactly because it paves the way to value-dependent value isolation. And existing isolated-params, are a limited form of value-dependent isolation. So, IMO, in the long-term we should be aiming at depreciating isolated arguments, global actor attributes and other limited tools, with a more universal @isolated(arg) syntax.

E.g, func foo(param: isolated MyActor) should become @isolated(param) func foo(param: MyActor).

But this looks weird, because @isolated(param) refers to param before it was declared. So maybe this should be among effects, and not attributes?

func foo(param: MyActor) isolated(param) throws(MyError) async -> Bool {
    …
}

wadetregaskis · 2024-04-30T22:47:11.461Z

Nickolas_Pohilets:

But this looks weird, because @isolated(param) refers to param before it was declared.

I don't think it looks weird objectively; I think this is just something you'd get used to, easily. The function can only be isolated to one parameter (as far as I understand it?) so at most you have to remember one [parameter] name as you're reading left-to-right.

Putting it after is technically fine too, but makes the ergonomics harder because it's more awkward to line wrap after the parameter list.

Either way, I think this makes way more sense than the current isolated parameter modifier, as this isolation requirement is a function-level attribute, not a parameter-level attribute, and a particularly important one at that - so it makes a lot of sense to have it declared more prominently and in a more predictable place in the grammar.

stephencelis · 2024-05-02T21:52:29.956Z

I pulled down a recent development snapshot to take this for a spin, but even with the experimental feature enabled I can't seem to access the closure.isolated parameter. Is there a different syntax being used in the implementation?

jamieQ · 2024-05-03T12:57:48.607Z

it's not clear to me from the proposal in exactly which contexts one should expect that new value to be accessible, but the following appeared to work locally (4/21 dev snapshot):

let op: @isolated(any) () async -> Void = { @MainActor in
    // ...
}
print(op.isolation ?? "nil") // Swift.MainActor

hborla · 2024-05-03T15:45:14.769Z

stephencelis:

I pulled down a recent development snapshot to take this for a spin, but even with the experimental feature enabled I can't seem to access the closure.isolated parameter. Is there a different syntax being used in the implementation?

The syntax is .isolation, not .isolated. Does closure.isolation work for you?

Douglas_Gregor · 2024-05-04T15:04:25.836Z

Thakn you for the great discussion everyone, this proposal has been accepted with modification to remove @Sendable inference.

Doug

John_McCall · 2024-05-04T17:11:45.947Z