[Concurrency] Evolving the Concurrency design and proposals

What would be the point of such function ? If it does call an async method, it doesn't have to be async itself.

If you're talking about writing your own async primitive, you still have to call runtime async functions like Task.withUnsafeContinuation, else you are not writing an async function.

  1. The function is a placeholder for which you haven't written any asynchronous guts for, yet.
  2. The function no longer needs to be asynchronous, but you can't change the public API for it.
  3. The function is only asynchronous on some paths through its code.
4 Likes

Yes Im talking about writing my own async primitive. Sounds like something we will be doing all the time..? So why is it so unclear how to do it?

Task.withUnsafeContinuation Huh? Why is the Task namespace involved? Why is static method call unsafe? What with continuation refer to exactly?

Why isn’t it:

Async.new

?

Or Async.create?

Because async is a facility to make function with completion callback usable like blocking functions, not to make blocking function magically asynchronous.

And it requires that the completion callback be called exactly once. As the system has no way to make sure this contract is enforced, it considers this call unsafe. Your primitive must be able to tell the runtime that the callback has been called, this is what continuation provides, a proxy object to notify the runtime.

Writing async primitive is considered advanced usage, and requires some knowledge about how async works under the hood.

What do you expect Async.create to do exactly ?

What Sajjon may be getting at, and what I'm now wondering myself, is how to wrap a closure based asynchronous function in a native Swift async function. This is something I'm doing all the time with promise/future implementations. But here, I have no explicit promise to fulfill:

func request() async -> SomeResult {
    requestWithClosure { result in
        // now what?
    }
}
2 Likes

This is exactly the use case of withUnsafeContinuation:

func request() async -> SomeResult {
    Task.withUnsafeContinuation { continuation in
        requestWithClosure { result in
            continuation.resume(returning: result) }
        }
    }
}

The unsafeness is unavoidable without extra language features to support the usage restrictions: continuations must be invoked exactly once and there must be no code executed after the continuation.

With such restrictions, it might be possible to provide a “safe” version that could be used in the simplest use cases, but see the section “The challenge with diagnostics for Unsafe” in Structured Concurrency.

3 Likes

Two safe versions could be provided, though: one that checks that the continuation has been resumed only once, and ignores further resumes, and one that performs the same check and fatal errors in case of multiple resumes. For the first one, this could give:

func request() async -> SomeResult {
    Task.withContinuation { continuation in
        requestWithClosure { result in
            continuation.resume(returning: result)
        }
    }
}

The misuse of not resuming is not caught. I think we can still remove the "unsafe" qualifier, since all future/reactive/promise libs out there also just rely on documentation in order to prevent such a mistake. Maybe this last sentence belongs to the "old world", filled with unsafety and fragile conventions, and some would consider those convenience checks as no less unsafe than the unchecked version. I don't know.

2 Likes

I think so. Avoiding this is a large part of the benefit of structured concurrency, and marking the escape hatch with Here Be Dragons seems desirable and even necessary.

2 Likes

This is the easy path.

One also has to consider the developers who will wrap existing and well-behaved completion-based apis. Do we really want their code to be riddled with "unsafe" calls, when they do nothing wrong, and they have no way to get rid of them expect writing the convenience wrappers I just described, possibly in a wrong way ?

Unsafety-in-your-face can just be pedantic without benefits, and even have plain bad consequences. I suggest an open mind.

1 Like

I think there was a comment on another thread saying that the second one, dynamically enforcing at-most-once behaviour, was intended to be baked in.

However, transitioning from async semantics to continuation-that-might-not-resume semantics is unsafe, and requires appropriate care. I don’t see any benefit to an API that glosses over this.

1 Like

Never mind. The rationale for an API without "unsafe" in its name is there, for anyone interested. Have a nice day!

It does show the "entry into the async world":

// can be not-async world
let dinnerHandle = Task.runDetached { // () async -> Dinner in 
  // async world
  await makeDinner()
}
// can be not-async world

We can add this implicit type of the closure to the writeup though to make it more clear, thanks.

3 Likes

Ah I see. As a Swift user who only casually follows Swift's evolution, I had expected this to be expressible explicitly with the async/await syntax, somewhat like this:

func request() async -> SomeResult {
    async var asyncResult: SomeResult
    requestWithClosure { result in
        asyncResult = result
    }
    return asyncResult
}

I'd love async to extend to properties in that way, but there are probably a million technical reasons for why that's impossible :cry:

Why not go the other way, allowing "throws async" instead of banning "try await"? We would probably need to allow arbitrary orders, including have the call site be in a different order than the declaration site, when/if we move to a general effects mechanic. (Hmm, should we do the general effects system first, then add async/await?)

I think enforcing a clear order is good idea. This would save community from writing linters/formatters normalising the order. But it is not clear to why try await was preferred over await try.

I'm reading await and try as prefix operators, applied from right to left.

  1. await try foo() means "try to start an asynchronous operation, and if it starts wait for the result". Signature of foo being equivalent to foo() -> Result<Promise<T>, Error>.
  2. try await foo(), means "launch an asynchronous operation which may fail, wait for it to finish and try to get its result". Signature of foo being equivalent to foo() -> Promise<Result<T, Error>>.

Proposal models the second case, so IMO, try await and throws async make more sense.

2 Likes

How to turn a long running (e.g., compute-intensive) synchronous function into a non-blocking async primitive (to take full the advantages of structured concurrency, run multiples in parallel)? Do we need to manually resort to DispatchQueue/Operation/OperationQueue? Or there’s a Task API ready for that?

Can we still Task.checkCancellation() during the long running synchronous function?

1 Like

I thought I might have overlooked something (there is much text about concurrency), but as you didn't get a simple answer yet, there really seems to be a hole here...
Has anyone already thought of allowing the following?

let resultA = async takesALongTime(100000000)
...
return await min(resultA, resultB)

(I think there is no explanation needed: Either the idea is obvious, or it does not fly ;-)

Exactly, there is an API for that.

There’s one thing still confuses me:

  public static func withUnsafeContinuation<T>(
    operation: (UnsafeContinuation<T>) -> ()
  ) async -> T { ... }

Will the this operation be running on a different thread than its caller? Otherwise it will block the calling thread.

If the answer is yes, does withGroup have similar semantics? Will the body passed to withGroup also be running on a different thread?

Thanks!

It’s packaging up the “rest of the current function” into a continuation (the UnsafeContinuation instance) that you can use in a completion handler closure. When that completion handler gets called, the rest of your function continues. It’s glue for completion-handler APIs.

withGroup is very different. It helps you manage a set of child tasks that run concurrently.

Doug