Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to the review manager via the forum messaging feature. When contacting the review manager directly, please keep the proposal link at the top of the message.
What goes into a review?
The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:
What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
More information about the Swift evolution process is available here.
I think the "future direction" of ~Discardable could not actually be adopted by Continuation since "I can prove to the compiler that I always consume this object" is quite different from "dynamically, I always consume this object", and I wouldn't expect it to be able to be done source-compatibly.
The usefulness of this proposal is tied to the acceptance of SE-0527, since without UniqueArray the stdlib has no solution for storing a collection of Continuations
Until Mutex.withLock accepts a once-callable closure (#81546), most uses of this type will have to jump through silly hoops like:
let continuations = Mutex(UniqueArray<Continuation<T, E>>())
...
try await withContinuation(...) { continuation in
var optionalContinuation = Optional(continuation)
continuations.withLock { array in
array.append(optionalContinuation.take()!)
}
}
Overall, I don't think that this actually, right now, gets us to a situation where the compiler can ensure we're using continuations safely. The runtime crash for discarded continuations is glaring, and the necessity of using optionals and force-unwrap to deal with the lack of once-callable closures means that in practice, there are likely to be more runtime checks associated with using Continuation than are currently required by CheckedContinuation. Certainly I don't think I have any uses of CheckedContinuation that could be migrated and the code made better as a result.
It's a "yes, but not yet" from me. Let's get SE-0527 approved, once-callable closures added, and a concrete decision on ~Discardable before going here. This proposal will be all the better for the wait, and there's no clamor for it now.
Arguably we need even more ā call-once closures to make it truly āgenerally usefulā. But I donāt think this is holding off the introduction of this type. Continued improvements in type system will just incrementally allow these types to be used in more places.
Thatās also why we have the conversion to the copyable continuations.
Realistically, I think that feature is pretty unlikely. Future directions donāt actually mean āwe will do thisā but are just open ended ideas which could be explored. I wouldnāt overly focus on the ~Discardable. And actually the dynamic crashing is a pretty neat tradeoff, itās hard to get in that situation to be honest
Even without a full-featured modeling of ~Discardable as a protocol, another direction we could consider is warning at compile time when types like Continuation are obviously dropped without being consumed when they shouldn't be. This wouldn't be foolproof, since we would lose the ability to track that through generics, existentials, closure captures, and other abstractions, but could be helpful to immediately catch obvious mistakes.
Yeah, given that the compiler probably can't prevent a ~Discardable type being caught in a retain cycle and leaked, I didn't think that ~Discardable was all that practical ā should it even be in the "future directions" though, if nobody thinks that it's realistic? Certainly I interpreted it as "this is still actually under consideration", not "nobody seriously thinks this is a possibility".
I do like Joe's idea of a best-effort @mustConsume annotation for the type, though. I agree it will catch stuff enough to be useful. But I don't know how you communicate the best-effort-ness of it.
This new type would be an immense improvement for concurrency implementations, e.g., _Concurrency and Async-Algorithms: improved safety while maintaining maximum performance, @safe type, proper typed throws, and ~Copyable elements. I would very much welcome this addition.
Itās unfortunate that the safety aspect canāt be fully enforced statically and that, without call-once closures, we have to rely on some awkward workarounds. However, in my opinion, the benefits outweigh the drawbacks. I agree with @ktoso here that the lack of call-once closures shouldnāt hold us back. In fact, I hope this serves as further motivation to implement such feature.
The only concern I have is that, with the addition of this type, we would end up with a total of five continuation types and potentially a sixth in the future.
It was thoroughly discussed during the pitch phase, but I'd like to reiterate my concerns here as well.
While I completely agree with the motivation, I must respectfully lean toward -1 on inclusion in the Standard Library at this time for the following reasons.
I believe we should keep the standard library's API as tidy as possible and avoid landing "half-way-there" solutions.
There is no strict need for this particular implementation to live in the standard library. It does not rely on any internal standard-library-only compiler features. It's essentially a highly useful wrapper over UnsafeContinuation. Because anyone can implement this exact type in their own codebase or as a standalone package today, there is no urgency to lock this specific set of tradeoffs into the stdlib.
Furthermore, a fatalError inside a standard library type's deinit produces one of the most opaque crashes for users.
I also strongly believe we should explore ~Discardable types first. The language already has just enough features today to support an MVP of ~Discardable (in classes, actors, and non-copyable structs).
Additionally, Continuation is simply too cool of a name to burn on a compromised implementation. It would be highly unfortunate to find ourselves capable of building a perfect, compile-time-safe continuation in the future, only to find the Continuation namespace already permanently taken by a type that relies on a deinit trap.
Another thing I'd like to add is that the introduction of "called-once closures" won't solve the integration problem.
Continuations are often used with third-party and legacy callback-based APIs. That legacy code has a very slim chance of ever being updated to adopt new "called-once" closure annotations. So, even if the language gets that feature, it won't immediately help us in the exact places where continuations are needed most.
Instead of relying on called-once closures, I think it would be highly beneficial to explore two other extensions for closure captures to handle non-discardable/non-copyable types:
Mutable captures: Allow syntax like { [var innerVar = consume outerVar] in ... }. This makes innerVar unavailable to the outer scope while allowing it to be mutable inside the closure. I don't think this would be too difficult to implement. If I'm not mistaken, weak captures are already mutable.
Closure deinit blocks: Allow a deinit block to be attached directly to a closure's capture list. SE-0429 brought the ability to consume fields in a deinit without reassigning them, based on the guarantee that nothing can access them afterward. Theoretically, a similar behavior is achievable for variables solely owned by a closure. It will probably be tricky since, if I'm not mistaken, captured mutable variables are individually boxed with lifetimes separate from the closure itself, but it represents a much more robust future direction.
If we had ~Discardable types combined with these two closure enhancements, integrating a perfectly safe Continuation into a legacy callback API would look like this:
try await withContinuation(of: Data.self, throws: (any Error).self) { continuation in
legacyFunction {
[
var innerContinuation = Optional(consume continuation),
deinit {
// innerContinuation is solely owned by this closure,
// so we can safely consume the ~Discardable payload here.
if let cont = consume innerContinuation {
cont.resume(throwing: CancellationError())
}
}
] result in
if let cont = innerContinuation.take() {
cont.resume(returning: result)
} else {
assertionFailure("Callback invoked multiple times")
}
}
}
This approach would give us total compile-time safety without hardcoding unrecoverable traps into the standard library, and would force users to handle edge cases thoughtfully.
I'd love to see us hold off on Continuation until we can build it on top of ~Discardable, or until ~Discardable gets rejected by the LSG.
To clarify on this point: You canāt implement this (exact) type including its support for non copyable values outside the standard library. It needs direct access to builtins and it required changes in compiler specific builtins to achieve this.
Updating existing continuation types to support typed throws and non copyable values is still a goal but is a heavier liftā¦ weāll want to do it for sure, but we cannot and will not make them be non copyable themselves for exampleā¦ so the type is definitely uniquely valuable and not some half baked version like some folks seem to be implying.
To say it another way, I donāt think this type is in any way āhalf doneā, it is exactly how we want to spell this kind of type.
Perhaps the future direction is misleading people into thinking that this direction would be absolutely required, which I donāt think is actually desirable tbh.
Yes and no, the crash would point at the exact method where the mistake happens. Because unlike just a class deinit, a noncopyable types deinit is going to happen exactly in one place there the unintentional drop happened
I donāt really want to be going into speculating about what thisānot yet designedāfeature might be like. But I do think it is understood that ācallbacks that run exactly onceā is a very common thing both in Apple sdk and just other libraries out there.
I would not be shocked if we allowed use site annotating closures this way and inject a dynamic check to guarantee this property, or annotate a use site with some āonce(unsafe)ā.
Again, this is entirely speculative and not part of this review. But looking at the situation of many existing libraries itās something I can see come about, in that way, yes, those once closures would solve the interop with unannotated APIs problem for Continuation.
// ps sorry for the reply by reply posts, not at my computer but wanted to reply to some of the points today :-)
At risk of derailing further, can we reuse any concepts we already have to get this functionality instead of inventing new ones? Could a "callback that runs at most once" come by having some way of forming a ~Copyable function type that must be consumed when it's called, and then a "callback that runs exactly once" is that + something like @mustConsume that's been mentioned before?
There are probably even more places where ~Copyable types fall short but I don't think this should hold up this proposal. If anything, it is actually the opposite. This proposal is a great motivation to fix those issues. There exist workaround for any of those problems, especially because you can convert the ~Copyable version to a Copyable version.
I think those other features will compose very nicely with this new type. I don't see a dependency here as nothing about this type or proposal will change at all.
One downside compared to CheckedContinuation is that previously you could easily find out where a continuation was created from by enabling malloc stack logging which will include the backtrace of the withCheckedContinuation call because it allocated a class. It also included the name of the calling function through function: String = #function.
Does the implementation store enough metadata to recover the backtrace at the point of creation? Is there any integration with the memory debugger to find all alive continuations?
I am suspicious of features that are almost useless without language extensions, but this doesn't strike me as being the case here. CheckedContinuation traps on deinit if the continuation expires before it's resumed, and the sky isn't falling.
I agree that collections of unique arrays make this more useful, but I think it's useful as-is, and it has its place in the standard library because people who need to pass around continuations should use Continuation at API boundaries instead of CheckedContinuation or UnsafeContinuation.
Besides, arrays of non-copyable types aren't a done deal, but they're at least in review, whereas ~Discardable is an idea without a pitch and no clear path forward. I think that we should evaluate this feature in the context that ~Discardable might never happen, and whether that would be fine.
With the evidence of CheckedContinuation being fine, and without additional evidence that Continuation is at a higher risk of causing issues, to me, the answer is "yes".
I find the design of such ~Escapable Continuation highly useful. That said, I donāt see a strong case for incorporating it into the standard library at this stage.
The design itself is solid, and it should certainly be adopted and preferred where appropriate.
In the meantime, there are several practical paths forward:
Introduce it in AsyncAlgorithms.
Maintain a local implementation within individual codebases.
Consider it as an experimental feature, waiting for complete statical compile-time checks become possible.
Right, and I apologize for not checking the diff since the pitch phase. I had assumed UnsafeContinuation would be updated first, since it's a lower-level type. It's good to know UnsafeContinuation is on the list, even if the order of updates is a bit unexpected.
I think it comes down to a philosophical difference regarding where we want to shift responsibility. Should we empower users to control edge cases, or should we explicitly take the responsibility of handling a missing invocation away from them? If ~Discardable types were in the language today, would we still hardcode a trap into this type?
Could you clarify this? What exactly should the stack trace show in this scenario? I tried a simple reproducer on 6.3, and the resulting stack trace isn't particularly clear about what closure and where it was actually dropped:
I have tried to write such a type myself in the past and hit the limitations that @ktoso described where it requires built-ins to implement. I am very much in favor of adding this type to the standard library and think the alluded alternative of ~Discardable seems easy on the surface but would be a huge new language feature similar to what ~Copyable and ~Escapable are. Similar to what @fclout has said, such a feature hasn't been pitched nor does it look like anyone is working on it right now. The proposed Continuation type will add real value and unblock cases where we currently cannot support ~Copyable such as in various swift-async-algorithm types like the latest MultiProducerSingleConsumer type.
Generally speaking, I think we will see more types that are ~Copyable with consuming methods and a deinit that preconditions that the type was consumed.
If there is a real engineering need for this type right now (as opposed to, for example, a desire to make the language more clean or accessible for newcomers) then can it be named something more explicit and then when the final decision is ultimately made about ~Discardable, depending on what that decision is, we either add a typealias to allow granting the name Continuation to this type or thank our past selves for not burning the clean name on this type?