This is a pitch to introduce single-yield functions to the language. Single-yield functions are declared by writing yield in the result position:
func f() -> yield T // yields an owned value
func f() -> yield borrow T // yields a borrowed value
func f() -> yield inout T // yields a mutable value
A single-yield function is a coroutine that runs until it reaches a yield statement. It then yields a value to its caller and suspends. When the caller is finished accessing the value, the coroutine resumes and runs to completion, performing any necessary cleanup.
This sounds like a great pitch. The only bikeshedding I have is that I think yielding would be better than yield in the function declaration (not the yielding func alternative, but still keeping it next to the returned type), i.e.:
func f() -> yielding T // yielding an owned value
func f() -> yielding borrow T // yielding a borrowed value
func f() -> yielding inout T // yielding a mutable value
This would also make it more consistent with the yielding syntax we currently have for accessors.
Very cool, this is something I've wanted for a long time.
Defers
It isn't called out in the proposal, but do defer statements work as one would expect with yield?
func f() -> yield T {
let t = getAT()
defer { t.release() }
yield t
// t.release() should happen here
}
Multiple yields
Since it's listed as a future direction, could you also provide some more insight on what more is needed to support multiple yields? Generators are one of the big things I'm interested in here, which this proposal doesn't yet support—which is fine, we don't need to bite off the entire feature at once if it's too complicated, but it'd be nice to know what some of the limitations or challenges are.
with* functions
I haven't thought this completely through yet, but there are a ton of closure-taking with* functions where functionally the closure invocation could be replaced by yielding the value and callers could reduce the pyramid of doom on their end. I wonder if you've given any thought to which standard library APIs could be improved with yielding versions, or if (crazy idea here) there's a path where for certain kinds of with* functions the compiler could auto-synthesize a yielding version, similar to how ClangImporter uses a bunch of heuristics to auto-translate Obj-C completion handler functions into async.
I'd be hesitant to pursue this because it depends on the body—there are several criteria ("yields exactly once", "does not throw after yielding") that aren't visible just from the declaration signature. It would be nice™ and maybe we can make it easier in the future, but the "auto" part is probably not going to fly.
Agree that yielding seems a better choice both grammatically and for reuse of the existing keyword.
yielding borrow T is also ungrammatical and doesn't reuse the borrowing keyword (where yielding inout T does reuse the existing keyword). yielding borrowing T isn't great but I think it's better.
The proposal doesn't seem to address where in the calling function these coroutines are resumed, and whether or not they form a strict stack. Maybe that behavior is already specified elsewhere for yielding borrow accessors?
e.g., if I write
func f() -> yielding Something { ... }
func g() {
let a = f()
let b = f()
a.method()
// does the continuation associated with `a` resume here?
b.method()
// the continuation associated with `b` resumes here
// or does the continuation associated with `a` resume here?
}
This also has the advantage of admitting a return type (distinct from the yield type) if we decide that's wanted in the future. As I understand it, this debate has (amongst other things) paralyzed Rust's coroutine work for a number of years.
On the other hand, it marks a split from yielding accessors, where I feel like this is intended as a "natural generalization" of those.
I did call the idea crazy I don't think it's something that this proposal has to address immediately, but I'd love to see an audit of, say, the standard library and Foundation to see how close we could get with good heuristics.
Regarding called-exactly-once, I would argue that we didn't let that stop us for bridged Obj-C completion handlers; if a closure is called more than once in a yielding overload, we could just trap at runtime, I think? (And then if @called(exactlyOnce) or something like it ever happens, we can eliminate that trap door. Or we just wait a bit and see if @called(exactlyOnce) gets proposed/added, since its sibling is already in pitch and there's been some discussion about it.)
For potentially tricky error-throwing paths, I would wager that the function either being rethrows or generic with the same typed throws(E) clause would be enough to guarantee that errors could only be thrown before/during the yield (unlikely that an implementation would generically get an arbitrary E to throw after the fact). And then maybe a runtime trap for the misbehaving ones again.
But all that would depend on whether the audit shows it makes sense. Maybe I'll tilt at this sometime (depending on how this pitch evolves); if we had evidence that the heuristics could be good enough that the default could be opt-out (annotate the functions that would match the heuristic but shouldn't) instead of opt-in, then that would be a pretty powerful feature.
If we can get this feature, Swift Testing can implement safe attachment buffers to replace the current unsafe interface Attachable.withUnsafeBytes(for:_:):
I like the idea but not the direction. I think this doesn’t really need to be such a distinct language feature. Normal types should be able to achieve the same thing.
Rough idea:
struct MyYielder: ~Copyable, ~Deinitilizable {
let value: Ref<Something>
let extraContext: Whatever
consuming func mustCallThisWhenDone() async throws {}
}
The function call can return a type like that. There could also be a “generic” default type that simply takes a closure to perform when done, so not everyone has to introduce their own type.
We might need to have that lang feature which has been discussed a few times, which would require users to explicitly consume the value by a function call or something when they’re done.
I think this is a logical extension of the current accessors and I have run into the limitation of only having yielding accessors but not methods.
[quote="allevato, post:3, topic:90319"]
I haven't thought this completely through yet, but there are a ton of closure-taking with* functions where functionally the closure invocation could be replaced by yielding the value and callers could reduce the pyramid of doom on their end. I wonder if you've given any thought to which standard library APIs could be improved with yielding versions
[/quote]
I had a similar thought and as I understand this proposal I think there are a few that could adopt this such as the existing methods that vend Output[Raw]Span into closures to instead use these single-yield functions.
Many with* functions from the Concurrency module can't adopt this since they are either doing throwing or asynchronous (or both) work after the user's closure ran such as the withTaskGroup APIs where we have to wait for all the child tasks to finish. At least for the Swift Server ecosystem the same applies where many of the with* functions do asynchronous clean-up.
A future proposal could allow single-yield functions to be declared async.
Such a function could suspend with await before it yields, for example to
wait for a resource to become available. The caller would mark the call with await, as with any other async call:
This would be great to model asynchronous I/O APIs such as reading from a file where the buffer is owned by a buffer pool and must be returned once the user is done with it.
Using these for concurrency with... APIs is tempting, but the trouble there will be not the asnc part which at least is acknowledged as future direction, but the throwing which is quite a bigger design task
(1) We'd need "the caller has thrown" which the single-yield function would need to get:
{
var group = TaskGroup.make(...)
throw boom()
}
func make() -> yield TaskGroup<...> {
do {
yield &taskGroup
} catch { // "the caller has thrown"
taskGroup.cancelAll(...)
}
await taskGroup.awaitAllReaminingTasks()
taskGroup._destroy()
...
}
so far so good.
Today's groups also rethrow the first child tasks failure; which would be quite weird
var group = TaskGroup.make(...)
group.add { throw Boom() }
return
// group make continues -- awaits -> needs to throw
but perhaps we'd make a throwing group ~Escaping, ~Deinitable, ~Copyable and force developers to call the:
try await group.waitForAll() // consumes group
That could work I guess? Either way, we'd likely need to figure out the throwing somehow that the group or similar APIs know they're on failure path and may need to do something about that...
Fully aware this isn't part of the pitch today, but wanted to give a short summary what it would take to make groups be able to use this, in case we'd like to discuss this future direction some more. Previously I was planning to propose a more "with resource = ..." style concept but this is actually quite similar, if only we solved the error path as well
I don't think it's something that this proposal has to address immediately, but I'd love to see an audit of, say, the standard library and Foundation to see how close we could get with good heuristics.
