I recently landed a series of changes to fix various limitations with closures and parameter packs.
Background
Swift 5.9 introduced the core model for parameter packs, and initially you could only iterate over a pack with a single expression. Doing anything more complicated required introducing a local generic function:
func f<each T>(_ t: repeat each T) {
func g<ElementOfT>(_ e: ElementOfT) {
// complex multi-statement body here
}
repeat g(each t)
}
Now, of course with Swift 6 for ... in repeat loops (Swift.org - Iterate Over Parameter Packs in Swift 6.0) this can be expressed directly:
func f<each T>(_ t: repeat each T) {
for e in repeat each t {
// complex multi-statement body here
}
}
Closures
The other limitation that required the local function workaround concerns the case where the pack expansion includes a closure. So the following didn't work:
func transform<Arg, Result>(_ t: Arg, _ fn: (Arg) -> Result) -> Result {
return fn(t)
}
func f<each T>(_ t: repeat each T) -> (repeat Array<each T>) {
return (repeat transform(each t) { [$0] })
}
And you had to write this:
func transform<Arg, Result>(_ t: Arg, _ fn: (Arg) -> Result) -> Result {
return fn(t)
}
func f<each T>(_ t: repeat each T) -> (repeat Array<each T>) {
func localTransform<E>(_ e: E) -> Array<E> {
return transform(e) { [$0] }
}
return (repeat localTransform(each t))
}
This workaround is no longer necessary. To understand what's going on here, consider what exactly are the types appearing in the expression (repeat transform(each t) { [$0] }). The sub-expression each t refers not to the entire pack, but some element of the pack. So the closure { [$0] } receives the pack element type, and returns an Array of this element type.
The pack element type is different on each iteration, so it depends on the loop index. We model this sort of thing (as well as opened existentials) as a local archetype in the compiler, which is basically a type that depends on a value (in this case, the pack and the index).
The former limitation was that neither the closure's type, not the type of any expressions in the closure's body, could refer to a local archetype that was defined outside the closure.
Closure conversion
This situation where the closure body refers to a local archetype outside the closure's scope is analogous to how closures can freely capture values from the outer scope. Most compilers create this "fiction" that values can be captured with a technique called closure conversion:
- For each closure, we collect its captures; a list of variables referenced within the closure body which are not defined in the closure itself.
- For each capture, the compiler introduces a new hidden argument to the closure's implementation. The lowered function type of the closure includes these additional parameters.
- References to local variables from outer scopes are replaced with references to these hidden arguments inside the closure's implementation.
- When actually forming the closure value, the captured values are collected and partially applied to the closure's function. The remaining arguments are then provided when the partially-applied closure is finally invoked.
After closure conversion, captured values no longer "exist".
The point is that this sounds a lot like the "local generic function" workaround, except at the level of values instead of types. So of course, we now allow closures to capture local archetypes (and values of local archetype type) by doing the following transform:
- For each closure, we collect its captured environments; this describes the stack of local archetypes introduced by outer pack expansions.
- For each local archetype, the compiler introduces a new hidden generic parameter in the closure's implementation. The lowered generic signature of the closure adds these additional generic parameters to the generic signature of the outer function.
- References to local archetypes from outer scopes are replaced with references to these hidden generic parameters inside the closure body.
- When actually forming the closure value, the captured local archetypes are collected and passed in as generic substitutions to the closure's function.
The previous workaround was just this, done by hand.
Appendix: Opened existentials
This transformation can enable another hypothetical feature that has come up in the past. Today, we can open existentials within a single expression, by passing an any P to a some P. To open an existential and do things that span multiple statements, you have to---drumroll---write a local function:
func foo(x: any P) {
func f(_ y: some P) {
let closure = { print(y) }
}
f(x)
}
Now, it would also be nice to open an existential into a local variable:
func foo(x: any P) {
let y: some P = x
let closure = { print(y) }
}
You'd then run into the same issue where the closure captures a local archetype (this time, it depends on the existential value), but thankfully, the new logic I added should already cover this case.
Parameter packs are a major extension of the type system and they intersect with almost every other language feature. In no particular order, these are the main implementation limitations that remain to be addressed:
- Same-element requirements
- Pack mutation, lvalue packs, etc
- Noncopyable packs and tuples
- Tuple conformances