[Pitch #2] Value and Type Parameter Packs

Slava_Pestov · 2022-10-13T03:55:12.297Z

Earlier thread: [Pitch] Parameter Packs
Related threads: Variadic generics and tuple shuffle conversions, [Pitch] One-Element Tuples

Here is an updated version of the "variadic generics" pitch that @hborla and I have been working on: swift-evolution/NNNN-parameter-packs.md at aec6725e1a67b6217e2c8c607294f8c0a5fbe8b6 · hborla/swift-evolution · GitHub

Please excuse the Github link; the pitch is now too long for Discourse, which rejects any post longer than 32,000 characters.

Major changes from last time:

We decided to subset out variadic generic types (structs, enums, classes and typealiases) for now. While this is certainly a very useful feature, we want to really nail down the semantics of variadic generic functions first.
Perhaps the biggest change is we decided to change the rules around tuples containing pack expansion types. We no longer require a pack expansion type to be followed by a labeled element, so you can now write (T..., U...) as long as T and U can be inferred by other means (since matching (T..., U...) against (Int, String, Float) or whatever is ambiguous).
Furthermore, our current thinking is that a tuple element with a pack expansion type actually cannot have an argument label. Imagine if you could write (t: T...) and you substitute T := {Int, String}. Now you get (t: Int, String), and accessing .t on the resulting tuple would just give you the first Int element. That's kind of weird, so we decided to disallow that.
We've removed the explicit spelling of same-length/count/shape requirements, but the concept still exists. They're just inferred implicitly from pack expansion types. A syntax for writing them might return later, but we feel that at least for variadic generic functions, the inference behavior should cover most reasonable use-cases.
Other than that, the updated pitch mostly nails down edge cases, makes everything much more rigorous, and adds more examples.
The surface syntax (..., etc) is unchanged from last time, and the pitch doesn't discuss any of the alternatives in more detail. This is not meant to foreclose the possibility of changing the syntax, it's just a consequence of our current focus on nailing down the formal semantics.

There are still a ton of open questions, marked as such in the pitch. As we get further along in the implementation we'll be able to actually write real code that uses variadic generics, and hopefully answer some of the open questions. And of course all of your feedback is welcome.

I also want to thank @John_McCall for reviewing a draft version of the updated pitch internally and giving some really helpful feedback.

MPLewis · 2022-10-13T05:00:10.226Z

There's a minor typo under "Generic requirements":

A same-type requirement where one side is a type parameter pack and the other type is a concrete type that does not capture any type parameter packs is interpreted as constraining each element of the replacement type pack to the same concrete type:
func variadic<S...: Sequence, T>(_: S...) where S.Element == Array<T> {}
This is called a concrete same-element requirement.

A valid substitution for the above might replace S with {Array<Int>, Set<Int>}, and T with Int.

The example function signature should probably not say the Element is an Array:

func variadic<S...: Sequence, T>(_: S...) where S.Element == T {}

You also have some numbering weirdness in the same section (repeated 3s for numbering requirement examples).

AliSoftware · 2022-10-13T08:12:46.223Z

Another small typo I found in example code, in "Label matching" section:

while the following is not:

func bad<T..., U...>(t: T..., U...) -> (T..., U...)
// error: 'T...' followed by an unlabeled parameter

bad(1, 2.0, "hi", [3])  // ambiguous; where does T... end and U... start?

I think there's a typo in both the method declaration signature (syntax for unlabeled parameter) and the example call site (missing the t: label):

func bad<T..., U...>(t: T..., _ u: U...) -> (T..., U...)
// error: 'T...' followed by an unlabeled parameter

bad(t: 1, 2.0, "hi", [3])  // ambiguous; where does T... end and U... start?

Similarly, in the next "Type sequence matching", the unlabeled parameter correctly have _ for the external parameter name but is missing an internal parameter name to be able to use it in the implementation:

func variadic<T...>(_: T...) -> (Int, T..., String) {}

Should be:

func variadic<T...>(_ t: T...) -> (Int, T..., String) {}

That last one appears similarly in other places later in the proposal in other examples too

DevAndArtist · 2022-10-13T09:18:50.876Z

I think this update is going into the right direction. I strongly feel that ruling out the basic things first is a better first step towards completing the whole big picture here and removing the variadic generic types from the initial proposal was the right decision, especially because we should also consider orthogonal expansion as I previously mentioned in the other thread with the example of enum cases.

While the first topic by Slava clearly states that the ... naming scheme is unchanged I think we should seriously consider alternative naming schemes towards the end of evolution of this proposal. I just quickly scanned the proposal and the given example gives me some headaches when my eyes and brain has to parse so many dots.

func variadic<T..., U...>(t: T..., u: U...) -> ((T) -> (U...)...)

For the type itself I still think the pack keyword would fit very well and be more human readable then .... As for the expansion side, maybe we could consider unpack as a fitting keyword. That's not urgent and I don't want to derail the main conversation with this topic, but I would very much appreciate if we could discuss this topic towards the end of the process.

xAlien95 · 2022-10-13T12:14:42.366Z

Thank you! As a quick note, the TOC contains an "One-element tuples" section (under "Detailed design" > "Type matching") that is absent in the body. Is it meant to just be a redirect to the appropriate pitch or there are changes in that area?

xwu · 2022-10-13T15:21:12.352Z

Slava_Pestov:

Furthermore, our current thinking is that a tuple element with a pack expansion type actually cannot have an argument label. Imagine if you could write (t: T...) and you substitute T := {Int, String}. Now you get (t: Int, String), and accessing .t on the resulting tuple would just give you the first Int element. That's kind of weird, so we decided to disallow that.

Admittedly, it is kind of weird at first blush.

However, on the other hand, it seems unfortunate that I won't have to run into ambiguities inferring T and U for a function f<T..., U...>(t: T..., u: U...), while I'd run into trouble with a tuple because (t: T..., u: U...) would be banned.

Moreover, if accessing t inside f to give the first Int argument isn't confusing, accessing .t on the tuple to give me the first Int element seems analogous?

Jumhyn · 2022-10-13T15:24:09.087Z

xwu:

Moreover, if accessing t inside f to give the first Int argument isn't confusing

t inside the function here here would refer to the entire pack, no?

A value parameter pack represents zero or more function arguments, and it is declared with a function parameter that has a pack expansion type. In the following declaration, the function parameter values is a value parameter pack that receives a value pack consisting of zero or more argument values from the call site:
func tuplify<T...>(_ values: T...) -> (T...)

_ = tuplify() // T := {}, values := {}
_ = tuplify(1) // T := {Int}, values := {1}
_ = tuplify(1, "hello", [Foo()]) // T := {Int, String, [Foo]}, values := {1, "hello", [Foo()]}

xwu · 2022-10-13T15:28:57.658Z

Jumhyn:

t inside the function here here would refer to the entire pack, no?

You are right...oops.
Is there a reason this wouldn't be possible for .t in a tuple?

Jumhyn · 2022-10-13T15:45:07.491Z

Slava_Pestov:

We've removed the explicit spelling of same-length/count/shape requirements, but the concept still exists. They're just inferred implicitly from pack expansion types. A syntax for writing them might return later, but we feel that at least for variadic generic functions, the inference behavior should cover most reasonable use-cases.

Haven't done a full pass through the new proposal yet but had a thought on this. I am slightly concerned about introducing this inference behavior in the first iteration of the feature—introducing inference has come back to bite us before (e.g. with associated types) so I would be wary about introducing it unless we really feel like the ergonomics require it for variadic generics 1.0. Of course, if we don't infer the same length requirements then we will need a way to spell it explicitly, so we would need to come up with something we're happy with in that regard.

Maybe this inference behavior is closer to inferring transitive protocol constraints than it is inferring associated types, but would love to hear the authors' thoughts.

Slava_Pestov · 2022-10-14T01:37:24.028Z

Jumhyn:

t inside the function here here would refer to the entire pack, no?

xwu:

You are right...oops.
Is there a reason this wouldn't be possible for .t in a tuple?

Sorry for the confusion, I meant accessing .t on the substituted type, so eg the result of a call:

func makeTuple<T...>(_ t: T...) -> (t: T...)

let result = makeTuple(1, 2, 3) // result: (t: Int, Int, Int) - only first element has label
let resultT = result.t // resultT: Int
print(resultT) // prints 1

This is the "weird" behavior we're thinking of.

Jumhyn:

Haven't done a full pass through the new proposal yet but had a thought on this. I am slightly concerned about introducing this inference behavior in the first iteration of the feature—introducing inference has come back to bite us before (e.g. with associated types) so I would be wary about introducing it unless we really feel like the ergonomics require it for variadic generics 1.0. Of course, if we don't infer the same length requirements then we will need a way to spell it explicitly, so we would need to come up with something we're happy with in that regard.

Maybe this inference behavior is closer to inferring transitive protocol constraints than it is inferring associated types, but would love to hear the authors' thoughts.

I wouldn't worry too much about it. The inference behavior is already implemented and uses the same "requirement inference" mechanism as this example which has worked since Swift 1.0:

// infers `T : Hashable` from Set<T>
func foo<T>(s: Set<T>) {}

Associated type inference is problematic because it's a global analysis, whereas this just looks at what pack expansion types appear in the signature of a single function.

juliand665 · 2022-10-18T17:12:52.146Z

It's very exciting to see this progress! I can't wait to use variadic generics in my code :)

In the context of parsing ambiguity, have you given any thought to the fact that trailing closures eschew argument labels? As such, calling e.g. a func variadic<T..., U...>(t: T..., u: U...) could be ambiguous:

variadic { 0 } could leave T..., U... being { () -> Int }, {} or {}, { () -> Int }
variadic(t: { 0 }) { 1 }, similarly, might be { () -> Int, () -> Int }, {} or { () -> Int }, { () -> Int }

Intuitively, it feels like trailing closures should not be part of the same parameter pack as any of the arguments within parentheses, though I'm not sure how exactly that'd work out in practice.