[Pitch] Parameter Packs

Yeah, I think Holly’s post does a good job of showing both sides of the coin and has left me feeling more conflicted than I was initially. Unfortunately I can’t think of a good way to handle the fact that plural/singular is clearly the more natural choice depending on the context in which you use the identifier, other than allowing users to introduce arbitrary synonyms for pack values/types, e.g.:

func f<Element(Elements)...>(element(elements): Element...) where Element: Comparable {
  for element in elements... { ... }
}

I wonder if what’s going wrong here isn’t that sequences is misnamed, but that the . here looks like just a normal member access when it’s really a sort of projection operation over the pack.

I could imagine us requiring a different syntax here, so that rather than sequences.count you’d write sequences->count or something.

Of course, doing something specific for member accesses does nothing to address the fact that sequences is unusual as an expression value, not just as the base of a member access. I think this is a discomfort I have with the pattern expansion syntax as-proposed, beyond the use of . Even if you know you’re dealing with a pack expansion plan ellipsis, it won’t necessarily be obvious what is getting expanded because the pack looks like a normal identifier.

Perhaps that suggests something like the following would be good:

{sequences}.count...

This would make it clear that sequences is a pack we’re expanding rather than a normal expression value, and it would also make the pack easier to find inside a complex expansion pattern.

ETA: Since my phone insists on correcting ... to a unicode ellipsis character, here's a proposal: ... retains its current meaning an we use U+2026 for variadic generics.

2 Likes

This is an interesting concept for sure and judging by the many comments already a welcomed one, but I have a very simple question about this:

Very well possible I am missing something, but supposed we have such a function declaration along the first point:

func withTypePacks(label1 foo: T..., label2 bar: U...)

How can I define a property or local variable for it? Just via type inference? That would be a hard limitations I don't have for any other function. I mean, I can basically have a variable for a closure and usually always assign it a function instead, correct?

func justOnePack(easy foo: T..., peasy bar: U) // for this I could do:

var localVariable: (T..., U) -> Void
localVariable = justOnePack(easy:peasy:)
localVariable(value1TypedT, value2TypedT, valueTypedU) // calling the closure

func withTypePacks(label1 foo: T..., label2 bar: U...) // legal function declaration according to bullet 1

var anotherVar: (T..., U...) -> Void // this is illegal according to the last bullet point, right?
anotherVar = withTypePacks(label1:label2:)
anotherVar(value1TypedT, value2TypedT, value1TypedU, value2TypedU) // but this should work, right?

I am sure I am just completely missing something here, but I am, apparently, blind. It would be a pretty big thing if all of a sudden there's such a distinction between closures and "proper" functions...

Edit: Just realizing it, I believe: Is the issue with the same type, i.e. if T and U would be the same? So just the following would be illegal:

func withTwoSameTypePacks(label1 foo: T..., label2: bar: T...)
// callable as:
withTwoSameTypePacks(label1: value1TypedT, value2TypedT, label2: value3TypedT, value4TypedT)
// in the functions body foo would be the first two values, and bar the last two

var asVariable: (T..., T...) -> Void
asVariable = withTwoSameTypePacks(label1:label2:)
asVariable(value1TypedT, value2TypedT, value3TypedT, value4TypedT)
// how can the function figure out which values are in foo, which in bar?

I assume that's it, but I am inclined to then rather forbid defining functions with more than one identical type packs in general instead of basically disallowing them to be stored in variables until we maybe get labeled closure calls?

There’s been some discussion of this issue up-thread (though no great solution offered).

1 Like

Oh, thanks, and I apologize. I actually overlooked that... :sweat_smile:
I mean, I just got it, but as my above example illustrates this applies to the same type packs only, correct? (Obviously that can quickly become way more messy if generics are involved and ultimately the resolved concrete types turn out to be the same...)

It is also the case for all generic functions, whether it involves variadic generics or plain generics:

func someFunction<C>(_: C) where C: Collection { }

type(of: someFunction)
// error: generic parameter 'C' could not be inferred

let x = someFunction
// error: generic parameter 'C' could not be inferred

let y = someFunction<Array<Int>>
// error: cannot explicitly specialize a generic function

Generic closures are high on the list of things I'd like us to add to Swift.

2 Likes

There's an innate conflict here between the declaration of a pack, where the name stands for a sequence and it feels right to pluralize it (e.g. Widgets), and the uses of a pack, where the name is always part of an expansion and stands for one component and so feels like it ought to be singular (e.g. Widget). This is a place where experience with a similar feature (e.g. from C++) could potentially be very helpful; unfortunately, as someone with that experience, I can tell you that I've never found an appealing resolution to that conflict, so my experience is in fact not helpful at all.

Now, this conflict would be resolved if we allowed the use site to bind a different name to the pack element. I can think of a couple different syntaxes for that:

  • for A in Arguments { A? }
  • each A in Arguments { A? }
  • Arguments.map { A in A? }

I don't love any of them, though. And in context, I worry that people won't understand that this is an expansion, so maybe they also need a trailing ..., which just makes the syntax worse. They're also a little awkward when it comes to expanding multiple packs in parallel — the first two, I can imagine something with parens, but for .map I really don't know.

Maybe the biggest problem is that this feels really heavyweight when you're just trying to expand the pack directly, which is very common. That's especially true in variadic generic functions, where we probably declare the type parameter pack T and then immediately turn around and declare a value parameter pack of type T..., or a single parameter of type Stream<T...>, or whatever. It's interesting to note that the naming issue disappears here, too: there's nothing misleading or confusing about just writing Arguments.... So if we can settle on a binding/mapping expansion syntax, maybe we can hybridize that syntax with ..., so that ... when the operand is directly a pack reference means a direct pack expansion without any mapping, and otherwise you have to use the binding/mapping syntax.

(This would also eliminate the formal ambiguities with ..., since you could only use it directly on pack references, which would otherwise be ill-formed. If you wanted to write a pack expansion of half-open ranges, you could just write each lowerBound in args { lowerBound... }.... It would still be visually ambiguous, of course.)

This would also give us a syntax for the dreaded "type of pack expansion no longer refers to a pack" problem, e.g. the type of collections.count..., where every component is concretely an Int but the expression is still a pack expansion.

3 Likes

Sadly, to me this looks like yet another case where it is considered more important to integrate some new shiny feature as soon as possible instead of striving for the best long-term solution.
Copying what C++ did might be cheap, but I don't think it's a good fit — and imo deciding about some smaller pieces like labels for generic parameters and value generics should happen before landing a huge change which could benefit from such preparation.

1 Like

I have only used variadic generics in other languages for educational purposes, but when writing out the examples in Swift, I found my self almost always using the singular name for type parameter packs, and the plural name for value parameter packs. I think I gravitated toward using plural names for value packs because 1) I didn't think to write a separate argument label and wanted to use the plural name at the call-site, and 2) with iteration, you typically use the singular/plural names together, so it's hard to figure out what to call the local variable when you're iterating over a direct pack expansion, e.g. for value in values.... But otherwise, I have a much easier time understanding the code when patterns use the singular name.

4 Likes

There is a relatively extensive alternatives considered section as far as a first pitch goes, and we've been exploring those alternatives for over a year now. It's fine to leave a drive by comment that you don't like the design, but it sounds like you might have thoughts on what would be a better fit for Swift. If you do, that's great and I would appreciate elaboration.

These features are both orthogonal to the fundamental model of abstracting over a variable number of type parameters.

12 Likes

We could always introduce a language model into name lookup so that Widgets... would implicitly introduce the Widget name as well, and Indices... would properly singularize to Index. :slight_smile:

Yeah, I was thinking about something similar in an early variadic generics thread, though only as a tool for writing constraints:

I'm not totally sold either.

Yep, this is my feeling as well, and I wouldn't be opposed the type of regime you're proposing where we have a shorthand syntax for the simplest case and then fall back to a heavier syntax as soon as you want to do anything more complex. I think I'd still prefer something that makes it clear in elements... that we're working with a pack rather than a range bound, but at least it doesn't have the issue of the pack reference being buried deep inside a complex expression.

I feel like we can only really resolve this with one of two things:

  • Explicit pack syntax to make it visually obvious when you're referring to the full pack
  • Explicit destructuring syntax that allows you to introduce a singular name

As you (and John) noted in your previous post(s), there's a fundamental conflict between different usages of a pack that make a single choice for singular or plural look silly in some contexts.

I think the silliness (totally agree with @hborla, by the way, about being drawn to the plural for values and the singular for types) is a symptom of the underlying cause where there are currently different usages sharing the same syntax, and perhaps @John_McCall's diagnosis is correct here:

2 Likes

This feature has been at the top of many wish lists essentially since Swift was released, so I don't think that's a fair characterisation.

4 Likes

I think I disagree that direct expansion vs expansion+map should have a separate syntax/mental model. The mental model that I find intuitive for an expansion of length N is "repeat this pattern N times". That mental model works for both plain pack expansions and an expansion where the pack is wrapped in some other type or expression.

Take this WeatherKit API as an example, written using parameter packs (under this pitch) instead of 7 overloads. The developer documentation uses dataSet1, dataSet2, etc, as the parameter names in the overloads, so in this case dataSet describes a single value of WeatherQuery:

func weather<T...>(
    for location: CLLocation,
    including dataSet: WeatherQuery<T>...) async throws -> (T...)
)

This makes sense to me as "repeat the dataSet: WeatherQuery<T> parameter N times". This, on the other hand, is way harder for me to understand:

func weather<Ts...>(
    for location: CLLocation,
    including dataSets: for T in Ts { WeatherQuery<T> }) async throws -> (Ts...)
)

I don't know how to explain this. It's hard to describe that dataSets is a parameter pack, i.e. separate parameters, repeated. I also don't know that I can get behind putting a for-in syntax or any other mapping syntax into a parameter list :face_with_spiral_eyes:

I think the parameter label issue I brought up can be solved with an API design guideline that for function parameter packs, you should use a separate argument label from the name of the pack. That really only leaves for-in loops as the problem:

for data in dataSet... {} // ???

for-in loops are kinda hard to think about under the repetition pattern model. I think what you really want is just a body of code that is repeated for each value in the expanded pattern. Maybe that suggests we should find a different way to express iteration, or maybe we should just let you call the binding the same name as the pack if you're expanding a pack directly.

6 Likes

We could also take the position that expansion patterns are permitted to consist of multiple statements, so that you could write:

  {
    result = result && array.isEmpty
    doSomething(with: result)
  }...

or, if we feel that we really need some introduction of the expansion maybe you can expand a value pack into a statement block:

array... {
    result = result && array.isEmpty // within this block, `array` refers to a single element of the pack
    doSomething(with: result)
}

or maybe somehow extend do?

do... {
    result = result && array.isEmpty
    doSomething(with: result)
}

of course, all of these preclude using array as a pack within the repeated block, so you couldn't do something like:

for element in elements... {
  print(element, bestMatchNotEqual(to: element, from: elements))
}

I can't really think of a concrete situation where this would be useful, but the for-in at least makes it possible to do, even if naming the pack/element is a little funky.

This motivation is exactly what I consider to be wrong: Features should be added when they fit into the big picture, and not in the by their rank in some wishlist.
Returning to an old comparison ([planning] [discussion] Schedule for return of closure parameter labels (+ world domination ramble) - #7 by Chris_Lattner):
You usually really want a roof when you build a house, but still you finish the walls first.

I don’t necessarily agree with this. The element type of packs could be considered the union of all types making up the pack. Since the types making up the pack are unknown in the generic context, they could be opaque types that expose the constraints placed on the parameter pack’s elements. In pseudo Swift

typealias H = (Bool, Int, String)

H.Element.self // (Bool | Int | String).self

func hello<pack Values>(values: expand Values) {
  (expand Values).Element.self // some Any
}

To be clear, I’m not proposing a union feature, just exploring a conceptual model for the type of pack elements.

1 Like

Picking an opaque type that maximally generalizes a bunch of concrete types is actually a very difficult problem that’s prone to instability (and thus source incompatibility) around API changes.

Doing it for an abstract pack is stabilized by the generic signature the user specified.

4 Likes

The proposal references zip, which already has variants (zip2, zip3, etc.) in frameworks like Combine. Builder APIs, like SwiftUI has already demonstrated the need for variadic generics. Multi-dimensional arrays would also be a nice performance improvement compared to just nesting arrays. Further, math frameworks, especially those used in ML would significantly benefit from variadic genetics (and integer generic parameters but that’s a different discussion). Overall, I think there are already several examples of how parameter packs will be useful.

Yeah, I was mostly referring to the generic case. I think it’s fair to make the hypothetical Element type of a concrete tuple unutterable or, at least, some Any. Making it unutterable would have the benefit of the user explicitly specifying which constraints they want to enforce, by writing something like the following (which is admittedly a mouthful).

(Int, String) as (expand some pack Hashable)

My point was that it can hardly be claimed that this feature has been rushed. But concretely then, why is now not the right time, what is missing that needs to precede this, that this builds on?

1 Like

I don’t think John was pushing for that sort of spelling as-is, nor I. In the original discussions the distinction was made using prefix rather than postfix ...:

func weather<T...>(
    for location: CLLocation,
    including dataSet: WeatherQuery<...T>) async throws -> (...T)
)