SE-0393: Value and Type Parameter Packs

Discussion has quiesced a little, but I had some more thoughts around the proposed each syntax, having to do with names.

That said, I’m going to fold it away because (1) it’s more bikeshedding, and (2) it’s an argument about what’s ‘natural-sounding’ to a native English speaker, and while that can encourage correct usage it should never be our most important goal. After all, we have many non-native-English-speaker users of Swift.

The repeat syntax is pretty obviously important: it indicates the type or expression being repeated. You could spell it differently, but some form of it is needed. each, however, has felt redundant in many of the examples; if it were omitted, the compiler would easily detect that something went wrong, and even know exactly how to correct it if repeat were already present.

The cycle seems to be 'we need these symbols to clarify what types of things we're referring to!' followed by 'wait, it turns out words already do that.' [xkcd: Sigil Cycle]

A lot of these examples have used each T as the name of the type parameter pack, and yeah, sometimes there’s no better name. But usually Swift tries to go for something better than that, and we therefore have a choice of whether a name should be singular or plural. (This was brought up in the pitch phase as well, when the working syntax still used ellipses like C++.)

There are three relevant names: the declared type parameter pack; the “nouny” name of a value parameter pack, property pack, or local binding pack; and the “verb-or-prepositiony” name of a “fluent”, “reads-like-a-sentence” function name or argument label. The last one, fortunately, is trivial in English: both verbs and prepositions do not change based on whether the object is plural (“consume an apple” vs “consume apples”; “into the box” vs “into the boxes”). The first one reads very well with each if it’s written using a singular noun: each Subview: View (“each subview is a View”), (repeat Optional<each Input>) -> Bool (“a function where you repeat Optional<_> for each input and return a boolean”).

But the second one is a problem, because parameter names (and nouny argument labels) really want to be plural:

/// - subviews: One or more views to include within this view
findFirstMatching(candidates: apple, orange, goat) { preventsScurvy($0) }

Both of those would read worse to a native English speaker in singular form. We could allow writing parameters as each subview in documentation, but nouny argument labels still show up enough that I don’t think we should disregard this.

This is too bad, because I like each. It has pretty good connotations for me within repeat and in the repeat-less syntax proposed for constraints (though now that it’s been pointed out that does seem odd to me too). But I think these things want to be plural for clarity in documentation and at the call site, and that means each no longer fits for me.

(Unfortunately, I cannot think of what would; remember, the important use is not when you declare the type pack, but within repeat, that goes from a plural to a singular. Perhaps this is why Python forces you to name the element in a list comprehension, instead of making the expression based on the name of an existing iterator.)

// Terrible exploration showing where 'each' is used
func process<pack Subviews>(subviews: repeat Optional<unpack Subviews>)
where repeat unpack Subviews: View {
  let labels: repeat String = repeat (unpack subviews)?.label ?? ""
  // …
}
1 Like

Thanks for catching the issues in the proposal text, I clarified those in [SE-0393] Minor proposal clarifications. by hborla · Pull Request #1994 · apple/swift-evolution · GitHub.

I actually considered early on whether we should write pack element requirements inside of a "requirement pack expansion", but decided against it because it seemed unnecessary. I hadn't thought of the local generic function inside a closure pattern case! I think it makes sense to require applying repeat to a generic requirement when the requirement conceptually represents N distinct requirements, one for each element of the substituted pack.

I'm concerned about this suggestion because of the impact it would have on pack expansion expressions. If we still require precise application of each in expressions, there would be a difference between the way you reference packs at the type-level and at the value-level. Relaxing the parenthesis requirements in expressions means that it's no longer syntactically clear which value in a member access chain is a value pack, so it has to be inferred (by both the compiler and the programmer). If you have repeat f(each x.y().z), it's not obvious which part of the subexpression produces a value pack, which is critical for understanding which parts of the subexpression are evaluated once before the loop vs at each iteration over the pack elements.

This is a similar problem to associated type packs, and it manifests in expressions as soon as we have stored property packs or tuple/array expansion. It's a problem that we need to confront now, because we already have an experimental implementation of expanding tuple values that contain pack expansions and the pitch is imminent!

After reading through the reviews at the beginning of this thread, I agree that the fixed-arity overloads should be considered more specialized than an overload with a parameter pack. @xwu pointed out that this will make it easier for existing APIs to add a new overload using parameter packs while allowing existing callers to still resolve to the old APIs, which is a compelling practical justification.

I think pack iteration is relatively straightforward to implement, but I'm fretting about the spelling of pack iteration in light of the repeat spelling for pack expansions. That's solvable, though. Do you feel strongly about including pack iteration in this initial proposal?

3 Likes

FWIW I wrote a note on naming convention in the parameter pack vision document/extended future directions writeup from a few months back, where I recommend explicitly using either no argument label or a plural argument label when writing an API that uses parameter packs. I do think it's unfortunate that the default will be the singular name in the absence of a separate argument label, though.

An interesting aside on this document

I never updated this vision document for the repeat/each spelling proposed here because most of the content was incorporated into this proposal, but you can kinda see how we arrived on repeat/each from the explanations in the document :slightly_smiling_face:

In the section on patterned pack expansion

The ellipsis is called the expansion operator , and the type or expression that the expansion operator is applied to is called the repetition pattern . The repetition pattern must contain pack references. Given a concrete pack substitution, the pattern is repeated for each element in the substituted pack.

Consider a type parameter pack T and a pack expansion Mapped<T>... . Substituting the concrete pack {Int, String, Bool} will expand the pattern Mapped<T> by repeating it for each element in the concrete pack, replacing the pack reference T with the concrete type at each position. This produces a new comma-separated list of types Mapped<Int>, Mapped<String>, Mapped<Bool> .

In the note on naming convention

In the return type, the repetition pattern Optional<Element>... means there is an optional type for each individual element in the parameter pack. When this method is called on List<Int, String, Bool>, the pattern is repeated once for each individual type in the list, replacing the reference to Element with that individual type, resulting in Optional<Int>, Optional<String>, Optional<Bool>.

2 Likes

Documentation comments are my main remaining concern; maybe we can indeed allow (require??) each in the parameter list there?

This is pure bikeshed I’m afraid, but have you considered distinguishing the pack declaration from the pack reference in spelling?

func variadic<repeating S>(_: repeat each S) where each S: Sequence { ... }

func < <repeating Element: Comparable>(lhs: (repeat each Element), rhs: (repeat each Element)) -> Bool

I like this idea. I started a discussion thread over in the Swift Documentation category to get some input from the folks working on documentation tools: SE-0393: Doc comments for parameter packs

1 Like

Putting my reviewer hat back on—

For those of you who haven't yet, the nightly development snapshots now include a substantial implementation of this feature under the flag above. Give it a whirl and share what you think after some hands-on experimentation!

6 Likes

@hborla, @John_McCall , @Slava_Pestov

It is so awesome to get to play with this!

It is so much easier for me to find heads and tails in the proposal when actually having to solve the type puzzle of a function signature!

I find the keywords quite intuitive.

I've 'implemented' sorting Arrays on variadic number of keypaths and a standard zip functionality with any arity.

It is very easy to imagine the iteration over value packs and also being able to define local packs.

Thank you so much for all of the work so far. I can't wait for this amazing feature to become part of the language!

7 Likes

Can you maybe share some of the code you wrote? Most of the functions I've tried to write so far have resulted in a compiler crash for me with the latest nightly snapshot.

1 Like

Certainly! So far only the function signatures - and calls to the functions compile.

I'm running Xcode 14.3 RC2 with the development snapshot 2023-03-27.

I got crashes too, but those were all related to actually calling functions returning tuples - which I guess is supported by this proposal already, but perhaps not completely working in this build. Removing the calls to these functions fixed the crashes.

So, here are my implementations

extension Array {
    func mySorted<each Component>(_ keyPaths: repeat KeyPath<Self.Element, each Component>) -> [Self.Element] where each Component: Comparable {
        // Just return something for now
        return self
    }
}

struct User {
    var name: String
    var age: Int
    var points: Int
}

func myTest() {
    let users: [User] = [.init(name: "Morten", age: 45, points: 10000)]
    let sorted = users.mySorted(\.name, \.age, \.points)
}

This compiles! For the actual implementation, I imagine the following to work once the packs can be iterated over:

extension Array {
    func mySorted<each Component>(_ keyPaths: repeat KeyPath<Self.Element, each Component>) -> [Self.Element] where each Component: Comparable {

        return self.sorted(by: { a, b in
            for keyPath in repeat each keyPaths {
                if a[keyPath: keyPath] < b[keyPath: keyPath] {
                    return true
                } else if a[keyPath: keyPath] > b[keyPath: keyPath] {
                    return false
                }
            }
            return false
        })
    }
}
  • or something similar.

With regards to the zip, the declaration that compiles is:

func myZip<each T>(_ arrays: repeat [each T]) -> [(repeat each T)] {
    fatalError()
}
func test() {
    let a = myZip([1], ["sko"], [2.0])
}

This actually type checks and gives [(Int, String, Double)] as the type of a.

I'm not completely certain about the implementation here, but based on an example with counts of arrays in the proposal text, I assume that you can do:

func myZip<each T>(_ arrays: repeat [each T]) -> [(repeat each T)] {
    var result: [(repeat each T)] = .init()
    var length: Int = .max
    for array in repeat each arrays {
        length = min(length, array.count)
    }
    for i in 0..<length {
        let tuple = (repeat (each arrays)[i])
        result.append(tuple)
    }
    return result
}

So - not so certain about the actual implementations since those are not part of this proposal, but just seeing the pieces of the puzzle fit together is pretty awesome!

Ah okay, so you don't actually have the function implementations yet. The implementations that you gave would indeed not work with the current proposal, because pack iteration is only a future direction for now.
I actually came up with an implementation for sorted which should work in theory but actually crashes the compiler right now:

enum Ordering {
    case lessThan
    case equal
    case greaterThan
}

extension Collection {
    func sorted<each T: Comparable>(by keyPath: repeat KeyPath<Element, each T>) -> [Element] {
        self.sorted {
            var ordering = Ordering.equal
            repeat ({
                guard ordering == .equal else { return }
                let keyPath = each keyPath
                let lhs = $0[keyPath: keyPath]
                let rhs = $1[keyPath: keyPath]
                if lhs < rhs {
                    ordering = .lessThan
                } else if lhs > rhs {
                    ordering = .greaterThan
                }
            }($0, $1))

            return ordering == .lessThan
        }
    }
}
1 Like

Unfortunately I was able to think of one case this proposed syntax doesn't cover, and that's with nested generics. For example,

struct Outer<each T> {
  struct Inner<each U> {}  // how do I spell shape(T) == shape(U)?
}
2 Likes

This is an excellent point!
Your point has somewhat expanded my concern that it may be difficult to extend the proposed syntax directly to Variadic generic types.
Isn't it problematic to discuss variadic generics in function declarations separately from variadic generics in type declarations?

We support nested variadic generic functions too, so this concern with the proposed shape requirement syntax is not specific to types per se.

I plan on sending out a proposal for variadic generic types very soon -- it will be quite short compared to variadic generic functions because most of the concepts generalize immediately to different kinds of type declarations. Type aliases with type parameter packs are completely described by substitution model in the functions pitch already. Structs/enums/classes just have a couple of additional behaviors that will be spelled out.

2 Likes

Since each is only a contextual keyword, each (T, U) would also still parse as a function call, wouldn't it?

  • What is your evaluation of the proposal?

Great proposal! I don’t have much experience with variadic types but I do think repeat and case introduce clarity and expressiveness. It’s clear what packs are captured (which I understand as “iterated over”).

However, I think functions with variadic types should be ranked higher than functions with variadic (non-pack) parameters in overload resolution. We should leave a door open towards deprecating variadic parameters at some point. Such functions can be expressed using functions with variadic parameter packs without having to resort to a (probably heterogeneous) array, and therefore have better performance. For resiliency reasons (ABI?), a library vendor (and even the stdlib with its print function) might want to keep an overload with variadic parameters next to their new function using parameter packs.

  • Is the problem being addressed significant enough to warrant a change to Swift?

I’ve been waiting for this for years! This will greatly simplify writing code that work with tuples of variable size, such as buildBlock in result builder types or functions like zip.

  • Does this proposal fit well with the feel and direction of Swift?

The syntax is not trivial but new Swift coders should be able to find documentation and articles using the keywords repeat and each. Moreover, they don’t need to define such functions and types. This feature remains mostly out of sight at the call site or at the place of spelling out a type. For me, it passes the progressive disclosure test.

It also fits with Swift’s evolution of a stronger, more informed, and more helpful type system and away from relying on existential types where it’s not really needed.

  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

It’s clearer than C++’s spelling for the above-mentioned reasons.

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

A quick reading but I’ve been following this topic on the side.

Blockquote
It seems odd to use each inside the generic angle brackets, where it's a reference not to 'each' scalar type, but to the pack itself. On the other hand, each makes perfect sense as syntax for the pack expansion, where the pack will be expanded to a list of each scalar type from the pack. And since that meaning is marked by each, I'm not sure that repeat is necessary at all.

I have a similar reaction. 'each' conveys expansion, or iteration, not declaration. The only point where I differ with @cmonsour is that, instead of pack as the alternative, I like many, which was one of the suggestions (not mine) in the initial pitch discussion. I see that choosing many as the syntax would have the downside of being an additional keyword, but, many has a sense of being a keyword in English too, whereas pack does not.

Edit: after another look at the alternatives considered section of the pitch, it looks like the ambiguities with the ... operator are all at the expansion site, not the declaration site. So another possibility might be: using the ... operator at the declaration (i.e., in the angle brackets) and the each keyword at the expansion. Maybe this is preferable if adding another keyword is unacceptable.

3 Likes

Fantastic proposal, thank you for pushing variadic generics forward!

Admittedly, I wasn't pleased with repeat each at first given its length, but after spending some time with the last snapshots, I have to say it started growing on me. In a sense, if repeat Int where allowed, then an hypothetical repeat 10 Int could be seen as a "natural" extension to spell out [Pitch] Improved Compiler Support for Large Homogenous Tuples - #3 by Michael_Gottesman.


For clarification purposes, code that was working with the 2023-03-30 snapshot, isn't working with the 2023-04-01 one:

typealias Map<Base, Result> = Result

protocol P { var foo: String { get } }
extension Int: P { var foo: String { "integer!" } }
extension Double: P { var foo: String { "floating point number!" } }
extension [Int]: P { var foo: String { "list of integers!" } }

func foos<each S: P>(of p: repeat each S) -> (repeat Map<each S, String>) {
  return (repeat (each p).foo)
}

let r = foos(of: [1], 2.0, 3)
print(r)

The Map typealias is there only to provide a same-shape requirement between (repeat each S) and (repeat String). Is this the intended (or an intended) way to procede? Is it expected to work?

1 Like

The syntax is not trivial but new Swift coders should be able to find documentation and articles using the keywords repeat and each .

an important point and indeed amongst the motivating reasons for the choice of keywords over operators

2 Likes

Yes, I expect your example to work under this proposal. We anticipated type aliases with parameter packs to be used as a way to explicitly name the shape of a concrete pattern type, exactly like you wrote. Thanks for trying out the implementation and reporting this bug!

1 Like