Pitching The Start of Variadic Generics

Evolution Pitches
#17

This will be great to have!

Thinking on ways to address some of the challenges, a thought came up—what if we address the current limitation by allowing arguments (variadic or not!) to have opaque generic types?

func print<Item>(_ items: some Item...)
  where Item: CustomStringConvertible {
  
}

The trick, of course, would be the type of items: [some Item]—an array with an opaque element type, which is currently disallowed. On the plus side, it does (imho) feel like a more direct semantic interpretation of heterogeneous variadics.

I’m aware this is quite different from @codafi’s pitch (and would warrant a separate proposal), but I think many of the described type sequence affordances translate moderately well to arrays of opaque types (with a couple of exceptions I can immediately see).

There are many aspects, implications, and variants to explore, but being quite new to this type of discussion, I wanted to see whether someone can correct me if it’s obviously problematic, or actually worth diving in deeper on and writing up? :blush:

#18

I really like the idea and where this is going, thanks for writing this :+1:

I agree that ... syntax for type sequences might be confusing and ambiguous. I for one got a bit confused as I was reading some of the examples as if they were declaration for variadic arguments instead of type sequences sometimes, and I feel it'd be too easy to confuse them, while it's important to make clear that the different types in a T... type sequence are not necessary the same types each as opposed to variadic arguments.

I also found it a bit confusing to see typealias Element = (Sequences.Element...) in the zip example, as I was expecting the syntax to be more something like (Sequences...).Element or needing some usage of _mapTypeSequence here, as were effectively wanting the element of each archetype here.

Some alternatives I have in mind as an alternative to ... and to avoid ambiguity:

  • postfix * — ie T*. Would also maybe allow things like typealias Element = Sequences*.Element
  • postfix [] — ie T[] and typealias Element = Sequences[].Element

Personal opinion on those:

  • I like * because it's common and would work well in conjunction to introducing a potential splat operator too using * too (a bit like in Ruby).
  • I like [] because when I see func debugPrint<T[]>(_ x: T[]) I think of the type sequence as being "a tuple (T0, T1, T2,… Tn)" or (T[0], T[1], … T[n]) — as if those were mathematical subscripted variables, which acts as a reminder that each Tn can be different (as opposed to T... used as a variadic generic where they'd be all the same).
  • It would also open the door in the future to allow specifying an arity or even a range, eg T[5] for a type sequence / tuple with 5 (possibly different) types, T[1...] for a type sequence of size at least 1, etc
  • But otoh for some subjective reason I feel like (_ x: T[]) is less suggestive of the method allowing a variable number of (potentially heterogeneous) arguments and instead suggests an arity of one there, while I somehow feel like T* would make it more natural to read it as variadic arity (maybe that's my Ruby bias speaking here?)
#19

Could we have some clarification on (or removal of) the word “archetype”? As far as I can tell it’s implementation-detail jargon. Slava was going to explain it in part four of his two-part series back in 2016…

7 Likes
#20

I think this is wonderful, but I wonder if the use case that I'm personally most interested in is possible to express, namely homogeneous tuples of variadic length? It's very useful in math related applications, especially of course with vectors. Something like:

struct Vector<S> {
    let value: S...
}

Also, I imagine other will be interested in the opposite case, heterogeneous tuples of a given length. Yeah, that's just tuples, but with the proposed syntax you don't have to explicitly name each type.

struct Quadruple<S[4]> {
    let value: S[4]
}

I don't mind the syntax, I'm sure it will be fine, but I suppose we should first make existing generics accept this shorthand (which is very useful):

extension [String] { } // = "Array where Element == String"

By the way, I didn't really understand what _forwardTypeSequence does, could you perhaps add another example or a few words of explanation?

1 Like
#21

many T suggest that they are all the same, which is an important use case, but perhaps it would be less clear for the general case.

#22

I also like the "zero or more" regular expression connection that the T* spelling admits (and could we add T+ as a lightweight "one or more" spelling in the future...?).

On the whole, though, I don't think we need to optimize for super terse syntax here, so I'd be fine with the variadic T spelling as well.

Another thing I've been mulling over is whether we need to have a bright-line distinction between "type sequences" and "tuples". That is, if we add all the special operators (_countTypeSequence, _mapTypeSequence, etc.), would it be sufficient to say that a generic parameter T* declares a type T which stands for a tuple of arbitrary arity? Then, we could have a tuple-splat operator for turning a T value into its constituent parts (for e.g. function application). This might also dovetail nicely with @expanded to give us a way of expanding a variadic tuple into a parameter list, e.g.:

func variadic<Tuple*>(args: @expanded Tuple)
// or
func variadic<Tuple*>(args: #splat(Tuple))
// or, maybe 'Tuple' means the tuple version of the 
// type, and 'Tuple*' the splatted version
func variadic<Tuple*>(args: Tuple*)

all of these could indicate that variadic is callable as:

variadic("Hello", 1, 2.0)

This could also imply (though it wouldn't have to) that all the special type sequence operations would be applicable to fixed-size tuples as well:

for elt in ("Hello", 1, 2.0) {
  // elt is 'Any' here?
}

Is there any reason we need to treat type sequences as their own first-class "thing" as opposed to making them "just" tuples of variadic arity, and improving support for operating on arbitrary tuples?

11 Likes
Variadic Generics
#23

Triple-dots already play many distinct roles in Swift’s grammar given that they are the spelling for variadics, a user-defined operator part, partial ranges, etc. It may make sense to adopt a different delimiter.

I've previously suggested using the triple-comma for variadics.

// Variadic parameters (or type sequences).
func debugPrint<Items,,,>(
  _ items: Items,,,
  separator: String = " ",
  terminator: String = "\n"
) where Items: CustomDebugStringConvertible

// Variadic arguments (or tuple-splat operations).
let items: (String, Int, Double) = ("Hi", 42, 3.14)
debugPrint(items,,,)
debugPrint(items,,, separator: "\t")
2 Likes
#24

That's a neat suggestion (and I like it better than ... FWIW), but I think your example shows one potential issue with this syntax:

func debugPrint<Items,,,>(
  _ items: Items,,,        // <- syntax error here, or only ',,' required
  separator: String = " ", // in comma-separated lists?
  terminator: String = "\n"
) where Items: CustomDebugStringConvertible

This is, of course, a purely aesthetic preference, but I'm slightly bothered by both quadruple commas ,,,, and ,,, serving as both a "variadic" indicator and comma separator. I'd still prefer we choose something that doesn't visually with existing parameter list characters/syntax.

1 Like
#25

The ASCII comma isn't available to custom operators, so single , and triple ,,, would both be valid list delimiters. (A quadruple ,,,, wouldn't be valid or necessary.)

1 Like
#26

I'm excited for this work, and agree with many of the points made here about legibility. This is an advanced feature and we should consider less experienced developers encountering it for the first time. "Swift generic type postfix star" seems like an ineffective search term and I'd wager folks can come up with a bunch of equally SEO-unfriendly variations on this for each of the symbol-based approaches suggested here. Using a keyword seems much better in this regard, something like "Swift many expanded keywords" seems more likely to yield good results. Super +1 if we can come up with keywords that can be broadly understood without searching.
Also, because this is an advanced feature, I see little downside to erring on the side of verbosity. This is especially true when you consider the syntax it is replacing is 10 overloads of the same method with N type parameters each.

This may be a bit out-of-scope as this proposal explicitly excludes associated types, but I think the design would benefit from discussing how this would affect a potential implementation of static structural reflection. Something like:

protocol Structural {
  static var storedProperties: many StoredProperty<Self, _> { get }
}

allowing the user to define (using @Jumhyn's notation):

protocol OnlyIntProperties: Structural where forEach(T in storedProperties, T.Value == Int) {
}

It would be great if at least this specific use case could leverage semantics similar to whatever we decide on here (for the record, I'm not suggesting we implement this as part of this proposal, just that the semantics are extensible enough to support it).

3 Likes
#27

Same question. It would be nice to have a list of the pros and cons of having (unlabeled) tuples as the foundational structure for variadic generics in an "alternatives considered" section at the end of the pitch.

For the pros I guess we have:

  • the ability to use postfix ...;
  • the ability to add .count and .map to values of Type Sequence type in the future.

For the cons:

  • a tuple splat function/directive/operator would be needed to pass tuples where variadic type sequences are expected and since postfix ... cannot be used, we would end up with an asymmetric notation in the point of declaration and point of use, as well as when splatting/collecting.
1: func debugPrint<T...>(_ items: T...) 
2: where T: CustomDebugStringConvertible {
3:   for (item: T) in items {
4:     stdout.write(item.debugDescription)
5:   }
6: }

Here, T at line 1 and 2 is different from the T at line 3. The third one is inferred and declared at the same time, right? If it were the same, you wouldn't be able to nest multiple loops over items. Can it be clarified with an ad-hoc syntax? Something like infer T maybe?

func debugPrint<T...>(_ items: T...) 
where T: CustomDebugStringConvertible {
  for item: infer U in items {
    stdout.write(item.debugDescription)
  }
}

Per the proposed design, T... is a Type Sequence Archetype and T and U are particular opened archetypes. T... canonicalizes to a tuple/value/Void depending on its arity during specialization, but we would have no way to convert a tuple to a Type Sequence Archetype, is that correct? Therefore looping over tuples, mapping tuples or checking the arity of tuples wouldn't be allowed.

I would personally be against bare words preceding a generic type, since that seems to be the best location to place generic type parameter labels:

struct Tupleish<label1 Type1, label2 Type2> { … }
typealias MyLabeledTuple = Tupleish<label1: Int, label2: String>

// generic type with label, inline conformance (and default type?)
struct F<label1 Type1: Protocol1 = Int> { … }

func f(label1 parameter1: Int = 0) { … }

The difference between generic type lists and function parameter lists would then be that for functions an omitted parameter label f(type1: Int) is equivalent to f(type1 type1: Int), while for types an omitted parameter label F<Type1: Protocol1> would be equivalent to F<_ Type1: Protocol1>.

1 Like
#28

Thanks for sharing the writeup! Here are a few inconsistencies I noticed:

  1. In the example of measure:
func measure<T...>(_ xs: T...) -> Int { return _countTypeSequence(xs) }

I think it needs to explicit forward the parameter of type T... (just like invocations of _mapTypeSequence)?

func measure<T...>(_ xs: T...) -> Int {
    return _countTypeSequence(_forwardTypeSequence(xs))
    //                        ^ added
}
  1. More on the differentiation between Tuple vs Type Sequence. In the example of Last and Init, it appears that value directly store a type sequence T.... On the other hand, ZipSequence and its Iterator seems to store a tuple (i.e. (Sequences...) and (Sequences.Iterator...)) that needs forwarding. Are there any differences between the 2 spellings?
#29

You seem to have understood what _forwardTypeSequence does, could you explain it to me?

#30

I think T... also suggests this (a list of T). When explaining this feature to a beginner, it's important to point out that T has a different meaning depending on its place on a declaration.

When writing

func foo<T>(x: T) { }

the T on the left (type position) means "a type locally called T", while the T on the right (value position) means "a value of type T". In the value-variadic form

func foo<T>(x: T...) { }

it's exactly the same, but because there's T... it doesn't mean just "a value of type T" but "a list of values of type T".

Putting ... in the type position will mean "a list of types locally called T", which doesn't mean that they are all the same.

This is rather complicated (it took me a while to understand the same concept in Haskell), but I don't think it can be further simplified. You can see the same difference if you used the many keyword in the value position:

func foo<T>(x: many T) { }

This would mean the same as before, "a list of values of type T".

#31

I see what you mean, but I wonder if it wouldn't be clearer with a keyword than just dots. I'd find various even clearer than variadic, because it's not just that the arity is flexible. I'd say we need different syntaxes for those case. Maybe various T vs many T.

In a generic T is sort of flexible, but at least it's fixed within the function context. With variadics T won't even stay the same in your own specific context, which I think makes it one level more complex to understand.

1 Like
#32

What about the notation that could be found in other languages proposals?

func foo<T>(x: ..T) { }
1 Like
#33

How about adding types(n) for type parameters declaration?

// declare 3 type parameters at once
func foo<types(3) Types>(t0: Types.0, t1: Types.1, t2: Types.2) {}
foo(t0: true, t1: 42, t2: "Hello")

// declare 3 type parameters and use them at once
func foo<types(3) Types>(values: Types...) { print(values.0, values.1, values.2) }
foo(values: true, 42, "Hello")

When n is an identifier, then treat it as a parameter:

// declare n type parameters and use them at once
func foo<types(n) Types>(values: Types...) { print("length = \(n), values = \(values)") }
foo(values: true, 42, "Hello", 3.14, [true])  // length = 5, values = true, 42, "Hello", 3.14, [true]

Since variadics is the generalization of length from fixed length, unlike generics which is the generalization of type from fixed type, using number here seems more straight way to express variadic generics for me.

1 Like
#34

You could generalize this idea even further by making it a range, however it might become very clunky.

<types(…2) T> // 0 - 2 
<types(2…) T> // 2 - infinity
<types(2 … 2) T> aka. <types(2) T> // 2 
<types T> // 0 - infinity

Having the ability to control the number of types would be cool, but I don‘t think we need that with the initial pitch. However it just signals that this feature would rather need a custom keyword instead of the … or * postfix, unless we want to add regex like ranges. T*{2}, T*{2,100}.

1 Like
#35

I’m confused about a couple things:

  1. The proposal says you can’t have more than one declared variadic generic and then uses examples with two. Am I misunderstanding something?

  2. What is the function of this oft-mentioned splat operator? What would it’s return type be?

  3. I also find the overloading of ... confusing. Spelling of T... and T... meaning two different things in the same declaration is a head-scratcher. I prefer the aesthetics of T* but the future extensibility of T[] here.

I’d also like to second @Jumhyn’s suggestion of establishing a convention of pluralizes generic placeholder names for variadic generic types. It definitely makes it clearer.

#36

As I understand it, the "only one variadic type param" is only a restriction on generic types since you parameterize types explicitly:

struct Zip<Xs..., Ys...> { ... }

Zip<A, B, C, D, E>(...) // where do the Xs stop and the Ys begin

But no such explicit parameterization syntax exists for functions, as functions have named parameters so can have their parameters explicitly differentiated. (which is also why we were able to add support for multiple variadic parameters).

However, we allow generic parameters to be inferred in many (most?) cases, so if multiple variadic parameters are allowed in Zip's init it seems reasonable that we could write:

let z = Zip(xs: a, b, ys: c, d, e)

And have the generic parameters to Zip inferred. Then we could worry about the syntax for explicit parameterization later.

We haven't ever gotten around to an explicit parameterization syntax for generic functions, but maybe it's more important for types?

The splat "operator" (as I've been using the term, at least) wouldn't be a proper Swift operator, and its type isn't really representable in Swift (though maybe it could be represented with the right variadic generics design?). It would be a piece of syntax for saying "take this aggregate value and expand it into its constituent parts." For instance, the tuple splat "operator" would let you do this:

func foo(_ x: Int, _ y: String) { ... }

let t = (5, "blah")
foo(t) // error: 'foo' expects two separate arguments
foo(#splat(t)) // implicitly: 'foo(t.0, t.1)'
4 Likes
Variadic Generics