Variadic Generics

[Edit 2] Link to detailed document (status: not completed yet).
The informations contained in the document supersede the contents of this opening post.


Hello everyone, I was reading the Generics Manifesto document and saw the Variadic Generics section. I would like to start a pitch about them!
This post mainly contains elements directly taken from the Generics Manifesto plus some personal thoughts and comments.

Disclaimer: I'm not a native English speaker so my writing could be less-thant-clear in some instances; sorry for that, I'll eventually try to explain anything that is not clear.

Basics: variadic generics are tuples

The document refers to C++ parameter pack and ... (ellipsis) syntax, that I personally do not particularly dislike, and then at the end of the section refers to a tuple splat operator.

I think that variadic generics can be represented as tuples once "instantiated", and there is a similar precedent in the language too with variadics function argument becoming Arrays. I say "I think" because I'm not actually sure that this is what the manifesto suggests - given the iterators example in which it talks about "zero or more stored properties" instead of a tuple of iterators instances.

struct Variadic<...T> {
  var (...items): (T...)
}

let v: Variadic<Int, Float, Any> = ...
// `v` is
// Variadic<(Int, Float, Any)> {
//   var items: (Int, Float, Any)
// }

This means that a type with variadic generics is actually a type with a single generic parameter of tuple type, whose number of elements depend on the actual usage. This even accounts for the case in which no generic are specified: the variadic parameter is represented by the empty tuple.
In this way one can directly reference any member via tuple access, and can use common members (if any) on the compound value itself.
In the previous example one cannot use any member on items, because T is unconstrained and in Swift there are no members common to all types. But if the declaration was T: Collection you could use first on items and obtain a tuple containing the first item in each sequence (if any):

struct Variadic<...T: Collection> {
  var (...collections): (T...)
}

let v: Variadic<[Int], [Float], [Any]> = ...
// `v` is
// Variadic<([Int], [Float], [Any])> {
//   var collections: ([Int], [Float], [Any])
// }

let result = (v.collections.first...)
// result has type (Int, Float, Any)

Pattern matching

The Generics Manifesto uses the following example:

var (...iterators): (Iterators...)

[...]

guard let values = (iterators.next()...) else {   // call "next" on each of the iterators, put the results into a tuple named "values"
  reachedEnd = true
  return nil
}

If the syntax (variadicPack.someMember...) means get someMember from all the items of variadicPack and put them, in order, in a new tuple, then (iterator.next()...) if actually of type (Element1?, Element2?, ..., ElementN?) and this is not an optional that can be used in a guard statement. So maybe I'm missing something from the document or we might want to bless the ... operator with some special meaning regarding optionals (i.e. the result is nil if any element is nil, otherwise it is a tuple of non-optional elements).

If giving special meaning to ... for optionals is not desired, something like the following might be more appropriate:

switch (iterators.next()...) { // switching over a tuple of N optional elements
  case (.some(let v)...): // matching the case of all elements being non-`nil`. The variable `v` contains all the elements 
    return v
  default: // default case: at least one element is `nil`
    reachedEnd = true
    return nil
}

Tuple splatting

We could use the same ... operator for tuple splatting. Other than the example of the Manifesto, in which a function with variadic generics is called expanding the arguments passed to it, the feature might be useful to allow one to write something like this:

let (...head, tail) = someTuple
// `someTuple` is a tuple containing N elements
// `head` is a tuple containing the first N-1 elements of `someTuple`
// `tail` is a single value (not a tuple) containing the last element of `someTuple`
//
// If `someTuple` contains only one element, `head` will be the empty tuple and `tail` will contain such element
// This is not possibile in current Swift because single-element tuples are not allowed
//
// If `someTuple` is empty ww might want this code to not compile and raise an error

What do you think? Any thought about the implementation level of this? Let's start the discussion!

edit: link to previous discussion.


[Edit 2] Link to detailed document (status: not completed yet).
The informations contained in the document supersede the contents of this opening post.

11 Likes

Did you read the previous threads about variadic generics? I got the impression that it’s an often discussed feature, so it would be great if new threads about it could also post a short summary of previous discussions. Otherwise we could easily repeat what has already been said.

2 Likes

Good thinking! Here's a link to the other major thread, in which I link to a more minor thread.

This is a feature that's near and dear to my heart, so I'm definitely going to write something up once I'm off my phone and back onto a laptop.

1 Like

Yeah, I've seen the other thread but it was two and a half years old and I didn't feel to resurrect it a second time... I guess I should've linked it in my OP though, I'll edit it.

@twof, I'd love to hear your thoughts about this topic, please share with us what you think!
Maybe @taylorswift would like to add something to the mix, given her recent discussion => Vector manifesto in which she talks a little about variadic generics.

Also, anyone can tag who he / she thinks may be interested in this topic!

I don’t think i talked a lot about variadic generics in that manifesto,, mainly i was just saying that variadic generics aren’t really relevant to implementing vector and fixed-size array types even though people always bring them up.

p.s. the woman in my icon isn’t me, it’s Hand of the Queen Tree Paine :joy:

1 Like

I see, I quickly scan the document because I was interested in the topic and saw the reference to VG, so I thought you might have something to say about them.

Oops :sweat_smile::upside_down_face:

1 Like

What about using an annotation instead of using lots-of-dots? So that instead of e.g.

struct ZipSequence<...Sequences: Sequence>: Sequence {
  var (...sequences): (Sequences...)
}

func zip<...Sequences: Sequence>(...sequences: Sequences...) -> ZipSequence<Sequences...> {
  return ZipSequence(sequences...)
}

one could more easily write

struct ZipSequence<Sequences: @pack Sequence>: Sequence {
  var sequences: Sequences
}

func zip<Sequences: @pack Sequence>(sequences: S) -> ZipSequence<Sequences> {
  return ZipSequence(sequences)
}

I used @pack because it conveys the fact that Sequences is a "parameter pack" of generic types, but discussion is welcome.

1 Like

cc @Douglas_Gregor

I thought I might tag you in this thread too in case you want to participate in the Swift-level discussion about this feature.

Rather than pack, it might make sense to do @variadic which is inline with other discussions to rename T... function arguments to @variaidc T. Although one is multiple types and the other is a group of the same type.

2 Likes

@variadic might in fact serve the purpose better, and the fact that is shared between the two concepts should not be a problem because in the variadic arguments case it will appear in the function parameter list "( )", in the variadic generics case it will instead appear between the generic argument clause "< >".

This also makes me think that we might not want to allow both kind of variadics in the same expression.

1 Like

Try not to get too caught up with the spelling. We'll almost certainly need some way to identify that we have a parameter pack containing type parameters. Whether that's introduced by ... or @pack or @variadic is less important than how it behaves.

Well, that depends. With your Variadic definition, you almost surely want to be able to implement an initializer, which would likely have a function parameter pack whose types are described by a type parameter pack:

struct Variadic<Types...> {
  init(_ values: Types...) { }
}

Doug

4 Likes

I sorta regret writing the example in that way. Swift shouldn't need the prefix "...", and that definition of items is probably wrong. If we're using "...", that example should likely be written as:

struct Variadic<T...> {
  var items: (T...)
}

Same thing here:

we don't need the prefix "...", and probably don't even want the notion of it in the language. iterators can be described as:

var iterators: (Iterators...)

so we're forming a tuple type from the types in Iterators and storing that.

You are correct; my example here was bogus. I'll take a pass through this part of the generics manifesto so at least it's not as misleading and wrong. Thank you!

This may not work so well in a generic context unless you had a way to say "I know that someTuple has at least one element".

When it comes to tuple splatting, I think what we really want is a way to take a tuple and expand it into N separate arguments, e.g.,

let someTuple = (1, 3.14159, "Hello")
f(someTuple...). // equivalent to f(1, 3.14159, "Hello")

It's plausible that we could integrate that with variadic generics if, say, a "function parameter pack" was represented as a tuple and all tuples could be expanded like that.

Doug

9 Likes

Could the syntax be ambiguous in this case? We today have that func(_ someParam: SomeType...) is a function that accepts a variable number of arguments of the same type SomeType, so:

func foo(_ v: Int...) {
 // v is an array here
}
foo(1, 2, 3, 4) // this is valid Swift

let v: Variadic<Int> // Specialising variadic generic argument with a single parameter "Int"
  = Variadic(1) // This is ok
  = Variadic(1, 2, 3, 4, 5, 6, 7) // This might be unclear to users, but is relatively similar to calling `foo`

let v: Variadic<Int, String> // Specialising variadic generic argument with two parameters "Int" and "String"
  = Variadic((1,"Hello")) // Everything's ok
  // = Variadic((1, "Hello"), (2, "World"), (3, "!")) // a user might not realise this is a valid call to the initialiser!

This is the reason I thought we may drop the "..." to unpack the variadic generic: to me, in a (variadic) generic context the type T declared as <T...> is already a tuple. So there will be no ambiguity in the aforementioned case.
My reasoning holds unless we want or need the syntaxes (T...) and T to actually represent different things - with the first one representing the tuple expansion and the second one (insert suggestion here).

1 Like

I have not given much thought to this argument (tuple splatting / expanding), but your suggestion is very interesting because IIRC this feature is something that we lost when we dropped the concept of a function's parameters being (also) a tuple.

Edit: fixed link to point to Chris's post, not the whole thread.

@Chris_Lattner3 shared his thoughts on my other thread (about potential implementation-level details of this feature), suggesting that we might want to treat a variadic generic like a standard generic type with the addition of being constrained to be a tuple.

This made my think that we may widen / shift the focus a little bit, and instead of variadic generics we might want to introduce variadic tuples altogether. This means that the syntax (...T) is (maybe) not limited to appear in generic contexts but may be used "anywhere" to indicate a tuple of variadic length. In this way a variadic generic type is simply a type which has one (or more?) generic parameters which are variadic tuples, so that with this shift in focus we are at least getting the same functionality for free.

struct Variadic<A, B, (...Collections: Collection)> {
  [...]
  var collections: Collections
}

We have then to decide how code referring to a variadic tuple works. For example, considering the previous example in which Collections is constrained to the Collection protocol, we might allow calls to members of Collection to be applied to the collections variable in order to return a tuple containing the result of applying the member to all of the tuple elements, or in code:

extension Variadic {
  func interestingFunc() {
    // the next statement is equivalent to the following:
    // (collections.0.first, collections.1.first, collections.2.first, ...)
    var firstElements = collections.first
    [...]
  }
}

What do you think?

In my free time I'm trying to write a document to sum up what we can do with Variadic Generics. Currently I'm in researching (just starting) the problem i.e. "what are we trying to solve with VG? Why would we want them?"

The primary and prominent use case I see is "I actually need a variadic function, but one whose arguments are not all equals to the same type T but similar between each other i.e. they are different concrete types conforming to a (set of) protocols".
In code this means the following:

// I want to pass *any number* of *any type* that conforms to Sequence
func zip<Ss: Sequence>(_ sequences: Ss...) -> ZipSequence {
   [...]
}
// This is not allowed because [Int] and [Int: Int] are not the *same* type
// even if they both are Sequence<Int>
// zip([1, 2, 3], [4: 4, 5: 5, 6: 6])

// I want to pass *any number* of KeyPath<T, *any comparable property*>
func sort<T, C: Comparable>(_ array: [T], by cps: KeyPath<T, C>...) {
   [...]
}
// This is not allowed because KeyPath<Person, String> and KeyPath<Person, Int> are not the *same* type
// even if they both are KeyPath<Person, Comparable>
// sort(anArrayOfPeople, by: \.name, \.age)

Can you suggest any other different use case? If possibile, use a real-world example that you or someone else could not solve because of this missing feature. But a made-up example is obviously welcome as well!

2 Likes

In the reactive world, you have the combineLatest operator, which turns several streams of values of various types into a single stream. In a way, it is like an asynchronous zip (or zip + map, when the host language does not support heterogeneous containers like Swift tuples).

Due to the lack of variadic generics, Swift libraries that provide such facilities can't support an arbitrary number of input streams. They must provide explicit support for up to N inputs (N being limited by some arbitrary decision).

RxSwift, for example, currently supports up to 8 inputs: https://github.com/ReactiveX/RxSwift/blob/fc6030ea9fe9b8af0351fe195b4095a9f2339757/Rx.playground/Pages/Combining_Operators.xcplaygroundpage/Contents.swift#L88

5 Likes

ReactiveSwift has a similar operation that allows up to 10 generic parameters: ReactiveSwift/Signal.swift at master · ReactiveCocoa/ReactiveSwift · GitHub

2 Likes

Thank you for your suggestions! The reactive examples are very interesting ones, because they are apparently similar to the zip one but have two main differences:

  • they have a different parameter other than the variadic one, and this parameter comes after in order to allow trailing closure syntax
  • more importantly, the last parameter is a closure, and the shape of its input depends on the number of generic parameters

Very nice catch guys, highlighting the last point was particularly interesting!

More examples!

2 Likes