Pitching The Start of Variadic Generics

Evolution Pitches
1

Hello Swift Community,

As part of our efforts to improve the ergonomics of the generics system, as well as providing better support for abstractions that use tuples, I wanted to approach y’all with a sketch for the surface syntax and preliminary semantics. Because this is a large topic area with a lot of impacts on the direction of both the language and later proposals, your feedback is critical at this stage for shaping the direction this feature set is taken in.

I want to thank (in no particular order) Alejandro Alonso, Doug Gregor, and Slava Pestov for shaping my thinking on this subject, and for advancing a lot of the foundations here.

You can find the link to the text of the pitch here TypeSequences.md · GitHub

80 Likes
2

Fantastic. Will have to mull over the idea that these are sequences. I think it’s fine that (T...)... composes weirdly as it’s a bit of a weird thing to want to compose.

(Re the last point, we can borrow Python’s splatting syntax, I think, since we’re not using C’s pointer operators.)

3 Likes
3

Excited to see this taking off! Any chance you could add syntax highlighting to the doc? It makes it much easier to read in the browser, especially on mobile.

1 Like
4

I have added swift annotations to the document.

8 Likes
5

Awesome to see this moving forward.

I've always found the ... spelling for variadic generics somewhat problematic given the noted extensive overloading of this piece of syntax. I don't know if I agree with @xwu's intuition that (T...)... is particularly strange. With an appropriate operator for an explicit tuple splat, it seems not-outlandish to me to have a function such as:

func apply<T..., U>(fn: (T...) -> U, _ args: (T...)...) -> [U] {
  return args.map { fn(#splat($0)) }
}

The bigger issue for me when it comes to the ... spelling is how it pans out for generic types. From the body of the example ZipSequence type:

public init(_ seqs: Sequences...) {
  self.seqs = seqs  
}

The only way we can tell for sure that Sequences is a variadic type sequence rather than seqs being a variadic parameter is to inspect the generic signature where Sequences is declared. While this is not so much of an issue for functions (since the generic parameter will always be fairly local to its use), type members may be defined in a separate extension, which means they could be in a disparate part of the file, another file, or another module entirely.

I'd really like a spelling here that indicates at the point of use of a particular type sequence that we're dealing with a type sequence. It shouldn't be something that's hard for the reader to determine. I think previous pitches have used prefix ... for this reason, which I don't love because it's not super distinctive from postfix, but at least it has the advantage that variadic parameters and generic type sequence parameters can be differentiated locally.

Somewhat relatedly, I think it would be good to think about how the syntax will relate to conventions around the naming of these type sequences. In some places the name associated with the type sequence is plural (Sequences, Views, Xs, Ys) but in others, it's a singular noun describing the entirety of the sequence (Tail, Init), and in still others we're using just single letter generic names (T, U).

The pitch says that when T... is a type sequence parameter,

T refers to a particular opened archetype of an element of the type sequence T...

which suggests to me that there are situations where the plural spelling might make a function signature more difficult to understand, e.g.:

func allPresent<Sets..., Elements...>(haystacks: Sets..., needles: Element...) -> Bool
 where Sets == Set<Elements>

I love that Swift has really leaned in to meaningful, more-than-one-letter names for generic parameters, so the extent to which the variadic generics design is able to encourage meaningful names as opposed to Ts... and Us... is something I think is important.

I could imagine us introducing, for instance, a new type of generic constraint that would allow us to supply a name for each element in a type sequence. Something like:

func allPresent<Haystacks..., Needles...>(haystacks: Haystacks..., needles: Elements...) -> Bool
  where forEach(Haystack in Haystacks, Needle in Needles, Haystack == Set<Needle>)

Minor note, but for this portion:

This also allows for stored properties of type sequence type within these nominal types.
[...]
As with functions, stored properties in specializations of these types canonicalize to tuples

IMO a stored property of type T... should be illegal, and the user should have to write (T...) to get the tuple behavior. Bare T... in a function parameter list doesn't mean "tuple" and I don't think we should confuse the two.

12 Likes
6

I’m very happy for all those involved in the process here so far, and eager to see the end result of this some day. The amount of error-prone boilerplate and/or high-maintenance code generation setups this has the potential to help unfurl is tremendous, and a lot of engineers at all layers of the stack will get to feel benefits here.

The dragons on the horizon in terms of identifying vocabulary for what this will imply are very intriguing, because so many of the most obvious choices to describe these situations are occupied by other language features. I’m really excited to see how everyone is able to push this forward.

4 Likes
7

Plus all the 1s!!!

I just got to work so I’ll have to actually read it later, I just saw the title and couldn’t contain my excitement is all.

:grin::grin::grin:

8

I think @xwu is right about the splatting operator. There's no reason we can't use a prefix * for it. I'm not sure about the spelling for type sequences, but I can't really think of anything better than either T... or ...T off the top of my head.

Everything else seems great, especially with parameterized extensions.

9

I'm super excited about this feature, so thanks so much for championing it!

This pitch makes sense to me overall, but one aspect confused me: In the allPresent example, I didn't expect the where T == Set<U> clause to map the constraint element-wise. I'm not sure what it would do otherwise though, and more explicit syntax is probably needlessly verbose, so maybe this is fine.

1 Like
10

In terms of the spelling for a type sequence, if using a prefix * for the splat operation, might it not make sense to use * on the other side, too? So instead of T... use T*. Then you could spell that potentially undesirable matrix example as func matrix<T*>(_ xss: (T*)...) if I'm understanding it correctly.

I don't know if that's really better than the dots or not, but... maybe?

14 Likes
11

I like this a lot, though I suspect that the details will evolve significantly over time (especially, I think, for the reasons/objections that @Jumhyn laid out).

A couple of comments specifically about the pitch, though:

— The allPresent2<T...> example is unnecessarily confusing, I think, because it uses T where the allPresent example used U. It seems like a trap for the unwary.

— The statement "This means we have the ability to construct generic functions that don’t just take type sequences, but return them as well." is followed by an example that doesn't actually return a T..., but a tuple (T...) that wraps the T....

The document doesn't actually show whether ( … ) -> T... is possible, it just says it is.

So, I think there is a different discussion to be had here, about storing and (perhaps) composing T... values. It's also interesting to consider whether there might be something analogous for non-generic variadics, too, so that we don't have to turn cartwheels to bridge between (say) Int... and [Int].

1 Like
12

I really want to see variadic generics in the Swift, so I'm very happy to see this pitch.

Composition of type sequences would be hard with this pitch. For example, I can imagine a type witch takes variadic (T) -> U arguments. But maybe it is difficult:

// desired behavior
foo(closures: { (a: Int) in a + 1 }, { (a: String) in print(a) }, { (a: Bool) in a ? "true" : "false" })

// maybe error, since the length of T... and U... is not always equal
func foo<T..., U...>(closures: ((T) -> U)...) {}

// however, since there is no relation ship between T... and U..., constraint like `T == Set<U>` cannot be added

It's just an idea, but maybe some (T, U)... type sequences can solve this problem.

// maybe error, since the length of T... and U... is not always equal
func foo<(T, U)...>(closures: ((T) -> U)...) {}

With it, allPresent example changes like this:

// it ensures that T... and U... have the same length
func allPresent<(T, U)...>(haystacks: T..., needles: U...) -> Bool
  where T == Set<U>  // such constraint can only be added to (T, U)... type sequence
{ /* ... */ }

Does T... allows zero-length type sequence? Considering current variadics allows zero length parameters, it is natural to allow them.

func foo<T...>(_ value: T...) {}
foo()

However, I think then it is difficult to express some constraints. Consider creating HomogeneousTuple struct.

struct HomogeneousTuple<T...> {}

Of course this is not correct, since T... allows heterogeneous types. Therefore, you have to write like this:

@dynamicMemberLookup
struct HomogeneousTuple<Element, T...> where T = Element {
    var tuple: (T...)
    subscript<U>(dynamicMember keyPath: WritableKeyPath<(T...), U> -> U {
        get { tuple[keyPath: keyPath] }
        set { tuple[keyPath: keyPath] = newValue }
    }
}

However, on the use-site, it becomes strange type:

let hTuple: HomogeneousTuple<Int, Int, Int> // this is two-length HomogeneousTuple actually!!

Maybe some solution must be prepared for it. I thought generic where I described here can be one solution, but there will be neat way.

Some thought:

  • Is there any possibility to consider variadic - reverse generics? Returning opaque length of type sequence would be intresting.

  • I don't think the strategy to have _countTypeSequence, _mapTypeSequence, and other build-in operations is enough. Maybe it turns endless addition of reserved name for zip, reversed, rotated, and so on? But I don't have any idea to enable such operation without built-in :cry:

    • In the allPresent example, zip(haystacks, needles) naturally appears, and I'm confused. Do type sequences conform Sequence?
13

+1. Interesting to hear if tuple fusion can be possible using this technique (like TypeScript's variadic tuple types).

// NOTE: This signature is probably wrong...
func fuseTuple<T...>(_ xs: (T...)...) -> (T...) { return ??? }

fuseTuple( ("Hi", 42, 3.14), ("foo", true) ) == ("Hi", 42, 3.14, "foo", true)
2 Likes
14

Compound sequences would be nice, yes.

func concat<T…, U…>(_ ts: (T…),  _ us: (U…)) -> (T…, U…) { /**/ }

Unfortunately a little too advanced for the first go.

3 Likes
15

Personally I wouldn’t mind the variadic T spelling instead of T..., it’s not shorter but much pleasant to parse as a human.

17 Likes
16

I like the keyword idea. Also something like many T could be interesting.

11 Likes
17

This will be great to have!

Thinking on ways to address some of the challenges, a thought came up—what if we address the current limitation by allowing arguments (variadic or not!) to have opaque generic types?

func print<Item>(_ items: some Item...)
  where Item: CustomStringConvertible {
  
}

The trick, of course, would be the type of items: [some Item]—an array with an opaque element type, which is currently disallowed. On the plus side, it does (imho) feel like a more direct semantic interpretation of heterogeneous variadics.

I’m aware this is quite different from @codafi’s pitch (and would warrant a separate proposal), but I think many of the described type sequence affordances translate moderately well to arrays of opaque types (with a couple of exceptions I can immediately see).

There are many aspects, implications, and variants to explore, but being quite new to this type of discussion, I wanted to see whether someone can correct me if it’s obviously problematic, or actually worth diving in deeper on and writing up? :blush:

18

I really like the idea and where this is going, thanks for writing this :+1:

I agree that ... syntax for type sequences might be confusing and ambiguous. I for one got a bit confused as I was reading some of the examples as if they were declaration for variadic arguments instead of type sequences sometimes, and I feel it'd be too easy to confuse them, while it's important to make clear that the different types in a T... type sequence are not necessary the same types each as opposed to variadic arguments.

I also found it a bit confusing to see typealias Element = (Sequences.Element...) in the zip example, as I was expecting the syntax to be more something like (Sequences...).Element or needing some usage of _mapTypeSequence here, as were effectively wanting the element of each archetype here.

Some alternatives I have in mind as an alternative to ... and to avoid ambiguity:

  • postfix * — ie T*. Would also maybe allow things like typealias Element = Sequences*.Element
  • postfix [] — ie T[] and typealias Element = Sequences[].Element

Personal opinion on those:

  • I like * because it's common and would work well in conjunction to introducing a potential splat operator too using * too (a bit like in Ruby).
  • I like [] because when I see func debugPrint<T[]>(_ x: T[]) I think of the type sequence as being "a tuple (T0, T1, T2,… Tn)" or (T[0], T[1], … T[n]) — as if those were mathematical subscripted variables, which acts as a reminder that each Tn can be different (as opposed to T... used as a variadic generic where they'd be all the same).
  • It would also open the door in the future to allow specifying an arity or even a range, eg T[5] for a type sequence / tuple with 5 (possibly different) types, T[1...] for a type sequence of size at least 1, etc
  • But otoh for some subjective reason I feel like (_ x: T[]) is less suggestive of the method allowing a variable number of (potentially heterogeneous) arguments and instead suggests an arity of one there, while I somehow feel like T* would make it more natural to read it as variadic arity (maybe that's my Ruby bias speaking here?)
19

Could we have some clarification on (or removal of) the word “archetype”? As far as I can tell it’s implementation-detail jargon. Slava was going to explain it in part four of his two-part series back in 2016…

7 Likes
20

I think this is wonderful, but I wonder if the use case that I'm personally most interested in is possible to express, namely homogeneous tuples of variadic length? It's very useful in math related applications, especially of course with vectors. Something like:

struct Vector<S> {
    let value: S...
}

Also, I imagine other will be interested in the opposite case, heterogeneous tuples of a given length. Yeah, that's just tuples, but with the proposed syntax you don't have to explicitly name each type.

struct Quadruple<S[4]> {
    let value: S[4]
}

I don't mind the syntax, I'm sure it will be fine, but I suppose we should first make existing generics accept this shorthand (which is very useful):

extension [String] { } // = "Array where Element == String"

By the way, I didn't really understand what _forwardTypeSequence does, could you perhaps add another example or a few words of explanation?

1 Like