Variadic Generics

Notice that I said sum and specifically not union. This would be the sum type equivalent of a tuple, with cases 1, 2, 3, etc. This is different from a union type. Union types have been deemed too big a risk for type system performance to be considered.

But isn‘t a sum type also referred as a tagged union which would be a refinement of a union type?

I specifically don‘t want any A | B syntax for that. A tuple like enum with numbered cases corresponding to the index in the variadic pack is enough for me.

I don't know what the syntax for these types would look like. Do you have another idea?

This is exactly what I was suggesting.

What exactly do you mean now? I know that A | B syntax is rejected and I‘m not proposing to reconsider it.

I meant the same, but I didn‘t mean to refer to the general 'union types' but rather called the variadic enum Union.

I think I used

(; label0: Type0, Type1, label2: Type2 ;)

for a sum-tuple syntax. We just use semicolons on the inside side of the tuple parentheses. Like regular tuples, an unlabeled entry gets an index number for its access label. (In the above example, the second member gets ".1" as its access case.)

For a broader discussion, I want to cross-link this thread.

1 Like

I don't agree with making variadic items real collections. Variadic items are figments of the compiler's imagination; the object code never sees them, only their post-exploded results (which should be handled the same way as if a developer manually wrote out the same code).

2 Likes

@technicated, I just gave the variadic generics document a read and it really looks great! Thank you for putting all of this together. I'm very excited for what the future has in store for Swift generics.

A couple questions... I was wondering if one will be able to write extensions on variadics (sort of like the project(_:) method AFAIU) like it is a regular protocol conforming to Collection? Also, is there any value in being able to have more fine-grained control over the number of arguments that a given variadic can accept (i.e. restricting a variadic parameter to only take a max number of arguments or a minimum number of arguments)? Something like that would allow you to leave out the first two parameters of you Variadic Generic version of the zip function by restricting the variadic parameter to 2 or more arguments.

I just wanted to bring your attention to a few minor fixes I found while reviewing the current version of your Variadic Generics proposal:

  • Spelling Errors (12):

    • Correct "thells" to "tells" and "wether" to "whether" in comments in collectionApi(t:u:) method in Variadic Value API section.
      // thells wether `T` was specialized with 0 parameters
      // (and therefore `t` contains no elements at all)
      print(t.isEmpty)
      
    • Correct "regualr" to "regular" in last line of Motivation section.

      Let's take a look at some examples that are hard or impossible to "scale up" today with regualr Swift.

    • Correct "clutterness" to "clutter" in Example 3: variadic sorting.

      SwiftUI and its ViewBuilder function builder type suffer from the same problem. ViewBuilder has a fixed amount of buildBlock methods that take C0 , C1 , C2 and so on subviews, leading again to code duplication and clutterness.

    • Correct "toghether" to "together" in Proposed Solution section.
       > This means, for example, that if there exists a function parametrised by a Variadic Generic conforming to the  `Collection`  protocol you can pass to this function an  `Array` , a  `Dictionary`  and a  `String`  - all toghether - because all these types conform to  `Collection` .
      
    • Correct "postifx" to "postfix" in Proposed Solution section.

      A variadic value can also be passed as the variadic argument of a variadic function (yay!). I mildly suggest the operation is again performed by using the ... syntax to explicitly indicate what's happening (but here, much more than in the tuple context, the ... can be mistaken for a postifx operator... suggestions are welcome):

    • Replace first "of" with "as" in the final portion of the Proposed Solution section.

      What can be noted in this example is that the code is practically the same of the current, non-variadic zip implementation. The only real difference is that zip can now be used to zip any number of arbirtary and different sequences together!

    • Correct "arbirtary" to "arbitrary" in the final portion of the Proposed Solution section.

      What can be noted in this example is that the code is practically the same of the current, non-variadic zip implementation. The only real difference is that zip can now be used to zip any number of arbirtary and different sequences together!

    • Correct "previos" to "previous" in link in Introduction section.

      This document is kind of a follow up to a previos one by Austin Zheng - "kind of" because I've read the document, but not too in depth - and this was intentional.

    • Correct "parametrised" to "parameterized" in Proposed Solution section.

      This means, for example, that if there exists a function parametrised by a Variadic Generic conforming to the Collection protocol you can pass to this function an Array , a Dictionary and a String - all toghether - because all these types conform to Collection .

    • Correct "Variadig" to "Variadic" in comments under Declaring and using Variadic Generic functions section.
      // The following code will cause a compile-time error like "Variadig generic
      // argument does not need `...`. Remove it." - a fixit can also be suggested
      func wrongVariadicGenericFunction<T: variadic Any>(ts: T...) { }
      
    • Correct "declarered" to "declared" in comments under Declaring and using Variadic Generic functions section.
      // =============================================================================
      // The `(T...)` syntax is also taken into account, allowing a tuple to be passed
      // as a parameter to a Variadic Generic function. Two functions declarered with
      // both `T` and `(T...)` syntax are different will participate in overload reso-
      // lution.
      // =============================================================================
      
      func overloaded<T: variadic P1>(ts: T) { }
      func overloaded<T: variadic P1>(ts: (T...)) { }
      
    • Correct "trasforming" to "transforming" in first comment under Variadic Value API section.
      // =============================================================================
      // Variadic values conform to the `RandomAccessCollection` protocol and enhance
      // it in various ways. All the standard `Collection` API can be used, and all
      // the standard functionality (for ... in loops, etc) is available. Moreover,
      // all the "trasforming" members are overloaded to return another variadic value
      // instead of an `Array`, and due to how overload resolution works these func-
      // tions will be the preferred one when no type context is given.
      // =============================================================================
      

  • Grammatical Errors (20):

    • Insert comma after "compiler" in comments under the Declaring and using a Variadic Generic section.
      // When there is ambiguity in case of multiple Variadic Generics, the type sys-
      // tem prefers to assign the maximum number of types to the first generic param-
      // eter. The parameter name can be used to manually resolve the ambiguity, and
      // for the happines of the compiler this concept may be generalized to all
      // generic parameters even if this is not strictly needed. This new syntax does
      // not allow the reordering of the generic parameters. The compiler is updated
      // to show new and more detailed informations about generic parameters.
      
    • Hyphenate "in depth" in Introduction section.

      This document is kind of a follow up to a previos one by Austin Zheng - "kind of" because I've read the document, but not too in depth - and this was intentional.

    • Insert comma after "Obviously" in Introduction section.

      Obviously the information contained in the mentioned document and collected on the Swift Forums are going to influence what's presented there.

    • Insert comma after "unfortunately" in first part of Motivation section.

      Today it is impossible or very difficult to express patterns related to an unbounded amount of generic types or values. Maybe the closest thing we have is variadic functions, but unfortunately such functions require all of their arguments to be of the same concrete type

    • Insert comma after "way" in Example 3: variadic sorting.

      In this way animals are sorted by name, and if their name is the same they are sorted by their age, and so on. But if we try to declare the function in the naïve way we get an error:

    • Insert commas before and after "in this case" in Example 3: variadic sorting.

      This example might again seem similar to the zip one. The difference in this case is that Variadic Generics are not used "directly" in the function signature, but are used instead to construct another type that depends on them i.e. KeyPath<Element, T>.

    • Insert comma after "in other words" in Example 5: curry.

      A curried function is one that takes multiple arguments but one at a time - in other words a curried function takes the first argument and returns a new function taking the second argument and so on until all arguments are used and the result is returned.

    • Insert comma after "context" in Proposed Solution section.

      Inside of a generic context a Variadic Generic can be directly used as a type. A value whose type is a Variadic Generic is called a variadic value, and is a collection of zero or more values (more on this later) implementing the Collection protocol:

    • Insert comma after "section" in Detailed Design section.

      In this section we are going to see how Variadic Generics can be declared and used in various contexts.

    • Insert comma after "case" in comments above vg3 declaration in Declaring and using a Variadic Generic section.
      // The name of the generic parameter can be used to specify how the
      // specialization should be done; note that in this case all the labels are
      // specified for clarity but not all of them are really required
      let vg3: ComplexVariadic<A: Double, B: Int, T: Int, String, U: String, String>
      // vg3: ComplexVariadic<A: Double, B: Int, T: <Int, String>, U: <String, String>>
      
      </details>
      
      Insert comma after "because" and ", they" after "generic context" under Variadic Values section.
    • Insert "elements" before "passed" in comments in collectionApi(t:u:) method under Variadic Value API section.
      // Get one of the `P1` passed to `t`
      print(t.randomElement())
          
      // The `Index` type is an `Int` and subscripts are 0-based
      // The same as `t.first` or a nice crash
      print(t[0])
      
    • Insert "`ts`" after "first function" in comments under Declaring and using Variadic Generic functions section.
      func wrongVariadicGenericFunction<T: variadic Any>(ts: T...) { }
      
      // =============================================================================
      // The two functions declared above are somewhat equivalent from the outside be-
      // cause `T` was not constrained in any way. The only difference is that in the
      // second function `ts` is of type `[Any]` while in the first function is of
      // type `variadic Any`.
      
    • Insert commas after "second function", "type `[Any]`", "first function", and "snippet" in comments under Declaring and using Variadic Generic functions section
      func wrongVariadicGenericFunction<T: variadic Any>(ts: T...) { }
      
      // =============================================================================
      // The two functions declared above are somewhat equivalent from the outside be-
      // cause `T` was not constrained in any way. The only difference is that in the
      // second function `ts` is of type `[Any]` while in the first function is of
      // type `variadic Any`.
      //
      // In the following snippet we instead actually have a difference: the second
      // function can inded accept parameters of any type, but only if all of them are
      // of the **same** type! The first function does not have this limitation.
      // =============================================================================
      
    • Insert "and" after "different" in comments under Declaring and using Variadic Generic functions section.
      // =============================================================================
      // The `(T...)` syntax is also taken into account, allowing a tuple to be passed
      // as a parameter to a Variadic Generic function. Two functions declarered with
      // both `T` and `(T...)` syntax are different will participate in overload reso-
      // lution.
      // =============================================================================
      
      func overloaded<T: variadic P1>(ts: T) { }
      func overloaded<T: variadic P1>(ts: (T...)) { }
      
    • Replace "The usage of the" with "Note, the" in comment under Declaring and using Variadic Generic functions section.
      // =============================================================================
      // A Variadic Generic can directly be used as the result type of a function, and
      // it will be automatically considered a `Collection` in concrete code. The us-
      // age of the `(T...)` syntax is used to make the result type a tuple.
      // =============================================================================
      
    • Hyphenate "fixit" in comment under Declaring and using Variadic Generic functions section.
      // The following code will cause a compile-time error like "Variadig generic
      // argument does not need `...`. Remove it." - a fixit can also be suggested
      func wrongVariadicGenericFunction<T: variadic Any>(ts: T...) { }
      
    • Replace "another" with "a" and add comma after "overload resolution works" in first comment under Variadic Value API section.
      // =============================================================================
      // Variadic values conform to the `RandomAccessCollection` protocol and enhance
      // it in various ways. All the standard `Collection` API can be used, and all
      // the standard functionality (for ... in loops, etc) is available. Moreover,
      // all the "trasforming" members are overloaded to return another variadic value
      // instead of an `Array`, and due to how overload resolution works these func-
      // tions will be the preferred one when no type context is given.
      // =============================================================================
      
    • Replace "the other" with "another", remove comma after "fair", insert comma after "However", replace ", and for this reason variadic" with ". For this reason, variadic" in comment under Variadic Value API section.
      // =============================================================================
      // The "transforming" functions need a little bit more attention. We have two
      // overloads of `map`, one that returns an `Array` and the other that returns a
      // variadic value - but those overloads do not allow modifications to the under-
      // lying collection element. And this is fair, because programmers expect a
      // `map` to be pure.
      // However sometimes it might be useful to mutate the original variadic value
      // while creating a new one, and for this reason variadic values offer an origi-
      // nal `project` method.
      //
      // *NOTE:* the name is subject to debate, `project` is the best (and sound) that
      // I could find.
      // =============================================================================
      
    • Remove "Obviously", replace semicolon with comma, and remove comma after "topic" in Introduction section.

      I've never worked too much with variadic generics in my programming life and I wanted to start "fresh" and as unbiased as possible on the topic, to see if I could come up with new or interesting ideas; but it's also possible that this document ends up sharing a lot of similarities with Austin's! Obviously the information contained in the mentioned document and collected on the Swift Forums are going to influence what's presented there.

    • Capitalize "values" in "Variadic values" header in Variadic Values section.
      // =============================================================================
      // Variadic values are somewhat "special", because both inside and outside of a
      // generic context act like a `Collection` - or better *are* a `Collection`. The
      // `Element` of this collection is the constraint of the Variadic Generic. In-
      // side a generic context, the `...` syntax allows users to convert this collec-
      // tion to a tuple whose shape and types will be the concrete types passed to
      // the Variadic Generic parameter.
      //
      

  • Other Errors (2):

    • Add "swift" after "```" to enable syntax highlighting under code block under Variadic Values.
      // =============================================================================
      // Variadic values are somewhat "special", because both inside and outside of a
      // generic context act like a `Collection` - or better *are* a `Collection`. The
      // `Element` of this collection is the constraint of the Variadic Generic. In-
      // side a generic context, the `...` syntax allows users to convert this collec-
      // tion to a tuple whose shape and types will be the concrete types passed to
      // the Variadic Generic parameter.
      //
      
    • In the example function, overloadedApis, under the Variadic Value API section, you make reference to a value, t, that I think you just forgot to add as a parameter to the function.
      func overloadedApis() {
          // No type specified - this is a variadic value of `Any`s
          let aVariadicValue = t.map { $0.someMember }
      
          // A type (`[Any]`) was specified - this is an array of `Any`s
          let anArray: [Any] = t.map { $0.someMember }
          
          // `SubSequence` is another variadic value, whose type is still `variadic P1`
          let subVariadicValue = t.dropFirst()
          
          // This works, subVariadicValue is unpacked and passed to the parameter
          // `values` of the called function
          _ = explicitlyMakeTuple(subVariadicValue)
       }
      
4 Likes

Thank you for the effort put in reading the document!

First, I never thought about the possibility of extending this "variadic value" type, but I think that at least for now it should not be possible.

Your second point is instead very very interesting, but I think it can be deferred to future discussions and can be implemented on top of "bare" Variadic Generics.

And finally, thank you very much for all the errors reported!

2 Likes

After some time, it seems to me that people (included myself) prefers Variadic Generics to be a "static" feature and not a "dynamic" one, and that variadic vales are not Collections but some other different type with some map / project / for ... in support...

With that in mind, please consider the following question:

// =============================================================================
// What should the type of the following statement be? And why should that be
// the case?
// =============================================================================

struct S<variadic Params1, variadic Params2> {
  let p1: Params1
  let p2: Params2
}

let s = S(p1: 1, "hello", p2: 42.42)

// Case 1: Flattened - no explicit binding
// s: S<Int, String, Float>
//
// Case 2: Flattened - explicit binding
// s: S<Params1: <Int, String>, Params2: <Float>>
//
// Case 3: Grouped - automatic transform(-ation?) into tuples
// s: S<(Int, String), (Float)>

It cannot be case one, because that would not distinguish between:

let s = S(p1: 1, "hello", p2: 42.42)

and

let s = S(p1: 1, p2: "hello" , 42.42)

but they must have different types. In fact, because variadics are greed, S<Int, String, Float> would have to bind all of the type parameters to Params1, leaving Params2 with no types.

It cannot be your third option because variadic value packs and tuples are not the same. The third option would be the case only if the type was unpacked as p1: (Params1...).

Adjacent variadics that have potentially overlapping bindings would require a label at least for the second parameter in order to associate the type arguments with the second type parameter.

4 Likes

That's exactly what I thought!

I have an update!

In this round I rollbacked the idea of VGs being solely dynamic and restored their static-ness, but I retained the Collection interface at the value level. Please note that the preamble of the document is not up-to-date, only the "Detailed Design" section is.

I tried to be more specific, without going too deep into it, about VG's representation in the compiler and in the type system.

I feel this time we have something more interesting to talk about!

#lovingswift :orange_heart:

12 Likes

Just gave the latest version of the document a read. Looks great! I love the idea of this being a STATIC feature. I hope we are all on the same page how this limits expressiveness. For instance, you can't express infinte lists like (0...). Also, one can't exchange the underyling type of an element of the list (with dynamic existential types, you could). But these are small prices to pay for the optimizations that the compiler can do if this is a static feature.

There's one thing I would like to add to the discussion. Suppose, you want to build (for whatever reason) a function composer with a similar syntax like the view builder. Function composition requires that the input of the next function in the list is equal to the output of the current function. So, in order to write code like

Chain {
f //(A) -> B
g //(B) -> C
h // (C) -> D
}

with arbitrarily many arguments, we would need some syntax to establish relations between successive elements in the variadic list.

If we had all that plus the ability to constrain associatedtypes of opaque return types, that could solve a looot of problems :smiley:

So... when will that come? :smiley:

5 Likes

Ok, I thought about my above example and now I think that it would actually be a bad idea - at least if you do it naively.

Writing some kind of chain would look something like this:

func chain<variadic T>(_ list: T...)
-> Function<T....First.Input,T....Last.Output> 
where T : Applicable, T.Sucessor.Input == T.Output

Applicable being a protocol implemented by Function.

We see here, that we need to refer to some First, Last and Successor types. These would inevitably be some meta-optional types, i.e. types that can be there or not (as opposed to values that can be there or not). Ok, that's not a big deal, we could find ways to handle that.

Now, Function<T....First.Input, T....Last.Output> requires that the last and the first type are actually there. Ok, still not a big deal: we could create some Nonempty protocol ensuring that those types are there and having our variadic type (not the individual types) conform to this protocol.

Here's the major problem: T.Successor would inevitably stay optional. That means that the last type equality constraint would compare T.Output to a nonexisting type. In order for the above code to compile, we would need an overload for == between types that evaluates to true if one of the types isn't there. This is exactly the opposite behaviour of the overload for T? where T is equatable. This may be unexpected. Hence, a less naive approach is needed if we want to make my example work.

Hi @Anachron, unfortunately I’ve not worked on this since my last post... A lot of things have changed in my life that absorbed me “full time”, so there are no updates on VGs.

I started to implement the basics of the feature but this was almost a year ago and so much has changed in Swift since then! The code is not up-to-date and I don’t think I have the ability today to start a new implementation, and this is really unfortunate because we need to try this feature in code and not only on paper to fully grasp what’s possible and how to shape this correctly :confused:

Oh NO! Really bad news...

It seems we can't get VG full implementation even Swift 7. :stuck_out_tongue_winking_eye:

Otherwise, if there're NO updates out of one year whether VG needs to be re-proposed and reviewed again?

I am surprised no Apple engineer from the SwiftUI team has taken up a proposal for variadic generics, since the framework limits you to 10 subviews due to this. It seems most proposals need an internal Apple use case to get them across the line.

2 Likes

In that case, how can I help? I have absolutely no experience working on the compiler, but I'm an ok programmer and I'd love to learn this and this particular feature is interesting to me.

Well chains could be handled by some kind of Previous and Next keyword that could only be used in a where clause but would have to apply to all successive pairs of types, plus the possibility to require T to be a nonempty list. Perhaps the pairwise requirement seems very niche, but I'm not so sure. In a variadic list comparing successive elements is pretty much the only thing you could do.

func chain<variadic non-empty T>(_ list: T...) -> Function<T.First.Input, T.Last.Output> where T: Applicable, Next.Input == Previous.Output