Variadic Generics

This looks great!

First of all, thank you for pushing further on this! Everything seems well thought out and I find the new terminology very clear.

Is this a pragmatic limitation or a design decision? My gut feeling is that protocols having associated packs might be a useful future direction, specifically for providing an improved form of Codable.

5 Likes

Oh, and what would a pack’s Element type be?

One additional clarification request, the example above seems to be the only one which has a non-trivial pack expansion. Could you elaborate a bit on what the underlying mechanism here is, is it just "any type expression which has the pack parameter in it"? For examples, users might want to define their own functions with signatures like the following (admittedly contrived) examples.

func getTypes<T...>(_ pack: T...) -> T.Type... { … }
func map<T...>(_ pack: T...) -> T.SomeAssociatedType... where T: SomeProtocol { … }
func wrap<T...>(_ pack: T...) -> SomeWrapperType<T>... { … }

Also, would it be possible to express relationships between packs?

struct Foo<A, B> 
where 
  A: SomeProtocol, B: SomeProtocol 
  A.SomeAssociatedType == B.SomeAssociatedType
{
}
func merge(a: A..., b: B...) -> Foo<A, B>...
where
  /// A and B satisfy Foo's requirements
{ … }
1 Like

A small comment on the proposal text:

It seems clear to me from the detailed description that the only viable "spelling" for generic type placeholder symbols is the singular form, not the plural. So, T..., not Ts..., or Arg..., not Args.... "Pluralizing" the symbol works especially badly in this:

  for (item: Args) in items {

For that reason, I'd suggest changing the proposal text to use the singular forms everywhere.

On the other hand, using a plural form in parameter/keyword symbols — items: — seems correct. However, I find it takes work to grasp that xs is intended to be the plural of x. It bothered me every time, not just the first time I encountered it.

I'd suggest changing the proposal text to use an actual word instead of xs:, such as values:.

These textual changes wouldn't affect what's being proposed, but would I think make it significantly easier to read.

15 Likes

Thanks for your continued work on this, @codafi!

I remain somewhat averse to the ... spelling for reasons mentioned in the previous thread. Most concerning to me is:

IMO, although the ... syntax is nice and terse, the potential breakdown of local reasoning ability is a good reason to prefer something else.


I don't know where exactly the Swift API design guidelines sit with regard to Swift evolution, but I think that as part of the variadic generics design we should settle on some conventions which could eventually be subsumed as part of the API guidelines. In particular, I am still bothered by the interchangeable use of plural and singular terminology for generic parameter packs. It looks like this proposal has mostly settled on plural terminology (though we still have some T, U usages rather than Ts and Us), but IMO that becomes a bit problematic given:

because it means that the parameter name is used in contexts that refer to the aggregate pack (plural) as well as the individual constituent pack types. I think this makes for some confusing signatures such as:

Here, Sequences: Sequence is really saying "each constituent type of Sequences is itself a Sequence, but IMO a perfectly plausible reading of this is "the Sequences pack is a Sequence itself." This particular example is made even more confusing by the fact that the erroneous interpretation is in fact true, since the pack as a whole does conform to Sequence.

This strangeness can also be seen in:

The type of item is really a single 'arg' from the Args... pack, but the use of the plural terminology reads as if item is itself some sort of aggregate type.

I'd really like it if we could have a design which allows for explicit differentiation between the singular usage and the aggregate usage of the variadic parameter names. In the previous pitch I came up with a somewhat verbose forEach syntax that would allow for the explicit introduction of the 'singular' name of a variadic parameter:

but I'd also be fine with some sort of syntax that indicates "we're really talking about an arbitrary element of this pack, maybe something like:

public struct ZipSequence<Sequences...>: Sequence
  where Sequences[_]: Sequence 
{

ETA: @QuinceyMorris's suggestion of always preferring the singular spelling for variadic generic parameters is also compelling to me, and wouldn't require the invention of new syntax to refer to a singular element. The bare name (e.g., Arg) would always refer to a singular element, and the aggregate would always be spelled with the trailing ... (e.g., Arg...). That also agrees with the existing usage of ... for variadic parameters, for which we write the singular type name and use ... to 'pluralize' it, e.g., Int....

Regardless of which direction we choose, though, I think the proposal should take an opinionated stance on the 'right' way to design variadic generic signatures.


I'm also still wondering if we really need to introduce the new "generic parameter packs" to the language:

If we were to 'superpower' tuples, could we get away with not introducing an entirely new concept to the language? Is there a fundamental expressivity issue with tuples that prevents us from realizing everything we would want from variadic parameters?


Regarding the "only one variadic parameter in a generic type" rule, is this a restriction we want to lift eventually? If so, could we allow such types to be defined and rely on generic parameter inference to fill in each pack appropriately?

11 Likes

A couple more questions:

  1. What is the expected syntax around empty parameter packs for functions. E.g.:

    func foo<T..., U...>(vals: T..., vals vals2: U...)
    
    // impossible to call with `(T...) == ()`?
    // should implicitly empty packs be allowed?
    foo(vals: 1, 2, 3) // what are `T...` and `U...` here?
    
    // do we need a way to specify explicitly empty packs?
    foo(vals:, vals: 1, 2, 3)
    

    I suppose the answer could just be "the author of foo did a bad thing by repeating the argument label", but in that case should we just disallow such functions from being formed (e.g., "variadic parameter packs in function signatures must all have distinct labels" or something?)

  2. Similarly, what about empty parameter packs for types?

    struct Foo<T...> {
      init(ts: T...) {}
    }
    Foo<>() // ok?
    Foo() // implicitly 'Foo<>'?
    
    
  3. How do we want properties of generic parameter pack tuples to interact with the synthesized initializer? E.g.,

    struct Foo<T...> {
      vals: (T...)
    }
    Foo(vals: 1, 2, 3) // ok? or do I have to...
    Foo(vals: (1, 2, 3))
    
4 Likes

Please could you add syntax highlighting to the pull request.


Would a type-safe print only avoid boxing the items as Any, but still need runtime checks for the Custom[Debug]StringConvertible and TextOutputStreamable conformances?


How would this (and the examples in Future directions) be able to compare across different types in the pack? The < and == operator functions have Self operands.


The semi-colon delimiter (Broken<String; Int, Void>) is probably available.

7 Likes

I am assuming that something like this will be allowed:


struct X <U…> {
    private let somePack: (U…)
    init ( someInput: (U…) ) { 
       somePack = someInput // is this the forwarding ?
    }
}

Are packs just heterogeneous tuples?

I am having trouble understanding how packs ( which can be heterogeneous ) are Iterable ( for loop into them).

In the above example the Args’s type is potentially different. In todays code that would have to be Any.

1 Like

The type would be different for each argument in the pack, but each type would be known statically at compile time. I imagine it would be equivalent to writing out...

do {
    // obviously, there's no proposed syntax for getting the nth element of a pack
    // so this is just for exposition.
    let item = pack.0
    print(type(of: item), item)
}
do {
    let item = pack.1
    print(type(of: item), item)
}
// etc.

Until you've covered each item in the pack. Although the compiler would potentially do something more efficient than that.

This is great!

I’ve been looking forward to variadic generics for a long time, and I like the direction you’ve gone here.

I’m sure I’ll have more thoughts and questions after I’ve digested the proposal a bit, but for now I just want to ask about one of the listed restrictions:

I think this could be useful, especially for constrained extensions:

struct Foo<T...> {
  // something
}

extension Foo where (T...) == (Int, String, Void) {
  // some particular things
}
13 Likes

I was just thinking about this, and boom yesterday someone made a proposal. The universe is funny like that!

Specifically, what I would like to do is something like this:

func foo<Root..., Value...>(keyPaths: KeyPath<Root, Value>...) { 
    func process<Root>(_ keyPath: KeyPath<Root, Int>) { ... }
    func process<Root>(_ keyPath: KeyPath<Root, String>) { ... }
    /// Gets called if value isn't `Int` or `String`.
    func process<Root, Value>(_ keyPath: KeyPath<Root, Value>) { ... } 
    for keyPath in keyPaths {
        process(keyPath)
    }
}

Can we update your proposal such that this would be handled, or would it be non-feasible?

You've discussed "packs" at length but not given an example of how a pack is declared in Swift. What is it exactly?

E.g.

struct Pack: Packable { ??? what goes in here ??? }

Questions about packs:

  1. How is pack defined exactly?
  2. How is a pack different from a collection exactly?
  3. How can you programmatically access the individual members of a pack?
  4. How does a pack interact with reflection and debugger introspection?
  5. Why is it called a "pack" (a word that usually refers to a group of things of identical type to one another, like a pack of cards or a pack of wolves) as opposed to using a word that better reflects the fact that it can be a group of completely diverse types, such as "medley"?
  6. If "pack" refers to the group of generic types in the generic params, then what refers to the type of the collection of various-typed objects in the function parameters...? (Are these also a pack, or still a collection?)
  7. how do you deal with capture groups and packs?

E.g.:

func foo<T..., U...>(_ t: T..., u: U...) {
    let myClosure: () -> Void = { [t, u] in
        ...
    }
}
  1. What types are captured t and u inside the closure? I.e. is t a "pack" or a "collection"?
  2. Is t a copy of the original pack t or is it the same instance, just retained, because packs are reference types?
  3. What happens if you print(type(of: t))?
  4. What if some T's and U's are reference types and others are not... ?
  5. Can weak modifier be applied to the capture group argument in a way that applies only to objects among the various things in t or u?
  6. Does this proposal require any dynamic behaviors or can it rely purely on static behaviors as current generics do?

Please disregard if you've already discussed elsewhere how these "packs" deal with memory concerns when they contain some value types and some reference types.

How about just allow variadic generic parameters to have parameter labels, e.g.:

struct Broken<A: T..., B: U...>
typealias Bar = Broken<A: String, B: Int, Void> 

... in which case, the type of A is something like, PackWrapper<T...> and the type of B is PackWrapper<U...>. Thoughts?

1 Like

First things first, thank you very much for this updated proposal on the subject!

Then, what if we treat "packs" as Collections, with a capital "C"? You can have all the benefits of map & co. (and even more, if we can / want to give them conformances to BidirectionalCollection and / or RandomAccessCollection), and moreover they can be blessed with a little compiler magic so that:

  • in static contexts the compiler knows order and position of all the members, and maybe can allow direct access using (checked?) indexed or dot notation
  • in dynamic / completely generic contexts you can rely on Collection operations to do all the necessary work

In this way you are not forced to immediately convert Variadic Generics to tuples, but can do so if needed, and you need not introduce a "special" map operation.

Speaking of map, I think the operation should take into account mutability: as currently defined you cannot, for example, invoke Iterator.next() because it is a mutating method. However, would we define the parameter to be inout, I'm not sure about map being the correct name (not sure about the relationship between inout and pure functions).

5 Likes

Thanks for actively working on this, @codafi! This is a long-awaited feature.
A couple of comments:

This example requires all Ts in the pack being the same type, as Comparable has Self type requirement, so this is not a case of variadic generics, but a regular generic function with a variadic parameter:

func max<T>(_ xs: T...) -> T? where T: Comparable

The variadic generic format would allow writing max(“Hi”, 42, 3.14, Date()), which technically satisfies the function signature (since all of String, Int, Double, and Date do conform to Comparable), but they’re all of different types, therefore the implementation of the function max is semantically impossible.
Perhaps a better example would be to constraint T… to Numeric instead.


I also find the idea of pack expansions a bit problematic. Apart from the fact that it could produce single element tuples, makes the syntax T... ambiguous for the reader, because if refers to different concepts in each position. In the first position (between the angle brackets), it signals that this generic parameter is something that is not a concrete, single type, but a sequence of those. The second (in the parameter position) is the already existing variadic syntax. While the last tuple-like one refers to a concrete return type of an anonymous sequence of existential types. This mixture can become a source of confusion. I’d prefer if these 3 occasions were written differently, as they stand for different concepts. For example

func tuple<T*>(_ xs: T...) -> pack T // similar to some T and any T
1 Like

I'm happy this is moving forward. I read the current proposal and came away with the following 3 questions:


It sounds like in func buildBlock<V...>(views: V...) -> (V...) where V: View, each V can be an independent type that conforms to View (all views don't need to be the same type). However, right at the start of the Proposed Solution section, we see the following example:

func max<T...>(_ xs: T...) -> T? where T: Comparable

which isn't workable if each T is allowed to be separate since Comparable can only compare Self objects.

It seems to me that max could be currently written as func max<T>(_ xs: T...) -> T? where T: Comparable (except that it would use monomorphic variadic parameters instead of the new parameter packs), so I'm guessing that the first interpretation is right and this example needs to be updated?


How do you explicitly specify the generic parameters of functions with multiple variadic generic parameter packs? In double<Ts..., Us...>, if I want a "function pointer" closure to double that has Ts=(Int, Int) and Us=(Double, Double), how do I spell that? double<Int, Int, Double, Double> is ambiguous for the same reasons that types are.


How does iteration work with protocols with associated types? Is for (item: Args) in items magic, given that item: Args is invalid typing as of the current Swift? (Are there interactions with SE-0335 and SE-0309?)

2 Likes

You cannot do that even with normal generic functions.

If you want to reference a generic function you have to use either as or a type hint, e.g.:

func foo<T>(_ t: T) {}

let foo2 = foo<Int> // Error: Cannot explicitly specialize a generic function

let bar = foo as (Int) -> ()
// or
let baz: (Int) -> () = foo

However, this breaks down as well for functions with multiple generic parameter packs:

func foo<T..., U...>(ts: T..., us: U...) {}

let bar = foo as (Int, Int, Double, Double) -> () // What is T and U supposed to be?
// or
let baz: (Int, Int, Double, Double) -> () = foo // Same problem

So the problem remains that we could not reference such a function under this proposal.

3 Likes

How about just allow variadic generic parameters to have parameter labels, e.g.:

struct Broken<A: T..., B: U...>
typealias Bar = Broken<A: String, B: Int, Void>

... in which case, the type of A is something like, PackWrapper<T...> and the type of B is PackWrapper<U...> . Thoughts?

Why not directly have the ability to (optionally) specify the parameter name in the specialisation (this may also be generalised to all generic parameters), e.g.

struct Broken<T..., U...>
typealias Specialised = Broken<T: Int, String, U: Double, Void, String>
10 Likes

The problem with treating them as Collection or more accurately RandomAccessCollection is that the Iterator would need to have some sort of annotation of the return type of next that permits the heterogeneous type signature that may be applied. Consider a pack of Numeric things: that could be Int, Int32, Int8 and so on. The iterator would need to have some way of addressing that return type to be different per iteration call. Maybe some types might work?

It would be reasonable however to still access a pack in an indexed fashion. I have distinct need for this behavior for similar APIs to zip.

One could easily conceive of count being implemented as:

func count<T...>(_ items: T...) -> Int { 
  var count = 0
  for _ in items { count += 1 }
  return count
}

But more complex algorithms need some more distinct access: for example making a combinatoric of states:

Lets say you have some types that are operations:

protocol Operation { 
  associatedtype Output
  func run() async -> Output
}

Then you want to pass a variadic of operations into a structure and for each call invoke the output resultant of that operation that happened next out of the group. This means there are a few different states associated with that - they all could be idle, the first could be actively running in a task and the rest could be idle, the second could be actively running and the others would be idle and so on, and then the final state is terminal.

Given a non variadic situation I would write this as for two items as an enum as such:

enum State {
  case allIdle(Operation1, Operation2)
  case firstPending(Task<Operation1.Output, Never>, Operation2)
  case secondPending(Operation1, Task<Operation2.Output, Never>)
  case terminal
}

The proposal as such does not seem to address this type of usage. Indexable packs might be able to allow for this type of affordance (perhaps with a downside of some states that are unreachable).

In that same vein of "higher kinded type" variadics - if the example of zip was to store elements for asynchronous processing of a temporary; would there be a way to map types across?

Say I needed to store the iterators and some values to compose the resultant. How can I map the type of the pack of iterators to a pack of corresponding Iterator.Element?

From what I can tell this does not allow for that - which seems like a missing part that still poses me reaching for .gyb files.

1 Like

You might be right, but I thought that when using Iterator, forEach and the like you only get a view on items based on the constraint, e.g. if you have

struct SomeStruct<Ns…> where Ns: Numeric {
  let numbers: Ns
}

let v: SomeStruct<Int, Double, UInt32> = …

then iterating over v.numbers will give you instances of Numeric, but a “direct access” (with some unspecified syntax) will give you the exact type.

If that was not the case, how would I even write the body of the iteration to use the specific type of the “current item” in a more interesting way than generically? At the moment (I just woke up) I can’t imagine :thinking: