Variadic Generics

Evolution Pitches
42 
func foo<T>(_ t: T) -> [T] {
    Array(repeating: t, 42)
}

Does that count? I wouldn’t expect unconstrained generics to be common, for precisely that reason.

1 Like
43

That's true, that example is definitely broken. But his if we take the proposal's (as far as I can tell correct) implementation of zip then you can implement this:

func == <T...>(lhs: (T...), rhs: (T...)) -> Bool where T: Equatable {
    for (left, right) in zip(lhs, rhs) {
        guard left == right else { return false }
    }
    return true
}

Which would not only replace six different handwritten implementations of tuple equality in the standard library, but would also allow the compiler's synthesized conformance of tuples to Equatable to support any arity.

Note: it doesn't matter that each value in the pack is a different type, because both the left hand side and the right hand side use the same pack.

3 Likes
44

You can do the same with variadic generics:

func foo<T...>(_ ts: T...) -> ([T]...) {
    ts.map { (t: T) -> [T] in
        Array(repeating: t, 42)
    }
}

let arrays = foo(5, "Hello") // arrays has type '([Int], [String])'
print(arrays.0) // prints "[5, 5, ... , 5]"
print(arrays.1) // prints "[Hello, Hello, ... , Hello]"

let arrays2 = foo(5, "Hello", true) // arrays2 has type '([Int], [String], [Bool])'
print(arrays2.0) // prints "[5, 5, ... , 5]"
print(arrays2.1) // prints "[Hello, Hello, ... , Hello]"
print(arrays2.2) // prints "[true, true, ... , true]"


let arrays3 = foo() // arrays3 has type '()' or 'Void'

There is definitely no type erasure going on here...

Obviously this needs a special map function but that is also part of the proposal.
But, IMHO, the syntax of the map function feels not completely right. It is definitely a function that could never be expressed in Swift (like the move function from the other pitch thread), because the body closure gets different types somehow. I'm currently thinking about how we could express such an operation better...

45

Even this example would have to be explicitly covered by the compiler, as left and right would change types with each iteration and thus violates the type system.

Equatable was specifically noted as an example where this sort of thing is impossible, as recently as last May.

46

What would have to be covered by the compiler is not the whole function but only the special for loop. It's just not a normal for loop looping through a normal Sequence, but another kind of for loop that gets another type in each iteration. Just like we got a new type of for loop for AsyncSequences, we would need one for parameter packs.

1 Like
47

But they didn’t get a special for loop, did they? That’s just shorthand for using the AsyncIterator, which simply requires an asynchronous version of the conventional Iterator’s next method.

Now, in theory, if the caller stored the type information (which would have to be public) along with each argument, the implementation could consist of attempting to cast each value to the stored type, then proceeding with the comparison if successful. But that would be abysmally slow, generally useless, and still type erasure.

48

It's just the case that the default for loop as well as the async for loop are just syntactic sugar for creating an (Async-)Iterator, and then calling next on it in a while loop until it returns nil. In other words a normal for loop can be completely replaced with other swift code.
A parameter pack for loop cannot be replaced by other swift code and would therefore also not be syntactic sugar for anything. But that doesn't mean, that it should not exist...

1 Like
49

Now returning to actual feedback about the pitch at hand:

I'm still somewhat uneasy about the map function on parameter packs, so I have two alternatives to offer:

Firstly we could go the way of c++ which allows basically every expression containing a parameter pack to be expanded as a whole (the first entry in the alternatives considered of this pitch).
Some examples from the pitch would look like follows (I chose ... here because that's how you do it in c++, but we probably would have to use another postfix expression, as this one is already used in Swift, as is noted in the pitch, so don't take the examples too literally):

func flattenAll<T...>(_ arrays: [T?]...) -> ([T]...) {
    (arrays.compactMap { $0 })...
}

func forwardWithABang<T..., R>(_ args: T?..., body: (T...) -> R) -> R {
    body(args!...)
}

For consistency we would probably need to use this postfix operator (whatever it might be) for normal forwarding as well:

func tuple<T...>(_ ts: T...) -> (T...) {
    ts...
}

func forward<T...>(_ ts: T...) { return onwards(ts...) }
func onwards<T...>(_ ts: T...) { /**/ }

However, I really dislike that in these expressions the value on which you operate is not the pack but only one of its values but you use the name of the pack regardless. E.g. in flattenAll the value on which you call the compactMap method is not actually arrays, but instead you call it for each value inside of arrays separately.

Therefore I propose another alternative which makes things a lot clearer, IMHO: We could basically use the syntax of python's generator expressions for this.

Here are the same examples as above but with this syntax instead:

func flattenAll<T...>(_ arrays: [T?]...) -> ([T]...) {
    (array.compactMap { $0 } for array in arrays)
}

func forwardWithABang<T..., R>(_ args: T?..., body: (T...) -> R) -> R {
    body(argument! for argument in args)
}

With this approach we could also use forwarding as it is described in the pitch.

IMHO, this way really clearly conveys what is being done, while not needing a magic map function that could never be written in Swift itself.

50

So far we've only been talking about tuples. Let me promote a different use-case.

Take a look at any popular dependency injection framework out there, which uses type information to resolve instances (e.g. Resolver or Swinject). They usually register instances into their registry like this:

register { resolver in
    MyService(dependency1: resolver.resolve(), 
              dependency2: resolver.resolve(), 
              dependency2: resolver.resolve()) 
}
Class definition 
class MyService {
  init(dependency1: Dependency1, 
       dependency2: Dependency2, 
       dependency3: Dependency3) { ... }
}

// this class has an initializer
let initializer: (Dependency1, Dependency2, Dependency3) -> MyService = MyService.init(dependency1:dependency2:depdency3) // or just MyService.init for short

Here, resolve is a generic function on a Resolver class of the DI framework that uses return type inference to query an instance for a particular type. func resolve<T>() -> T

Instead of resolving each parameter manually, we can define a helper method on the Resolver to call resolve for each parameter and return an instance.

extension Resolver {
    func resolve<T, P1, P2, P3>(_ factory: @escaping (P1, P2, P3) -> T) -> T {
        factory(
        	resolve(),
        	resolve(),
        	resolve()
		)
    }
}

This will let us use the initializer or any factory method or closure as its parameter. We can use it like this, with a much more lightweight syntax than before:

register { resolver in
    resolver.resolve(MyService.init) // the type of MyService.init is `(Dependency1, Dependency2, Dependency3) -> MyService` so T is substituted with MyService, P1 is Dependency1 and so on.
}

If we define a helper function for each arity (similarly to what ViewBuilder.buildBlock does), we can make this a future proof helper function, which will adopt the growing number of dependencies of the initializer automatically. The implementation remains similar, it will only contain more resolve() calls in its body.

Given that this list of static types in the function signature is not a tuple but part of a closure, how would this idea fit under the variadic generics theme? As far as I'm aware there's currently no way to turn a tuple into a closure's parameter list.

I can only theorize on a syntax.

extension Resolver {
    func resolve<T, P...>(_ factory: @escaping (P...) -> T) -> T {
        factory(P.self) // ???
    }
}

I have a feeling that variadic generics is a much broader idea than just dealing with tuples.

6 Likes
51

That’s a classic example of when to use a result builder, but I wouldn’t say that can’t be expressed in the form of tuples.

I’m not a compiler specialist, but I don’t think it is possible to make an interface for an unknown quantity of heterogeneous parameters without first reducing them to a common type (that is, performing type erasure).

I feel it is worth noting that SE-309 will make existentials less constrained than they currently are. They should still be avoided if possible, and contravariant use of Self will still make them unusable, but they remain a potent last resort.

52

Never say never… I think a map over a pack could be written in Swift, with some additional general-purpose features:

  • If we could split a concrete pack and its corresponding pack type into a head and tail, and join a value with a pack to produce a left-extended pack type, the map could be written recursively. (Incidentally, these operations would be a natural extension of tuples, if a good syntax could be found.)
  • The transform closure for the map would need to be a generic closure, inheriting the constraints of T and U.
extension <T...> (T...) {
    // This spelling is from the pitch, but transform is effectively a <T2, U2>(T2) throws -> U2 where
    // constraints on T... and U... are somehow propagated to T2 and U2
    public func map<U...>(_ transform (T) throws -> U) rethrows -> (U...) {
        // if packs are tuples, this is equivalent to if (T...).self == Void.self.
        if _Arity(T...) == 0 { return () }

        let mappedHead = try transform(_Head(self))
        let mappedTail = try _Tail(self).map(transform)

        return _Join(mappedHead, mappedTail)
    }
}

(If packs are tuples, this is also a generic map over tuples, although T... would effectively be Any... in heterogeneous non-pack use cases)

2 Likes
53

You're absolutely right.

However, the issue still remains that we will need the behaviour of the map function from the very beginning or otherwise there really isn't much value in this pitch, as you cannot do much with variadic generics without it. That poses the problem though that we cannot even write the signature of this function with the features that will be included in this pitch. We'd at least need parameterized extensions and generic closures for that, both of which are way out of scope for this proposal.

Sure, we could just add the map function for packs even if we cannot write its signature in Swift, but, AFAIK, this would pretty much be unprecedented in the language...

54

AFAIUI, this bit of code:

should actually mean that A is being expanded like this:

Otherwise there wouldn't be an equal-arity constraint between T... and U....

I would suggest that such a construct which needs an equal-arity constraint should only be allowed, where this constraint is already established.

In other words, I would think that the example should only be allowed if it is written e.g. like this:

struct X<U...> {}

struct AmbiguityWithFirstMeaning<T...> {
    struct Inner<U...> where (T...).count == (U...).count {
        typealias VariadicFn<V, R> = (V...) -> R
        typealias A = X<VariadicFn<T, U>...>
    }
}

or like that:

struct X<U...> {}

struct AmbiguityWithFirstMeaning<T...> {
    struct Inner<U...> {}
}

extension AmbiguityWithFirstMeaning.Inner where (T...).count == (U...).count {
    typealias VariadicFn<V, R> = (V...) -> R
    typealias A = X<VariadicFn<T, U>...>
}

(with the difference being that the first version requires U... to always have the same arity as T... while the second version only makes A available when that is the case, but allows other uses of Inner where the arity doesn't match).

About implicit equal-arity constraints for functions I'm a bit uncertain.

E.g. we have one in this example from the pitch:

We should probably allow this and, as a bonus, dropping the where T: Hashable should probably also be allowed, to be consistent with current implicit constraints for normal generic functions...

55

Just found a few more examples in the Combine framework for the Motivation section:

  1. struct CombineLatest4<A, B, C, D> docs link

  2. struct Zip4<A, B, C, D> docs link

  3. the various merge(with:::::::) functions where each one returns a corresponding type
    struct Merge8<A, B, C, D, E, F, G, H> docs link

  4. struct MapKeyPath3<Upstream, Output0, Output1, Output2> with properties keyPath0, keyPath1, keyPath2 docs link

I imagine the implementations are all duplicates of the ones with lower arity.

3 Likes
56

I'd advocate for generic parameter labels (as well as default values) being added as just a general language feature, aside from their utility in this context:

Off-topic example of how I'd like to use them 
struct Matrix<Rows M: SomeLengthType, Cols N: SomeLengthType, Of Value: SomeAppropriateConstraint = Double> { /**/ }
struct Vector<Length N: SomeLengthType, Of Value: SomeAppropriateConstraint = Double> { /**/ }

// Yes, I know integer literals can't conform to a protocol... I still have hope for this syntax working some day
let a = Matrix<Rows: 5, Cols: 6>() // 5 by 6 matrix of Doubles
let b = Matrix<Rows: 3, Cols: 3, Of: Vector<Length: 3, Of: Int>>() // 3 by 3 matrix of length-3 Vectors of Ints

Anyway, is the lack of a delimiter the only reason the we're limited to one variadic generic parameter pack per type? If so, would adding generic parameter labels and allowing an arbitrary number of variadic generic parameter packs per type be in-scope for this proposal, or should that be a follow-on proposal?

6 Likes
57

I'd really love to get variadic generics support in Swift, so a big :+1: from my side on the direction.

I can't really discuss on the detail level (as I don't understand compilers & grammar well enough), but from a Swift users point of view I find the T... syntax kind of confusing. It's kind of easy to confuse them with variadic parameters which look exactly the same, but have a different meaning. I kind of prefer either T* or even the very explicit variadic T over T... on first impression basis.

I personally think of the variadic parameters of something like "an array written without brackets". If I apply this to generics from a Swift developers point of view who has never seen variadic generics before, let's say a method func doSomething<T...>() -> T then I would expect the return type to be some kind of collection (and hence I'd ask myself why it doesn't just return an array, which is effectively the same on return types).

From this point of view, I actually consider any T to be the most intuitive name. For example:

func max<any T>(_ xs: T...) -> T? where T: Comparable { /**/ }

To me this would intuitively tell that "any type T that conforms to Comparable" is accepted.

1 Like
58

What about something like :

func max(_ xs: many T) -> T? where T: Comparable

edit: the code is missing the generic brackets and should be

func max<T>(_ xs: many T) -> T? where T: Comparable
1 Like
59

Personally, I think Swift 5.6’s paradigm is sufficient for expressing generics:

  1. ExampleProtocol is a protocol (not an implementation thereof)
  2. some ExampleProtocol is a specific implementation of a protocol (this is almost always what you want, and most forms of generic programming traffic in it)
  3. any ExampleProtocol is an existential: for each value, an implementation of a protocol exists. This should be avoided whenever possible, as it immediately reduces everything to the lowest common denominator.

Since Comparable only requires support for comparing values of the same type, that must be a form of #2. In fact, you could express it right now:

func max<T: Comparable>(_ xs: T...) -> T?

Note that the result being optional is due to a current language limitation: it’s nil iff zero values are passed, and there’s no way to impose arity constraints of variadic parameters. It’s possible to work around this, and the Standard Library actually does just that[1]:

func max<T: Comparable>(_ first: T, _ rest: T...) -> T

  1. The Standard Library actually has four parameters, the last of which is variadic. I believe the second is required because there’s no real reason to call it with one parameter (even if it would be technically correct). I think the third non-variadic parameter is required as a performance optimization: calling it with two values is extremely common, and a third variadic parameter would make that ambiguous (you could be calling the function with two values and a zero-arity variadic). ↩︎

60

My suggestion was about replacing the suggested ... with many, not a new generic paradigm.

func max<T: Comparable>(_ first: T, _ rest: many T) -> T
3 Likes
61

I’ve been thinking about equal-arity constraints for generic parameter packs, and I’m not sure we should make it implicit. I think it would be better to have an explicit syntax for specifying “these variadic generic parameters must have the same number of types”.

To that end, perhaps something like this is worth considering:

struct Foo<(T, U)..., V...> {}

That would mean the parameter packs T and U must have equal arity, but V has no such constraint.

• • •

Another option would be a magical “Arity” (or Count) property on parameter packs, so you could write the constraint:

where T.Arity == U.Arity
2 Likes