Variadic Generics

Evolution Pitches
82

The proposed new call-me-anything-except-a-real-tuple type seems a bit complicated to me. Someone earlier in the thread suggested that variadic tuples - on which variadic generics could be built - would be more straightforward, and I agree. They also seem more broadly useful and to require less change to the language (at least from a Swift user’s perspective - I have no idea what it means for the compiler’s implementation).

An example is the proposed overloading of the . operator to mean map only on these special not-a-real-tuple-tuples. That’s a bit magical to me - why is that necessary? Why not just have these new things (whether tuples or whatever) just have a map method, so that they are more consistent with existing Sequences. In fact, my (perhaps naive) question is why can’t these collections - tuples or otherwise - of metatypes genuinely conform to Collection, and support all the normal methods?

Or alternatively, if we want some syntactic sugar that translates .map { $0.foo } to something shorter, how about a proper new operator that then also applies anywhere else, e.g. self.values..foo or somesuch? (riffing, syntactically, on the pointwise operators introduced for SIMD recently, which have defined . as also an operator modifier, essentially, so the prior example reads as “self.values pointwise .foo”) That avoids ambiguity to the reader as to whether “.foo” is referring to a property of the tuple itself.

If it helps for understanding where I’m coming from, I’ve been doing a lot of metaprogramming in Python lately, and for all Python’s warts I do find its metatyping system to actually be pretty elegant, particularly w.r.t. to the fact that its metatypes (by which I mean instances of the type type) can be treated just like any other object. Maybe something, in Swift, like:

// “T” becomes a tuple of 0 or more Metatypes, each of which represent
// a type that conforms to Sequence (but are not necessarily the same
// types).
struct Zipped<variadic T: Sequence> {
  private let values: T // used as a type here, in a property declaration...

  // ... and here in a function signature
  init(_ values: T) {
    self.values = values
  }

  // Used here directly as an object.
  static var typeName {
    return “Zipped<“ + “, “.join(T.map { “\($0)” }) + “>“
  }

  var description {
    return self.typeName + “(“ + “, “.join(self.values.map { “\($0)” }) + “)”
  }
}

Maybe it makes more sense if you presume constexpr support too… e.g. if you think of the variadic types as constexpr tuples.

1 Like
83

Re. the syntax for variadics (... vs #pack vs variadic etc), has borrowing from glob or regex syntax been considered? I’m spurred in particular by the notion that a variadic would have a length of zero or more, which is useful sometimes but other times makes for a lot of boilerplate in having to write, if you want to require at least one parameter, something like:

func foo<T1: Sequence, variadic T2: Sequence>(_ first: T1, _ rest: T2) where T1 == T2 {
  // Or whatever the syntax is for meaning T1 must be the same type as T2.
  ...
}

What if instead you could write:

func foo<T: Sequence+>(_ values: T) {
  ...
}

In cases where you do want to allow for zero parameters:

func foo<T: Sequence*>(_ values: T) {
  ...
}

Then more powerfully, where you require at least two but not more than eight:

func foo<T: Sequence{2,8}>(_ values: T) {
  ...
}

Or as a shorthand for a specific number of parameters:

func foo<T: Sequence{4}>(_ values: T) {
  // I require exactly four parameters, all conforming to Sequence.
  ...
}

// Today this would have to be:
func foo<T: Sequence>(_ value1: T, _ value2: T, _ value3: T, _ value4: T) {
  ...
}

// Or (depending on your true intent):
func foo<T1: Sequence, T2: Sequence, T3: Sequence, T4: Sequence>(_ value1: T1, _ value2: T2, _ value3: T3, _ value4: T4) {
  ...
}

There might be a distinction between using the modifiers on the generic name’s declaration vs in the parameter list - to me the former suggests T could be distinct types sharing some common trait (e.g. all Sequences), while the latter could be more naturally a way to say a given argument is actually a tuple of some number of elements all of which are the same type.

I haven’t thought it through further than that, but even if it doesn’t prove sensible, I recommend it be included in the ‘Alternatives considered’ section for future reference.

2 Likes
84

@wadetregaskis
Regard your first post, I think that in another thread there was a discussion about the variadic-are-tuples argument. It seems to me that there is desire (and advantage) to keep built-in tuples and this new concept of "variadic-tuple" separated.

What you suggest I briefly mentioned in the #head, #tail, #length, #reduce, #ifempty section of the latest version of the document. The problem I mention there (conflict between the built-in Collection methods of the variadic tuple and members with the same name on the common supertype) obviously only occurs if we have the magic . syntax access, which you suggest to remove. I have not explored a design in which variadic tuples are only Collections and nothing else, it might be interesting to reason about.

About your second post, personally I think that the gains we might have using glob syntax are not worth the implementation required, especially because I cannot find (but that's just me) a reason to limit the number of variadic parameters in a specified range. The * vs + to indicate 0... or 1... is interesting, but I'm not sure if a single character really stands up to indicate what's happening (variadic generic plus specification on the minimum number of parameters required). Maybe if this is a real problem while designing APIs we should find another solution.

Sure your suggestions may be worth the addition in the Alternative Considered section! Thank you for your feedback!

1 Like
85

Instead of tweaking the generic parameter's text with a regex, we could move the restriction to a where clause.

func fooAtLeastOne<T: variadic Sequence>(_ values: T) where #length(T) >= 1

func fooBetween2And8<T: variadic Sequence>(_ values: T) where 2...8 ~= #length(T)

func fooExactly4<T: variadic Sequence>(_ values: T) where #length(T) == 4

func fooExactly4Identical<T: Sequence, U: variadic Sequence>(_ values0: T, _ values: U) where #length(U) == 3, U == T

where the appropriate operators are constexpr. The default is "#length(Whatever) >= 0," of course.

1 Like
86

Requiring specific lengths of variadic parameters makes the type system a whole lot more complicated for very little gain, IMO.

3 Likes
87

I wasn't necessarily saying it's a good idea, but that the way I gave to express it is much better than a surprise!regex syntax at the end of a formal parameter. At least the way I gave it works with the current parser and expands on the user's knowledge of Swift; the surprise!regex way does neither.

1 Like
88

Indeed, that works too. I can understand an aversion to regex-style syntax, though I do find its brevity very appealing (Swift type definitions are becoming dangerously verbose as it is, with stacks of keywords and where clauses and all this other stuff).

An advantage of the more flexible where clause approach is that you could then presumably construct arbitrarily complex length constraints - e.g. where #length(T) % dimensions == 0 (where dimensions would presumably be some other compile-time parameter).

Though, I really don't like the #pragma syntax - IMHO it's ugly and C++y. More objectively, it's more cognitive burden since it reuses less of existing syntax & concepts. It'd be much simpler to have T.length instead.

Though, one downside of T.length is that it could be misconstrued as meaning a class method / property of T named length. Having distinct syntax does reduce the potential for that confusion. Though not entirely - #length(T) could still plausibly be misunderstood as just some weird syntax equivalent to T.length. Either way there remains a problem that it's not as clear as it could be that T is a collection of types, not a singular type.

So I've been toying with syntax revolving around that concerns, with approaches like:

// 'T' is more clearly a collection of types conforming to Sequence,
// but the fact that 'values' is a variadic tuple of those types is less explicit.
func sum<T: (Sequence...)>(_ values: T) where T.length > 1 …

Or:

// 'T' is now clearly a _singular_ type, and 'values' is more explicitly a
// tuple, but it's also now implied that 'values' is a tuple of caller-determined
// length but with the _same_ type in every position.
func sum<T: Sequence>(_ values: (T...)) where values.length > 1 …

(note also that this is a realistic example where it's useful to have variadic length options beyond "zero or more" and "one or more")

I haven't thought them through much further than that, though, and as the comments note they're all imperfect. But they feel more aesthetically pleasing to me, at least. Maybe there's a better alternative or compromise that I'm not seeing yet.

P.S. Another concern with the 'variadic' keyword as suggested is that it's unnatural grammar in English, where adjectives precede nouns. It's implying that T is of type of some variadic Sequence - i.e. the 'variadicness' is an attribute of the Sequence, not of T. Though easily fixed by reversing the order, i.e. <T: Sequence variadic> or <variadic T: Sequence>. I still prefer a syntax which makes it more explicit that T is a collection of types, though. Following the principle of "things should look like what they are".

1 Like
89

If I can bike-shed for a moment, it seems like it really should be #count(T) or T.count since Swift has never used length for the number of items in a collection.

I see an advantage to the #pragma style in that it is a departure from current Swift syntax and concepts. It seems to me that variadic generics is a foray farther into metaprogramming than Swift currently has done, and I think it's a good thing to have sufficiently differentiated syntax to clearly indicate the additional level of abstraction.

2 Likes
90

My original suggestion was "#count." My philosophy in using a # operator is that the variadic item is only in the compiler's imagination; it doesn't (directly) correspond to something that will end up in the binary. The conditional compilation items are the same way. It's the same reason I suggested "#explode" for translating an Array object to its individual elements to pass in a T... parameter; T... isn't (and shouldn't be) a real type with a real calculus, it's just a convention and #explode would be a meta-conversion.

2 Likes
91

I want to add another bit of information: what if we have variadic values actually conform to Collection?

Access to shared variadic members will occur like it does with dynamicMemberLookup, and in case of conflict (e.g. let's say we have a Variadic Generic of Collections and we want to access the count property of the wrapped collections) a new property wrappedValues (placeholder name here) can be used to disambiguate the access:

let vg = createVariadicGenericOfCollections([42], [:], "Hello")

// This accesses the `count` property of `vg`'s `Collection` interface
// and returns 3
// Also note that in this way we are not going to need `#count` and
// other `#magicStuff` anymore
vg.count

// This accesses the `count` property of the wrapped `Collection`s
// and returns a new variadic value containing the values {1, 0, 5}
vg.wrappedValues.count

// This works out of the box like it does for any `Collection`
for collection in vg {
  // Use `collection`
}


What do you think? Potential benefits / drawbacks?

1 Like
92

Collection requires that all Element be of the same type.

// This works out of the box like it does for any `Collection`
for collection in vg {
  // Use `collection`
}

What the collection type is supposed to be ?

93

I think that it should be the type to which you constrained the VG or Any if no constraint was given:

struct NonConstrainedVariadic<variadic Ts> {
  let ts: Ts

  func test() {
    for elem in ts {
      // The type of `elem` is `Any`
    }
  }
}

struct ConstrainedVariadic<variadic Cs: Collection>
  where Cs.Element == Int {

  let cs: Cs

  func test() {
    for elem in cs {
      // The type of `elem` is `Collection` (of `Int`s)
    }
  }
}
94

Is this wrappedValues necessary? In what situation would it be superior to the existing map?

95

Only to be consistent, I guess... I thought it would be weird that you can directly access all members of the base type but some (i.e. those shadowed by Collection).

But maybe we should not include this dynamic member access at all and only rely on map at this point?

96

I know it has been a while since this thread has been active. I just wanted to ask if you have any update on the state of your variadics generics proposal?

Also, I want to commend you on the hard work that has already went into this so far, though I do not possess extensive knowledge regarding the internal workings of the compiler, I have read enough about variadic generics in Swift in anticipation of this feature to know about the extreme complexity required to implement such a major change to the Swift generics model, so ya... keep on keeping on :wink:

5 Likes
97

TL;DR No, as of today I have not made any progress proposal-wise or implementation-wise.

Long answer: unfortunately I had health issues, I had (have) a disk herniation that blocked me from doing anything, even the most basic tasks, because of the pain. These days I am feeling better and I'm event thinking to go back to my work a couple of days before the Christmas holidays, but the full recovery is still going to take a long time.
Moreover, me and my girlfriend (sounded so childish :sweat_smile:) partner were thinking to move together but obviously this task had to be deferred because of my health problem. But the time has come, and from a couple of weeks until the end of the holidays (at least) we're going to be working on this.

So sad as all of my problems are, there actually are some news on VG's.

I have been following all the new topics: SwiftUI, View Builders, Generics UI improvement, etc. The existence of Group in the new UI library is yet another evidence of the need for this feature, and the new advances in Swift might help us better implement VG's.
What I can say is that in this span of time I've been thinking of a new way to represent Variadic Generics "inside" the generic scope, one that can better integrate in how Swift works, and I hope to clean my mind and update the document during the holidays (but, no promises!).

One more thing: is Apple "secretly" (i.e. internally) working on Variadic Generics, due to the lack of updates on this topic? Maybe someone like @Joe_Groff can give us some hints about that!

18 Likes
98

I thought about variadic generics recently because it was anounced that Swift 6 will have this function at here.

I found this thread and read draft of proposal.
Then I want to improve expression of curry.

One existing idea is this.

func curry<A, B, C, variadic D, Result>(_ fn: @escaping (A, B, C, D...) -> Result) -> ???

It seems to infer type of ???.
But it is not appropriate for Swift which always describes signature of function.

Another is this.

enum CurryHelper<variadic T, Result> {
#ifempty T
  typealias Fn = Result
#else
  typealias Fn = (#head(T)) -> CurryHelper<#tail(T), Result: Result>.Fn
#endif
}

I think its good for point of that it is succeeded to express.
But rule of evaluating #ifempty happens when application of generic parameters is not consistent with existing evaluation of #if.
#if is evaluated statically only once if it is used in declaration like enum.
This is confusing behavior for programmer.

So I propose #condtype as a new operation to represent branching in compile time.
It can be expressed as below with this.

enum CurryHelper<vairadic T, Result> {
  typealias Fn = #condtype(#ifempty(T), 
    Result,
    (#head(T)) -> CurryHelper<#tail(T), Result>.Fn)
}

Or we can eliminate namespace by CurryHelper.

typealias CurryFn<variadic T, Result> = #condtype(
    #ifempty(T), 
    Result,
    (#head(T)) -> CurryFn<#tail(T), Result>
)
1 Like
99

Lo and behold! The new revision of the document is ready!

@omochimetaru unfortunately, at least for the moment, I've removed the "support" for curry from the proposal...

10 Likes
100

Bike-shedding ahead.

The "variadic" part should precede the rest of the formal parameter declaration. That way, when using a parameter without a constraint, we use a "variadic T" instead of the more awkward "T: variadic Any". (Although your code uses the latter, the formal grammar description uses the former.) After all, it is the "T" part being multiplied, not the constraint.

4 Likes
101

Agree. I haven’t made it through the whole revised draft yet, but this jumped out at me in the bit I did read. It took me a while to figure out how unconstrained variadics were intended to be spelled. Is there a reason for the current design placing variadic before the constraints instead of before the type parameter?

2 Likes