Starting to implement Variadic Generics

(Andrea Tomarelli) #6

That is super great advice!
I think I understand how I can start, but I have a question (for now).

When you mention the GenericParamDecl node to wrap the generics stuff do you actually mean a new entry to DeclNodes.def or it's just a separate class used in GenericParamList & friends?
In the former case, where in the node hierarchy should we put it? At the moment GenericTypeParamDecl is an AbstratcTypeParamDecl, that is a TypeDecl, but parameter packs (probably) and value parameters (of course) are not types.
I have this doubt because you mention the eventual factoring of functionalities in the new GenericParamDecl type so I am (wrongly?) assuming a type/class hierarchy may be involved.

Sorry for the errors but I'm writing this reply in a little bit of hurry!

(Chris Lattner) #7

Just out of curiosity, instead of making packs be a special new concept, have you considered making them be a generic type that is known to be a tuple? Just like we have "class constraints" on generic types, we could have "tuple constraints", and tuple constrained generic parameters could have accessors on them for getting the tuple length etc.

This could allow a more dynamic model for variadics that might fit better into the overarching swift design than the static C++ model.

-Chris

3 Likes
Implementing ExpressibleByTupleLiteral: how hard?
Variadic Generics
(Chris Lattner) #8

Basically, I'm thinking that it is very likely to be interesting over time to investigate making tuples be more first class: e.g. tuples conforming to protocols, the ExpressibleByTupleLiteral thread, allowing types to conform to a protocol that gives them participation with tuple patterns, etc.

If/when we do those sorts of thing, having variadics is essential, but instead of making variadics a heavy core language feature, maybe it is just light sugar for working with tuples.

In any case, I'm certainly not an expert on this stuff and this isn't exactly a baked plan :-) I'm just curious if you've thought about it.

-Chris

3 Likes
(Joe Groff) #9

In my thinking, we kinda already closed off that "variadics are just tuples" avenue once we committed to breaking up function arguments. That means that there are now at least two expression contexts we'd want to be able to "unpack" into. Nonetheless, we can probably limit the number of places you need variadic declarations compared to C++, since we ought to make it easy to pack into and unpack from tuple values.

One benefit of tuples preserving their built-in status is that we can maintain the "unlabeled single-element tuple types are equivalent to their element type" invariant (in other words, (T) and T are always the same type), which is important for the ergonomics of variadic returns or fields, since you don't want to have to unwrap them in the single element case.

1 Like
(Joe Groff) #10

As a forum etiquette thing, I wonder whether it'd be a good idea to start pitch threads on this and the other "tuple literals" thread, so that these development threads can remain focused on implementation questions.

1 Like
(Andrea Tomarelli) #11

Regarding this, in reality I initially thought (like Chris) that the variadic generics could be sugar for tuples, as I expressed in my other thread; but this is obvious subject to discussion and your point on (T) and T is a good reminder of the implications one should consider when relating new features to tuples.

I'm not sure if this is a reply to Chris's post or mine, but in any case I agree with you and would like to invite anyone interested to participate to my other thread for discussion / pitching so that this thread can "only" collect informations about implementation-level stuff :smile:

(Joe Groff) #12

It wasn't really directed at anyone in particular, I was mostly concerned about your own tolerance for design discussion.

(Andrea Tomarelli) #13

I'm sorry, I don't understand what you mean here - not saying this in a sarcastic way, I literally do not understand what you mean by "tolerance for design discussion".

(Joe Groff) #14

Maybe I'm being paranoid, but I was only worried that you had come in here asking for implementation guidance, and then we all started arguing about aspects of the design. I could see some people being put off by that.

1 Like
(Andrea Tomarelli) #15

Ah, now I understand!
I'm personally not too concerned about this, but obviously for reference and discussion simplicity I'd like to keep the two aspects separated because like you say not everyone might be happy with things being mixed.

(Chris Lattner) #16

Oh yes, very good point. Does this mean that variadic parameter packs will be "what tuples should have been"? Variadic parameter packs don't need to be varargs (in the current sense) so that is one bit of complexity they wouldn't have to eat. I think the only other things that are missing are the ownership modifiers and autoclosures.

Does ownership give us a new way to introduce inout elements to tuples? What will the fundamental difference be between a parameter pack and a tuple? We don't have partial specialization, so we can't use the C++ model for variadic templates, we need some sort of "dynamic" model.

-Chris

(John McCall) #17

Random thoughts about representations:

  • I don't know if the runtime representation of a variadic pack of types should be an ArrayRef or a pointer to "Pascal-style" array or what, but I know it should not be flattened into the surrounding types.
  • That's when storing the pack in type metadata; as an argument to a function I think an ArrayRef is the way to go, especially if recursion is going to be important somehow.
  • If we want to do piecemeal destructuring of the value representation, and we use a tuple for the representation of a variadic pack of values, then we should promote pulling elements off the end rather than the beginning or else we'll have to move the elements in order to make the padding right.
  • That said, it might be more efficient to use an array of pointers to values.
(Andrea Tomarelli) #18

@Douglas_Gregor @Chris_Lattner3
I was writing my document about Variadic Generics when I stumbled across Chris's post on the Introduce (static) callables thread. In the post, Chris mentions a long term goal of unifying nominal and structural types. If I'm correct, this desire is shared among multiple Swift users and other thread refers to this feature, too.

At the moment I'm thinking to represent Variadic Generics as tuples inside types and function bodies, much like variadic function arguments are passed as Arrays.
But if / when we have structural types for tuples and / or functions, because they will need Variadic Generics (I assume), they might look like this:

struct Tuple<T...> {
  // Tuple implementation
}

class Function<Args..., Result> {
  // Function implementation
}

But at this point it seems to me that we have a chicken-and-egg problem: Tuple definition requires Variadic Generics, but Variadic Generics are represented as tuples inside the type using them, so...

struct Tuple<T...> {
  // This is... a tuple?
  private var members: (T...)

  init(_ members: (T...)) {
    self.members = members
  }
}

So, looking at the (far) future, what can we do here? The only solution I can think of ATM is to actually not represent Variadic Generics as tuples, but as something else like a "parameter vector" or C++'s "parameter pack".

2 Likes
(Adrian Zubarev) #19

In case of Tuple<T...> I think we can rely on the stdlib internal capabilities and hide the inner storage from the user. For the end-user Tuple<T...> would conform to some kind of ExpressibleByTupleLiteral protocol to achieve the old tuple syntax. The inner storage tuple of Tuple struct does not need to be the swifts (T...) current tuple, which avoids the paradox.

(Andrea Tomarelli) #20

So you suggest that some magic could go on only with the Tuple type? Why not, this can be an acceptable trade-off.

This still leaves me with a doubt, let me clarify one thing just to be sure:

struct ZipSequence<Sequences... : Sequence> {
  // For what concerns `ZipSequence`, `Sequences` is a tuple of unknown arity (S1, S2, ..., Sn)
}
func zip<Sequences... : Sequence>() {
  // Also here, `Sequences` is a tuple of unknown arity (S1, S2, ..., Sn)
}

Simply by declaring a generic T to be variadic, I would like the type / function to see it as a tuple, and this is independent of any member declared to be of that type. So if we have this in the stdlib:

struct Tuple<T...> {
  // Here, the compiler would treat `T` as a tuple like it would do
  // for every other type, even if there are no explicit members and
  // magic storage is going on
}

part of the issue will still remain. Maybe I'm missing part of your reasoning?

(Adrian Zubarev) #21

I'm not sure we're on the same page, but for me Sequences... nor T... is a tuple. These are 'just' generic parameter lists of a 1 ... n range. You can create a tuple 'only' if you write either (Sequences...) or Tuple<Sequences...> (the former falls back to the latter as we already have it today with [T] fallback to Array<T>). Also a small note, an empty tuple aka. Void cannot be represented by Tuple<T...> unless we say that T... can also have the size zero, but that would likely create more issues than helping here.

(Andrea Tomarelli) #22

Ah yeah, that was the problem! To me T was already a tuple inside a VG context, but not to you. And when I said:

I was saying exactly what you intended, that T is a group of parameters and another syntax - like (T...) - is its "tuple expansion".

About T... allowing for empty parameters, I'd like it to be true, but I've not thought about the potential issues. Do you have something specific in mind?

Thanks for your feedback!

(Adrian Zubarev) #23

On the first glance I have no usability issues that I can foresee, but I could guess that it might potentially require more language support and would complicate the already complex feature set we're talking about here.

On the other hand some people asked for being able to extend Void or conform custom protocol to it, which would typealias Void = Tuple<> let them do. Furthermore I think we have to be carful on how ExpressibleByTupleLiteral would behave as not every type that can potentially conform to the protocol can or should be representable by an 'empty' tuple.

(Matthew Johnson) #24

Being able to conform Void to protocols is really important (especially Equatable and Hashable).

Another edge case to keep in mind here is the fact that Swift doesn’t actually have single-element tuples. This will probably complicate the design of Tuple (as a nominal type). I even wonder if we would need to make Void a separate nominal type and have something like Tuple<First, Second, Rest...> in order to prevent formation of single-element tuples like Tuple<Int> (because (Int) is actually just Int in Swift). This may be a really bad idea, but I don’t have a better one off the top of my head.

1 Like
(Adrian Zubarev) #25

That is true but only because of the ambiguity the syntax has today.

// This should be just fine because it wouldn't be ambiguous for the compiler anymore.
let tuple: Tuple<Int> = (42)
print(tuple.0)

So my two cents are, keep the source compatibility but allow single element tuples in an unambiguous way like above.

1 Like