Variadic generics discussion

Sounds like more compiler magic... rather than trying to reduce it.

Think of it this way: The amount of compiler magic should be at most proportional to the amount of utility granted by that magic. Most of the magic in this design is not directly in service of variadic generics, but rather in service of more general, and more widely useful, features like Collection-like tuples and splatting.

In particular, a lot of the Collection-like tuple stuff—which at least has the largest *surface area* of magic in this proposal—is, in my opinion, *very* likely to be standard-library-implementable eventually, either with regular code or with macros. It's also the most broadly useful part of it; people have asked for this feature to make C buffers more accessible, to implement fixed-size arrays, and for various other purposes. As I said, I actually forgot that we hadn't gotten that through review already.

Personally, I think the gains for each feature I discuss are well worth their costs in magic.

Yes, 100% with you there. Just wondering if magic is the only way. You very much said it all when you said 'std lib ... eventually'. Considering it has been described as post-3 there will be time to refine the ideas.

···

On Jun 1, 2016, at 8:26 AM, Brent Royal-Gordon <brent@architechies.com> wrote:

--
Brent Royal-Gordon
Architechies

I agree that this is a better design for Swift than the monstrosity I started out with.

The "biggest" technical challenge I see is being able to type a reduction sort of operator on a heterogenous tuple based on on whatever protocols and constraints are common to its constituent members. For example:

// Every Tn in T... is Fooable and Barrable
let x : (T...)
reduce(x, reducer, startingValue)

func reducer<X : ???>(startingValue: U, eachMemberOfT: X) -> U { ... }

How do we bound ??? such that 'reducer' is useful while still being statically type sound? Honestly, that's the most interesting question to me. Generalized existentials might help with that.

If every T is Fooable and Barrable, couldn't `eachMemberOfT` be (inout U, protocol<Fooable, Barrable>) -> ()?

I feel like I might be overthinking this. The tightest sound bound is protocol<Fooable, Barrable> (or <T : Fooable where T : Barrable>, and the loosest bound is Any (or <T>), but it would be good to have a notion of ordering other bounds so that functions with more lenient bounds for which all the requirements are satisfied by the tuple members (e.g. protocol<Fooable>, not protocol<Fooable, Baz>) are admitted. This would become more complex with associated types and constraints thereof, where ordering doesn't just mean "at least some of the protocols the variadic parameter type bounds are required to conform to, but not any new ones".

Other questions (inherent to any proposal) would be:

- How do we resolve the impedance mismatch between tuples and function argument lists? Is it even worth trying to resolve this mismatch, given that argument lists are intentionally not intended to mirror tuples?

The impedance mismatch between function arguments and tuples is superficial. You ought to be able to splat and bind tuples into function arguments, e.g.:

  let args = (1, 2)
  foo(bar:bas:)(args...)

  func variadicFn<T: Runcible...>(_ args: T...) { ... }

I agree, I was thinking more of things like inout and default parameters. But that splat/bind notation should be enough for the majority of use cases.

- As you said, how do variadic generics work in the 0- and 1-member cases?

What problem are you referring to?

How does a type with zero type parameters work? I guess all the variadic tuples in question could just be the empty tuple.

Would there be tuple operations that work on variadic tuples which would break down in the 1-ple case (where the item isn't a tuple anymore, but a regular value)?

As Brent mentioned, having base cases for the 0 and 1-arity cases defines this problem away. so it's not really a big deal.

···

On Jun 1, 2016, at 6:18 AM, Joe Groff <jgroff@apple.com> wrote:

On May 31, 2016, at 6:49 PM, Austin Zheng <austinzheng@gmail.com> wrote:

-Joe

I agree that this is a better design for Swift than the monstrosity I started out with.

The "biggest" technical challenge I see is being able to type a reduction sort of operator on a heterogenous tuple based on on whatever protocols and constraints are common to its constituent members. For example:

// Every Tn in T... is Fooable and Barrable
let x : (T...)
reduce(x, reducer, startingValue)

func reducer<X : ???>(startingValue: U, eachMemberOfT: X) -> U { ... }

How do we bound ??? such that 'reducer' is useful while still being statically type sound? Honestly, that's the most interesting question to me. Generalized existentials might help with that.

If every T is Fooable and Barrable, couldn't `eachMemberOfT` be (inout U, protocol<Fooable, Barrable>) -> ()?

Other questions (inherent to any proposal) would be:

- How do we resolve the impedance mismatch between tuples and function argument lists? Is it even worth trying to resolve this mismatch, given that argument lists are intentionally not intended to mirror tuples?

The impedance mismatch between function arguments and tuples is superficial. You ought to be able to splat and bind tuples into function arguments, e.g.:

  let args = (1, 2)
  foo(bar:bas:)(args...)

  func variadicFn<T: Runcible...>(_ args: T...) { … }

Going this direction is relatively straightforward. The impedance mismatch is trickier when you want to do more than that. For example, it would be extremely useful to come up with some way to wrap a function that has parameters with default arguments without requiring callers of the wrapped function to supply arguments for the defaulted parameters. Any ideas on how to solve that?

As Chris noted, that sounds like a great use case for a macro. Trying to do that with the type system feels like it's on the wrong side of the complexity threshold to me.

-Joe

···

On Jun 1, 2016, at 6:26 AM, Matthew Johnson <matthew@anandabits.com> wrote:

On Jun 1, 2016, at 8:18 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

On May 31, 2016, at 6:49 PM, Austin Zheng <austinzheng@gmail.com> wrote:

- As you said, how do variadic generics work in the 0- and 1-member cases?

What problem are you referring to?

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Perhaps inout params of type T could be somehow passed as a pair (T, (T) -> ()), where the first item is the input value and the second item is a writeback function representing the 'out' part of inout.

The simplest solution, I think, is to require the entire splatted tuple to be `inout` if any of the parameters it's being splatted into are `inout`. This doesn't give you the same granularity as the original, but I don't see that as a serious problem.

I've no ideas for default values.

Ignoring the defaults is not a terrible solution. Leaving that aside, we could match the tuple against the parameter list, figure out (from positions and types) which parameters are missing, and fill in their default values. Or we could punt this to the overload selection layer, and make it so that writing, say, `print(_:)` implicitly creates a closure which always uses the default values for the `separator` and `terminator` parameters.

···

--
Brent Royal-Gordon
Architechies

1 Like

I agree that this is a better design for Swift than the monstrosity I started out with.

The "biggest" technical challenge I see is being able to type a reduction sort of operator on a heterogenous tuple based on on whatever protocols and constraints are common to its constituent members. For example:

// Every Tn in T... is Fooable and Barrable
let x : (T...)
reduce(x, reducer, startingValue)

func reducer<X : ???>(startingValue: U, eachMemberOfT: X) -> U { ... }

How do we bound ??? such that 'reducer' is useful while still being statically type sound? Honestly, that's the most interesting question to me. Generalized existentials might help with that.

If every T is Fooable and Barrable, couldn't `eachMemberOfT` be (inout U, protocol<Fooable, Barrable>) -> ()?

Other questions (inherent to any proposal) would be:

- How do we resolve the impedance mismatch between tuples and function argument lists? Is it even worth trying to resolve this mismatch, given that argument lists are intentionally not intended to mirror tuples?

The impedance mismatch between function arguments and tuples is superficial. You ought to be able to splat and bind tuples into function arguments, e.g.:

  let args = (1, 2)
  foo(bar:bas:)(args...)

  func variadicFn<T: Runcible...>(_ args: T...) { … }

Going this direction is relatively straightforward. The impedance mismatch is trickier when you want to do more than that. For example, it would be extremely useful to come up with some way to wrap a function that has parameters with default arguments without requiring callers of the wrapped function to supply arguments for the defaulted parameters. Any ideas on how to solve that?

As Chris noted, that sounds like a great use case for a macro. Trying to do that with the type system feels like it's on the wrong side of the complexity threshold to me.

Makes sense. Looking forward to having macros someday… :)

···

On Jun 1, 2016, at 8:29 AM, Joe Groff <jgroff@apple.com> wrote:

On Jun 1, 2016, at 6:26 AM, Matthew Johnson <matthew@anandabits.com> wrote:

On Jun 1, 2016, at 8:18 AM, Joe Groff via swift-evolution <swift-evolution@swift.org> wrote:

On May 31, 2016, at 6:49 PM, Austin Zheng <austinzheng@gmail.com> wrote:

-Joe

- As you said, how do variadic generics work in the 0- and 1-member cases?

What problem are you referring to?

-Joe

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution

1 Like

A regular value is a 1-tuple of itself. In the type system, we know what abstraction level a value expects to work at. It shouldn't be a problem for a variadic type parameter to be bound to 0, 1, or more types.

-Joe

···

On Jun 1, 2016, at 9:35 AM, Austin Zheng <austinzheng@gmail.com> wrote:

On Jun 1, 2016, at 6:18 AM, Joe Groff <jgroff@apple.com> wrote:

On May 31, 2016, at 6:49 PM, Austin Zheng <austinzheng@gmail.com> wrote:

I agree that this is a better design for Swift than the monstrosity I started out with.

The "biggest" technical challenge I see is being able to type a reduction sort of operator on a heterogenous tuple based on on whatever protocols and constraints are common to its constituent members. For example:

// Every Tn in T... is Fooable and Barrable
let x : (T...)
reduce(x, reducer, startingValue)

func reducer<X : ???>(startingValue: U, eachMemberOfT: X) -> U { ... }

How do we bound ??? such that 'reducer' is useful while still being statically type sound? Honestly, that's the most interesting question to me. Generalized existentials might help with that.

If every T is Fooable and Barrable, couldn't `eachMemberOfT` be (inout U, protocol<Fooable, Barrable>) -> ()?

I feel like I might be overthinking this. The tightest sound bound is protocol<Fooable, Barrable> (or <T : Fooable where T : Barrable>, and the loosest bound is Any (or <T>), but it would be good to have a notion of ordering other bounds so that functions with more lenient bounds for which all the requirements are satisfied by the tuple members (e.g. protocol<Fooable>, not protocol<Fooable, Baz>) are admitted. This would become more complex with associated types and constraints thereof, where ordering doesn't just mean "at least some of the protocols the variadic parameter type bounds are required to conform to, but not any new ones".

Other questions (inherent to any proposal) would be:

- How do we resolve the impedance mismatch between tuples and function argument lists? Is it even worth trying to resolve this mismatch, given that argument lists are intentionally not intended to mirror tuples?

The impedance mismatch between function arguments and tuples is superficial. You ought to be able to splat and bind tuples into function arguments, e.g.:

  let args = (1, 2)
  foo(bar:bas:)(args...)

  func variadicFn<T: Runcible...>(_ args: T...) { ... }

I agree, I was thinking more of things like inout and default parameters. But that splat/bind notation should be enough for the majority of use cases.

- As you said, how do variadic generics work in the 0- and 1-member cases?

What problem are you referring to?

How does a type with zero type parameters work? I guess all the variadic tuples in question could just be the empty tuple.

Would there be tuple operations that work on variadic tuples which would break down in the 1-ple case (where the item isn't a tuple anymore, but a regular value)?

This is good to know, thanks for the clarification!

Austin

···

On Jun 1, 2016, at 9:38 AM, Joe Groff <jgroff@apple.com> wrote:

A regular value is a 1-tuple of itself. In the type system, we know what abstraction level a value expects to work at. It shouldn't be a problem for a variadic type parameter to be bound to 0, 1, or more types.

-Joe

Bumping this thread. Myself and some folks I've been talking to would still very much like to see variadic generics in Swift, but it looks like discussion died after it was decided that the feature wasn't within the scope of Swift 3. Let me know if necroing a thread this old is against forum policy, or if I should open a new pitch instead :)

I'll link here to another thread where I posted my thoughts on this. It's only IMHO, but I would like to see this etiquette established where "bumps" stop happening. The feature doesn't exist today not for lack of interest, but for lack of engineering resources to make it happen.

Are you saying that people should stop trying to push features they want? Don't get me wrong, I clearly understand your point about the lack of implementers, but this was also one major point to move to a forum so that people that are interested in certain features can create groups and eventually find someone who is willing to build it. But on the flip side of the coin if no one will ask for something, nothing will happen at all. That said, reviving some older topics isn't necessarily a bad thing. If there is no interest right now, then it will be deferred again, but how can you know that? ;)

2 Likes

I think you missed the point of my message. Wouldn't you agree that there's clearly interest in the feature, given the extensive discussion? Do you think that another "bump" was necessary to demonstrate interest?

One major point of the forums was to make it easier to find all the archives so that the same conversations don't need to be repeated. If someone thinks they're the first to come up with an idea and discovers 70 previous messages about it, they can see what's already been said and then chime in only with the points that haven't been said. It saves that person time and energy, and it saves everyone else time and energy, and we can all contribute more effectively. I think it's rather unfortunate that, for so many threads where the previous conversation has been very detailed and thorough, the additional contributions that a forum format have made possible consist of: "bump."

1 Like

Yes I think the "bump" is actually necessary, because that demonstrates that there are folks that are willing to sacrifice there spare time in order to push things forward. The discovery of an older topic won't get that particular feature anywhere. I'm not saying that we should "ping" topics that we share interest in everywhere so that those are moved to the top. If you have something to add to a particular topic so this is a good bump enough for me, even if it's a similar opinion to what have been previously said (this is how our reviews already work, we share either similar or different opinions). That is my opinion and I don't think someone can convince me to change it. ;)

1 Like

I would love to see this added to Swift and know a specific case where this would be super useful. Type safe routing in server side Swift frameworks. This would allow declaring routes in a super type safe way e.g.

func get<T: RouteParameter>(_ paths: T..., _ handler: (Request, T...) -> Response

And calling it would be something like:

get(“user”, String.self) { req, userId in }

Not sure exactly how the syntax would look but also after reading the thread this might also be solved by tuple splatting if I am not mistaken? I know Vapor at one point was using code gen to generate a lot of these in the past

2 Likes

That doesn't mean that I won't try to change your opinion, however. I am surprised that you think that collecting similar opinions is how reviews are supposed to work. Usually, I do not review proposals, even if I feel very strongly for or against them, if the same viewpoint has already been put forward by someone else. As has been said here before, this forum isn't a democracy, posts aren't votes, and saying the same thing multiple times doesn't make it more important or convincing.

1 Like

Yes I do. It's been two years and two Swift versions since there was any real discussion on the topic at least as far as I could tell from a forum search. I think a lot has changed since then and we've learned quite a bit that might feed into new opinions. I'd also love to see if people feel this is something that is within scope for Swift 5. If not, then discussion can still happen knowing that variadic generics will need to be pushed to Swift 6 or beyond.

There is one other discussion on variadic generics that happened last year, but it looks like it mostly devolved to a discussion on whether or not tuples should exist:

I'm interested in this because I'm working on a command line tools framework that would benefit. Currently, if I want a command line tool to be able to take parameters and have those parameters be type-safe, I have to restrict the tool's parameters to a single type.

Command<U: LosslessStringConvertible>(
    triggers: [String],
    help: String,
    numParams: NumberOfParams = .any,
    options: [ProtoOption]=[],
    onExecution: @escaping ([U], State) throws -> ()
)

In the current state someone couldn't accept a command like newperson Alex 10 passing in parameters for name and age, unless they wanted to also cast 10 to a String and deal with it at runtime.

However, with variadic generics, the type could look something more like (Using C++ syntax)

Command<...U: LosslessStringConvertible>(
    triggers: [String],
    help: String,
    numParams: NumberOfParams = .any,
    options: [ProtoOption]=[],
    onExecution: @escaping (State, U...) throws -> ()
)

The feature required here is the ability to take a variadic generic type signature, and unpack it for use as the parameter type of an instance, static, or anonymous function.

I'm interested to see what other use cases people can think of :grinning:

1 Like

I assure you, there's a lot of interest in this feature and many use cases. The feature doesn't exist because there's not the resources to make it exist, not because people can't imagine why they'd want it.

If you haven't familiarized yourself with it yet, I'd encourage reading through the following document:
The Generics Manifesto

Of features considered "major extensions to the generics model," Swift is still in the process of implementing the first item on the list (conditional conformances). This work has taken almost two versions of the language to come to fruition, and may yet take more time to complete, just to give you a sense of the timelines involved here. Variadic generics are also listed; it would be a minor miracle to have a proposal before Swift 6 or 7 and an implementation before Swift 8 or 9.

I read the proposal and began reading the thread until I realized how long it is and that I don't have time to read every comment, so I'm sorry if my question is already answered somewhere that I have missed.

I found my way to this page because a feature which I am currently in need of could be very nicely served by variadic generics, but from what I could understand from the proposal, the proposed implementation would not meet my need. What I want (I think this is a coherent request) is to be able to use a variadic associated type to declare a set of functions, one for each type in the vector. For example:

protocol Eater {
    associatedtype ...Foods: Digestible
    var carbs: Int { get set }
    var fats: Int { get set }
    var proteins: Int { get set }
}

extension Eater {
    mutating func ...eat (_ food: Foods...) {
        self.carbs += food.carbs
        self.fats += food.fats
        self.proteins += food.proteins
    }
}

Notice the elipsis before the name of the function, but not before the function parameter name. This is because the parameter is singular, and it is the function declarations which are multiple in order to match the arity of the vector. I would define some Digestible types (pretend that they conform):

enum Fruit: Digestible {
    case apple
    case strawberry
    case banana
    case kiwi
}

enum Vegetable: Digestible {
    case broccoli
    case spinach
    case zucchini
    case squash
}

And then could define a type which conforms to Eater:

struct Vegetarian: Eater {
    typealias ...Foods = <Fruit, Vegetable>
    var carbs: Int = 0
    var fats: Int = 0
    var proteins: Int = 0
}

And use it like this:

let joe = Vegetarian()
joe.eat(.zucchini)
joe.eat(.kiwi)

In this toy example, the Digestible protocol does not have associated types and therefore the Eater type could of course just put the same implementation under this function signature:

func eat (_ food: Digestible) { ... }

or if it did have associated types it could be written like this:

func eat <Food: Digestible> (_ food: Food, using recipe: Food.IdealRecipe) { ... }

but the MAJOR drawback here is that the types are no longer constrained, which leads to a few problems for me, the most dire of which is that autocomplete will no longer help me. The workflow which I have been working extremely hard to refine and perfect is one which makes heavy use of autocomplete and type inference, such that when I am figuring out which input to provide to a given function, I just type a . and autocomplete immediately springs into action and gives a concise list of my exact options with this function in this particular context. I cannot understate the leaps and bounds in workflow that this produces. Without this, my code in this example would be:

joe.eat(Vegetable.zucchini)
joe.eat(Fruit.kiwi)

and I would have to remember the names of the types I want to use.

Has anyone put thought toward this type of use case for variadic generic parameters? To me, this is not a niche or trivial ability that is gained. If this were possible, my library would enable such a new level of effortlessness in development due to the beauty of autocomplete showing me my exact options all over the place. Right now I'm resorting to code generation to get the job done - perhaps when my library is in more of a working condition I can post about it to hopefully prove that there is great utility in this use case.

P.S. I'm aware that on top of the main syntax I'm proposing here I have also introduced this syntax typealias ...Foods = <Fruit, Vegetable>. As far as I can tell from the proposal, the only way to satisfy the requirements of a protocol with a variadic associated type is for the conforming type to also have a variadic generic parameter list, but what if I want to create a non-generic type which conforms to a protocol with a variadic associated type, as in my example? There's gotta be a way to do that.

1 Like

@Austin I don't know if you receive notifications for every comment in this thread but I thought I'd tag you just in case because I'd love to hear your opinion on this ^^^

Please avoid resurrecting three-year-old threads like this. Everyone who's ever participated gets emailed and many will have moved on to other things, Austin among them (and he is missed).

2 Likes