Ah, great! :-)
That's a good point. In the case I presented it should be possible to infer this from the usage: i.e. the position an argument appears relative to the function signature should unambiguously tell you which VG pack the corresponding type would fall into. But I haven't thought about it enough to know whether this would be true in the general case.
It might be an unpopular opinion, but if we are going to simply copy from C++, I‘d rather prefer Swift not to have variadic generics at all.
I can think of alternatives that feel less alien for me, but all what I have seen so far involved some unintuitive usage of dots, and I don‘t think this is a good fit for Swift.
This is not unpopular to me, and I agree that we should and we can to build this feature so that it feels as Swifty as possible. I generally do not like C++'s cumbersome syntax, and in fact my usage of ... is not derived by its existence in C++ or other languages.
To me, (T...) syntax is nice because it feels like a tuple of unspecified length, like in a list of things or numbers: 1, 2, 3, 4, .... This is the reason why I initially went for this syntax. I intentionally avoided #sharpStuffSyntax because I feared that it could involve too much noise, but I might reconsider it.
So please share with us why the syntax feels unintuitive and / or alien to you, I'm really open to everything!
Moreover, I'd also like to hear other suggestions because the ... syntax might be a problem for another reason: it's a valid operator in Swift, so much that the Standard Library actually declares and implements it for numbers:
1 ... 5 // ClosedRange<Int> = 1...5
That means that, even if this is unlikely, the ... operator could be defined to operate on tuples:
postfix func ...<T, U>(lhs: (T, U)) -> T { return lhs.0 }
This means that in the section "empowering tuples" of my document we have a problem:
let t1 = (a: 1, b: 2)
let t2 = (c: 3, d: 4)
// ~~ HERE IS THE PROBLEM ~~
// --- Unpack (using `...`) and repack
let bigTuple = (t1... , t2...)
// What do `...` mean here? The result is `(a: 1, b: 2, c: 3, d: 4)`
// [using unpack / repack] or `(1, 3)` [using `...` operator]?
So I've spent a bit of time with your document: thank you for working on this by the way, I think it's a much-needed feature in the language, and you do a very good job laying out the case for the feature as well as how VG might work.
It's possible I'm just not grepping this from the document, but it seems like there are several important use-cases which are not covered in the design which is laid out so far:
1. It should be possible to operate on the list of types in the variadic type-pack.
So for instance, in the existing generics implementation, it's possible that a generic function operates on a type itself rather than a value of a type. For example, 'UnsafeMutableBufferPointer.withMemoryRebound(to:_:)' infers it's generic argument from a Type rather than a value of a type.
For the following example let's consider this (totally contrived) protocol and function:
protocol P {
static func foo()
}
func myGenericFunction<T: P>(type: T.Type) {
type.foo()
}
We can imagine that we might also want to implement a variadic version of this function:
func myVariadicGenericFunction<T...: P>(types: T.Type...) { // Not sure what this syntax would be
// I want to be able to do this, but I am not sure how iteration over a type-pack in the proposed solution
for type in types {
type.foo()
}
}
2. I want to be able to perform sequence-like operations over variadic tuples
So as was hinted at in the above example, as I think through potential use-cases for this feature, it seems like in order to do almost anything interesting with variadic generics, it would be neccecary to perform sequence-like operations on variadic tuples and variadic type packs.
For this protocol, let's consider another another generic function:
func myGenericFunction2<T>(value: T) { ... }
It might be the case that in a variadic generic type, we would want to call this function on each member in a variadic tuple:
struct MyVarStruct<T...> {
let storage: (T...)
func foo() {
for value in storage {
myGenericFunction2(value: value)
}
}
}
Reduce over a tuple is another operation which jumps out as being useful:
func myGenericFunction3<T>(value: T) -> Int { ... }
struct MyVarStruct2<T...> {
let storage: (T...)
func total() -> Int {
return storage.reduce(0) { $0 + myGenericFunction3(value: $1) }
}
}
I can also imagine it might be useful to be able to map from one variadic tuple to antother:
func myGenericFunction4<T, U>(value: T) -> U { ... }
struct MyVarStruct3<T...> {
let storage: (T...)
func mapped<U...>() -> (U...) {
return storage.map { return myGenericFunction4($0) }
}
}
So I can see that there is some way in the design document to operate over tuples, i.e:
// The type of this expression is `([T.AT]...)`
let associatedValues = (storage.getAssociatedValues()...)
// The type of this expression is `(T.AT?...)`
let firstAssociatedValues = (storage.getAssociatedValues().first...)
But I'm not sure if I understand the intent and limitations of that syntax, and whether this would cover the cases above.
2 Likes
We use a simple in-house promise implementation in the project I'm currently on, and it has a function Promise.all which takes an sequence of promises, and resolves with an array of results. This works for homogeneous sets of promises.
But we also have a few overloaded versions of Promise.all which takes a few promises and returns a new promise which resolves with a tuple for heterogeneous sets. These are defined up to arity 3. These are actually used far more often, as the common use case for us is to kick of a few network requests, and once all of them completes, update the UI. The dependant requests are almost never of the same return type.
Currently, in order to make the implementation of each of these overloaded functions minimal, we have a file-private asAny on Promise which turns a Promise<T> into a Promise<Any>, so we can use the sequence based Promise.all, and in its completion handler we force-cast each return value back into their corresponding types and construct a tuple from it, like so:
extension Promise {
static func all<A, B, C>(
_ prA: Promise<A, Error>,
_ prB: Promise<B, Error>,
_ prC: Promise<C, Error>
) -> Promise where Value == (A, B, C) { //swiftlint:disable:this large_tuple
return Promise<[Any], Error>
.all([prA.asAny(), prB.asAny(), prC.asAny()])
.then { ($0[0] as! A, $0[1] as! B, $0[2] as! C) } //swiftlint:disable:this force_cast
}
}
Typically it looks like this at call-site:
Promise.all(
fetchUserProfile(for: currentUser),
fetchFriendList(for: currentUser)
).onSuccess { user, friends in
// update ui
}.onFailure { error in
// show error message
}
It works well using the manually written overloads with different arities, but we could reduce boiler plate and support an arbitrary number of arguments using variadic generics.
3 Likes
We also have a Promise.promisify which takes a void-returning function with n parameters where the last is a completion handler, and returns a new promise-returning function with n - 1 parameters.
extension Promise {
static func promisify<A, B, C>(_ callbackFunc: @escaping (A, B, C, @escaping (Result) -> Void) -> Void) -> (A, B, C) -> Promise {
return { a, b, c in Promise { done in callbackFunc(a, b, c, done) } }
}
}
At call-site we use it like this, to convert older parts of our code base into promises:
func fetchFriendList(for user: User) -> Promise<[User]> {
let fetcher = Promise.promisify(apiClient.getFriendList)
return fetcher(user)
}
These are also defined for a number of arities. It think up to 6.
1 Like
Vapor does something similar with its Futures.
@Spencer_Kohan thank you for your comments about the document! I'll try to quickly address your points (I'll stick to the syntax of the current version of the document).
Point #1
You are right, this is not explicitly stated in the proposal. I imagine it working the same way it will work for values, because AFAIK types are values themselves (whose type is their metatype):
// === For values ===
protocol P {
func createInt() -> Int
}
func vgFunc<T... : P>(_ values: T...) {
// This applies `createInt` to every member
// The result is `(Int, Int, ...)`
let ints = (values.createInt()...)
}
// === For types ===
protocol P {
static func createInt() -> Int
}
func vgFunc<T... : P>(_ types: T.Type...) {
// This applies `createInt` to every member
// The result is `(Int, Int, ...)`
let intsAgain = (types.createInt()...)
}
The next example may be a little bit crazy, but I can see this work even for the following:
protocol Initializable {
init()
var name: String { get }
}
func createStuff<I... : Initializable>(_ types: I.Type...) {
let stuffNames = (types.init().name...) // `(String, String, ...)`
// let stuff = (types.init()...)
// let stuffNames = (stuff.name...)
}
I think the for type in types example fall under point #2.
Point #2
Starting from the bottom, the syntax (storage.varOrFunc...) is used to create a new tuple containing the result of applying varOrFunc to every member of the original tuple i.e. this is like doing (storage.0.varOrFunc, storage.1.varOrFunc, ...). However, like you noted in "point 2.1", the syntax does not allow to call an arbitrary function on every member of storage - for this you need something else like the for ... in loop you proposed. In this regard the syntax presented in the document is actually very limited.
Points 2.2 and 2.3 are very interesting topics that I've not addressed at all to this day, and I have two opposing thoughts about these collection-like features. The problem is that I'd like all the new syntax to not be limited to Variadic Generics (e.g. the ... syntax to unpack / repack tuples can be applied to any tuple), but if we introduce for example this mapping operator / function we run into an issue. What happens to "standard" tuples that have a member named map? There will be ambiguity here...
Maybe it's just that tuples are not the right answer? Maybe we need a special new concept of "parameter pack", unrelated to tuples, to support all these features?
@sveinhal and @twof in the Promise case I think what I've proposed until now should work, but unfortunately I cannot be sure because I do not know how Promise.all<T>(Array<Promise<T>>) (I suppose the signature is this) works. If the representation as Array is mandatory there might be a problem, because ATM there is no way to convert a variadic tuple to an array.
For reference, the promisify function could look like this (correct me if I'm wrong):
// Note: The `...` syntax without the `()` just "expands" the
// variadic tuple into the surrounding scope
extension Promise {
static func promisify<T...>(
_ callbackFunc: @escaping (T... , @escaping (Result) -> Void) -> Void
) -> (T...) -> Promise {
return { values in Promise { done in callbackFunc(values... , done) } }
}
}
I'm not sure what the use cases might be, but I can imagine there might be some use for a generic constraint that says all types in the pack are the same. With a constraint like that you'd be able to use that T on its own and do some interesting things. For example, pass a pack of Ts to a normal variadic method, hopefully reduce the pack to a single value, etc.
This seems like a serious limitation. Essentially this would mean the only way to operate on a parameter pack would be to add members to the constituent types. A pattern I employ pretty often in Swift is to work primarily with pure-data types which are operated on by pure functions, and this limitation would make it mandatory to subvert this design pattern if you wanted to make use of Variadic Generics.
Also, I can imagine this leading to a fair amount of coding gymnastics to get around the limitation. For instance, to emulate that for ... in loop I can imagine writing something like this:
protocol P { .... }
func myGenericFunction<T: P>(value: T) { ... }
extension P {
func fakeForLoop() {
myGenericFunction(value: self)
}
}
func myVariadicGenericFunction<T...: P>(values: T....) {
_ = (values.fakeForLoop()....)
}
To me the for ... in version communicates the intent more clearly.
Yeah I can see the problem there. I guess one solution would be to make these global functions which operate on tuples (similar to math functions like max and atan etc.) but this would seem like a deviation in style to a majority of the API.
I can see some merits to this approach. I can understand the motivation for wanting to use Tuples for variadic generics, since at the end of the day both are fixed-length ordered collections of values with heterogeneous types. However it seems to me that the usage of these two concepts is quite different.
In the case of Tuples, currently you are always working with them with full knowledge of the structure: you are always operating on the values either by name, or by index.
By contrast, in a variadic generic context, it is going to be much more similar to working with a sequence. You have a series of values of unknown length, and it seems in general you're going to be applying generic operations over each of those values.
I wonder if this would be another good use-case for the ExpressibleByTupleLiteral proposed in the Vector Manifesto.
1 Like
Yeah sorry, what I meant was not that I wanted to stick with this limitation, only that at this moment I have not explored this "issue". Sorry if I wasn't clear!
Regarding the topic "tuples vs. other-data-struct" for VG I now want to explore this a bit, it may really be worth it to add a new built-in type / data structure in order to support VG if this new thing carries is weight! Especially because of what you say here:
No need to apologize! I understand this is a work-in-progress, and I only wanted to lend perspective on why I think this is a design consideration which is worth taking into account. Sorry if my comment came off as pointed!
1 Like
I think I may have figured out a way to support the example below using variadic generics without requiring any kind of static metaprogramming.
The idea builds on another generics feature I have often wanted: labeled type arguments. If we had labeled type arguments there could also be packs of labeled variadic type arguments which are able to use the label when they are unpacked. This allows us to introduce case and property declarations by expanding a labeled type argument pack.
The one part I’m not sure of is what the syntax would be - it would need to be distinguished from ordinary variadic type argument packs somehow.
1 Like
I have thought about that too actually, but did not mention it that much explicitly to not derail the main topic into a discussion about labels on generic type parameters. (IIRC I mentioned it previously in context of Tuple<T...>.)
The main issue still remains, it's almost impossible to express it nicely. With the current set of features VG would only allow enum cases with variadic payloads, but not variadic cases. Variadic cases with labels generated for each generic type parameter would be just perfect and extremely powerful tool at our sleeves.
Sure, but we’re talking about enhancements here. This could easily fit on the VG roadmap, which is pretty cool. It’s important to point out that the labels would be come part of the type. GenericPath<a: Int> would be a different type than GenericPath<b: Int>. (I believe row types are a related topic, is that correct @Joe_Groff?)
Do you have an idea how this would also fit in non VG generic context? Also there is this potential feature, which I never fully understood as the example contained let which I find confusing in that context:
@anandabits @DevAndArtist do you think that the ability to build an enum with a variable number of cases is something that can / should fit in VG or that it is something independent that can be introduced separately in the future? Because I'm not totally sure of this... I mean, if one needs a variadic enum but which cases all have (say) an Int payload? This is not going to be generic! So really I'm not sure.
In regard to Generic Value Parameters, I think it is a way to parametrize a type using a constant expression like an Int or maybe a String, i.e. something that is ExpressibleBySomeLiteral. It seems that the parameter can then be referred in the type as it was an instance property. In code I think it can be like:
struct Vector<N: Numeric, let Dimensions: Int> { }
func sum<N: Numeric, let D: Int>(v1: Vector<N, D>, v2: Vector<N, D>) -> N {
// perform sum on vector of same dimension
}
let v1: Vector<Int, 2>
let v2: Vector<Int, 2>
let v3: Vector<Int, 3>
dot(v1, v2) // ok
dot(v1, v3) // not, ok, `D` does not match
Do you think that VG and Generic Value Parameters can mix?
1 Like
I think it depends. If we have labeled generic type arguments already when a variadic generics is going through implementation and review it should be considered. But I don't think it's a requirement for an initial proposal. Variadic generics is substantial enough on its own. This could be a future direction.
I am hoping we eventually have this feature with support for any type that has an @compilerEvaluable initializer.
Absolutely! If we had both I can imagine two kinds of labeled packs of variadic values
One would work like variadics in functions do where the type specifies a value parameter type and the user provides multiple values:
static Struct S<let n: Int...> {}
S<n: 42, 43, 44>
Another would work more like the labeled type packs I described above, taking both a label and a value of any type meeting the necessary constraints (using row as a strawman syntactic marker for this labeled pack):
static Struct S<row let n: Numeric...> {}
S<first: 42, second: 43, third: 44>
This row syntax is completely spitballing. I don't know if it's any good or not but can't think of anything else to use at the moment. For reference, here is the example from upthread written with this syntax:
enum GenericPath<row T...>: Hashable where T: Drivable {
/* case per generic type parameter - made up syntax */
case ...$(PartialKeyPathSet<T.Driver>)
/* optional set per generic type parameter to extract the payload */
}
In the past few days I've worked on a copy of my document, fancifully named YYYY-variadic-generics.md; this doc does not replace the original one and is sitting alongside it in my brach.
In this document I changed the Detailed design section (and the zip re-implementation example) to explore an alternative syntax and model for Variadic Generics, one that does not require modifications to tuples and aims to be more minimalist with less ... and visual clutter. This required the introduction of a new concept to represent variadic types and values.
The document, like the original one, still lacks some things and examples like recurrent Variadic Generics, compile-time support for VG length etc.
What do you think of this new proposal? I might actually like it better...
1 Like