Enhanced Variadic Parameters

(Matthew Johnson) #1

While I was looking at examples of EDSLs while considering the comma elision proposal I hit upon another idea that would open up some interesting possibilities for EDSLs. The idea is to significantly broaden the scope of what can be done with variadic parameters by lifting the requirement that variadic arguments be turned into an Array . Even more interesting is expanding variadics further to include labeled variadic arguments which are treated as a dictionary literal.

Here is a simple example of what labeled variadics can do in conjunction with comma ellison:

enum ColumnType { case int, string, ... }
struct Table {
    init(_ columns: variadic [String: ColumnType]) { ... }
}
Table(
    firstColumn:  .int
    secondColumn: .string
)

// without labeled variadics and comma ellision:
Table([
    “firstColumn”:  .int,
    “secondColumn”: .string
])

It is hard to argue that the additional syntax in the latter example aids clarity. In fact, it is pretty easy to argue the opposite: the additional syntax gets in the way.

This syntax for declaring a table scheme is extremely lightweight. It is

I will keep an updated copy of the proposal at this gist. I am also including the initial draft below.

Enhanced Variadic Parameters

Introduction

This proposal enhances variadic parameters by introducing support for user-defined types, labeled variadic parameters, and a choice at the call site of whether to provide a variadic argument list or a collection value.

Swift-evolution thread: Discussion thread topic for that proposal

Motivation

Variadic parameters are a nice feature in Swift. They allow us to elide array brackets at a call site. Unfortunately this elision comes with a cost: even though variadics formally take an Array one cannot be provided at a call site. This often results in an overload set to support direct arrays as well:

func compute(ints: Int...) -> Int {
   return compute(ints: ints)
}
func compute(ints: [Int]) -> Int { ... }

Further, Array is priveleged as the only type that is supported by variadic parameter syntax. This is at odds with Swift's otherwise pervasive support for user-defined types in its literal syntax.

Proposed solution

We should reimagine variadic parameters as simple syntactic sugar for optionally eliding collection literal brackets at the call site.

Syntax

Swift magically turns the current T... type syntax into [T] as the type used in the body of a function. Having the Array assumption baked directly into this syntax means it will not work with other types.

In order to support types beyond Array the variadic parameter modifier is used:

func compute(ints: variadic [Int]) -> Int {...} 

// callable in any of these ways:
compute(ints: 42, 43, 44)
compute(ints: [42, 43, 44])
compute(ints: arrayOfInt)

ExpressibleByArrayLiteral

All ExpressibleByArrayLiteral types are supported. The compiler will use the type's init(arrayLiteral:) initializer to create a value from variadic arguments provided at the call site.

func compute(ints: variadic Set<Int>) -> Int {...} 

// callable in any of these ways:
compute(ints: 42, 43, 44)
compute(ints: [42, 43, 44])
compute(ints: setOfInt)

ExpressibleByDictionaryLiteral

ExpressibleByDictionaryLiteral types are also supported with a variadic list of labeled arguments . The labels in this list are key literals and the arguments are values in a dictionary literal.

In order to use the type's init(dictionaryLiteral:) initializer arguemnt labels must be turned into values of type Key . This proposal includes two ways to do that.

ExpressibleByStringLiteral keys

When the key is ExpressibleByStringLiteral , the argument label can be passed to init(stringLiteral:) to create a value of type Key .

func compute(pairs: variadic [String: Int]) -> Int { ... }

// callable in any of these ways:
compute(first: 42, second: 43, third: 44)
compute(pairs: ["first": 42, "second": 43, "third": 44])
compute(pairs: dictionaryFromStringToInt)

Notice that this sugar allows not only the dictionary literal brackets to be ellided, but the string quotes and the eternal pairs label as well. pairs is only used when a collection value is passed directly and variadic syntax is not used.

This feature is not limited to Dictionary :

// assume OrderedDictionary: ExpressibleByDictionaryLiteral
func compute(pairs: variadic OrderedDictionary<String, Int>) -> Int { ... }

// callable in any of these ways:
compute(first: 42, second: 43, third: 44)
compute(pairs: ["first": 42, "second": 43, "third": 44])
compute(pairs: orderedDictionaryFromStringToInt)

Enum keys

When the key is of an enum type that does not have any associated values keys can be required to match the identifiers of its cases.

enum Foo { case foo, bar, baz }
func compute(pairs: variadic [Foo: Int]) -> Int { ... }

// callable in any of these ways:
compute(foo: 42, bar: 43, baz: 44)
compute(pairs: [.foo: 42, .bar: 43, .baz: 44])
compute(pairs: dictionaryFromFooToInt)

Notice that this sugar allows not only the dictionary literal brackets to be ellided, but the dot prefix on the enum case identifier and the eternal pairs label as well. pairs is only used when a collection value is passed directly and variadic syntax is not used.

This feature is not limited to Dictionary :

enum Foo { case foo, bar, baz }
func compute(pairs: variadic OrderedDictionary<Foo, Int>) -> Int { ... }

// callable in any of these ways:
compute(foo: 42, bar: 43, baz: 44)
compute(pairs: [.foo: 42, .bar: 43, .baz: 44])
compute(pairs: orderedDictionaryFromFooToInt)
8 Likes
SE-0257: Eliding commas from multiline expression lists
[Accepted with Modification] SE-0253 - Callable values of user-defined nominal types
(Tony Allevato) #2

I think there's some interesting potential here. Your String-based examples can be hacked together with @dynamicCallable today, but with big caveats:

enum ColumnType { case int, string }

@dynamicCallable
struct _Table {
  func dynamicallyCall(withKeywordArguments args: KeyValuePairs<String, ColumnType>) {
    print(args)
  }
}

let Table = _Table()

Table(
  firstColumn:  .int,
  secondColumn: .string
)

As you can see, it's quite awkward—the @dynamicCallable attribute attaches to the type, but the function application is on an instance of that type, and it doesn't support non-String keys from what I can tell.

So, your proposal would be a nice improvement on that, and would be a big win for people who want to use Swift in "configuration language" use cases. It's a nice parallel to how dynamicallyCall already works, and perhaps could even reuse some of the existing compiler implementation?

Possible enhancement: bounds for array-based variadics

Variadic arguments are always zero or more, period. Have you ever had a situation where you wanted a function to require, say, one or more items? You have to do something like this:

func compute(items first: Int, _ rest: Int...) { ... }

But now you awkwardly have the elements split across multiple variables and you have to re-combine them or tweak your implementation to handle that case.

What if the variadic keyword could accept a compile-time-computable non-negative RangeExpression and the compiler would enforce that the callers pass only valid numbers of arguments in that range?

// Cleaner representation of the example above
func compute(items: variadic(1...) [Int])

// If omitted, treat it as 0...
func compute(items: variadic [Int])

// Can only pass 0, 1, 2, or 3 items
func compute(items: variadic(..<4) [Int])

// Sure, why not...
func compute(items: variadic(17...94) [Int])

I don't know if this could easily extend to the dictionary case—I suppose the equivalent there would be providing a list of required keys in the variadic() modifier. I'm less enthusiastic about that, just because it doesn't seem as elegant.

Bold source-breaking suggestion that probably won't happen

Deprecate and remove the T... syntax after this?

3 Likes
(Slava Pestov) #3

One possible direction here would be to extend @dynamicCallable to allow the dynamicallyCall function to be static, and also to accept an enum type in addition to String as the key type of the list of pairs.

3 Likes
(Andrea Tomarelli) #4

This seems to be a very nice addition to the language! And I totally agree with:

In my opinion this design also fits well with my latest document on variadic generics, in which I'm proposing the variadic keyword to create a VG:

// Old fashioned way: a function that accepts one or more `P`s, all of the
// same concrete type. `values` is an `Array` inside the function.
func variadicFn1<P: SomeProto>(_ values: P...) { }

// Proposed syntax: a function that accepts a list of `P`s as a real list, as
// an array literal or as an `Array` instance (all values have the same
// concrete type). `values` is an `Array` inside the function.
func variadicFn2<P: SomeProto>(_ values: variadic [P]) { }

// Proposed syntax: a function that accepts a dictionary from `String`s to
// `P`s as a real list (with labels), as a dictionary literal or as a `Dictionary`
// instance (all values have the same concrete type). `values` is a
// `Dictionary` inside the function.
func variadicFn3<P: SomeProto>(_ values: variadic [String: P]) { }

// Variadic Generics syntax: a function that accepts one or more `P`s,
// all of potentially different concrete types. `values` is a "pack of values"
// (strawman WIP name) inside the function.
func variadicFn4<variadic P: SomeProto>(_ values: P) { }
(Adrian Zubarev) #5

I'm still in the process of understanding the whole pitch but I want to start with one concern I already have.

func foo(_: variadic Set<Int>)

This seems great at first glance but I think it's course is set against ambiguity, because I think it could prevent some retroactive extensions as they might cause ambiguity. I said could/might on purpose as I'm not 100% sure.

Here is one case I would like to be valid in Swift one day.

enum E {
  case something(Set<Int>)
  static func something(_ values: Int...) -> E {
    return .something(Set(values))
  }
}

If we would now only allow variadic instead of T... then the above will never be possible, which means that if the case payload itself is not declared as variadic Set<Int> I will never gain that convenience.

Here is one more concrete example:

extension NSLayoutConstraint {
  func activate(_ constraints: NSLayoutConstraint...) { ... }
}

The original API already has an [NSLayoutConstraint] overload which allows me still retroactively extend the type with a variadic overload. However if my custom extension would be variadic [NSLayoutConstraint] then passing an array literal or a concrete array instance would be ambiguous because variadic [NSLayoutConstraint] allows these cases but NSLayoutConstraint... does not.

let array = [42]

func bar(_: [Int]) {}  // #1
func bar(_: Int...) {} // #2

bar([42])  // okay #1 called
bar(array) // okay #1 called

func foo(_: [Int]) {}          // #3
func foo(_: variadic [Int]) {} // #4

// With the pitched *new* rules in mind:
// Why should the compiler prefer #3 here? I think it would be ambiguous.
foo([42]) 
foo(array)
#6

My understanding of the pitch is that the new syntax informs the compiler how to consolidate a list of parameters into a single collection. In these examples, the element type is Int, not [Int], so the only overload that matches is #3. The effect of variadic [Int] would be identical to Int..., including with respect to overload resolution.

(Adrian Zubarev) #7

That is what I would wish for, but I think the proposal says that variadic [Int] will allow us to pass instances of [Int] to a variadic function, which is different from the current behavior of Int....

ExpressibleByArrayLiteral

All ExpressibleByArrayLiteral types are supported. The compiler will use the type's init(arrayLiteral:) initializer to create a value from variadic arguments provided at the call site.

func compute(ints: variadic [Int]) -> Int {...} 
// callable in any of these ways: 
compute(ints: 42, 43, 44)   // ✅ okay, but not allowed for all `ExpressibleByArrayLiteral` types
compute(ints: [42, 43, 44]) // ❌ this is new and different from `Int...`
compute(ints: arrayOfInt)   // ❌ this is new and different from `Int...`

I think only compute(ints: 42, 43, 44) should be allowed as this won't create any issues with retroactive overloads.

#8

I missed that the first time around. I agree with you that the ambiguity would be problematic.

1 Like
(Dante Broggi) #9

Swift already allows enums as the key type in @dynamicCallable if the enum is ExpresibleByStringLiteral, however the compiler does not constant-fold the enum init, so the error for invalid argument names is deferred to runtime.

#10

This wouldn't address the other issue with variadic parameters (probably even more common than “I want a set and not an array” and oft-discussed on these forums), that you often want to call a variadic function with an existing array as well as using variadic syntax. You currently have to write two functions and forward the variadic version to the other.

Now, as to ambiguity, if this behaviour was restricted to variadic [T] and didn't apply to T... then it wouldn't be a backwards compatibility issue, though it may be considered confusing. Deprecation of T... could make sense in that context.

This particular idea has been discussed in similar forms a few times, but I think adding dictionary literals is a new and interesting touch.

1 Like
(Adrian Zubarev) #11

I don't understand. The only rule that makes sense to me for variadic compared to ... is the expansion on every ExpressibleByArrayLiteral types. Then you can write variadic Set<Int> and use it the same way you would use it with Int... but the underlying type will become Set<Int> instead of Array<Int>, I don't see any problems here. The other two rules that are discussed on SE quite often would be source breaking as they will create ambiguity of previously valid Swift code.

On the other hand we could say that if you have two functions like foo(_: [Int]) and foo(_: variadic [Int]) that the compiler will prefer the former if you pass a literal or an array instance to foo, but this completely defeats the purpose here as we then gained nothing from having the extra too rules on variadic.

#12

If allowing you to pass an array or array literal as well as using variadic syntax only applies to variadic [T] and not T... then it cannot be source breaking, because variadic [T] does not exist yet. If you already have a [T] or T... function and you decide to add a variadic [T] function then you've created a new ambiguity, which is not a source compatibility issue and can be handled with warnings, errors, or “specificity rules for disambiguation” as appropriate

(Adrian Zubarev) #13

And where would I end up if T... is deprecated? I will need to use T... to have the convenience I want but would need to abandon variadic as it does not play well with retroactive extensions. You simply lose the ability to extend a type you don't own that has a non-variadic member with a variadic custom extension as this will generate ambiguity.

// module A
public struct S {
  public func makeSomeMagic(_ values: Set<Int>) { ... }
}

// in your module 
import A
extension S {
  func makeSomeMagic(_ values: variadic Set<Int>) { 
    self.makeSomeMagic(values) // 🚧 infinite recursion!!!
  }
}  

let s = S()
// great 😓
s.makeSomeMagic([42]) // error: ambiguous without more context
#14

I don't know what “does not play well with retroactive extensions” means exactly. Retroactive extensions can already cause issues with clashing names/signatures. Are any of your concerns here specific to this proposal?

extension Array {
  subscript(_ i: Int) -> Int { return 99 }
}

let a = [0, 1, 2, 3]
a[0] // error: ambiguous use of 'subscript(_:)'
(Adrian Zubarev) #15

In one sentence: The two extra rules prevent you making non-variadic API you don't own variadic.

#16

Oh, with the same base name, sure, without some “specificity rule for disambiguation” like I mentioned. I don't see how this holds though, then

This is a possible disambiguation rule, and I don't see how this “completely defeats the purpose” because it allows you to write a single function instead of two in the common case.

(Adrian Zubarev) #17

I said it multiple times already, this is only true for code you own. I think the general rule should be that you should design your API non-variadic first, if you users then want to make some parts of it variadic that should be able to without any need of creating alternative names. That is already possible with [T] and T... today. This should remain possible with R: ExpressibleByArrayLiteral and variadic R. If on the other hand you decide to go variadic form the beginning, fine there won't be any issues for your users as they still can choose how to use your API, either in variadic or the non-variadic way, but that requires you to provide a little overload, the same way your users could transform your non-variadic API into variadic API.

If on the other hand variadic would have the two extra rules, you will lose that ability.

I think the ability of making non-variadic API retroactively variadic is a fair trade-off and worth a little custom overload.

#18

Code you own is the common case, yes. In what way would a disambiguation rule like the one you mentioned hurt either “code you own” or “retroactively making someone else's code variadic without creating alternative names”?

(Adrian Zubarev) #19

Let's say R is a type that conforms to ExpressibleByArrayLiteral, for simplicity we ignore any Element type.

As currently proposed a single variadic construct should cover three cases:

  1. You should be able to omit []
  2. You should be able to pass an array literal to variadic R
  3. You should be able to pass an instance of R to variadic R

In current Swift T... allows only (1) and it generates always an instance of [T].

A library author can decide if the library should require only R or can make it variadic R. In case of variadic R there won't be any issues at all, not for the library itself nor for its users. However if the author goes with non-variadic approach then the library user may want to overload existing non-variadic API with a variadic version.

Rules (2) and (3) will generate an ambiguity if the overloaded API share the same signature except for the now variadic type. I mentioned above that we could say that the compiler can in this case prefer the non-variadic API. That however makes these two rules less useful. Furthermore the variadic overload can potentially be impossible because it will cause an infinite recursion (like showed above), but that is again only caused by rules (2) and (3).

So far these issues arise for functions/methods, but what about an enum payload?
If the enum case for enum from a module you don't own is non-variadic, the user won't be able to extend it to make the payload seem variadic, it will require awkward new names.

extension Optional where Wrapped: ExpressibleByArrayLiteral {
  static func some(_ wrapped: variadic Wrapped) -> Optional {
    // with presence of rules (2) and (3) this is an infinite recursion
    return .some(wrapped) 
  }
}

// with only rule (1) the use can now write
Set<Int>?.some(1, 2, 3, 4, 5) 
// if rules (2) and (3) are there the following two examples ambiguous 
Set<Int>?.some([1, 2, 3, 4, 5]) 
Set<Int>?.some(intSet)

That said, allowing rule (1) is a huge enhancement, but I do not support rules (2) and (3) because of the disadvantages they come with.

#20

I think you've made a great argument for a disambiguation rule to better support retroactive conformance, but I'm still not convinced that such a rule is impossible. I only just saw the recursion example above (did you edit it in?), but:

With your proposed disambiguation rule, essentially preferring the non-variadic overload when otherwise ambiguous (a nice match to other existing “more specific” disambiguation rules), wouldn't the “infinite recursion” line actually call the version from module A, it being the more specific match for Set<Int>, and therefore not be an infinite recursion? What am I missing here? And how does this make the rules “less useful”? Similarly for your example in this post (why wouldn't .some(wrapped) prefer the non-variadic version, and it would leave neither of the last two examples ambiguous).