Produce a new type out of type parameter packs

Using Swift
1

Is it possible to join two type parameter packs to create another type?

Let's see the issue with a sample. Given the following protocol:

protocol Node {}

I create some concrete types:

struct Heading: Node {}
struct Paragraph: Node {}
struct Quote: Node {}
struct Image: Node {}
struct Table: Node {}

I also have a simple grouping type:

struct Group<each N: Node> {
  let nodes: (repeat each N)

  init(_ nodes: repeat each N) {
    self.nodes = (repeat each nodes)
  }
}

If I create two groups:

let lhs = Group(Heading(), Paragraph(), Quote())
// Group<Pack{Heading, Paragraph, Quote}>
let rhs = Group(Image(), Table())
// Group<Pack{Image, Table}>

Is it possible to join the two groups in a new group? I am not able to create a group eventhough the compiler is inferring the types correctly:

func joinNodes<each N1: Node, each N2: Node>(
  lhs: Group<repeat each N1>,
  rhs: Group<repeat each N2>
) -> (repeat each N1, repeat each N2) {
  (repeat each lhs.nodes, repeat each rhs.nodes)
}

let nodes = joinNodes(lhs: lhs, rhs: rhs)
// type(of: nodes): (Heading, Paragraph, Quote, Image, Table)

The compiler fails to compile the following line with error: Type '(Heading, Paragraph, Quote, Image, Table)' cannot conform to 'Node'.

let group = Group(nodes)

As far as I understand, the compiler is telling me that the type parameter pack is not conforming to Node, which is true. But each individual type does conform to Node, which it is what the initializer expects.

2

Interestingly, if I let the compiler generate the initializer for Group:

struct Group<each N: Node> {
  let nodes: (repeat each N)
}

Then the compiler just crashes:

let lhs = Group(nodes: (Heading(), Paragraph(), Quote()))
let rhs = Group(nodes: (Image(), Table()))

let nodes = joinNodes(lhs: lhs, rhs: rhs)
let group = Group(nodes: nodes)
1 Like
3

Not quite. joinNodes() returns a tuple, so you're calling Group.init() with a pack of length one, and the one element in that pack is your single tuple type, which is probably not what you meant here.

You can instead expand the tuple's elements into a pack:

let group = Group(repeat each nodes)

Or, you can change joinNodes() to return a Group instead of a tuple:

func joinNodes<each N1: Node, each N2: Node>(
  lhs: Group<repeat each N1>,
  rhs: Group<repeat each N2>
) -> Group<repeat each N1, repeat each N2> {
  return Group(repeat each lhs.nodes, repeat each rhs.nodes)
}

As for the crash with calling the synthesized memberwise init, so that it doesn't slip through the cracks, would you be so kind as to file a bug report? Issues · apple/swift · GitHub.

3 Likes
4

Thank you for the quick answer @Slava_Pestov.

Expanding the tuple's element into a pack doesn't work. It fails with 'each' cannot be applied to non-pack type '(Heading, Paragraph, Quote, Image, Table)'

func joinNodes<each N1: Node, each N2: Node>(
  lhs: Group<repeat each N1>,
  rhs: Group<repeat each N2>
) -> (repeat each N1, repeat each N2) {
  (repeat each lhs.nodes, repeat each rhs.nodes)
}

let nodes = joinNodes(lhs: lhs, rhs: rhs)
let group = Group(repeat each nodes)

However, changing the joinNodes() to return a Group instead of a tuple does work.

func joinNodes<each N1: Node, each N2: Node>(
  lhs: Group<repeat each N1>,
  rhs: Group<repeat each N2>
) -> Group<repeat each N1, repeat each N2> {
  Group(repeat each lhs.nodes, repeat each rhs.nodes)
}

let group = joinNodes(lhs: lhs, rhs: rhs)

I haven't still internalized when a tuple is a tuple and when it counts as expanded. It feels confusing. I guess I need to work a bit more with parameter packs. In any case, thank you very much. Your answer was very helpful.

Regarding the crash, I just filed it in the repo.

Regards,
Marcos

5

Here's another way to think of what's going on. In your original joinNodes, you have this:

func joinNodes<each N1: Node, each N2: Node>(
  lhs: Group<repeat each N1>,
  rhs: Group<repeat each N2>
) -> (repeat each N1, repeat each N2) {
  (repeat each lhs.nodes, repeat each rhs.nodes)
}

While the type of lhs.nodes (and rhs.nodes) is a tuple, by prefixing it with each you take the contents of the tuple out, which gives you a pack. Think of it like removing the outermost ( and ). You then use the repeat keyword to expand both of these packs into a single larger tuple type.

If you had instead written this,

(lhs.nodes, rhs.nodes)

Then joinNodes() would return increasingly-nested tuple types, where each tuple type only had two elements (one of either one could then be another tuple, and so on). In fact at this point you'd have nothing variadic, so joinNodes() could be a traditional generic function.

Similarly, if I have a tuple, then Group(myTupleValue), with Group.init declared as taking a parameter pack, will always call Group with a pack containing a single element, the tuple itself, because you haven't "removed" the outermost ( and ) using each.

By analogy with normal generics, the following two functions are different:

func f<T, U>(_: T, _: U) {}
func g<T, U>(_: (T, U)) {}

And f(1, 2) and f((1, 2)) mean different things (in fact the second is invalid of course, while g((1, 2)) is accepted).

5 Likes
6

I have re-read your answers a couple of times this morning and I now get the usage distinction between tuples and parameter packs. Thanks again.

I am interested in how to expand the tuple's element into a pack as you described in your 1st answer. Somehow, I don't manage to make it work.

I have re-rewritten the code to avoid confusion.

struct Group<each N: Node> {
  let nodes: (repeat each N)

  init(_ nodes: repeat each N) {
    self.nodes = (repeat each nodes)
  }

  func merging<each M: Node>(_ group: Group<repeat each M>) -> Group<repeat each N, repeat each M> {
    Group<repeat each N, repeat each M>(repeat each self.nodes, repeat each group.nodes)
  }
}

func joinNodes<each N1: Node, each N2: Node>(
  lhs: Group<repeat each N1>,
  rhs: Group<repeat each N2>
) -> (repeat each N1, repeat each N2) {
  (repeat each lhs.nodes, repeat each rhs.nodes)
}

The following snippet works (as we discussed previously):

let lhs = Group(Heading(), Paragraph(), Quote())
let rhs = Group(Image(), Table())
let groupA = lhs.merging(rhs)

However, your suggested snippet fails with 'each' cannot be applied to non-pack type '(Heading, Paragraph, Quote, Image, Table)'.

let nodes = joinNodes(lhs: lhs, rhs: rhs)
let groupB = Group(repeat each nodes)

Being able to expand tuple's elements would be incrediably useful.

7

A bit tangential to the topic at hand, but reading SE-0393 I see:

(Effect on ABI stability) This is still an area of open discussion, but we anticipate that generic functions with type parameter packs will not require runtime support, and thus will backward deploy. As work proceeds on the implementation, the above is subject to change.

Currently I need to specify a minimum requirement of .macOS(.v14) to use parameter packs. Can you give a bit more insight on this requirement and whether it is intended to change?

I am working in a new feature that fits the usage of parameter packs, but I wouldn't want spend to much effort on it if I will be unable to support macOS 12+ (which is my current requirement). If we are not having backward deployment, I will have to go the SwiftUI-like route with a lot of Tuple types instead.

8

Variadic generic structs and classes require runtime support, but not variadic generic functions or type aliases.

2 Likes
9

I raised this internally and @sophia pointed out that a tuple can only be expanded if it contains an "abstract" pack like (repeat each T); a concrete type (Int, String, Float) cannot be expanded yet. This is covered in future directions in https://github.com/apple/swift-evolution/blob/main/proposals/0399-tuple-of-value-pack-expansion.md#future-directions.

1 Like
10

I have continued playing with parameter packs trying to build more complex structures, but I am continuously hitting compiler issues (both regular errors and crashes).

Do you have any recommendation to better work with parameter packs?

I don't mind working with future toolchains. I did try main and Swift 5.10 Development, but those perform worst than the Xcode built-in toolchain in terms of stability when applied to parameter packs.

Some examples of issues:

Compiler is unable to correctly infer joint type pack.

In the following example, within the merging(_:) function, I must explicitly define Group<repeat each N, repeat each M>

public struct Group<each N: Node>: Sendable {
  public let nodes: (repeat each N)

  public init(nodes: repeat each N) {
    self.nodes = (repeat each nodes)
  }

  public init(@GroupBuilder _ builder: () -> Group<repeat each N>) {
    self = builder()
  }

  public func merging<each M>(_ group: Group<repeat each M>) -> Group<repeat each N, repeat each M> {
    // 🔥 This fails to compile
    Group(nodes: repeat each self.nodes, repeat each group.nodes)
  }
}

If I don't define it explicitly, the compiler fails with:

  • Cannot convert return expression of type 'Group' to return type 'Group<repeat each N, repeat each M>'
  • Cannot convert value of type 'repeat each N' to expected argument type 'repeat each N, repeat each M'
  • Extra argument in call
  • Pack expansion requires that 'each N' and 'repeat each N, repeat each M' have the same shape
Compiler fails when returning parameter pack value from result builder.

When trying to build a result builder using parameter packs, the compiler fails to match return types.

@resultBuilder public enum GroupBuilder {
  public static func buildExpression<N>(_ expression: N) -> Group<N> where N: Node {
    Group(nodes: expression)
  }

  public static func buildPartialBlock<each N>(first: Group<repeat each N>) -> Group<repeat each N> {
    first
  }

  public static func buildPartialBlock<each A, each N>(accumulated: Group<repeat each A>, next: Group<repeat each N>) -> Group<repeat each A, repeat each N> {
    // 🔥 This fails to compile
    accumulated.merging(next)
  }
}

The previous example funnily fails with:

  • Cannot convert return expression of type 'Group<repeat each A, repeat each N>' to return type 'Group<repeat each A, repeat each N>'
  • Pack expansion requires that 'repeat each A' and 'each A' have the same shape
  • Pack expansion requires that 'repeat each N' and 'each N' have the same shape

I can overcome the issue by simply adding a two step process like so:

public static func buildPartialBlock<each A, each N>(accumulated: Group<repeat each A>, next: Group<repeat each N>) -> Group<repeat each A, repeat each N> {
  let tmp = accumulated.merging(next)
  return tmp
}
Compiler crashes for previous examples in Swift 5.10 toolchain.

Not much to add, but the example above make the compiler crash in Swift 5.10 toolchain. I understand it is a snapshot, though. So regressions are to be expected.

I have been reading over the proposals (SE-0393, SE-0398, SE-0399, SE-0408), but they get too "involved" and I find them hard to follow up.

Is there other type of resource to better understand parameter packs?

11

Can you please file these issues at Issues · apple/swift · GitHub?

12

I already had issue them here and here. I haven't filed the compiler crashes for Swift 5.10 since I am supposing crashes are to be expected in snapshot toolchains. Should I file those crashes as well?

Any thoughts about the questions above?

Also, another tangential question: What does it mean "Implemented (Swift Next)" in the SE-0408 Pack Iteration proposal? Will it be implemented in Swift 5.10 or Swift 6?

13

if you’ve got time and a small test program that reproduces the crash, please file a report about it, a lot more people than you think are using the 5.10 snapshots :slight_smile:

14

Done. It can be found here.

15

I'm still confused. How can we pass a value pack that is (forced to be) returned as tuple from function?

protocol P {}
struct A: P {}
struct B: P {}

func foo<each T>(_ value: repeat each T) -> (repeat each T) where repeat each T: P {
    (repeat each value)
}

let a = foo(A(), B()) // `a` is abstract tuple?
foo(a) // Invalid: Type '(A, B)' cannot conform to 'P'
foo(repeat each a) // Invalid: 'each' cannot be applied to non-pack type '(A, B)'
16

You have to take the tuple apart by hand:

let (a, b) = foo()
bar(a, b)
17

This seems like a serious limitation (if I don't know the shape upfront, or is just too complex). Is it temporary or by design? I'm playing with a result builder that would flatten all components into one value pack but there is no way how to pass the final result into a function that takes pack.

18

It’s supported. In your example, foo() shouldn’t return a tuple, but just expansion result (that is, without parenthesis). I think @Slava_Pestov said by hand because you asked about tuple.

19

As far as I know (and compiler is telling), "Pack expansion 'repeat each T' can only appear in a function parameter list, tuple element, or generic argument of a variadic type". I think a pack expansion cannot be returned without being tupled.

20

Sorry, my mistake. I just checked my code and found I used a pattern like the following:

func foo<each T>(_ value: repeat each T)  {
    ...
    repeat bar(each value)
}

func bar<each T>(_ value: repeat each T)  {
    repeat each value
}

The key point is to never save pack expansion result in a tuple. Not sure if it helps in your case though.