[Review] SE-0408: Pack Iteration

ensan-hcl · 2023-09-10T00:16:57.825Z

Can "for expression" be a future direction?

As the length is statically known, I believe it makes a lot of sense to have something like

let sum = for left, right in repeat(each left, each right) { left + right }

If we adopt then statement (Introduce `then` statements by hamishknight · Pull Request #67454 · apple/swift · GitHub) officially, we will be able to write this.

func allUnwrapOrNil<each Element>(over element: repeat (each Element)?) -> (repeat each Element)? {
    let unwrappedValues = for value in repeat each element {
        if let value {
            then value
        }
        return nil
    }
    return unwrappedValues
}

simanerush · 2023-09-10T00:23:58.631Z

Thank you for such detailed feedback! However, this post is intended to be a place where people can experiment with the feature. The syntax and the direction of this feature were discussed in a [Pitch] Enable pack iteration post.
As per what this proposal suggests, we are aiming to allow Swift users to iterate over packs and explicitly bind an element to a local variable. As @hborla mentioned, one of the important purposes of this proposal (and something that your syntax doesn't support) is to allow pattern matching with an element of the expansion. Also, the proposed syntax follows the progressive disclosure paradigm, where we let users use the familiar for-each syntax for pack iteration. Since you mention that your syntax can be equivalent to the normal for-each with a Sequence, it is debatable to add a new syntax for essentially the same logic as the existing for loop conveys.
Since the feature you are proposing seems to be orthogonal to this proposal, I would encourage you to move this discussion to a separate thread where you can explicitly propose the changes you'd like to make

simanerush · 2023-09-10T00:31:31.410Z

ensan-hcl:

As the length is statically known

Actually, the length of a pack is not statically known since it conveys generic context. In your sum example, we cannot make any assumptions about the length of a pack.

As for enabling for expressions, I think it is worth considering it as a general future direction, not for this proposal itself.

John_McCall · 2023-09-10T00:36:17.046Z

simanerush:

Thank you for such detailed feedback! However, this post is intended to be a place where people can experiment with the feature. The syntax and the direction of this feature were discussed in a [Pitch] Enable pack iteration post.

To be clear, design feedback is definitely still on-topic during the review. If you agree that the feedback is suggesting a better design, you're welcome to make changes to your proposal; similarly, the Language Steering Group might decide that we agree with the feedback and ask you to make those changes. But of course both you and the LSG are also entitled to disagree with the feedback and not make any changes at all.

ellie20 · 2023-09-10T00:54:43.500Z

ensan-hcl:

let sum = for left, right in repeat(each left, each right) { left + right }

I'm confused, isn't this particular example the same as let sum = repeat ((each left) + (each right))?

simanerush:

As @hborla mentioned, one of the important purposes of this proposal (and something that your syntax doesn't support) is to allow pattern matching with an element of the expansion.

It does support it though, albeit more verbosely:

repeat Element {
    guard case .one(let value) = each element else {
        continue
    }
    ...
}

simanerush:

Since you mention that your syntax can be equivalent to the normal for-each with a Sequence, it is debatable to add a new syntax for essentially the same logic as the existing for loop conveys.

I was only intending to show how they would have a similar level of clarity with regards to what is being iterated over and what values are updated with each iteration. There are still other important differences in my opinion.

simanerush:

Also, the proposed syntax follows the progressive disclosure paradigm, where we let users use the familiar for-each syntax for pack iteration. Since you mention that your syntax can be equivalent to the normal for-each with a Sequence, it is debatable to add a new syntax for essentially the same logic as the existing for loop conveys.

But the syntax feels like a special case that clashes with the rest of the language design. There's no other place in the language where packs and tuples are treated analogously to runtime collections like arrays; even the indexing syntax is different (x.0 vs x[0]).

The value of progressive disclosure, in my opinion, is extending concepts you already know in more advanced ways. In my opinion, the for-each syntax doesn't really expand or generalize to anything, it's kind of just a "dead end" in being a special case. People who use value and type packs would be familiar with the repeat keyword, and the statement form would extend that to code blocks rather than just expressions. The repeat statement would then naturally extend to using unpacked values as lvalues, unpacking different value packs throughout the code block, etc.

simanerush · 2023-09-10T01:38:02.204Z

The proposal includes an example implementation of == for tuples. How would it look like with the syntax you're proposing?

More specifically, I'm wondering what the rules are on what should go after the repeat keyword in your syntax.

ellie20 · 2023-09-10T01:41:22.686Z

func == <each Element: Equatable>(lhs: (repeat each Element), rhs: (repeat each Element)) -> Bool {
    repeat Element {
        guard each lhs == each rhs else {
            return lhs
        }
    }
    return true
}

The thing that goes after the repeat syntax is the pack type that's being iterated over. The pack type is written in order to be more obvious about the "shape" that's being iterated over, since otherwise that information can be lost in the potentially complex body.

anthonylatsis · 2023-09-10T02:12:25.169Z

ellie20:

The thing that goes after the repeat syntax is the pack type that's being iterated over

How would this rule scale to pack expansion expressions involving multiple different abstract packs, e.g. for _ in repeat (each t, each u), for _ in (repeat each t, repeat each u, "hello") or for _ in repeat f(each x, g(each y))?

ellie20 · 2023-09-10T02:24:59.386Z

anthonylatsis:

How would this rule scale to pack expansion expressions involving multiple different abstract packs, e.g. for _ in repeat (each t, each u), for _ in (repeat each t, repeat each u, "hello") or for _ in repeat f(each x, g(each y))?

Your first and third examples would be written like this:

repeat T {
    _ = (each t, each u)
}

repeat T {
    _ = f(each x, g(each y))
}

If t and u are of the same type pack, you write down the type pack. If they're different type packs but the type packs have the same shape, I don't know. Either you just pick one of them arbitrarily, or we require that people list them all. The same issue would arise in a (repeat T in f()) or (repeat<T> f()) syntax; it would be arbitrary whether to pick T or U if both have the same shape.

I don't think there's a better way unfortunately, unless we were to introduce "shape parameters" or something instead of having same-shape requirements solely be inferred. Making the type pack inferred (like in expressions) instead of having it written explicitly would cause clarity issues like @rdemarest and @simanerush talked about.

Your second example is something I never considered, that not all packs have an easily nameable type. In that case, the logical conclusion of my idea would be repeat (repeat each T, repeat each U, String), but that's cumbersome (and since it's only the shape that matters, it doesn't matter whether it's String or Int, so if we go down this route we should probably just have people type _ for single elements) So maybe we should make explicitly writing the type pack optional, at least for the simple cases, so that it can simply be written like this:

let a = (repeat each t, repeat each u, "hello")
repeat {
    _ = each a
}

ensan-hcl · 2023-09-10T04:10:29.824Z

ellie20:

I'm confused, isn't this particular example the same as let sum = repeat ((each left) + (each right))?

My point is the latter example rather than the first one. You are right that repeat is enough in the former example.

I don't support repeat each Element syntax you proposed. for statement can be extended for raw tuple like for value in repeat each (1, true, "swift") {}, but your syntax seems not suitable for that purpose.

Edit: Ahh I’m really sorry but I meant to reply to @ellie20

ellie20 · 2023-09-10T04:50:32.260Z

For an actual tuple without any pack expansions, the pack-iterating for loop would be written as for value in (1, true, "swift"), which would be syntactically identical to a normal for loop, making it appear as if a tuple is a Sequence.

But I've just now realized that the original SE-0393 proposal has pack expansion types as something separate from tuple types, with "Modeling packs as tuples with abstract elements" listed as a rejected alternative. I don't think it's actually possible to unpack an arbitrary tuple (including a tuple made from a pack like (repeat each t, repeat each u, "hello")) to a pack expansion type, and therefore use it in pack iteration, without introducing a new feature for it. I assume not because that feature is talked about as a future direction in the "Value expansion operator" section.

If that feature is ever added, then you could write something like this:

repeat {
    let i = each (1, true, "swift").element
}

// with tuples outside the loop and multiple tuples
let tuple1 = (1, true, "swift")
let tuple2 = ("", 0, false)
repeat {
    let i = each tuple1.element
    let j = each tuple2.element
}

// with the pack-iterating for loop in the pitch
for i in repeat each (1, true, "swift").element {
    ...
}

If we do get the ability to make pack expansion values out of tuples, and thus have pack expansion types that are tedious to write out, then I would think my syntax idea might be better off with the written-out pack type being optional. Or with some other syntax. My guess is that the scenarios where a pack expansion type that's iterated over can't be spelled easily will be relatively uncommon.

ensan-hcl · 2023-09-10T07:33:01.907Z

ellie20:

without introducing a new feature for it

Yes, and this is mentioned as a future direction in SE-0399. That's why I said 'can be extended'.

ellie20:

If we do get the ability to make pack expansion values out of tuples, and thus have pack expansion types that are tedious to write out, then I would think my syntax idea might be better off with the written-out pack type being optional. Or with some other syntax. My guess is that the scenarios where a pack expansion type that's iterated over can't be spelled easily will be relatively uncommon.

Oh, I understood that you are suggesting the use of repeat keyword and not repeat T {} specifically. Then my point above is not applicable to your argument.

xAlien95 · 2023-09-10T10:51:02.235Z

Douglas_Gregor:

What is your evaluation of the proposal?

Highly positive. The proposal fills an important gap in language usability and expressivity.

Douglas_Gregor:

Is the problem being addressed significant enough to warrant a change to Swift?

Yes. While it's true that there are means to get the same behavior via local throwing functions, that approach isn't neither scalable nor intuitive. E.g., nested and labeled for loops involving continue <label> or break <label> statements aren't easily convertible to repeated invocations of throwing functions and there's no guarantee that those abstractions can be optimized by the compiler.

Douglas_Gregor:

Does this proposal fit well with the feel and direction of Swift?

Yes, it does. I would hardly think of an alternative syntax better that the one proposed.

Douglas_Gregor:

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

Python is the first language which comes to mind. You can use for...in loops with both lists and tuples:

for x in [1, 2, 3, 4, 5]:
    print(x)

for y in (1, 2, 3, 4, 5):
    print(y)

With this proposal and in a future in which concrete tuples can be implicitly converted to tuples of value packs, users may transition more easily from Python to Swift.

Douglas_Gregor:

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

A quick reading, but accurate enough to notice some cpp left-overs

github.com

apple/swift-evolution/blob/f48821cc2c74c888734a44227f3a627294f68dc9/proposals/0408-pack-iteration.md?plain=1#L27-L28


      
          if (left == right)
            return

simanerush · 2023-09-10T18:26:08.407Z

Thank you for the feedback and for catching the typo, I've updated the proposal!

Nobody1707 · 2023-09-10T23:43:37.283Z

+1 This is needed functionality, and this seems to me to be the natural spelling for pack iteration in Swift.

Also excited for the mentioned future extension to guard let.

jrose · 2023-09-17T22:56:00.478Z

Sidebar discussion about iterating over a pack and a collection at the same time

I think this is a pretty good solution, and sorry if this is so much of a Future Direction as to be off-topic, but is there space in this syntax to allow iterating over a pack and a collection at the same time? In particular, I can easily imagine it would be nice to have some equivalent of enumerated:

for (i, next) in repeat (each 0..., each element) {
  print(i, next)
}

I don’t think this exact syntax works because the compiler can’t prove 0... is at least as long as each element. Maybe it’ll have to use some function like

// UNTESTED

// A bit clunky, used to construct a pack
// with the same length as another pack
// but a different element type.
// No idea if it works.
typealias Second<FirstType, SecondType> = SecondType

func take<each PackElement, Items: Sequence>(
  from items: Items,
  matching packElement: repeat each PackElement
) -> (repeat Second<each PackElement, Items.Element>) {
  var iter = items.makeIterator()
  return (repeat (each packElement, iter.next()!).1)
}

for (i, next) in repeat (
  take(from: 0..., matching: each element),
  each element
) {
  print(i, next)
}

Now that I write that out I think that makes sense. It’s a bit verbose, but it makes the collection-to-pack conversion explicit.

simanerush · 2023-09-25T03:56:15.878Z

The untested code actually already works under this proposal! But because you're making a new iterator every time take is called, i will always be 0. Here's the modified version that will produce the desired behavior (it also already works):

func enumerated<each T>(_ t: repeat each T) -> (repeat (Int, each T)) {
  func advance(_ i: inout Int) -> Int {
    defer { i = i + 1 }
    return i
  }
  var index = 0

  return (repeat (advance(&index), each t))
}


func variadic<each Element>(element: repeat each Element) {
  let enumerated = enumerated(repeat each element)
  for (i, el) in repeat each enumerated {
    print("\(i): \(el)")
    // 0: first
    // 1: 3
    // 2: true
  }
}

variadic(element: "first", 3, true)

Also this example helped to uncover a bug in the current implementation: getting rid of enumerated variable and replacing it with the call to the enumerated function will crash the compiler😅 Will be working on fixing that.

jrose · 2023-09-25T13:48:23.386Z

And to do that with an arbitrary collection, you’d use an Iterator instead of Int! Thanks, Sima.

Douglas_Gregor · 2023-09-29T18:18:23.400Z

This proposal has been accepted, thank you everyone!

John_McCall · 2023-09-29T18:32:56.162Z