[Review] SE-0408: Pack Iteration

@Morten_Bek_Ditlevsen Could you please provide an example of how you ran sorted4? I'm having trouble reproducing the crash.

Certainly - I’m building with the provided macOS toolchain from Xcode. I can package up the project in the morning.

Excellent to hear that closures will be supported too! Thanks for the awesome work on this!

1 Like

Thank you!! I'd love to take a look at the Xcode project🙂

Hi again - here's a link through WeTransfer. Let me know if it's better to share it in some other fashion.

I'm running the project on the iPad Simulator running the latest 17.0 beta - and using the toolchain for macOS shared in this thread.

1 Like

This already means something under SE-0393; it's a pack expansion expression with a closure pattern.

A similar syntax, but using for instead of repeat, was discussed in the pitch thread as sugar for what is presumably the most common use-case for pack iteration: for each element { ... } could be sugar for for element in repeat each element { ... }. It was discussed as sugar for what's proposed here because for each element { ... } or repeat each element { ... } is not sufficient in the general case of binding a name to an element of a pack expansion, because pack expansion patterns are not always just a single pack reference, and this syntax also doesn't support pattern matching within an element of the expansion. There are several examples in the proposal demonstrating this:

for (left, right) in repeat (each lhs, each rhs) {

for x in repeat Generic<each Element>() {

for case .one(let value) in repeat each element {

for value in repeat printAndReturn(each t) {

In my opinion, the most important part of each of the above examples is binding a name to elements of the pack expansion. The repeat each Element suggestion which only names a type pack is more akin to explicitly parameterizing a pack expansion expression with a type pack to expand over. That is a feature I think would be valuable for pack expansion expressions, but I don't see any reason why a loop statement should be privileged with explicit parameterization but not expressions. And I think if we were to go in that direction, we would pick a syntax that makes it more clear that each Element is a parameter to repeat, such as repeat<each Element> f(), and this syntax also makes it possible to express concrete patterns that are repeated N times where N is the length of the each Element pack. To apply this to both expressions and your suggested new repeat statement, there also needs to be some syntactic difference between a repeat statement and a repeat expression with a closure pattern.

In any case, I think this is just a different feature from what's being proposed here. A major part of the goal here is to enable iteration in a way that's more natural to Swift programmers. Inventing a fancy new explicit parameterization syntax for repeat -- though a powerful feature that I think is worth exploring! -- undermines the specific goal of this proposal.

The future direction section mentions guard let. Supporting if let or guard let in the future relies on having local variable packs, which is a separate piece of functionality that would need to be proposed and implemented.

The downside is that you cannot clearly see the value packs that are expanded as part of the iteration; those are easily lost deep within the repeat loop body. I also don't think that a new syntactic form of a for loop is inherently bad. You won't encounter it unless you're inside code using parameter packs, where you will have already encountered the repeat and each syntax. The addition of the repeat keyword in for-in repeat makes it clear that the for loop is iterating over a different kind of value, and the body of the for loop is regular Swift code that's operating over opaque types. Operating over opaque types in the body of the loop is not different than if we were to allow implicit existential opening for for-in loops over collections of existential types (another valuable feature that I think is worth pursuing!).

6 Likes

The downside though is that every single unpacked value has to be listed at the top, which I worry might get cluttered for more complex code. Specifying the pack type that's being unpacked feels, to me at least, like a nice middle ground between being too verbose (explicitly listing each pack variable that's being unpacked) and not being sufficiently clear what pack is being iterated over. In my opinion, there wouldn't be much difference between explicitly specifying the pack type to iterate over, while using the keyword each before each unpacked value, and specifying a single iteration value i and then accessing the different properties of it.

// The collection is explicitly specified here
for i in someCollection() {
    // The `i.` indicates that the value varies with each loop
    let x = i.x 
    let y = i.y
}

// The iterated pack type is explicitly specified here
repeat Element {
    // The `each ` indicates that the value varies with each loop
    let x = each x
    let y = each y
}

Although, there is still the issue where if you want to iterate over different pack types that are related with a same-length requirement, it'd be arbitrary which of the pack types you'd choose.

I also worry that the analogy to a runtime for loop conveys the wrong intuition. Pack expansion comes with a lot more compile-time guarantees, and in some ways would be more powerful; for example, multiple packs can be statically guaranteed to have the same length, so you can use different pack variables all throughout the loop body (in a way that could otherwise be too cluttered if all the different packs were bound at the top of the loop). In other ways, it would be less powerful; for example, if you want to initialize each value of a tuple within a pack iteration (like in the allUnwrapOrNil hypothetical), it would be impossible to break out of the loop. I think the two types of loops might be more easily understood as separate/orthogonal features.

Yeah, that would be nice. If we were to add a repeat Element or repeat each Element syntax (which I'm not entirely sure of myself) then perhaps for consistency it could be repeat Element in f() or something like that, in an analogy to closure syntax.

Can "for expression" be a future direction?

As the length is statically known, I believe it makes a lot of sense to have something like

let sum = for left, right in repeat(each left, each right) { left + right }

If we adopt then statement (Introduce `then` statements by hamishknight · Pull Request #67454 · apple/swift · GitHub) officially, we will be able to write this.

func allUnwrapOrNil<each Element>(over element: repeat (each Element)?) -> (repeat each Element)? {
    let unwrappedValues = for value in repeat each element {
        if let value {
            then value
        }
        return nil
    }
    return unwrappedValues
}

Thank you for such detailed feedback! However, this post is intended to be a place where people can experiment with the feature. The syntax and the direction of this feature were discussed in a [Pitch] Enable pack iteration post.
As per what this proposal suggests, we are aiming to allow Swift users to iterate over packs and explicitly bind an element to a local variable. As @hborla mentioned, one of the important purposes of this proposal (and something that your syntax doesn't support) is to allow pattern matching with an element of the expansion. Also, the proposed syntax follows the progressive disclosure paradigm, where we let users use the familiar for-each syntax for pack iteration. Since you mention that your syntax can be equivalent to the normal for-each with a Sequence, it is debatable to add a new syntax for essentially the same logic as the existing for loop conveys.
Since the feature you are proposing seems to be orthogonal to this proposal, I would encourage you to move this discussion to a separate thread where you can explicitly propose the changes you'd like to make :slightly_smiling_face:

Actually, the length of a pack is not statically known since it conveys generic context. In your sum example, we cannot make any assumptions about the length of a pack.

As for enabling for expressions, I think it is worth considering it as a general future direction, not for this proposal itself.

To be clear, design feedback is definitely still on-topic during the review. If you agree that the feedback is suggesting a better design, you're welcome to make changes to your proposal; similarly, the Language Steering Group might decide that we agree with the feedback and ask you to make those changes. But of course both you and the LSG are also entitled to disagree with the feedback and not make any changes at all.

5 Likes

I'm confused, isn't this particular example the same as let sum = repeat ((each left) + (each right))?

It does support it though, albeit more verbosely:

repeat Element {
    guard case .one(let value) = each element else {
        continue
    }
    ...
}

I was only intending to show how they would have a similar level of clarity with regards to what is being iterated over and what values are updated with each iteration. There are still other important differences in my opinion.

But the syntax feels like a special case that clashes with the rest of the language design. There's no other place in the language where packs and tuples are treated analogously to runtime collections like arrays; even the indexing syntax is different (x.0 vs x[0]).

The value of progressive disclosure, in my opinion, is extending concepts you already know in more advanced ways. In my opinion, the for-each syntax doesn't really expand or generalize to anything, it's kind of just a "dead end" in being a special case. People who use value and type packs would be familiar with the repeat keyword, and the statement form would extend that to code blocks rather than just expressions. The repeat statement would then naturally extend to using unpacked values as lvalues, unpacking different value packs throughout the code block, etc.

1 Like

The proposal includes an example implementation of == for tuples. How would it look like with the syntax you're proposing?

More specifically, I'm wondering what the rules are on what should go after the repeat keyword in your syntax.

func == <each Element: Equatable>(lhs: (repeat each Element), rhs: (repeat each Element)) -> Bool {
    repeat Element {
        guard each lhs == each rhs else {
            return lhs
        }
    }
    return true
}

The thing that goes after the repeat syntax is the pack type that's being iterated over. The pack type is written in order to be more obvious about the "shape" that's being iterated over, since otherwise that information can be lost in the potentially complex body.

1 Like

How would this rule scale to pack expansion expressions involving multiple different abstract packs, e.g. for _ in repeat (each t, each u), for _ in (repeat each t, repeat each u, "hello") or for _ in repeat f(each x, g(each y))?

1 Like

Your first and third examples would be written like this:

repeat T {
    _ = (each t, each u)
}

repeat T {
    _ = f(each x, g(each y))
}

If t and u are of the same type pack, you write down the type pack. If they're different type packs but the type packs have the same shape, I don't know. Either you just pick one of them arbitrarily, or we require that people list them all. The same issue would arise in a (repeat T in f()) or (repeat<T> f()) syntax; it would be arbitrary whether to pick T or U if both have the same shape.

I don't think there's a better way unfortunately, unless we were to introduce "shape parameters" or something instead of having same-shape requirements solely be inferred. Making the type pack inferred (like in expressions) instead of having it written explicitly would cause clarity issues like @rdemarest and @simanerush talked about.

Your second example is something I never considered, that not all packs have an easily nameable type. In that case, the logical conclusion of my idea would be repeat (repeat each T, repeat each U, String), but that's cumbersome (and since it's only the shape that matters, it doesn't matter whether it's String or Int, so if we go down this route we should probably just have people type _ for single elements) So maybe we should make explicitly writing the type pack optional, at least for the simple cases, so that it can simply be written like this:

let a = (repeat each t, repeat each u, "hello")
repeat {
    _ = each a
}

My point is the latter example rather than the first one. You are right that repeat is enough in the former example.


I don't support repeat each Element syntax you proposed. for statement can be extended for raw tuple like for value in repeat each (1, true, "swift") {}, but your syntax seems not suitable for that purpose.


Edit: Ahh I’m really sorry but I meant to reply to @ellie20 :scream:

For an actual tuple without any pack expansions, the pack-iterating for loop would be written as for value in (1, true, "swift"), which would be syntactically identical to a normal for loop, making it appear as if a tuple is a Sequence.

But I've just now realized that the original SE-0393 proposal has pack expansion types as something separate from tuple types, with "Modeling packs as tuples with abstract elements" listed as a rejected alternative. I don't think it's actually possible to unpack an arbitrary tuple (including a tuple made from a pack like (repeat each t, repeat each u, "hello")) to a pack expansion type, and therefore use it in pack iteration, without introducing a new feature for it. I assume not because that feature is talked about as a future direction in the "Value expansion operator" section.

If that feature is ever added, then you could write something like this:

repeat {
    let i = each (1, true, "swift").element
}

// with tuples outside the loop and multiple tuples
let tuple1 = (1, true, "swift")
let tuple2 = ("", 0, false)
repeat {
    let i = each tuple1.element
    let j = each tuple2.element
}

// with the pack-iterating for loop in the pitch
for i in repeat each (1, true, "swift").element {
    ...
}

If we do get the ability to make pack expansion values out of tuples, and thus have pack expansion types that are tedious to write out, then I would think my syntax idea might be better off with the written-out pack type being optional. Or with some other syntax. My guess is that the scenarios where a pack expansion type that's iterated over can't be spelled easily will be relatively uncommon.

Yes, and this is mentioned as a future direction in SE-0399. That's why I said 'can be extended'.

Oh, I understood that you are suggesting the use of repeat keyword and not repeat T {} specifically. Then my point above is not applicable to your argument.

Highly positive. The proposal fills an important gap in language usability and expressivity.

Yes. While it's true that there are means to get the same behavior via local throwing functions, that approach isn't neither scalable nor intuitive. E.g., nested and labeled for loops involving continue <label> or break <label> statements aren't easily convertible to repeated invocations of throwing functions and there's no guarantee that those abstractions can be optimized by the compiler.

Yes, it does. I would hardly think of an alternative syntax better that the one proposed.

Python is the first language which comes to mind. You can use for...in loops with both lists and tuples:

for x in [1, 2, 3, 4, 5]:
    print(x)

for y in (1, 2, 3, 4, 5):
    print(y)

With this proposal and in a future in which concrete tuples can be implicitly converted to tuples of value packs, users may transition more easily from Python to Swift.

A quick reading, but accurate enough to notice some cpp left-overs :stuck_out_tongue_closed_eyes:

4 Likes

Thank you for the feedback and for catching the typo, I've updated the proposal!

1 Like