SE-0231 — Optional iteration

The reason I see library solutions as superior is that I can hover over orEmpty and get a tooltip that gives me a type signature and explains what the functionality is. Additionally, if I'm looking to implement this functionality, my first thought is going to be "I wonder if there's a method for this" and see if autocomplete pulls anything up with an obvious name. It may not occur to me that there's syntax for it at all.

It has been proposed that both of these work—effectively, the for loop ends up "inside" the optional chain.

Yes they both would work, but isn't that just more confusing than simply allowing for e in arr, which is in all likelihood the expected syntax of a new-to-Swift programmer?

This is on the list as the "implicit" option. Personally, I am very opposed to implicitly allowing iteration over optional sequences. The optional should be acknowledge somewhere in the statement.

6 Likes

The proposal doesn't seem to give much space to this option, just dismissing it with

This decision was primarily based on inconsistency and potential confusion that an otherwise left without syntactic changes for-in loop could potentially lead to ("clarity over brevity").

This doesn't sound like a compelling argument to me, especially since we're considering a solution that also introduces "inconsistency and potential confusion". The idea that both for e in arr? and for e in obj?.arr could work is both inconsistent and confusing.

Is there actual evidence to suggest users would be confused by for e in arr working on optional arrays?

1 Like

I don't know how one would come by such evidence without shipping the feature first. IMO, implicitly swallowing a nil would be more inconsistent than any other solution. I can't think of any other place in Swift where an Optional is implicitly turned into a no-op without syntactic acknowledgement of the Optional. Explicitly acknowledging Optional is a core design principle that we should not deviate from.

5 Likes

Odd: I was pretty sure that this was the first choice in this iteration... but something in this thread seems to be broken: Another try at allowing optional iteration - #3 by anthonylatsis

In short: I think allowing iteration of Sequence? received the most backlash from those who don't want the concept at all, and a single ? isn't that terrible for those who want a concise solution.

13 optional sequences in 100k LOC (as previously reported), 1147 total for in loops. So near 1% in our admittedly avoiding-optional-sequences code base. (Which, has never struck me as any sort of limitation or bother.)

I don't think per-LOC is a good metric to make this decision. Compare to two other similar language features: optional patterns in case statements, and where clauses in case statements. A search yields a pretty similar level of usage to my search for iteration of optionals, so would give an indistinguishably tiny per-LOC number. Should we interpret that as an indication that these features are needless complexity we probably shouldn't have? I hope not! I love both those features and use them frequently.

From my perspective, the search of the compatibility suite is best seen as a counter to the claim "this doesn't ever come up". It comes up in almost a third of projects in a representative sample. Whether that makes it common enough is certainly subjective. I think trying to turn that into an objective judgement via application of a threshold would be a mistake.

Commonality is important, but it's just one measure we should use to consider a proposal. Some other ones include:

Does it help avoid a performance or correctness trap? We've seen how the frequently-advocated ?? [] idiom can easily lead to sub-optimal performance (unnecessary allocations due to lazy-defeating overload resolution) and in some cases, full-blown trapping in a way that is hard to understand and test for. It also encourages use of forEach, a method which is also subject to correctness issues.

Does it help readability and maintainability? This again is subjective, but personally I think the optional-chaining solution is much more readable than a nested if. Clearly the enthusiasm for ?? [] suggests a succinct form is important for people, but is a flawed technique because many collections are not literally expressible, in addition to the performance issues above.

Maintainability also brings up another thought: it is quite common to first build an interface by returning an array. Then later, you switch it to a custom collection (maybe a view that doesn't allocate memory or enforces the correct API). Much to your annoyance, this breaks call sites because your new collection is (maybe intentionally) not literal convertible, so you have to go back and change them to if statements. A single consistent way of iterating over optional sequences would avoid this.

Speaking of...

Consistency Optional chaining works well on function and method invocation, and on subscript access. It composes well with while let. It has a nice duality between ! (trap on nil) and ? (do nothing on nil). The fact that it doesn't really help with for loops seems like an exception to me. I raise this mainly as a counter to the idea that adding optional chaining for for adds complexity – rather I see it as doing the opposite, improving consistency.

Is this a feature people would not be able to add as efficiently themselves outside the language? This is sometimes cited as a reason to add something to the library, and I think in a way it applies here. Some have suggested adding a orEmpty method to optional sequences. This is a nice solution that ticks many of the benefits I outline above and if we decide against adding optional chaining support, I'd advocate for that as a backup option. But it's not quite so nice from a performance POV. It would need to return a MaybeEmpty wrapper collection, and I think that would need to check if the original collection was empty on every iteration of the loop. Chances are the optimizer will fix all this up, but better if the language guaranteed this optimization which I expect a solution of skipping the for with a nil optional chain would always do.

14 Likes

I'm of the same opinion, in this case I'd rather see this solved as a method instead of another syntactic deviation just for iteration. As mentioned there's also precedence for this.

...and they also tend provide an alternative method of a similar concept to yours.

Swift has already eschewed many such solutions, so at this point, some random method that doesn't match anything else doesn't really fit. Plus, we'd still get proposals to sugar around those methods anyway.

4 Likes

Problem; as already demonstrated is that these syntactic deviations are error prone; not so with a simple method + let's not just glance over ease of discoverability.

That also makes for a hugely verbose language which, again, is not Swift. You're talking about fundamentally changing the design principles behind the language, or having a one off use case for this feature.

Whew.... perfect case where it's best ignore unsubstantiated conclusions.

I was mainly pointing out that that particular solution was largely ignored because it doesn’t fit the language.

This is not accurate, at least for the problem I outlined. The error-prone solution of ?? [] was an entirely library-level solution, and the problem was not specific to operators. It can happen with methods as well.

@endofunk Why does the conversation feel like the above would be a valid solution for optional iteration? This is just another way of spelling ?? []. Regardless of it's benefits, you still have to find an actual orElse.

Without trying to upset this apple cart. Let's simply review the code sample you provided:

for i in maybeHugeRange?.reversed() ?? [] { // doom occurs here
  print(i) // your app will jetsam without ever reaching here
  break
}

...and consider an alternative method like Rust's unwrap_or. I'll use the simple code sample I provide earlier:

extension Optional {
  func ifSome(orElse: Wrapped) -> Wrapped {
    switch self {
    case .some(let wrapped): return wrapped
    case .none: return orElse
    }
  }
}

Trying to do this kind of this isn't possible re "Left side of nil coalescing operator '??' has non-optional type '[Int]', so the right side is never used"

for i in maybeHugeRange.ifSome(orElse: 0..<0).reversed() ?? [] {
  print(i) // your app will jetsam without ever reaching here
  break
}

So maybe I'm missing the obvious here... how exactly does the method end up with the same issue.

Hi @endofunk – since it isn't directly related to the review, I took my reply over to the discussion forum.

2 Likes

I'd like to add one more reason in favor of the optional chaining solution, one that looks a bit into the future.

It is likely that one day if and for will allow inout bindings, and you'll be able to write this:

if inout array = obj?.array {
  for inout x in array {
    x.mutate()
  }
}

Unfortunately, inout isn't possible with the result of ?? []. And unless a lot of things change, neither will it work with a function returning a wrapper collection.

But optional chaining already works well with inout. So if optional iteration is implemented in term of optional chaining, this should be fine:

for inout x in obj?.array {
  x.mutate()
}
7 Likes