[Pitch] Extending optional chains to include for loops

Then it would be interesting to know why this is such a huge problem for for-in loops but obviously not for all (?) usages of optional chaining. I guess you then also would not like to use forEach in your organisation as there you can already use optional chaining.

Sorry I do not want to bother you with those questions, I am really interested. No need to answer.

The pitch text has again been updated.

As I said, it is also problematic when function evaluation is at the end of the chain, and that I wish such call sites would complain that the result is unused. E.g. someOptionalCollection?.forEach { ... } returns nil if the the collection is nil, and .some(()) if the collection is non-nil, so there is a return value which is silently discarded and obfuscates the fact that there is a no-op branch in there.

However, most use cases for optional chaining is to walk a property path, evaluate each node, and return the leaf node wrapped in an optional. That is, we still have an optional.

The result of optional chaining is always an Optional, and using it almost always requires the usual ceremony where optional values are involved. That is, binding it, unwrapping it, guarding on it, pattern matching it, or whatever. This ceremony is usually very visible and easy to read:

if let foo = some.long?.winded.path?.of.optionals {
  // it's pretty obvious that we're dealing with branching here
}

But it still removes the clutter of unwrapping each node along the path:

if let long = some.long, let path = long.winded.path, let foo = path.of.optionals {
  // the branching is here is obvious as well, but the clutter doesn't add much in terms of clarity
}

I don't think all optional chains are bad.

However, the branching and implicit no-op here is obfuscated at a glance:

for item in some.long?.winded.path?.of.optionals {
  // it's clear that this is a loop, but not that it is wrapped 
  // in an implicit if-branch, with a no-op path
}

I don't think you can convince me otherwise. You can perhaps convince me that the feature is worth the cost, if the motivation is compelling enough, but so far I don't see it. Either wrapping the iteration in an if-let, or appending ?? [], or creating an extension on Optional where Wrapped: Sequence that returns some kind of OptionalIterationSequence<Wrapped.Element> or whatever, is probably a better solution, IMHO.

I think I have voiced my concern with this pitch, and I don't think it is fruitful for the discussion to spend much more time on me. Either let the discussion progress with other aspects of the pitch, or address my concerns with better motivations.

3 Likes

I tried to formulate the concern in the pitch. Thank you.

I just would like to note that currently the pitch draft is in some kind of hibernation:

  • The “Detailed design” section is still missing, and I cannot deliver that, I am not able to provide an implementation.
  • Support for the described idea by the community seems to be limited.

There seem to be quite opposite opinions about the proposed feature. For me, without this feature “for-in” loops are kind of unusable in many cases where I need some kind of “for” loop, while for others the usage of optional chaining in a “for-in” loop is too easily overlooked.

At this point there will not be any further action on my part. I tried to include all known objections into the pitch draft, so it should be a good overview over the pros, the cons, and the alternatives. I will leave the pitch draft available at the current URL for a long time.

I think this problem definitely merits addressing – optional collections do happen in the real world and are a real pain when they aren't expressible by array literals – and I think the proposed syntax would be a good solution.

However, I think the proposal could do with giving more serious consideration to the alternative of a orEmpty property on Optional<Sequence>, as Joe described in his rejection of SE-0231.

The reasons for not considering it in this proposal are not compelling to me:

  • Preserving symmetry with forEach: forEach is a bug not a feature (IMO at least :)
  • Discoverability: if .orEmpty is in the std lib, the compiler can produce a fixit to use it when attempting to loop over an optional.

Here are the pros/cons of the two options as I see them:

Pro for x in y?:

  • follows existing practice of sugaring optional chaining with ?
  • very succinct while (arguably) clear given the existing language conventions
  • potentially even more succinct if "chained" e.g. for x in try? y does not require parens unlike for x in (try? f()).orEmpty which is pretty noisy

Pro for for x in y.orEmpty:

  • easily searchable
  • can be implemented in the library today without compiler magic
  • (arguably) clearer
  • discoverable via completion (tenuous, especially if the fixit idea is pursued)
Implementation of orEmpty if you want to play with it
struct MaybeSequence<Wrapped: Sequence> {
  let wrapped: Wrapped?
}

extension MaybeSequence {
  struct Iterator { var wrapped: Wrapped.Iterator? }
}

extension MaybeSequence.Iterator: IteratorProtocol {
  mutating func next() -> Wrapped.Element? { wrapped?.next() }
}

extension MaybeSequence: Sequence {
  func makeIterator() -> Iterator { .init(wrapped: wrapped?.makeIterator()) }
}

extension Optional where Wrapped: Sequence {
  var orEmpty: MaybeSequence<Wrapped> { .init(wrapped: self) }
}
5 Likes

By your own logic, .forEach should not exist because for loops exist, despite sometimes requiring a substantial refactoring to switch. I think it is only consistent to argue that .orEmpty is similarly duplicative of first pattern matching away the optionality or using an inline ternary operator.

The argument in favor of .orEmpty also extends to things like Rust’s Result::is_ok. By that logic, Swift should have Result.isError. I don’t necessarily disagree.

I think the question mark — despite being short — is better recognizable as a “warning” that we are kind of iterating through an optional sequence than “orEmpty”, which looks like some normal property. The question mark is what we are looking for to discover optional things so to speak.

“orEmpty” is Alternative 4 in the pitch draft. I will change the name of the property accordingly and also list the pro-points that @Ben_Cohen mentioned (tomorrow, I will then add an according note in this topic).

BTW I have no problem if someone would like to rewrite the pitch draft (switching the author is often a good idea). But we should not loose any of the arguments or alternatives.

Well, this then is a pro-point in favor of this pitch I would say.

Could we do this (perhaps with language support)?

    for x in y ?? .empty {
    }

where empty is an "empty sequence of appropriate type".

We would probably allow it in other places as well:

func foo(_ array: [Int], _ dictionary: [Int: Int]) {...}
foo(.empty, .empty)
1 Like

This looks pretty good to me, and should be implementable in today’s Swift.

One difficulty would be defining CollectionOfOne.empty.

1 Like

I would not like to use this because, as I mentioned before, I use those optional chains extensively in iterations, so I see this formulation as a burden that to some degree destroys the conciseness of my code. But I will add this alternative to the pitch draft.

It should only be necessary for the last step, no? for x in some?.optional?.chain ?? .empty? Or are you objecting to having to put .empty on a substantial proportion of your for loops?

1 Like

Yes (… ?? .empty). I am happy if somewhere a question mark makes it clear that we are dealing with an optional sequence (and I am not the one who has to change the compiler :wink:). Do not want to add 30 times … ?? .empty in a source file. Well, this is a personal opinion of course. Other people think the question mark is not enough…

1 Like

I updated the pitch draft:

  • Added the bullet points by @Ben_Cohen above (regarding for x in y? , for x in y.orEmpty).
  • Renamed alternative 4 to ´orEmpty`.
  • Added ... ?? .empty as a variation of alternative 3.
  • Used the fact that forEach might be considered bad practice as an argument in favour of the proposed change. (And no more talk about a "symmetry" of forEach vs. for-in loops.)

Well, the pitch draft evolved over time and it is kind of difficult for me to judge the quality of it. As I already said, a fresh start might be helpful. But maybe it is OK as it is, I at least hope that all arguments and alternatives are mentioned.

The downside of such a pitch is of course that it is already written in favour of a certain solution. So even if alternatives and arguments against the proposed solution are mentioned, it is not a "neutral" text. So I am even not sure if a pitch is the right thing to do at this point.

Evolution proposals have an Alternatives Considered section, not Arguments Against. They have no obligation to be neutral. Other people can argue against your proposal in its pitch and review; maybe you can change their minds with a different argument, or maybe you just disagree about some judgment inherent in the proposal. Ultimately, you want the proposal document to make as strong a case as it can, but don’t get too disheartened if there are people who still aren’t convinced.

6 Likes

We shouldn't turn this into a forEach debate, since it's related but tangential. However,

Yes indeed.

Not so. Use of forEach is trivially replaceable with a for loop in all cases, other than the one being debated here.

Not really. I would say more that forEach should be ignored for our purposes. It's neither a reason for or against the two options discussed here.

Just to be clear: I'm not sure which of ? vs .orEmpty is better. Probably either would be just fine, since they both have advantages over the other and no major downsides. I just think both need to have the right amount of discussion to ensure the best choice is made.

I understand why one would not like to mix up those two issues too much, but: If forEach is considered bad practice, then for-in loops should be a good replacement for the cases where people still feel the need to use forEach, i.e. for-in loops should work fine in "all" cases without much additional burden on the syntax, but they do not. I would like to use for-in loops only, but not at their current state.

There are objections against changing for-in loops at all concerning optional chaining or optional sequences because of “non-obvious ways” of introducing non-optional sequences. .orEmpty even drops the question mark so I think this is even more “non-obvious”. I consider this a major downside.

1 Like

I believe this would be an inadvisable use of the ?? operator (which is already challenging from a type checking perspective, which I think that would rule this option out even if it was a good idea, but that's secondary to it not being a good idea).

The ?? operator currently means "here is an optional value of a wrapped type. If the optional on the LHS is nil, substitute this other value, giving you a value of the wrapped type."[1] So the type signature of ?? is (Wrapped?,Wrapped)->Wrapped.

But this is not what we want here. We do not want to produce the wrapped value. We cannot, in the general case, do this for collections we cannot create an empty instance of (like CollectionOfOne or an opaque some Sequence).

Instead what you're looking for is an operator that says "given an optional value, return a type that's either the wrapped type if it wasn't nil, or some other type if it was". So your ?? would be (Wrapped,OrElse)->Either<Wrapped,OrElse>.

This Either type is extremely useful, and totally implementable and I do think we should add it to the standard library. It's super useful because you can conform it to lots of protocols when both the types conform – like Sequence. But if we do, we should not overload ?? to turn optionals into it, and I also think we should still give a more bespoke affordance for iterating optional collections easily that doesn't rely on it, since it's a pretty advanced feature that too much of progressive disclosure to understand.

(Of course, a sequence-conforming Either could be the type returned by Sequence.orEmpty if that's the route we go down)

More on Either if you're interested

The benefit of using Either to solve this problem is that it allows more than just empty sequences...

// let's say ?| turns an Optional into an Either
infix operator ?|

func ?| <S: Sequence>(lhs: S?, rhs: [S.Element]) -> Either<S,[S.Element]> {
    if let lhs { .left(lhs) } else { .right(rhs) }
}

let maybeArray: [Int]? = nil 

for x in maybeArray ?| [99] {
    print(x) // prints 99
}

You might see though that I used a concrete array type for the rhs of ?|, rather than the more appealing solution of an arbitrary generic sequence, which I then happen to use Array with in the for loop. This is because I don't think you can make Either conform to ExpressibleByArrayLiteral because AFAICT the way that protocol works it can't be forwarded to the wrapped type.

EDIT: I was being way to over specific with the implementation of ?|, this just works:

func ?| <S,T>(lhs: S?, rhs: T) -> Either<S,T> {
    if let lhs { .left(lhs) } else { .right(rhs) }
}

Implementation of Either that makes the above snippet work:

enum Either<Left, Right> {
  case left(Left), right(Right)
}

typealias EitherSequence<L: Sequence, R: Sequence> =
  Either<L,R> where L.Element == R.Element

extension EitherSequence {
  internal struct Iterator {
    var left: Left.Iterator?
    var right: Right.Iterator?
  }
}

extension Either.Iterator: IteratorProtocol {
  internal typealias Element = Left.Element

  internal mutating func next() -> Element? {
    left?.next() ?? right?.next()
  }
}

extension EitherSequence: Sequence {
  internal typealias Element = Left.Element

  internal func makeIterator() -> Iterator {
    switch self {
    case let .left(l):
      Iterator(left: l.makeIterator(), right: nil)
    case let .right(r):
      Iterator(left: nil, right: r.makeIterator())
    }
  }
}

This is needlessly duplicating [] and [:], which already exist for this purpose.


  1. yes I know there's a second overload that takes an optional type on the rhs, too... this is part of the reason it's hellish for type checking ↩︎

7 Likes

There isn't a hard rule that every operation involving optionals must include a question mark (map and flatMap exist, for instance). I don't think the claim that for x in a.orEmpty is unclear is particularly credible (especially given you can just option-click it to see what it does the first time you hit it).

for x in a? certainly has the advantage of being more succinct, and I agree the symmetry with other optional chaining is nice. I'm just saying, lack of clarity of the .orEmpty option doesn't stand up as an argument against, to me.

2 Likes