Another try at allowing optional iteration

I do believe, as a general principle, explicit unwrapping is to be encouraged over implicit: that is one of the main wins of having Optional as part of the language. So in that respect, yes, I would write something to indicate unwrapping in preference to nothing.

My overall conclusion here is that I remained unconvinced that there is an ergonomics issue that I'd like to see solved. Even the fullest spelling if let elements = elements { for element in elements { ... } } is perfectly acceptable in my eyes. I'm therefore not in favor of any proposed change, whatever syntactic form it takes.

4 Likes

@gwendal.roue I see. However, isn't it the responsibility of the developer to be aware of the type and not follow an optional iteration with logic that does not account for the possibility of the optional sequence being nil? I wouldn't be against for?, but I'm afraid that will highly raise the bar for the change, even if I implement it myself. But I will try nevertheless, you can always reset. Thanks.

1 Like

I'm interested if something like if let function = optFunction { function() } would be also perfectly acceptable in your eyes in contrast to the existing optFunction?()

I've never had occasion to use either form, and would not have been able to tell you for sure that f?() was even possible until now. So, yes, it does seem perfectly acceptable.

Now, optional chaining with ?. solves a very particular issue (much like guard does) in terms of avoiding multiple layers of nesting, but I'm persuaded that optional chaining, if let, and guard let together provide sufficient ergonomics for working with optional types.

More generally, I'm coming around to the notion that the forums would improve dramatically for all participants, the language would benefit from more focus on much needed areas of improvement, and nothing of value would be lost if we were to say that all changes related to optional unwrapping are to be out of scope for Swift Evolution. This seems to be the new fileprivate of 2018.

1 Like

I pretty strongly disagree with that. If Optional has a natural conformance to the collection protocols it is as an unconditional RandomAccessCollection that always has zero or one elements. I am not proposing we add that conformance to the standard library but we certainly should not prevent people from adding it in their own code.

I agree with these questions. The approach I favored in the Set / Sequence discussion was to use a property to access a projected sequence when necessary. The same technique could work here as well when people want to view nil as equal to an empty Sequence.

A projection gives you easy access to iteration even when it would not be convenient to use ?? and it also provides access to the entire Sequence API. Even better, this is something you can add to your code today!

extension Optional where Wrapped: Sequence {
    public var nilIsEmpty: NilIsEmpty<Wrapped> {
        return NilIsEmpty(base: self)
    }
}
public struct NilIsEmpty<Base: Sequence> {
    let base: Base?
}
extension NilIsEmpty: Sequence {
    public struct Iterator: IteratorProtocol {
        private var base: Base.Iterator?

        fileprivate init(_ base: Base.Iterator?) {
            self.base = base
        }

        public mutating func next() -> Base.Element? {
            return base?.next()
        }
    }

    public func makeIterator() -> Iterator {
        return Iterator.init(base?.makeIterator())
    }
}

let array: [Int]? = [42, 42, 44]
for int in array.nilIsEmpty {
    /// prints 42, 43, and 44
    print(int)
}

let empty: [Int]? = []
for int in empty.nilIsEmpty {
    // never execued
    print(int)
}

The best part about this technique is that it scales up as the underlying type becomes more powerful. It can conditionally conform to Collection, BidirectionalCollection, RandomAccessCollection, MutableCollection, and RangeReplaceableCollection.

If you really want to propose a change to Swift itself I encourage you to pursue this avenue. But maybe it's enough to just start adding these few lines of code to your project (at least for now).

5 Likes

Yes, I didn't mean it looks good as a solution, but as a piece of code.

I understand your alternative approach, but I don't see in what way it surpasses a native built-in feature, be it for foo in optionalSequence or for? foo in optionalSequence. Compared to that, it looks like a workaround; compare optFunction?() to (optFunction.nilIsEmptyFunction)() as an analogue:


Once again, although it can be interpreted this way, it is important to understand the pitch is about allowing optional iteration, not making a nil optional Sequence behave as empty. These are not substitutable notions, because optional iteration doesn't imply a question on how a nil sequence should behave or how the user wants it to behave. It works in consistency with the language rules, meaning (here I can safely say without semantic loss) that an iteration over a nil optional sequence indeed behaves equally to an iteration over an empty sequence, regardless of whether the sequence can possibly be empty. We are talking primarily about optionals, not sequences (we don't talk about optFunction?() as a possibly empty function although it does behave accordingly if nil, right?).

I understand that your initial pitch doesn't discuss other uses of optional sequences but several commenters have questioned why we would special case iteration. There are certainly other use cases where treating nil as empty instead of unwrapping manually would be useful so this question has merit.

That depends on your perspective. If you are only interested in iteration this is true. If you are interested in a more generally useful design then this is not true.

I don't have a strong opinion about this topic but I do think it would be good idea to consider the projection approach to supporting iteration. The relative merits of that vs modifying the rules of for vs introducing for? should be considered carefully.

Sure, I have no doubt. Still, I believe this concrete use case that is always accompanied by a language statement should become built in sugar rather than a stdlib-side approach that yours and, in particular, the conditional conformance are. I suppose this pretty much answers why I could consider this a special case and would like it to land separately.

Keeping in mind the change is 'Swifty', has existing analogues, is purely additive and not source-breaking makes me confident I am being careful enough.

I think you have just made an argument for treating nil like an empty array.

Does anyone have a realistic use case where treating a nil array like an empty one in a for loop would crash/error, but not an empty array?

You don't know what is other people's reality, or what is your future reality.

It's important to remember that there are far more possible realities and assumptions than your imagination can hold. There are soooo many domains where the general-purpose stdlib collections can be used. Even domains you will never hear about, because your life is simply too short.

Yet it's entertaining to put one's imagination to good use. I gave you non-empty collections for free. Another one? Sure: odd-numbered collections. My little finger tells me that there are a lot of funny collections in maths, graphs theory, group theory, etc, that can not safely assume that nil is equivalent to empty. From there, I'm rather sure that crypto follows. Etc, etc. Now, it's your turn: run your imagination, it's fun.

Gwendal, please be careful about the tone of your response. I don't think you meant to be condescending, but what you wrote can come off that way!


For an Apple example, there are a (small) handful of APIs that take an Array where nil means "the default set, whatever is most appropriate" and [] means "literally nothing is allowed". It's subtle, but valid. Having for/in treat nil as empty could lead to mistakes in implementing this API.

That said, this case is super rare. It's not often that nil and [] have different meanings in an API.

That said, if nil and [] have the same meaning, it's better not to make the value Optional at all. (We put a good amount of effort into that with the Apple SDKs.)

That said, the examples people have brought up for for/in loops usually aren't directly about Optional types in an API; they're the results of using optional chaining or map.


I'm personally mildly against this feature because I think the subtlety outweighs the benefit ("clarity over brevity" etc etc). But only mildly.

6 Likes

One example of nil and an empty array being different from a project I've been working on: a whitelist of allowed values.

An empty array is treated as 'the policy is to allow nothing', while nil is treated as 'there are no policy restrictions, allow everything'.

The expression whitelist.contains(hostname) now could very well be hiding an issue in my logic (that I didn't take care of the optional case).

1 Like

I understand, Jordan. I'll say it in another way, because I think the general topic is important for the forums.

Carelessness, ignorance, lack of imagination, innocence, naïvety, and generally all manifestations of lack of knowledge are not a sane ground for reasoning. When one says "I don't see how fact F could occur, hence fact F does not occur, hence I can draw conclusion C", one has not proved anything: not seeing anything could just be the sign of a lack of knowledge.

There is no value in not being able to contemplate an eventuality.

"The only thing I know is that I know nothing" is attributed to Socrates. This is quite an applicable wisdom: one should be aware of the limitations of one's knowledge, and use this awareness to avoid drawing unsound conclusions.

In this particular thread, which wonders (amongst other things) if we could eventually iterate an optional sequence with the same syntax as an actual sequence, and make, iteration-wise, nil equivalent not only to [] but also to all collections attached to specific domain assumptions, ignorance is very harmful. We really don't want to ruin the safety of some code. Since we can't expect all present and future Swift developers from various domains to chime in and contemplate how this equivalence could harm their code, whoever wants to champion this change can't expect one's lack of knowledge to be filled by external sources. One thus has to show great care right from the start, and explore a lot before jumping on conclusions.

I hope my tone has moved from condescending to pontificating. Not quite the best one yet, but it should find more readers.

Edit: the target readers of this post are experienced developers and forum members, obviously. I wanted to put a name on a specific kind of post that I wish were avoided by our senior members, for the sake of discussion quality.

@dwaite There is a lot of people talking about errors and issues related to treating nil sequences as empty, and I am beginning to think this is all a huge misunderstanding.

On treating a nil sequence like an empty sequence in this thread

This thread isn't about treating a nil sequence as an empty one. Of course there is a general difference between an empty sequence and a nil optional sequence. I have said this quite a few times already – that we shouldn't be interpreting optional iteration as “treating nil like an empty sequence". This leads to confusion. Optional iteration means that in a for-in loop, if the sequence is nil, nothing happens at all, while an empty sequence, if it can possibly be empty, will iterate in a regular way, calling makeIterator(), next(). The sequence will not remain intact.


To begin with, whitelist doesn't seem to be optional since you're not using optional chaining. I suppose it was to meant to be optional, so I will continue in this vein.

Suppose it is optional. How does that create an issue? This means you had to take care of the nil case anyways. Also, I have the impression you are misunderstanding the proposal. The only purely additive change this pitch implies is optional iteration or an optional for-in loop. In other words, unless you use an optional sequence in a for-in loop (currently illegal), you can't possibly run into an issue related to this thread.

Thanks Jordan. I definitely agree that we shouldn't treat nil and the same in general... I think I explained my question badly.

For the record, I am not arguing for or against the proposal yet... just asking a question to inform how I should feel about it.

Let me try to clarify by restating my question in a different way:

Right now optional sequences cause a compiler error in for-in.

for x in optSequence {  //Error: Value of optional type is not unwrapped
}

If we make a variant of for-in which allows optional sequences, it would need to be sugar for:

if let compilerVar = optSequence {
    for x in compilerVar {
        //Do stuff here
    }
}

The question is: What types of bugs become possible because of this that would not also be a bug for an empty array?

I guess the argument here is that, with the compiler error, you are forced to explicitly consider the nil case, and as a result you are more likely to catch the mistake. I know it is a common belief that adding forcing functions (in this case "Lockout") will help people to avoid logical mistakes, but that effect really only comes from novelty. In practice, when forcing functions are applied liberally, it has the opposite effect, causing people to ignore important warnings. For example, the proliferation of "Are you sure?" dialogs causes users to build a motor program which automatically confirms before consciously thinking about it. The alert that cried wolf.

This is the primary design dilemma of forcing functions (especially lockout), they are powerful but annoying. If there is too much annoyance, it is counterproductive because the mind will eventually find a way to get rid of the annoyance and minimize the amount of thought it has to do.

All of that is my way of saying that I look at this through the opposite lens that most people are looking at it from. That is: Is the risk/cost of error great enough here that it is worth spending a limited design resource (lockout) on?

That is what I am trying to figure out by asking what types of errors can happen because of this...

1 Like

There has just been a decision for a proposal in problem space of how to deal with Optionals, and although it was rejected, it may still have some impact on Swift.

Although this is taken out of context, I think this sentence captures the spirit of the rationale:
Force unwrapping won't be banned, but it should be easier to use the alternatives.

Now, here we have a situation where force unwrapping is extremely easy and convenient, and the "better" alternatives are either much more cumbersome (if let), less powerful (?.forEach) or not universal and/or potentially less performant (?? []).
At the same time, there is an extremely obvious solution ("if there's nothing to iterate, don't iterate") which just has no syntax to be expressed.

I don't like treating a nil-value as an empty sequence, but like in many other cases where we make Optionals more convenient, a single question mark could be enough to solve the problem.

2 Likes

The draft Optional Iteration by AnthonyLatsis · Pull Request #878 · apple/swift-evolution · GitHub

https://github.com/AnthonyLatsis/swift-evolution/blob/optional-iteration/proposals/optional-iteration.md

I can see the logic because of the symmetry with forEach, but it is easy to write optionalSequence?.forEach { ... } so not a priority for me.

I can see one argument being that, if we ever get for-in-else syntax, this will allow something that isn't really possible (at least not without duplication) now:

for? x in optSequence {
    //Do something with X
} else {
    //The loop didn't run at all
}

Without the sugar, the closest we could come is:

if let sequence = optSequence {
    for x in sequence {
        //Do something with x
    } else {
        //The loop didn't run at all
    }
} else {
    //This most likely has the exact same code as the inner else statement
}

fwiw, in a real project, I just replaced some lines like

(field.text == nil || field.text!.isEmpty) ? .invalid : .valid

with a nilOrEmpty extension property defined on String? - and writing this, decided to declare it on Optional<Collection> instead ;-)

Due to Cocoa, nearly all important APIs today are build under the assumption that

[nil count] == 0 // true

As long as this doesn't change (and I don't see that change happen soon), the ability to easily treat missing collections as empty is quite convenient.