Who actually expects such big changes to happen?
I have the impression some of us recently developed an attitude of a lonely knight fighting off the barbarians who are searching for backdoors into the castle of Swift (see Default Implementation in Protocols - #77 by GetSwifty).
Imho it's great that people try to paint a bigger picture - but when you only use dark color, you won't end up with a bright result:
Changes are not only danger, they are an opportunity, and I don't think the core team will follow the logic of "you accepted SE-0231, now you also have to accept this change that a consider to be in the same spirit".
Maybe this specific issue is older than SE, and it's not for? which is breaks with composability and simplicity, but for itself:
We always had while, forEach might become more powerful in the future - and still a single kind of loop construct is sufficient to write any program you want (I bet many Swift developers don't even know about repeat ;-).
Of course, we already got our share of deprecating for ;-) and we can't expect big cleanups to happen anymore, so it seems unavoidable that Swift collects cruft, like all languages do.
Yes, the low-hanging fruits are tempting, and preferring small additive steps over big changes often naturally produces inconsistency... but do we have a realistic option for iteration of Sequence? that has a better convenience/cruft ratio than for x in seq? {}?
Well, there's always the effect on compile times. Of course, more lines in the code doesn't necessarily imply longer runtimes, but added complexity has a certain chance of making code perform worse. Swift compile times are already quite high.
Of course, compiler authors would have to judge the level of impact of such a feature, maybe it's miniscule. But we shouldn't disregard the negative effects of compiler complexity per se.
I think the option of making both for x in optSeq? { … } and eg let incrementedCount = optSeq?.count + 1
work would be interesting to hear more about as I (perhaps embarrassingly) can't think of an example of how this more general solution (that makes the latter example work) would …
Also, It'd be interesting to see some examples of how/why making all optional sequences conform to Sequence is not an option. Perhaps I'm not the only one who are not able to see those without having them spelled out for me.
Here is the implementation for that, which I posted above, repeated for completeness.
public struct FlattenedOptionalSequence<Base: Sequence>: IteratorProtocol, Sequence {
let baseNext: () -> Base.Element?
init(_ optionalSequence: Optional<Base>) {
switch optionalSequence {
case .none: baseNext = { return nil }
case .some(let s):
var baseIterator = s.makeIterator()
baseNext = { return baseIterator.next() }
}
}
public func makeIterator() -> FlattenedOptionalSequence<Base> { return self }
public mutating func next() -> Base.Element? { return baseNext() }
}
extension Optional: Sequence where Wrapped: Sequence {
public typealias Element = Wrapped.Element
public typealias Iterator = FlattenedOptionalSequence<Wrapped>
public func makeIterator() -> FlattenedOptionalSequence<Wrapped> {
return FlattenedOptionalSequence(self)
}
}
It's rather obvious what iterating nil would do, and assuming a count of zero might be straightforward too - but I don't think it's possible to have "default" values for all properties (and what about methods? (nil as Array?)?.contains(nil) == ?)
So you would most likely still have some things that aren't possible with nil, but the cut won't be very clean.
I don't think there is a compelling argument against this... it's just one character worth of to much convenience for many people ;-) FlattenedOptionalSequence imho feels more like a clever hack for private codebases, and not like something you want to have in the stdlib.
Also, why would this solution (or Sequence?.orEmpty) be better?
Because of the number of characters? Because you don't have to know C++ to understand it?
Isn't a new type a special case as well?
I'd really like to see all those synthesized protocol conformances thrown away and replaced with powerful Swift-features, but I have no idea where I would use FlattenedOptionalSequence besides loops.
That said, I think this single use case is already quite important, so it's good to have that clever hack as an alternative :-)
Why? Depending on whether optSeq is nil or not, two different things can happen in the example of for e in optSeq? { … }:
When optSeq is Optional<WrappedSequence>.none the for loop does nothing, exactly as if it had been given an empty sequence of type WrappedSequence, it's not iterating over "nil", it's iterating over an empty sequence of the same type as optSeq, ie WrappedSequence.
When optSeq is Optional<WrappedSequence>.some(seq) the for loop does exactly what it would do if it had been given seq, ie the value of type WrappedSequence that is wrapped within optSeq.
The above two rules are all that you need to think about, and they work exactly the same for all properties (including methods) of the type WrappedSequence. The properties of sequence types do work even when the sequence is empty.
Note that optSeq?, optSeq and seq are three different things (perhaps easily conflated) and that optSeq? and seq are of the same sequence conforming type S, and optSeq is the only one which is an optional, ie Optional<S>.
For CollectionOfOne things are not more or less straight forward with for loops than they are with eg optSeq?.count. We could argue that the count of a CollectionOfOne can never be zero, but then we have to explain how the for loop manages to iterate over the zero elements. It's exactly the same argument.
There would be no need for any "default" values for all properties, since the properties and methods would simply be the same as those of WrappedSequence, since myOptSeqThatHappensToBeNil? (note the question mark) is equivalent to an empty instance of WrappedSequence.
Optional can reasonably be viewed as an unconditional sequence of 0 or 1. This perspective is in conflict with a conditional conformance when Wrapped: Sequence. The current agnostic position taken by the standard library is probably best, especially since there are also reasonable arguments thatOptional should not conform at all.
There's certainly precedence for this in other languages (iter: Rust, iterator: Scala, ...), and your proposed behaviour is consistent with that. As mentioned before this probably merits a more general discussion on what's missing from the Optional type.
Ah, my fault - I think I misunderstood let incrementedCount = optSeq?.count + 1:
With Optional<Sequence>: Sequence, you could write let incrementedCount = optSeq.count + 1 (although I think count belongs to Collection... but I'm in constant struggle with Swift Collections, so I might be wrong ;-) and expect an result of 1 (non-optional).
But it's more about Int? than about Sequence?, isn't it?
I just checked, and there's no func +(lhs: Int?, rhs: Int?) -> Int? defined in the stdlib yet (and my intuition says it might be useful to have a general mechanism for this)
Well, if we had a chance to do it over again, having ? exit the current function monadically (instead of just the current postfix expression) might’ve been a better option, the way Rust does, since it composes more generally, and better tracks what ! does too. Like you said, that’d be a major compatibility break, but maybe having some notion of “idiom brackets” to scope an optional chain expression could fit into the existing language somewhere.
I know you meant this well, but it's best not to approach the line of suggesting that other people stop participating in a review, especially when it's apparent that they have strong opinions about the subject.
Is the problem being addressed significant enough to warrant a change to Swift?
No.
Avoiding a level of indentation is not that important, and when it matters, we have guard.
Raw loops are to be discouraged anyway, and algorithms can be optional-chained.
As @duan pointed out, most optional sequences should just be empty instead.
Optional chaining is an expression-level construct, and I am opposed to extending it to the statement level. At the limit you end up with code that is logically identical to the old Objective-C “plow ahead and let the nils propagate” idiom, but littered with ?s that you're supposed to mostly ignore. One of the reasons Swift doesn't behave that way is because that kind of code is hard to reason about. Explicit control flow and error unwinding are better ways to manage nils.
Does this proposal fit well with the feel and direction of Swift?
I don't think so. The extension of optional chaining beyond the expression level is precedent-breaking, and this strikes me as a very minor syntactic convenience to be adding at a stage where there are still many fundamental issues of design and correctness to work out.
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
The analogy to Objective-C's pervasive nil propagation is hard to ignore ;-)
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
Read through the proposal and skimmed the review thread.
Thanks for so thoroughly investigating your own codebase for this! As review manager, I would be interested in seeing more hard evidence of this sort from anyone else willing to spend some time mining their own private Swift codebases (I can look at Github well enough myself, though in our experience, open-source repos have not always been an accurate representation of all code). If you (or others) have time to refine your methodology and get better results, I'd also appreciate that.
One concern that comes to my mind with this sort of approach, though, is whether the existing language design, by imposing this impediment, forces developers to code around it. By not providing a direct means to iterate over optional collections, a developer could be pushed to change their coding approach more drastically in an attempt to avoid having to do so, perhaps by handling optionals more eagerly elsewhere. One could argue that's a good thing, but it might also obscure how much of an immediate frustration this really is if you try to divine it from the code. You mention anecdotally that it hasn't been a problem for yourself; do you happen to know whether your coworkers feel the same?
At a meta-level, I feel like we've had a good and thorough discussion of both whether iterating sequences wrapped in Optional is an important problem to solve, and many approaches to how to solve it if so. I'll try to summarize, and please reply if I've left your position out:
We could take the proposed for? syntax as is
We could use some other special case syntax, such as for x in? optional
We could include for loops in optional chains, so that for x in optional?.sequence works. Some people, including the proposal as an alternative, have suggested "put a ? after the sequence", for x in optional?, which can be seen as the degenerate case of optional-chaining the for loop. This has also led to discussion about whether optional chaining ought to be generalized in more ways, such as to arguments, binary operators, or other statement forms, particularly if optionalBool?.
We could allow for loops to accept multiple statement conditions, in the manner of if and while, using the final in form as the sequence to be iterated, for let xs = optional, x in xs
We could make for loops implicitly accept Optional with a Wrapped sequence.
We could leave the language as is, and make a library change to address this problem:
Introducing an orEmpty method on Sequence that produces a wrapper
Making Optional conform to Sequence when it has a Wrapped sequence
Continue to promote x ?? [] (or equivalent empty literal syntax) as the idiom for handling optionals containing sequences
Note that by listing these out, I'm trying only to summarize and confirm that these positions have been heard, not implying anything about the merits of any particular solution, or whether making any change is the desired outcome of this review. Personally, I'd like to see deeper analysis of the "Is this problem important to solve" angle, and doing more data analysis like what @algal has provided would be very interesting.
As another potential library change that could help address this issue, and would be more closely targeted than making Optional<Sequence> conform to Sequence, we could add a special overload for ??:
struct EmptySequence<Element> : Sequence, ExpressibleByArrayLiteral {
typealias ArrayLiteralElement = Element
public init(arrayLiteral elements: ArrayLiteralElement...) {
guard elements.isEmpty else { abort() }
}
struct Iterator : IteratorProtocol {
public mutating func next() -> Element? { return nil }
}
public func makeIterator() -> Iterator { return Iterator() }
}
func ??<S>(lhs: S?, rhs: EmptySequence<S.Element>) -> AnySequence<S.Element> where S : Sequence
{
switch lhs {
case .some(let s):
return AnySequence(s)
case .none:
return AnySequence(rhs)
}
}
With this addition, the existing ?? [] idiom would work in a for loop for every type of optional Sequence.