SE-0230: Flatten nested optionals resulting from `try?`

Evolution Proposal Reviews
81

Optional chaining isn't "shaving off" optionality in the case of nested optionals. It is, in all cases, ensuring that there's at least one level of optionality because it needs to be able to return nil.

You can certainly make a case that it would be conceptually cleaner to always add a level of optionality, but that conceptual cleanliness would in practice just create annoying nested optionals that users would have to flatten, like they do with try?.

If you're implementing some abstract operation where you want consistent behavior for things like optional chaining regardless of whether the result type is optional, you're probably going to write it as a generic function, in which case that's exactly what you get: consistent behavior regardless of whether the result type is optional, because the function doesn't know what the result type is. (People who say that we shouldn't have nested optionals aren't thinking of what that would mean for generic code like this.)

6 Likes
82

(The "shaving off" was a mistype that I edited to "adding", but never mind)

Right, "conceptually cleaner/simpler" rather than "introduce complexity in order to try and make the language do what most people are most likely trying to do in various specific situations" is what I'd vote for, because it would probably mean that more people would be able to fully get the concepts / rules, and therefore wouldn't have to try to do something, as things would work the way they expected more often than not, and when not, learning why would add a valuable contribution to their understanding of the system.

I think a lot of people's problems with optionals (and related magic) is that they can't form a simple consistent mental model of them, because optionals tries so hard to be easy that their defining features vanishes in a cloud of special rules.

There seems to be at least two different common ways of looking at optionals, and I think that @QuinceyMorris describes one of them above. I'm afraid I cannot really understand that way, but it seems to be closer to the concept of Obj-C message sending (to nil) than to the concept of a generic Swift enum with the two cases .some(wrapped: Wrapped) and .none.

I tend to see optionals as just the generic enum that it currently is defined as, although I('m a bit worried to) know that this should be thought of as just an implementation detail.

I have never understood nil in Swift, and I've never understood why people think nested optionals are any more problematic than nested arrays or any other nested generic type. It would be very weird if nested optionals were somehow not allowed (given how Optional is defined).

83

I guess you can be assured that won't happen ;-)

I have to admit that saddens me a bit, as it's a sign that the young and wild times of Swift are over now, and we entered the phase where major overhauls have to be surrogated with workarounds.

No one actually made a list, but I count three occasions where nested Optionals can/could appear:

  • Optional chaining (nesting is already avoided)
  • try? (we are about to avoid nesting here)
  • Dictionary<Optional<T>> (this seems to be the single "legitimate" use of double Optionals

I consider optional chaining to be the by far most most important of those situations (more than the other two combined...), and rarely encounter [Key: Element?] - but we are planning to add another exception for a common case, and don't even dare to talk about changing Optionals themselves to better align with how we want them to behave.

FWIW, I think (and always thought) the proposed change will actually make try? more convenient, and most developers might never have to worry about the increased complexity (and after all, probably all successful languages have a big baggage of compatibility cruft).

Finally, no matter wether SE-0230 will be accepted or rejected (seems unlikely now), I want to thank @bjhomer for his work:
Swift evolution is tough business, and I appreciate the corrections in the proposal which have been applied during the review phase.

1 Like
84

What is your evaluation of the proposal?

-1

Is the problem being addressed significant enough to warrant a change to Swift?

No. This is source breaking, and I don't think try lines should be so complicated that it isn't easily fixable.

Most of the problems here can be solved with ((try? throwingFunc()) ?? nil)

Does this proposal fit well with the feel and direction of Swift?

No. It does not feel orthogonal.

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

I read the proposal, wrote some code to test stuff, and checked my own code.

I think there are better alternatives. For example, a different keyword or changing the precedence of try?. The problem is that it's a prefix operator- the thought of comparing it to as? never crossed my mind.

85

I just finally read the review. I haven't followed the discussion or pitch threads closely, but I am +0.5 on this proposal (starting from an initially skeptical position). My opinion aligns very closely with Paul's summary below.

-Chris

2 Likes
86

  • What is your evaluation of the proposal?
    -1

  • Is the problem being addressed significant enough to warrant a change to Swift?
    No, aside from the fact that it's source breaking; the current behaviour is predictable and certainly not IMO a big enough problem to justify this.

  • Does this proposal fit well with the feel and direction of Swift?
    No.

  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
    For similar operations, it's normal to end up with nested types.

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
    Read the proposal and a few of the comments.

87

Quick double-check: have we run this against the source compatibility suite? Does everything work there, or does compilation fail in some understandable way?

88

I have not run my implementation PR against the source compatibility suite. I wasn't sure whether that was something that any old contributor could do, or whether it took a core team member to kick it off. But I agree that we should do so.

89

It's not restricted to the core team, just committers, but I don't think you have a commit bit yet. Rebase the pull request, make sure tests pass locally, and mention @brentdax in the Github comments and I'll kick it off for you.

90

Great, I'll take care of that later today.

1 Like
91

Looks like the swift compatibility tests passed.

(Caveat: Because the compiler is still set to version 4.2 by default, passing the compatibility suite means we haven't broken compatibility with any code that's using swift-version 4.2. It doesn't currently test whether we have source compatibility with Swift 5.)

1 Like
92

What is your evaluation of the proposal?
-1

Is the problem being addressed significant enough to warrant a change to Swift?
I don't believe so. In fact I think it will cause more problems and violates the Source Compatibility guidelines

  1. The new syntax/API must be clearly better and must not conflict with existing Swift syntax.
    and
  2. There must be a reasonably automated migration path for existing code.

In the original proposal it was acknowledged that

This is a source-breaking change for try? expressions that operate on an Optional sub-expression if they do not explicitly flatten the optional themselves.

but then immediately followed by an assumption

It appears that those cases are rare, though;

While this might be true for some use-cases, it doesn't change the fact that it breaks this use-case:

if try? doSomething() == nil {
    // ...
}

I think most would agree that this idiom is simple enough; we want to handle the failure case. Because the proposed change would cause this to behave differently only in some cases, and places where this was used would now need to reason their code if they can keep their current code or if they should migrate to

do { try doSomething() } catch {
    // ...
}

It is also mentioned in the proposal that generics would still behave the way they currently do.

Generic code that uses try? can continue to use it as always without concern for whether the generic type might be optional at runtime. No behavior is changed in this case.

This duality for me will cause a lot more harm, and will likely be sources of hard-to-find errors especially in apps that heavily rely on generics (e.g. RxSwift, etc). If we can reconcile these cases in one rule then I'd likely support a convenience change like this one.

There are also issues raised about as? already flattening optionality, but I think that is a completely different discussion because casting implies you are trying to convert one type to another. The semantic nullness of try? is not due to conversion, but due to the existence of an error.

Does this proposal fit well with the feel and direction of Swift?
No. I've wanted the try? syntax way back, but I feel many are using it as a way to ignore the result altogether. Instead it should be to inline error handling when the Error is not needed but distinguishing between failure or success is.

(Can we get a "try?" as a more leni… | Apple Developer Forums)
The purpose of 'try?' is not to allow you to ignore errors; it's to treat all errors the same way, inline. If you want to ignore the error, you have to say so explicitly. (And if there should never be an error, use 'try!'.)

I've seen some points in this thread that say the current status quo forces them to write "workarounds" or "boilerplate" code. Code is read more than it is written. In my opinion, these constructs are not "workarounds", but are there to communicate intent.

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
N/A

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
I have read the original proposal and about half of the discussion in this thread as of the time of writing. I also reviewed the original discussion during the implementation of try?.

1 Like
93
  • What is your evaluation of the proposal?

I totally support this. Current behavior usually forces me to write extensions to eliminate nesting.

  • Is the problem being addressed significant enough to warrant a change to Swift?

Yes, I have seen a lot of code fighting this problem in different ways, and almost none where this was used somehow. Every time I'm facing such nesting unpacking, it's a signal for me to reconsider surrounding code flow (decompose, make it throws, or wrap in do/catch) or just unwrap in-place.

  • Does this proposal fit well with the feel and direction of Swift?

This feature is what should be done years ago, in my opinion. And it will be easy to use the old behavior if someone needs this anyway.

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

I have read almost every comment. I am often using try? or do/try/catch where they appropriate. Also had some playground researches to understand double-optional behavior use cases.

1 Like
94

Question 1:

Are the following statements about the current behavior in Swift (4.2) correct?

  1. try? exprOfTypeT always result in type T?

  2. try! exprOfTypeT always result in type T (or traps)

  3. exprOfTypeU as? T always result in type T?

  4. exprOfTypeU as! T always result in type T (or traps)

(Please note that T and U are any type, which of course includes T and U being eg V? and W?, or X???? and Y??????? for that matter. So if T is V?? then T? is V???.)

Some code that provides details behind the above statements. 
let o0: Int = 123
let o1: Int? = 123
let o2: Int?? = 123
let o3: Int??? = 123

func throwingIdentity<T>(_ v: T) throws -> T { return v }

// NOTE 1: try? exprOfTypeT always returns a value of type T?
let tryQ0: Int? = try? throwingIdentity(o0)    // T = Int    -> Int?    = T?
let tryQ1: Int?? = try? throwingIdentity(o1)   // T = Int?   -> Int??   = T?
let tryQ2: Int??? = try? throwingIdentity(o2)  // T = Int??  -> Int???  = T?
let tryQ3: Int???? = try? throwingIdentity(o3) // T = Int??? -> Int???? = T?

// NOTE 2: try! exprOfTypeT always returns a value of type T (or traps)
let tryE0: Int  = try! throwingIdentity(o0)    // T = Int    -> Int     = T
let tryE1: Int?  = try! throwingIdentity(o1)   // T = Int?   -> Int?    = T
let tryE2: Int??  = try! throwingIdentity(o2)  // T = Int??  -> Int??   = T
let tryE3: Int???  = try! throwingIdentity(o3) // T = Int??? -> Int???  = T

// NOTE 3: exprOfTypeU as? T always returns a value of type T?
let asQ0: Int? = o0 as? Int        // U = Int    as? T = Int   ->  Int? = T?
let asQ1: Int? = o1 as? Int        // U = Int?   as? T = Int   ->  Int? = T?
let asQ2: Int? = o2 as? Int        // U = Int??  as? T = Int   ->  Int? = T?
let asQ3: Int? = o3 as? Int        // U = Int??? as? T = Int   ->  Int? = T?

// NOTE 4: exprOfTypeU as! T always returns a value of type T (or traps)
let asE0: Int  = o0 as! Int        // U = Int    as? T = Int   ->  Int? = T?
let asE1: Int  = o1 as! Int        // U = Int?   as? T = Int   ->  Int? = T?
let asE2: Int  = o2 as! Int        // U = Int??  as? T = Int   ->  Int? = T?
let asE3: Int  = o3 as! Int        // U = Int??? as? T = Int   ->  Int? = T?

Question 2:

Will the proposal, if implemented, introduce the context dependent symmetry break reflected by the following modification of the above statements?

  1. try? exprOfTypeT results in type T? unless T is statically known to be some type V?

  2. try! exprOfTypeT always results in type T (or traps)

  3. exprOfTypeU as? T always results in type T?

  4. exprOfTypeU as! T always results in type T (or traps)

Question 3

Can the proposal be described as changing the behavior of try? so that it moves away from
try!, as?, and as!
in order to align more with optional chaining, for which the following statement is correct:

  • exprOfTypeU?.propertyOfTypeT results in type T? unless T is statically known to be some type V?

?

(U must of course be some optional type W? there, or it would not be possible to use optional chaining, but T can be any type, including some optional X? or non-optional Y)

95

Yes, I believe all of those are correct.

96

Thanks. My humble view on this is that among the current behaviors of:

  • try?
  • try!
  • as?
  • as!
  • optional chaining

the behavior of optional chaining is the one that sticks out and causes most confusion (for users and and I guess compiler engineers and language designers). This is because it has achieved (at least part of) its convenience by introducing inconsistencies. I think it might be that it could have been as convenient as it is today, or even more convenient, while still having been more consistent.

I think we should be very careful and think many times about whether it might be possible to increase convenience while maintaining or even increasing consistency, before doing it in a way that definitely decreases consistency.

I think the following (current) behavior is easier to remember, implement, maintain and talk about:

  1. try? exprOfTypeT always result in type T?

than this (proposed):

  1. try? exprOfTypeT results in type T? unless T is statically known to be some type V?

which is exactly like optional chaining:

  • exprOfTypeU?.propertyOfTypeT results in type T? unless T is statically known to be some type V?

Even if the latter might be (dear I say superficially) easier to use in some contexts.

97

In fact, the dissymmetry with as? is one of the points that has come out of this thread. The current version of the proposal acknowledges that the goal is to make try? behave more like optional chaining, not like as?.

as? is a different beast because it takes an explicit destination type. However, the surprising behavior of try? throwingFoo() as? Bar not resulting in Bar? was one of the motivating issues behind the proposal in the first place, and is one of the most common pain points it will solve.

2 Likes
98

I am aware of that, and it is reflected in my above post and its 4 + 1 statements (and code example).

The point I tried to make with that post was that there currently is symmetry between try?, try!, as? and as!. And the one that sticks out is optional chaining. Look at those 4 statements again + the one for optional chaining, and see how the proposal will move try? from the list of 4 statements which it currently is consistent with, to that of optional chaining.

99

As has already been pointed out in this thread, the surprise is because most people's (including mine) intuition about the relative precedence of try and as is opposite to the one implemented in the language, as demonstrated by these parentheses which makes the expression work according to peoples intuition:
(try? throwingFoo()) as? Bar

Fixing this issue by introducing a special case (for when the type of the try?'s sub-expression is statically known to be an optional) like the one in optional chaining to try? instead of eg changing the precedence of try and as is imho not productive (note that I don't know if changing their precedence would be the right choice either).

While this solution will fix some pain points, it will do that by breaking something that isn't broken, which will cause unforeseen problems elsewhere. It would be better to further analyze this problem and find related ones, then try to find a solution to a common actual problem, and scrutinize that solution to make sure that it fixes more than it (perhaps hard to quickly see) breaks.

Note that I don't want to come off as offensive or unnecessarily negative. I do appreciate the time that go into making proposals, but I figure raising concerns about solutions so that they can be improved is an important contribution too, although I realize that it might look like I'm fighting rather than helping SE.

3 Likes
100

Changing the precedence sounds like a simple (although not complete) solution - but try? isn't an operator, is it? It might be rather hard to change something that has no precedence group...