SE-0230: Flatten nested optionals resulting from `try?`

Evolution Proposal Reviews
41

I'll certainly not try to go against a “how do we teach this?" in the proposal. Yet, in this very particular case, we enhance try? in order to make it do the expected thing. What goes without saying is the easiest thing to teach: you simply don't have to say anything.

Yet, in our particular case, we still have to care about teaching the breaking change. There should lie the teaching efforts.

BTW, teaching is not about describing the consequences of the proposal. That is already handled in the "motivation", "proposed solution", and "detailed design" chapters of the proposal. There's no point repeating them. Teaching is something different: it is about putting people in the state of mind that makes them able to put the proposal at work for their very own purposes. Have them "grok" the change. Make them think "this is exactly what I need", or "this change will finally make this Swift feature usable". It's very personal. It's about convincing users that a new tool has its place in their own toolbox.

I'd thus write the teaching section for this proposal as a purpose-oriented guide, which would talk about:

  • differences between Swift 4 and Swift 5.
  • an explicit note about how simpler Swift 5 code generally is.
  • how to "get back" the Swift 4 behavior in Swift 5 (with an explicit do { try ... } catch { ... })
  • how to change code when compiler version bumps from 4 to 5 (change may be handled by a migration tool (how?), but it may also have to be done "by hand" (how?)).
  • how to write code that is compatible with both Swift 4 and Swift 5 (with as? T).
3 Likes
42

-1 on the proposal. It's nice, but it's also source-breaking and, IMO, does not meet the "high bar" for such changes.

However, I do think that some kind of generalised optional flattening (for Int??, Int???, Int????, etc in other contexts) would be useful. They don't come up very often (mostly in generic code), but when they do it's nice to have a way to collapse them down to a single level of optionality.

I use something like this in my own projects (up to 4 levels of optionals):

public protocol OptionalType: ExpressibleByNilLiteral {
  associatedtype Wrapped
  var asOptional: Wrapped? { get }
}

extension Optional: OptionalType {
  public var asOptional: Wrapped? { return self }
}

// MARK: Compacting.

extension OptionalType where Wrapped: OptionalType {
  public var compacted: Wrapped {
    if let v = asOptional { return v }
    return nil
  }
}

extension OptionalType where Wrapped: OptionalType, Wrapped.Wrapped: OptionalType {
  public var compacted: Wrapped.Wrapped {
    if let v = asOptional?.asOptional { return v }
    return nil
  }
}

// ...etc
1 Like
43

What is your evaluation of the proposal?
I support it. Current behavior for try? is confusing for many users and might lead to overuse of try!, which can be dangerous for production code.

Is the problem being addressed significant enough to warrant a change to Swift?
Yes, as stated above.

Does this proposal fit well with the feel and direction of Swift?
Yes, it would make using try? pretty straightforward for most common scenarios.

If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
I've never seen a language with try? before. I believe is unique to Swift. I think it is a great language feature.

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
Quick reading of the proposal. However, I use exceptions in Swift on a daily basis and I feel inclined to give my support for this change. I haven't looked too deep into the nuances or edge cases of this change.

44

I think removing them is going to far, @bjhomer. They’re a compelling part of the proposal, and they belong in there.

It’s true that as? and try? are distinct features with distinct behavior.

It’s also true that the intent of this code is perfectly clear, and a user might naively expect it to work:

if let widget = try? thinger.makeDoodad() as? Widget {
    // Compiler says “value of optional type 'Widget?' must be unwrapped”:
    print(widget.size)
    // Huh?! I thought I just unwrapped it?
}

We’re thus in a situation where underlying complexity is in tension with perceived complexity. At the expense of making the details more complex, we can create a system that is more likely to match user intent, and thus allows users to think about fewer details on average. More details in the tools, but fewer details in the brain. (Slack’s notification logic (source) is a classic example.)

This is what I meant by “heuristic consistency”. The more complex option allows a simpler informal mental model: “I did a bunch of failable things, I used ? for all of them, and if I unwrap the result it succeeded.” No mention of static vs runtime types, levels of unwrapping, etc. That reasoning may be vague in its details, but it is entirely correct in its conclusion … under the new proposal.

My opinion is that heuristic consistency should be a design goal for Swift. In fact, my impression is that it already is.

4 Likes
45

Yeah, I'm not removing the mentions of things like try? thinner.makeDoodad() as? Widget. I just removed this part at the beginning. You can see my changes here: Remove comparison of 'try?' to 'as?' by bjhomer · Pull Request #922 · apple/swift-evolution · GitHub

46

To me, the simplest interpretation of this is that try? adds one layer of optionality, as does as?, so of course widget as defined above will have two layers of optionality.

That piece of code would be more clearly (and imo correctly) written as:

if let doodad = try? thinger.makeDoodad(), let widget = doodad as? Widget {
    print(widget.size)
}

I think rules should be as simple and as few as possible. And I have the feeling that sometimes Swift is increasing the number of rules in an attempt to make the language less confusing for those who are confused even by the current lower number of rules.

Let's try to make Swift simple rather than easy, or we might end up with a language that is so full of well-intended rules/special cases/exceptions that it will be confusing for everyone.

4 Likes
47

That design philosophy is IMO one of the primary reasons functional languages (especially ML family languages) have struggled so hard with uptake. Left untempered, it asymptotically tends toward “developers should have the same mental model as language designers.” That’s Haskell, it’s Clojure perhaps, but it’s not Swift.

Swift’s willingness to include conveniences that defer understanding of details, its philosophy of progressive disclosure, its general concern with user experience in general — these are defining features of the language. It’s the reason that Swift, Kotlin, Typescript, et al have have finally managed to give Maybe-shaped types traction in the mainstream where Haskell and friends failed to do so.

I’m often the one arguing for the “simple over easy” as well. “Easy” taken too far tends toward Clippy. Swift, however, is not anywhere near erring in that direction at the moment.

6 Likes
48

This is inaccurate. value as? T always returns T? regardless of how many optional layers value has. Rewriting it as

if let widget = (try? thinger.makeDoodad()) as? Widget { ... }

shows that as? doesn't add a layer of optionality.

That to me is like saying that

if let bar = optionalFoo?.optionalBar { ... }

should result in a double optional as well, and that the correct way of writing it is

if let foo = optionalFoo, let bar = foo.optionalBar { ... }

Optional chaining doesn't keep adding layers of optionality because it would be very inconvenient, even though it could be considered incorrect.

4 Likes
49

You two are saying the same thing. When the RHS of the as? operator is of type T, the result is of type T?: hence, adding one layer of optionality.

let x = 42
type(of: x as? Int??) // Int???
1 Like
50

I was sloppy with how I used the phrase "adds one layer of optionality".

My problem was that I didn't know that as? had higher precedence than try? so I thought that

if let widget = try? thinger.makeDoodad() as? Widget { ... }

was the same as (your):

if let widget = (try? (thinger.makeDoodad()) as? Widget { ... }

and not:

if let widget = try? (thinger.makeDoodad() as? Widget) { ... }

Now that I know that as? is evaluated before try? I would have written it with parentheses like you did. Just like I would have written (1 + 2) * 3 rather than 1 + 2 * 3 if I expected the result to be 9 rather than 7.

51

-0.

The proposal makes me nervous because it is changing the behavior of the expression based on compile time knowledge and not the underlying type of the data.

In addition to making things confusing (e.g. having to understand that try? behaves differently when doing certain types of code such as generic programming), I don't understand if there is any other desired future enhancement to flattening optionals which this might conflict with.

That said, it seems like a net win in isolation; if I had a vote I wouldn't vote against it based solely on my nervousness.

  • Is the problem being addressed significant enough to warrant a change to Swift?

I think so; nested optionals are a significant problem, and the amount of support for optionals within expression syntax warrants a solution that simplifies expression syntax

  • Does this proposal fit well with the feel and direction of Swift?

This is the area of my nervousness

  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

None, as others have nil as either a type or a value for all references.

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

A quick reading

1 Like
52

You're right, though try? (currently) adds a layer of optionality to a given expression, while as? doesn't add a layer of optionality to the expression it's given, but only to the rhs type. Those are very different things, despite both technically adding a layer of optionality to something. The resemblance to optional chaining is much clearer.

53

Currently, the language reference describes try? like this:

You use try? to handle an error by converting it to an optional value.
If an error is thrown while evaluating the try? expression, the value of
the expression is nil.

So try? converts the (throwing) error-handling layer to an optional layer, ie it wraps the type T returned by the throwing function in an optional, as an alternative representation of the original (throwing) error-handling layer:

  • An Error, ie no value: Optional<T>.none
  • No Error, ie a value: Optional<T>.some(value)

What this proposal says is that this should no longer be true for all types T. Instead, there should be a special rule that says that for some T, the error-handling-layer should be skipped, ie the type of the try?-expression should no longer always be T? / Optional<T>. And the rationale is that when T happens to be any Optional<W> type, then an error-handling layer is sort of already there, so it should be skipped.

Consider the throwing function f() throws -> T, and the relation between U and T in
let v: U = try? f()

The current behavior of try? is easy to describe (as above), remember, reason about and write code around. Because the relationship between U and T is simply:

U = Optional<T> for all T

If the proposed special case is introduced, the behavior of try? will be more complex and harder to describe, remember, reason about and write code around:

U = Optional<T> for all T, eg T == Array<E>, except T == Optional<W>

Optional is already too special/magic for me to use without having to stop and think about strange stuff like:
.none vs nil, "Optional being defined as an enum is only an implementation detail." (What, why?), etc.

Furthermore, If the proposed change is made, I'm not sure how code like the following would be handled:

let rndBool = Bool.random()

func foo<T>(_ v: T) throws -> T {
    if rndBool { throw NSError() }
    return v
}

func bar<T>(_ v: T) -> Optional<T> {
    return try? foo(v)
}

This (contrived but short) example program currently compiles, as expected.

But if the proposed change is made, the return type of bar would be Optional<T> only for "most" but not all T, and thus the bar function would AFAICS be impossible to write in a way that makes it compile. And my guess is that similar issues would definitely show up in real world code.

So, to those more knowledgable than me:

  • Would the proposed change make the type of
    try? throwingFuncReturningT()-expressions
    impossible to express in the language (as Optional<T> or T? or in any other way)? If so, isn't that quite a serious problem/show-stopper (the way it would affect generic programming)?

  • It seems to me that the real source of the problem that this proposal seeks to solve is that users (including me) don't intuitively expect the precedence of try? to be lower than that of as?, in situations like:
    if let x = try? somethingAsAny() as? JournalEntry { ... },
    so what is the rationale behind having those precedences like that?

3 Likes
54

Without commenting on the rest of it, this is not correct, and the proposal takes care to say so. The rules here are based on the static type T, not the dynamic type (which may be some Optional), just like they are for optional chaining and things like Dictionary subscripting.

55

If you read the example code again and try to imagine what would happen if the static type T is eg Optional<Int> when calling the bar function.

How could that work if the proposed change was introduced? It would be impossible to write out the return type of bar, since it depends on (the static type) T.

For clarity, this program currently compiles:

let rndBool = Bool.random()

func foo<T>(_ v: T) throws -> T {
    if rndBool { throw NSError() }
    return v
}

func bar<T>(_ v: T) -> Optional<T> {
    return try? foo(v)
}

typealias T = Optional<Int>
let myOptionalInt: T = 123
let hmm: Optional<T> = bar(myOptionalInt)
// The type of hmm is Optional<T> which is Optional<Optional<Int>> == Int??
// If the proposed change is made, it will not be Int?? but Int?,
// which makes the type of hmm and the return type of bar different,
// but only when T is Optional<W>, not when T is eg Int.
56

The return type of bar is Optional<T>. When T is dynamically Optional<Int>, the return type of bar will be Optional<Optional<Int>>. You're mixing up static and dynamic types.

57

In my example above, myOptionalInt is statically Optional<Int>:

typealias T = Optional<Int>
let myOptionalInt: T = 123

I'm truly sorry for the noise if everyone but me understands what's going on here but my example above involves no dynamic types at all, I never thought about any dynamic types.

The following example from the proposal doesn't have anything to do with dynamic types either:

func doubleOptionalInt() throws -> Int?? {
    return 3
}

// Swift 4: 'Int???'
// Swift 5: 'Int??'
let x = try? doubleOptionalInt()

To me, it clearly looks like the proposal's aim is to change the static type returned by
try? throwingFuncReturningTypeT()-expressions
from (as currently):

  • Always being Optional<T>

to (according to the proposal):

  • Being Optional<T> if T != Optional<W>
  • Being T if T is "already" an Optional<W>

No matter how many times I read the entire proposal, including the Generics section, I can't see what you mean. My example is not analogous to the example given in the generics section.

58

You can match the proposed behavior in Swift 4.2 by using generic overloads:

// Swift 4.2
func tryQuestion<T>(_ expr: @autoclosure () throws -> T) -> T? {
    do { return try expr() }
    catch { return nil }
}

func tryQuestion<T>(_ expr: @autoclosure () throws -> T?) -> T? {
    do { return try expr() }
    catch { return nil }
}

struct Foo {
    func bar() throws -> Int { return 3 }
}

let optFoo: Foo? = Foo()

let w = try? Foo().bar()
type(of: w) // Int?, current behavior

let x = try? optFoo?.bar()
type(of: x) // Int??, current behavior

let y = tryQuestion(try Foo().bar())
type(of: y) // Int?, matches proposed behavior

let z = tryQuestion(try optFoo?.bar())
type(of: z) // Int?, matches proposed behavior
3 Likes
59

You named several things "T", which doesn't help. Let's try again:

let rndBool = Bool.random()

func foo<A>(_ v: A) throws -> A {
    if rndBool { throw NSError() }
    return v
}

func bar<B>(_ v: B) -> Optional<B> {
    return try? foo(v)
}

typealias C = Optional<Int>
let myOptionalInt: C = 123
let hmm: Optional<C> = bar(myOptionalInt)

And then my comments:

The rules here are based on the static type B , not the dynamic type (which may be some Optional), just like they are for optional chaining and things like Dictionary subscripting.

The return type of bar is Optional<B> . When B is dynamically Optional<Int> , the return type of bar will be Optional<Optional<Int>> . You're mixing up static and dynamic types.

1 Like
60

Ah, thanks, I guess my problem was forgetting that bar<T>(_ v: T) makes the type of T only dynamically knowable within the function body. I see now that doing what bar does directly will (with the proposal implemented) result in a different type than doing it by calling bar.

My code example with bjhomer's simulation-code 
func tryQ<T>(_ expr: @autoclosure () throws -> T) -> T? {
    do { return try expr() } catch { return nil }
}

func tryQ<T>(_ expr: @autoclosure () throws -> T?) -> T? {
    do { return try expr() } catch { return nil }
}

let rndBool = Bool.random()

func foo<T>(_ v: T) throws -> T {
    if rndBool { throw NSError() }
    return v
}

func bar42<T>(_ v: T) -> Optional<T> { return try? foo(v) }
func bar50<T>(_ v: T) -> Optional<T> { return tryQ(try foo(v)) }

typealias T = Optional<Int>

let a: T = 123
print(type(of: bar42(a)))         // Optional<Optional<Int>>
print(type(of: bar50(a)))         // Optional<Optional<Int>>

// And doing exactly what bar42 and bar50 do, only here directly:
print(type(of: try? foo(a)))      // Optional<Optional<Int>>
print(type(of: tryQ(try foo(a)))) // Optional<Int>

And for T = Int:

...
typealias T = Int

let a: T = 123
print(type(of: bar42(a)))         // Optional<Int>
print(type(of: bar50(a)))         // Optional<Int>

// And doing exactly what bar42 and bar50 do, only here directly:
print(type(of: try? foo(a)))      // Optional<Int>
print(type(of: tryQ(try foo(a)))) // Optional<Int>

So people will have to keep in mind that the behavior will be different depending on whether
the type T in try? throwingFuncReturningT() is an Optional<W> or not, and wether it's statically knowable or not. Phew …

FWIW: As can be seen by my previous posts, I'm quite far from an expert Swift programmer, and I think try?'s current behavior is much easier to understand than the proposed one.

4 Likes