Make try? + optional chain flattening work together

Evolution Pitches
41

I gave an example upthread. The most common case I have seen is with APIs that throw and also return loosely typed data (often Any). It is not uncommon to see code that uses these APIs downcast the result using as? in the same expression where try? is used. When code does this there are exactly two code paths: one when where a value out with the expected type was available and the other where it isn't. Information about the reason you couldn't get the expected value is lost, but sometimes it is not necessary.

I think it's also worth noticing that as? performs the same kind of flattening we're discussing in this thread:

let helloAny: Any = "String"
let helloString: String? = helloAny as? String

let helloAnyOptional: Any? = "hello"
let helloStringOptional: String? = helloAnyOptional as? String // "hello"

let nilAnyOptional: Any? = nil
let nilStringOptional: String? = nilAnyOptional as? String // nil

As far as I know, try? is the only operator which involves optional and does not flatten. It is an exception to how Swift typically handles Optional which makes it unintuitive. try? will be more consistent with the rest of the language as well as being more useful if this work moves forward.

2 Likes
42

Oh, you mean the toy example? It's not so helpful to me because of course you can make up examples where this flattening would be useful, but I'm trying to get a feel for how often they occur in practice.

I don't find as? casting to be a great example to follow because of how confusing and broken feeling the system is when mixing Optional/Any/Any? because of the flattening. You can see a lot of confusion in this area if you look at any developer forum. We're probably stuck with it though, and hopefully it's rarely encountered.

Not exactly sure what the definition of “operator” is, but this doesn't seem true. ! doesn't flatten, optional chaining doesn't flatten (in that you need to use ??. instead of ?. to optional chain through a double optional), if let doesn't flatten, ?? (nil coalescing) doesn't flatten, and I've run out of optional operators/functionality that I remember off the top of my head.

1 Like
43

I don’t think it’s a toy at all. There is a lot of Swift code doing this. The frequency will go down over time as we get better APIs like Codable but the need for this kind of code will still exist.

I think the confusion here is primarily around the relationship of a top type to an optional type. The nuances of Objective-C bridging are also a source of confusion. I don’t think the behavior of the flattening behavior of the as? operator when going from one optional type to another optional type is the fundamental source of confusion.

My mistake, I wasn’t thinking of force unwrap. Optional chaining certainly does flatten. That is the whole point of it! ?? is a coalesces to remove a layer of optionality which is closely related to flattening. if let isn’t an operator but it does remove a layer of optionality in the scope it introduces.

44

I disagree, because it leaves you unable to distinguish between T and T? in some cases. And the only recourse, at least according to several of those linked threads, seems to be to go through the half-baked `Mirror API.

I wrote the exact sense in which I claimed optional chaining doesn't flatten, but I'll clarify. If your type is Optional<Optional<T>> then you need to use ??. not ?. to do optional chaining (so it doesn't flatten multiple Optionals down to a single one), and if let and ?? remove only one layer of optionality not both/all. If your optional chain ends in a method/variable that returns a double Optional then the resulting type is a double Optional not a single, etc. These are all situations where multiple layers of Optionals could be collapsed, but aren't.

1 Like
45

as? does what it's told to do, so I don't think its behavior supports a change for try?

let helloAnyOptional: Any?? = "hello"
let helloStringOptional: String?????? = helloAnyOptional as? String // "hello"
1 Like
46

These are all situations where the language facilitates removing exactly one layer of optionality. That is what is being proposed in this thread.

47

Given that virtually everyone uses try? to ignore errors, can you explain why the following behavior should differ?

func throwOpt() throws -> Int? {
    return 3 as Int?
}

if let x = try! throwOpt() {
    print(x)                       // 3 
}

if let x = try? throwOpt() {
    print(x)                       // Optional(3) 
}

I think the confusion boils down to this point: despite ! and ? being used for dealing with Optionals in other contexts, most developers seem to expect that with try, they are only analogous in behavior, and not actually directly related.

1 Like
48

The optionality is being conditionally removed, though, versus unconditionally removed. So it's actually removing either 0 or 1 layers of optionality, to be pedantic.

Okay, I'll try this from another angle to make what I'm trying to say clear. Imagine if we try to implement these Optional operators/functionality in Swift. Many of these are simple, with roughly these signatures:

!: T? -> T
??: (T?, T) -> T
if let: (T?, T -> ()) // okay, there's also some name binding, etc you can't express

Now, write down a rough signature for the proposed try? or ?.. They're more complex, maybe roughly something overloaded like:

try?: (throws -> T) -> T?
try?: (throws -> T?) -> T?

and I'm not even sure how to write ?: ((T?, T -> U) -> U? then something else for repeated chaining?). I'm only saying that this extra complexity/overloaded behaviour makes it harder to reason about and I think it needs more justification and proof of burden in practice.

I don't think I understand the question. try? doesn't ignore errors, it ignores the error type/particular failure mode. You still need to handle the error by dealing with the Optional in some way. try! is roughly (throws -> T) -> T so the behaviour is obvious to me (and the same as as! vs as?).

2 Likes
49

I didn't write that it ignores errors. I wrote that's what developers use it for. I have never once used for it anything else, and no one who has has chimed in. The entire purpose of this proposal is to get it to behave the way people use it. Or want to use it, if you wish to be pedantic.

[edit]

To clarify: I am expressing my opinion based on my usage and my knowledge. If enough people chime in to explain how and why they use the current behavior, I will not suggest that we change behavior to make it better for some, at the expense of others.

1 Like
50

This one is not hard either:

func flatTry<T>(_ body: @autoclosure () throws -> T?) -> T? {
    do {
        return try body()
    } catch {
        return nil
    }
}

func throwing() throws -> String {
    struct E: Error {}
    throw E()
}
let throwingResult = flatTry(try throwing()) // Optional<String>.none

func optionalThrowing() throws -> String {
    struct E: Error {}
    throw E()
}
let optionalThrowingResult = flatTry(try optionalThrowing()) // Optional<String>.none

func throwingAny() throws -> Any {
    return 42
}
let throwingAnyResult = flatTry(try throwingAny() as? Int) // Optional<Int>.some(42)
let throwingAnyNilResult = flatTry(try throwingAny() as? String) // Optional<Int>.none

func optionalThrowingAny() throws -> Any? {
    return 42
}
let optionalThrowingAnyResult = flatTry(try optionalThrowingAny() as? Int) // Optional<Int>.some(42)
let optionalThrowingAnyNilResult = flatTry(try optionalThrowingAny() as? String) // Optional<Int>.none
51

I presume you mean

func optionalThrowing() throws -> String? {
  struct E: Error {}
  throw E()
}

instead. Sure, that makes sense. I will only say that this relies on implicit conversion to Optional which dozens of threads in this forum and others have shown comes with its own set of confusions, particularly in generic contexts and/or when inference is done over multiple interacting signatures. And it's of course equivalent to the overloads I posted above. It's the conditional behaviour that I find complex, particularly when interacting with optional chaining and other functionality, not that you need two separate implementations.

Edit: And the compiler errors or warns you when you try to use these other Optional features with a non-Optional type, instead of just implicitly converting to an Optional, probably because the implicit conversion would be confusing or useless

error: cannot force unwrap value of non-optional type
error: initializer for conditional binding must have Optional type
warning: left side of nil coalescing operator '??' has non-optional type, so the right side is never used
52 
let w: Any? = 3
let x: Any = 3

let y = w as? Int
let z = x as? Int

print(y)   // Optional(3)
print(z)   // Optional(3)

I fail to understand why it makes sense to have this behavior, where as? does not wrap the Any? in another Optional, but it does make sense for try? to do so.

This is beside for the fact that no one has shown a reason why anyone would want the double-Optional.

53

My interpretation is that as? isn't about wrapping or unwrapping:
It just tries to convert its argument to the given type - and just because that may fail, it returns an Optional.

I don't think there is a real reason (reason in the sense of "it's useful because of...").
It is just consistency:
try? returns either Optional.none or Optional.some(T) - and if T happens to be an Optional itself, you end up with double optionality.

2 Likes
54

Note my little edit. What I changed it to isn't the case, of course, but if it were, it would be just as logical, no?

I fully understand that as? and try? do not work the same way. What I haven't seen yet is a good explanation as to why.

55

If I recall correctly, as? T was originally as T?, but it was deemed confusing so failable casts were separated out into their own operator. Meanwhile, try? has always just mapped errors to nil.

56

I have submitted a formal proposal for the changes discussed here. I have a working implementation (I hope!), though there's still a bit of work to do to handle Swift 4 compatibility and migration.

5 Likes
57

Thanks for writing this and particularly for looking into the impact on the compatibility suite. Do you have the ability to provide statistics on how often this change will have an effect in the compatibility suite (i.e. the counts of how often try? is used to with an expression that is statically known to produce an Optional vs a non-Optional)? This would really help me to understand the tradeoff between simpler behaviour and the burden of dealing with the nested Optional in this case.

Edit: I notice that in one part of the proposal you say

The above examples are contrived, but it's actually quite common to end up with a nested optional in production code

but in the source compatibility section you say

It appears that most uses of try? in the Swift Compatibility Library are with non-optional expressions. The code most liikely to be impacted is of the form try? foo() as? Bar, but even that is relatively rare, and can be migrated.

which is somewhat contradictory, so maybe some data would help here.

58

That’s true; those are contradictory, and data would be helpful. Unfortunately, github’s search functionality ignores punctuation, so I was searching for things like ”if let” “try” and let try as and manually scrolling through the search results to find cases that would be affected. That’s hardly a rigorous way of doing things, but the initial results suggested that the solid majority of try? statements are with non-optional.

In my own code base, nearly all the places I run into this are of the form if let x = (try? someCocoaAPI()) as? MyType. I have already manually fixed the problem here by adding the parentheses, so my code at this point would work the same either way. But every time I write one of these I’m annoyed by the current behavior.

A better way of determining the impact might be to check my implementation PR against the compatibility suite, which we can do now that the PR is up. My gut instinct is that this will affect most projects, but only a handful of times per project; relative to the size of the codebase itself, it would be fairly small.

I suspect that try? is used more by apps than by libraries, since libraries should generally preserve information about what errors occurred and let the calling code decide what to do about it. That may mean that try? is under-represented in the compatibility suite relative to the overall population.

I also suspect that try? is especially more common in ad hoc use cases like Swift Playgrounds, where convenience is more important than rigor. Those may be harder to quantitatively analyze, since they’re usually not checked in to version control.

1 Like
59

Yeah, it's tricky because people might have already rewritten their code to avoid the double-Optional and the compatibility suite is self-selected so is unlikely to be representative, as you note. Before I wrote my previous post I originally thought it might be quite unrepresentative, in that it might not have many projects that interact with Cocoa/UIKit/other Objective-C framewors, but I looked at the project list and it seems to contain a fair amount of UI components and similar. I still think it would be interesting to know the numbers, even with those caveats.

60

This isn't quite true. It can perform the flattening, but only if you ask it to. as? permits the writer to specify which type to convert to, and so the flattening only occurs when it is explicitly requested.
Note that given

let helloAnyOptional: Any? = "hello"

then the most direct way of getting rid of the Any part is by using the following:

let helloStringOptionalOptional: String?? = helloAnyOptional as? String?

The writer may however choose to do do the following:

let helloStringOptional: String? = helloAnyOptional as? String

which performs the unwrap as part of the cast, but it is importantly always clear which you are obtaining.

(also note that helloAnyOptional as? String?? is never valid when helloAnyOptional: Any? – confirming the first statement doesn't just "add an optional layer").