Make try? + optional chain flattening work together

This one is not hard either:

func flatTry<T>(_ body: @autoclosure () throws -> T?) -> T? {
    do {
        return try body()
    } catch {
        return nil
    }
}

func throwing() throws -> String {
    struct E: Error {}
    throw E()
}
let throwingResult = flatTry(try throwing()) // Optional<String>.none

func optionalThrowing() throws -> String {
    struct E: Error {}
    throw E()
}
let optionalThrowingResult = flatTry(try optionalThrowing()) // Optional<String>.none

func throwingAny() throws -> Any {
    return 42
}
let throwingAnyResult = flatTry(try throwingAny() as? Int) // Optional<Int>.some(42)
let throwingAnyNilResult = flatTry(try throwingAny() as? String) // Optional<Int>.none

func optionalThrowingAny() throws -> Any? {
    return 42
}
let optionalThrowingAnyResult = flatTry(try optionalThrowingAny() as? Int) // Optional<Int>.some(42)
let optionalThrowingAnyNilResult = flatTry(try optionalThrowingAny() as? String) // Optional<Int>.none

I presume you mean

func optionalThrowing() throws -> String? {
  struct E: Error {}
  throw E()
}

instead. Sure, that makes sense. I will only say that this relies on implicit conversion to Optional which dozens of threads in this forum and others have shown comes with its own set of confusions, particularly in generic contexts and/or when inference is done over multiple interacting signatures. And it's of course equivalent to the overloads I posted above. It's the conditional behaviour that I find complex, particularly when interacting with optional chaining and other functionality, not that you need two separate implementations.

Edit: And the compiler errors or warns you when you try to use these other Optional features with a non-Optional type, instead of just implicitly converting to an Optional, probably because the implicit conversion would be confusing or useless

error: cannot force unwrap value of non-optional type
error: initializer for conditional binding must have Optional type
warning: left side of nil coalescing operator '??' has non-optional type, so the right side is never used
let w: Any? = 3
let x: Any = 3

let y = w as? Int
let z = x as? Int

print(y)   // Optional(3)
print(z)   // Optional(3)

I fail to understand why it makes sense to have this behavior, where as? does not wrap the Any? in another Optional, but it does make sense for try? to do so.

This is beside for the fact that no one has shown a reason why anyone would want the double-Optional.

My interpretation is that as? isn't about wrapping or unwrapping:
It just tries to convert its argument to the given type - and just because that may fail, it returns an Optional.

I don't think there is a real reason (reason in the sense of "it's useful because of...").
It is just consistency:
try? returns either Optional.none or Optional.some(T) - and if T happens to be an Optional itself, you end up with double optionality.

2 Likes

Note my little edit. What I changed it to isn't the case, of course, but if it were, it would be just as logical, no?

I fully understand that as? and try? do not work the same way. What I haven't seen yet is a good explanation as to why.

If I recall correctly, as? T was originally as T?, but it was deemed confusing so failable casts were separated out into their own operator. Meanwhile, try? has always just mapped errors to nil.

I have submitted a formal proposal for the changes discussed here. I have a working implementation (I hope!), though there's still a bit of work to do to handle Swift 4 compatibility and migration.

5 Likes

Thanks for writing this and particularly for looking into the impact on the compatibility suite. Do you have the ability to provide statistics on how often this change will have an effect in the compatibility suite (i.e. the counts of how often try? is used to with an expression that is statically known to produce an Optional vs a non-Optional)? This would really help me to understand the tradeoff between simpler behaviour and the burden of dealing with the nested Optional in this case.

Edit: I notice that in one part of the proposal you say

The above examples are contrived, but it's actually quite common to end up with a nested optional in production code

but in the source compatibility section you say

It appears that most uses of try? in the Swift Compatibility Library are with non-optional expressions. The code most liikely to be impacted is of the form try? foo() as? Bar, but even that is relatively rare, and can be migrated.

which is somewhat contradictory, so maybe some data would help here.

That’s true; those are contradictory, and data would be helpful. Unfortunately, github’s search functionality ignores punctuation, so I was searching for things like ”if let” “try” and let try as and manually scrolling through the search results to find cases that would be affected. That’s hardly a rigorous way of doing things, but the initial results suggested that the solid majority of try? statements are with non-optional.

In my own code base, nearly all the places I run into this are of the form if let x = (try? someCocoaAPI()) as? MyType. I have already manually fixed the problem here by adding the parentheses, so my code at this point would work the same either way. But every time I write one of these I’m annoyed by the current behavior.

A better way of determining the impact might be to check my implementation PR against the compatibility suite, which we can do now that the PR is up. My gut instinct is that this will affect most projects, but only a handful of times per project; relative to the size of the codebase itself, it would be fairly small.

I suspect that try? is used more by apps than by libraries, since libraries should generally preserve information about what errors occurred and let the calling code decide what to do about it. That may mean that try? is under-represented in the compatibility suite relative to the overall population.

I also suspect that try? is especially more common in ad hoc use cases like Swift Playgrounds, where convenience is more important than rigor. Those may be harder to quantitatively analyze, since they’re usually not checked in to version control.

1 Like

Yeah, it's tricky because people might have already rewritten their code to avoid the double-Optional and the compatibility suite is self-selected so is unlikely to be representative, as you note. Before I wrote my previous post I originally thought it might be quite unrepresentative, in that it might not have many projects that interact with Cocoa/UIKit/other Objective-C framewors, but I looked at the project list and it seems to contain a fair amount of UI components and similar. I still think it would be interesting to know the numbers, even with those caveats.

This isn't quite true. It can perform the flattening, but only if you ask it to. as? permits the writer to specify which type to convert to, and so the flattening only occurs when it is explicitly requested.
Note that given

let helloAnyOptional: Any? = "hello"

then the most direct way of getting rid of the Any part is by using the following:

let helloStringOptionalOptional: String?? = helloAnyOptional as? String?

The writer may however choose to do do the following:

let helloStringOptional: String? = helloAnyOptional as? String

which performs the unwrap as part of the cast, but it is importantly always clear which you are obtaining.

(also note that helloAnyOptional as? String?? is never valid when helloAnyOptional: Any? – confirming the first statement doesn't just "add an optional layer").

I did not suggest that it always performs flattening, just as try? will not always perform flattening.

This is operation preserves the most information but I don't think I have ever seen real-world code that needed to preserve that information. In practice, people almost always use flattening when using as? with an optional value.

Mostly what I'm trying to say here is that because the behaviour with as? is explicit it is fundamentally a different mechanism to any kind of implicit unwrapping that try? might be given. I think it is useful (as you've mentioned) to point out that

In practice, people almost always use flattening when using as? with an optional value.

However I do not believe it to be accurate to say that the explicit unwrapping of as? is at all similar as a mechanism to the proposed implicit unwrapping of try?, even if there might be similar intent in most cases.

The main reason I brought up as? is not to argue for direct similarity. I brought it up in the context of other fundamental operations involving optional in order to point out that when flattening a single layer of optionality is possible it is performed. In other words, the current behavior of try? is inconsistent with the rest of the language.

1 Like

In order to evaluate whether try? is inconsistent in comparison to as? I think it is important to look at why the flattening occurs for as?.
Consider for the case of as? that by writing let a = foo() as? String, this always produces a: String? regardless of the return type of foo(). i.e. the type that the expression foo() as? String resolves to is not just some convenience of as?, but instead is intrinsic to its definition. If it produced String?? then it would be broken.

Orthogonally, let b = try? bar() requires one to be aware of the return type of bar() to know what to expect as the expression try? bar() to evaluate to. The primary purpose of try? is to convert throws into an optional wrapping.

It might be desirable to have try? collapse an optional layer, but clearly it is not definitionally broken in the same way that as? would be if it exhibited non-collapsing behaviour. Thus I don't believe this to be an inconsistency.

4 Likes

I agree that it would be broken for as? to return something other than its RHS?. I still think it's an inconsistency in the programmer model as it is experienced. If you disagree that is ok! :)

The most important point IMO is that the current behavior of try? makes the language harder to learn and use than it should be. BJ's proposal does a really nice job of walking through this in detail.

1 Like

Here are a few numbers from the 59 projects in the Swift Source Compatibility Suite. I tried to find cases of try? being used with an optional-typed sub-expression. (I haven't tried to compile them all yet with these changes, but I've downloaded them and am searching for common patterns.)

  • There are 613 total instances of try? in the compatibility suite. The vast majority of those appear to use non-optional sub-expressions, and would be unaffected by this proposal.

  • There are 4 instances of try? ... as? . Two in mapper, one in Kitura, and one in SwiftLint. All four of them wrap the try? in parentheses to get the flattening behavior, and would be source-compatible either way. They all look something like this:

    (try? JSONSerialization.jsonObject(with: $0)) as? NSDictionary
    
  • There are 12 cases of try? foo?.bar() across 3 projects.
    10 of those assign it to _ = try? foo?.bar(), so the resulting type does not matter.
    2 of those cases have a Void? sub-expression type, and just ignore it.

  • There are 6 instances in KeychainAccess of try? somethingReturningOptional(). They all flatten it manually using flatMap { $0 }

    (try? get(key)).flatMap { $0 }
    
  • As far as I can tell, there are zero cases in the entire suite where a double-optional is actually used to distinguish between the error case and the nil-as-a-value case.

If you'd like to inspect it yourself, I'm happy to send a list of all 613 cases. (I tried attaching it here, but it's too large to paste inline, and attachments have to be images.) Send me a direct message if you're interested in seeing it.

5 Likes

I have updated the proposal with the Source Compatibility Suite analysis results.

1 Like

Great work, thanks. That's helpful and strengthens the proposal regardless of whether the reader thinks the change should be made.

To me it seems like it could be a larger feature to implement with more changes than people may suspect.

struct Foo<T> {
  func bar(T) throws -> T { ... }

  func doSomething(_ arg:T) -> T? {
     return try? bar(arg)
  }
}

In some cases, this now will need to fail, because when T is an Optional, the return type of doSomething is now T, not T?.

Similar generic issue, if I did

let b = try? bar(arg) {...}

As part of the implementation of this generic class, it would mean two different things depending on if T was actually an optional. If it was non-optional, it would mean 'I am ignoring the why of the exception, just run this code if the method succeeded'. If it was optional, it would mean 'if the message succeeded and the argument was non-null give me this new inner type my code currently doesn't have a mechanism available in the language to handle'.