SE-0217 - The "Unwrap or Die" operator

This seems to be the cause of a big part of the discussion in this thread. IMO it's not ?? that crashes here, it's fatalError. To me this is like saying that

guard let foo = maybeFoo else { fatalError("an invariant condition has been violated") }

is bad because guard isn't "supposed" to crash.

4 Likes

I can see that line of thinking ("?? is sugar for an if ... else statement"), but I disagree with it, mainly because ?? is an operator, and guard... is not.

The comparison fails because of Postfix-!, which is also sugar for an if ... else statement, except it crashes:

let foo = maybeFoo!

is equivalent to:

let foo: Foo
if let f = maybeFoo {
    foo = f
} else {
    fatalError()
}

Yet no one seems to have a problem with Postfix-! causing a crash. Why is !! different?

But we alread doing that with ?? where one overload returns the rhs optional or the other overload a non-optional rhs. I think ?! would be the natural fit if something like this would be accepted with this proposal.

:point_up:️This is an important point. If we proposal authors had been notified prior to review, we would have revised this, and the name would likely have been changed to the "Unwrap Because" operator, because technically Postfix-! is already "unwrap or die".

2 Likes

But isn't that mainly so we can support chaining?

let foo = maybeFoo ?? maybeOtherFoo ?? maybeThirdFoo ?? defaultFoo

Yeah we can have it to pick one optional over another, but that (intuitively) seems far less useful to me than wanting to end up with a non-optional.

:man_shrugging: my style disagrees, for reasons I've articulated multiple times in this thread. (!! is analogous to ??, !! is "more dangerous" than ?!, etc)

I guess we'll have to see what the Core Team says.

2 Likes

I don’t understand why you seem to ignore the reality of prevalent misuse of the ! operator. The argument about it’s use for chaining falls squarely into the “power tool use”, which no one’s questioning. The reality of everyday Swift is that this single operator is responsible for complete negation of type safety benefit brought on by the Optional type in practice. It is the smoking foot-gun of Swift, see the evidence from StackOverflow. When this issue was raised in the Resolved: Insert "!" is a bad fixit thread, @rudkx asked what should be done instead, to not let users hanging with code that doesn’t compile. This proposal offers a concrete solution that is in line with rest of DSL for Optional handling in Swift which is operator based.

Fair point. Yes, it's redundant and heavy but it provides an important deployment fix.

I want a solution. Sugar seemed to be the easiest solution. At this point, I'm more wedded to a commitment to solving this issue than to sugar.

Yes, these are common enough to warrant a change. They are especially rampant in Cocoa Foundation but every failable initializer is a potential use-case.

I know the other authors (both named and unnamed) care a lot about the design. I'm motivated by the problem. I want a fix that is in keeping with the spirit and direction of the language and I don't want this proposal to be rejected and then the problem never brought up again because it's been debated and discarded.

That said, I have used the operator. I like the operator. For reasons stated upthread, it mimics ?? coalescing (and no, while I prefer !!, I would not throw a fit with ??), it is ergonomic, it is transformative in that it turns a code-level comment into a black-swan-level usable diagnostic.

Further, it ties into the work we've been doing on creating FatalReason. Individual types with failable initializers could extend prebuilt errors, allowing the rhs to be .unconstructableURL or .missingImageAsset, with autocompletion support.

This is focused on solving one real world problem rather than a full class of theoretical ones. It offers a narrowly focused utility with limited impact and measurable benefit.

The !! operator is so naturally related to ?? that it has a significantly lower cost to both recall and recognition than a completely novel operator. Just because new operators are usually the wrong answer does not mean they are always the wrong answer.

And even though I think Optional<Wrapped> ?? () -> Never is a slightly inferior design, I'd much rather have that than allow the language walk away from this common problem space.

Erica addressed your other points, but I'd like to think about this momentarily on a thread tangent.

I actually think that all operators should just be sugar for underlying method calls. I like this from a symmetry and completeness point-of-view, and I also like it because it would establish a baseline for what would be required in order to add a new operator (it has to be sugar for a commonly-used method).

This thread isn't the place to discuss this, but on the whole I would also love to see:

public extension Optional {
    public func unwrap(_ because: @autoclosure (() → String)? = nil, file: StaticString = #file, line: Uint = #line) → Wrapped {
        if let value = self { return value }
        let message = because?() ?? "Unexpected found nil value when unwrapping value of type \(Wrapped.self)"
        fatalError(message, file: file, line: line)
    }
}
5 Likes

Same here.

I very much like the spirit of this idea. I particularly like that it could cover several failures within a single expression.

I’d want the actual syntax & naming to favor readability a bit more: less noise, minimized boilerplate, clarity about how the message should be phrased, etc.

This is an excellent question, and I suspect the answer varies quite a bit between different programming domains. If we are going to do another design round on this problem, as seems likely, I'd want to assemble a nice set of concrete, real-world examples taken from a variety of kinds of project (iOS, server side, system tool, C interop, etc.) before finalizing and reviewing the design.

1 Like

I see a lot of people mentioning any sort of Cocoa interaction as a valid and frequent reason for force unwraps, but I'd also like to point out that using C APIs can also lead to potential force unwraps.

In a path library I'm working on, I try to make things as "safe" as possible before calling into C APIs, some of which may return NULL pointers. In several instances, I can eliminate any possible reason for the C APIs to return a NULL pointer (ie: the only C API requirement is for the path to be less than PATH_MAX), and so I use either a force unwrap or a guard let x = x else { fatalError("...") }.

There are plenty of APIs across C, Foundation, Cocoa, etc that return optionals, but that developers can relatively safely presume will not return nil and if they do return nil then a fatalError is the only appropriate solution.

I would like to see a solution to this that is less verbose than guard let x = x else { fatalError("...") }.

I'm not a big fan of:

x !! "Something went wrong"

because I'd like the ability to do more than just use fatalError. Sometimes it may be a recoverable error and so I'd just like to throw without having to write out a whole guard ... else ... block.

x !! fatalError("Something went wrong")

Is definitely more brief than the whole guard let x = x..., but still clear enough that anyone (beginner or veteran) should know what is going on.



As someone who is definitely not a Swift "expert", a ? with any kind of optional signifies a "safe" operation (AKA won't crash) and the nil-coalescing operator is taught on several beginner tutorials as "unwrap the optional or use the default value provided."

The ability to throw/crash to me just seems contrary to the intent of the ? which is why I am opposed to ?? using a Never return type. Doing so would require learning what Never is and why that means it's appropriate to use fatalError instead of the default value you were originally taught should go on the right-hand side of the ??.

On the other hand, ! is taught from the start as an "unsafe" operation (AKA may-crash). And so the !! operator makes logical sense to me.

Now once you get into some more advanced Swift topics you start to realize that these "beginner assumptions" aren't exactly true and that there is a lot more to the ! or ? than you originally thought, but as I read through this thread from the beginning it was brought up several times about how this proposal is meant to help beginners and how it shouldn't be confusing to beginners etc.

In my opinion, the ?? spelling with Never as the return type definitely makes this confusing for beginners who may only know that ! means unsafe and ? means safe. They may come across someone else's code and see that the supposedly "safe" ?? operation looks really dangerous with a fatalError on the rhs. It seems like a really bad idea to shatter their perceptions of ?-related operations in something as common as the nil-coalescing operator.



I actually really like @davedelong's idea:

I'm a huge fan of keeping ?? as "safe" as it currently is and introducing the ability to either throw or crash with ?! and !! respectively. If we had these 3 options, then I wouldn't be opposed to using just a string for the rhs of !!.

5 Likes

Another issue I have with force unwrapping is the poor error message/debugging on Linux (where I do most of my dev).

In my path library when I run swift test on Linux I occasionally get a "forced unwrap unexpectedly yielded nil" (or whatever the generic failed force-unwrap message is). There is no line number or anything so I can't tell which force unwrap is failing and I haven't figured out how to use lldb with swift test so I'll generally just clone the repo onto my Mac, run swift package generate-xcodeproj and run the tests in Xcode where it will tell me exactly where the force unwrap fails.

For the most part, I only use forced unwraps where it should be 100% safe and then in places where I'm less certain I use guard + fatalError.

Unnecessary Information

Unfortunately for me, the force unwrap that's failing seems to occur in a section of code that shouldn't even be executed during the test, so I've got more debugging to do. My test opens a lot of directories very quickly, and it seems like I'm running out of open file descriptors for the process and that's having unexplainable side-effects.

Another issue is that expressions aren't permitted at the top-level (where the failed unwrap is happening), so I'm rather stuck with my force unwrap (but this is a problem for another day) instead of a guard + fatalError.

1 Like

I am -1 on the proposed design. And -0 on the issue being addressed.

I have been lurking for a good while and read the pitch when it came about, the previous review posts, and heavily discussed the topic with co-workers. Through all of that I feel like the current proposed solution is a band-aid on what is (and needs to be) a sharp and powerful edge of the language.

I understand that optionals can be difficult for new Swift developers but having looked over the latests release of the swift programming language I feel like optionals are clearly described. I feel like fixing the fixit to suggest the use of ?? defaultValue will be enough to help alleviate this problem while helping users of Swift become more knowledgable.

It feels like the proposed solution is simply trying to force developers to write a comment that ideally shouldn't be needed in most cases. I would argue that knowing how and when to use ! to force unwrap is an important skill to have so you can fully understand the execution flow and architecture of the code you are working with. It should be clear from context that this should be a safe operation or else there is often bigger issues in the code you are working with and a crash is appropriate. When things aren't simple enough that a ! gives the context to understand the "safety" of unwrapping, a guard let with a fatalError() doesn't seem like a lot of boilerplate at all.

6 Likes

The intent of the proposal is to make it easier to explain, and in turn to understand, why we are using the force unwrap operator. The spirit of the proposal is both to save us from ourselves and to make our intentions clear to others, including our future selves.

The Two Use Cases of the ! Operator

This discussion has been hampered by a conflation of the types of reason underlying the two use cases for the ! operator. The proposal contemplates a function that takes a String, with that String being a “reason why”. However, the two use cases of the ! operator have fundamentally different types of reasons, and packing those two different types into a single function is confusing. In other words, our discussion is suffering from unsafe type usage of two different subclasses of the English-language reason class.

To illustrate, consider a pair of pseudo force unwrap methods on Optional, with exaggerated signatures:

The Favored Force Unwrap Use Case:
.unwrapAssured(thisValueWillNotBeBilBecause: “reason stated here”)

This use case is undertaken with deliberation and reason as to why nil cannot occur. One should take an extra moment to document that reason.

The Disfavored Force Unwrap Use Case:
.unwrapUncertain(ifItIsNilThenItIsBestToCrashBecause: “reason stated here”)

This use case is a product of haste or expediency, with the acceptance that a crash very well may occur. Here, the reason is not about why the crash will occur, but instead about why the user is willing to allow a crash to occur (e.g., this is a prototype, and we will build the sad path later; there is no good default value, and I don’t want to write error handling code for this edge case).

While the latter use of the ! operator is disfavored, it very well may be the dominant use case. While a ?? fix-it will reduce usage of the ! operator, the user often will prefer to implement the disfavored use case of the ! operator.

The Language Can Teach the User

The language is able to teach the user. Requiring the user to explicitly select a use case would clarify intent, and teach the user about the difference. Requiring the user to supply an explicit reason as to why the disfavored use case is being implemented would help the user learn to want better patterns that avoid the dead end.

Making a disfavored use case into an official part of the language is not desirable. Yet, that approach may be superior to the reality of millions of users shoving square pegs into round holes on a daily basis.

If there is to be a Standard Library solution, perhaps that solution should teach programmers to distinguish between the two use cases, and also prompt them to learn the bad reasons underlying their use of the disfavored case.

3 Likes

I like the ?! as well, I would prefer to use that than !! in my usage. You still need to put try out front so it does not eliminate the other variants where you just want it to die as fast as possible. The big plus is getting it to a single line, and making safe code easier to write. The ! makes it too easy to write unsafe crashing code. ?! makes it almost as easy as writing ! (this also seems similar to what I wrote above, although even more well thought out).

Think of this usage pattern, developer is just getting something working, so they don't deal with all the formalities of throwing on a nil or handling the error cases. I have seen thing quite often in the code base I work on with many developers of varying abilities. They are in a hurry. They never go back to fix these issues until it is in shipping code and we start getting crashes. It is easy to miss because ! is easily lost in all the other code inside expressions. If we can ban ! in most of our code base then we can eliminate one of the biggest causes of crashes.

I think we need to make doing the right thing as easy, or easier than it is now, doing the wrong thing should be harder. This seems to be in line with making Swift a safe language. Maybe we should not focus on !! but making it easier to not do a force unwrap.

+1 from me. This is what I am arguing for on the thread about improving operator type checking.

-Chris

2 Likes

+1. I would also particularly like to see a version of the motivation that focuses on the core aspects of the language, not a bug in a fixit.

-Chris

3 Likes

If we support operators expressing underlying functions, is it possible to implement an operator that relays to a function with the following signature: (lhs: T, rhs: U, file: StaticString = #file, line: UInt = #line) -> V, and then provide an operator association attribute @operator(valid_operator, fixity: infix)?

We already introduced use cases from code from a variety of developers into the proposal. The operator's use in the wild, to be clear, drove this proposal, in a variety of domains from Cocoa to iOS. Upthread, uses with C APIs were discussed by Jacob Williams. I don't believe the proposal is lacking a breadth of use-cases. Any developer who may experience an unexpected failure related to unwrapping an optional, who wishes to emit diagnostic information about that unexpected failure, is a candidate for this solution regardless of whether iOS, server side, system tool, C interop, etc.

Focusing on the fixit, which I know you specifically asked to be removed, may be pulling this discussion off-course as with any text relating to pedagogy, even though learning and naive users are major drivers of the problem domain.

That the solution has not been generalized to try! or as! does not mean this issue is not an important one that should be addressed. I believe this loss of focus is due to a fundamental issue of language expressibility and diagnostics. I would like to see operator-nonoperator coupling discussed further. I'd like to see fixits discussed further. I would like to see better diagnostics for try! and as!. But for now, I'd like to see a commitment to a solution to this particular problem: the unexpected deployment catastrophe encountered when unwrapping an optional that is otherwise guaranteed to hold a value.

7 Likes

Erica, looking back at our proposal, you’re quite right — there are plenty of use cases.

@Chris_Lattner3, per your wondering about support for try! and as!, I did a highly unscientific scan of the Swift compat suite, and came up with these estimates of how frequently they’re used relative to force unwrapping:

operation count
force unwrap ~2000
as! 394
try! 107

I excluded test suites from that count, because tests tend to use force unwrapping much more frequently (and because usage in tests tends to be a form of asserting expected results, and thus not to call for the same kind of explanatory comments this proposal is concerned with).

A skim of ! usage suggests it’s generally well motivated. I came across many examples where either (1) restructuring code would eliminate it, or (2) there’s probably a lurking bug. (Alamofire was notably loose with its use of force unwrapping.) However, in the solid majority of cases, usage of force unwrapping seemed justifiable. In many cases, developers already added comments of exactly the kind we're trying to encourage here. In a few, I wished they had.

7 Likes

To be clear, I'm speaking for myself only, the core team hasn't even spoken about this proposal yet:

Erica (and Paul, and others). I understand that you have a pragmatic goal here, and I totally respect your focus on solving specific user challenges. I really do get that, and I really understand where you are coming from. The concerns I'm raising are not from lack of interest. Here is a more detailed explanation of my position:

Swift is still a "young" language. It is missing key (large) pieces of functionality, including a baked resilience model, ownership model, concurrency model and reliability model. These are huge bricks that should make up the house of Swift, and each will introduce new complexity into the model. We (as a community) will do our best to factor that complexity and progressively disclose it, but these are all large things that have significant unknowns, which really do need to happen.

At the same time, the swift-evolution process naturally encourages smaller incremental improvements that are "gap fillers". These are really important to me because (for example) the standard library is missing a bunch of fairly obvious convenience functions, and I'm thrilled to see those get added.

At the same time, "gap filling" when it comes to core syntax and core syntactic sugar is really dangerous right now. Operators have global impact on the language and should only be added when there is a very strong rationale for doing so. In contrast, methods only affect the types they apply to, and deprecating them is not a big deal (having a deprecated operator in the core language would be pretty embarrassing).

Again, I understand your pragmatic goal, but this proposal is problematic to me for several reasons. Because it only applies to x! it undermines a core orthogonality of ! that we've fought to preserve (remember how controversial try! was?). This proposal makes swift more complicated not only because it introduces an operator, but also because that operator is a special case that breaks a unifying principle in the language (it also doesn't chain/compose well as I've mentioned, but that's a more minor detail).

Furthermore, this proposal is likely to be subsumed in the future by more general functionality. I keep mentioning a scoped failure handler approach because the reliability model is very likely to include something like it.

In short, I think it would be very unfortunate to introduce new core syntax for this, particularly given that it makes the language less orthogonal, does not solve the whole problem, and which may become legacy in ~two years. We've made it this far without this, and while I understand that your goals are well-meaning and pragmatic, I don't see the rush to make such a high impact change here.

To be clear, I'm not trying to impede progress, and I have given a different approach to consider: introducing a new method. @davedelong mentioned that we could introduce an ".unwrap()" method, make x! be sugar for it (at least when it is an rvalue) and then introduce an .unwrap(message: ) form. Such an approach solves the same problem as this proposal, is not invasive into the core language feel, and chains more nicely. I would love to see a discussion about why this approach is problematic.

-Chris

27 Likes

I’m minor -1 on this proposal. I would like to see a redesigned FatalError with built-in options for common error cases to come along with this proposal. For example, let x = URL("str") !! Fatal.invalidLiteralValue. Having to provide a custom string for everything seems cumbersome to me.