Crash during optional key path access. What is going on?

Using Swift
1

I'm encountering a very strange crash when appending key paths along optionals. Here's a reduced case:

struct Foo {
  var bar: Bar?
}
struct Bar {
  var baz: Int?
}
extension Optional {
  subscript(default default: Wrapped) -> Wrapped {
    self ?? `default`
  }
}

let foo = Foo()
let kp1 = \Foo.bar
let kp2 = kp1.appending(path: \.?.baz)
let kp3 = kp2.appending(path: \.[default: 5])
foo[keyPath: kp3]  // 💥

:boom: Could not cast value of type 'Swift.Optional<()>' (0x1df122378) to 'Swift.Int' (0x1df11e138).

Any idea what's going on? Any savvy folks know how to work around the issue?

(Filed here.)

2 Likes
2

I noticed that it (the example code in the original post) doesn't compile without the extension but it also doesn't trigger any breakpoints that I set in the extension. I could be missing something though.

With that said, it looks like the cast needs to happens before the subscript in the extension is exercised.

3

The optional extension, you mean? Yeah, that's part of the subscript, so it's required for things to compile. I don't think the breakpoint hits because it crashes before it hits that part.

There's no casting happening in my code. I think the cast is happening inside Swift's own key path logic.

4

the best explanation i've come up with thus far is that the runtime crash is due to this logic which seemingly attempts to handle the scenario where an optionally-chained key path component resolves to nil. it looks like there is perhaps an expectation that the presence of any optional-chain component implies that the 'leaf value' of the key path must itself be optional, which is not the case in the provided example. there are even some assertions to this effect in the stdlib, though i'm not really sure how to enable them (or if that requires a custom build of the runtime).

i think the expected behavior in this case would be to return nil rather than invoke the extension on Optional, since optional chaining is supposed to 'short circuit' evaluation when a nil is encountered in a chained expression. for example, if we consider a non-key-path formulation, the compound expression would produce an optional Int value:

let baz = Foo().bar?.baz[default: 5] // evaluates to nil since `bar` is nil
type(of: baz) // Optional<Int>

and indeed, if we modify the example slightly to use an extension that requires no arguments, then we can see that the full key path expression is inferred to yield an optional value when an optional-chain component is present:

extension Optional where Wrapped == Int {
  var defaultValue: Int { 5 }
}

let foo = Foo()
let kp = \Foo.bar?.baz.defaultValue
type(of: kp) // KeyPath<Foo, Int?>
_ = foo[keyPath: kp] // nil – no crash!

interestingly, using the original example, trying to provide a full key path expression statically rather than building it up dynamically using the appending(path:) methods does not appear to compile (at least in any configuration i could think of), which is perhaps another indication that this scenario is an unhandled edge case.

let staticKPE = \Foo.bar?.baz[default: 5] // 🛑 A Swift key path must begin with a type

so, putting the pieces together, it seems like the crash is likely due to the aforementioned forced cast to a non-optional value failing at runtime when the key path contains an optional chain, but ends in a non-optional terminal value. this suggests that a workaround may be to update the Optional extension to return an implicitly unwrapped optional, and indeed, if we make that adjustment the code no longer crashes.

extension Optional {
  subscript(default default: Wrapped) -> Wrapped! {
    self ?? `default`
  }
}

let foo = Foo()

let kp1 = \Foo.bar
let kp2 = kp1.appending(path: \.?.baz)
let kp3 = kp2.appending(path: \.[default: 5])
foo[keyPath: kp3] // nil

we don't get the 'nil coalescing' behavior that the extension is perhaps trying to provide, but i think that is expected in this case.

5

Can you explain why this should be expected? To me it's quite surprising.

The extension is on Optional, not the Wrapped value at the end of the optional chain, so I would expect things to parenthesize more like this:

(foo.bar?.baz)[default: 5]  // 5

Instead, it seems that any key path that is optional chained cannot be further appended. And that optional-chaining a key path breaks its further composability.

Note that even if I change the optionality of the baz things still crash.

In general I would expect a key path to be fully parenthesized before appending:

\.a + \.b + \.c

// equivalent to:

((\.a).b).c)

Edit: It looks like an open PR by @Alejandro removes that cast, so perhaps this crash is fixed on his branch: [DNM] [stdlib] Performance improvements for reading keypaths by Azoy · Pull Request #70451 · apple/swift · GitHub

6

i think the expectation that key paths behave this way can reasonably be drawn from what we observe when writing 'vanilla' optionally-chained expressions. for example,

let baz = Foo().bar?.baz[default: 5]
print(type(of: baz)) // Optional<Int>
print(baz as Any) // nil

let kp = \Foo.bar?.baz[default: 5]
print(type(of: kp)) // KeyPath<Foo, Optional<Int>>
print(Foo()[keyPath: kp] as Any) // nil

when accessing a property via an optional chain, the resulting value is itself an optional, independent of the optionality of the final component of the chain. the optional chaining section of the swift programming language book highlights this with an example:

... Note that this is true even though numberOfRooms is a non-optional Int. The fact that it’s queried through an optional chain means that the call to numberOfRooms will always return an Int? instead of an Int.

the key path handling in the stdlib appears to try and encode this as the expected behavior when evaluating key paths. i think the expectation that using the dynamic appending(path:) methods would produce evaluation semantics different from what one would get when writing out a key path statically would be surprising.

// these seem like they should be equivalent

let staticKP = \Foo.bar?.baz[default: 5] // KeyPath<Foo, Int?>

let dynamicKP = (\Foo.bar)
  .appending(path: \.?.baz)
  .appending(path: \.[default: 5]) // KeyPath<Foo, Int>

the fact that these don't currently produce equivalent key paths seems like a bug or oversight in how optional chains are handled in this context. from the (admittedly somewhat aged) key paths proposal, it seems like the intent is that the two methods of construction should produce key paths that behave the same way:

Optionals are handled via optional-chaining. Multiply dotted expressions are allowed as well, and work just as if they were composed via the appending methods on KeyPath.

if 'parenthesizing' the evaluation of the key path were supported, i would expect some additional sigil or method to be required, as it is in a non-key-path expression.

that all being said, this is just my opinion with some sprinkles of speculation – my expectations may not be shared. i was unable to really find much in the way of 'formal' descriptions of the intended semantics in this case, so i'm unsure what those more well-versed in this domain would expect.

based on the current implementation in that branch, i would guess the exact way this manifests probably changes, but fundamentally i don't think it resolves the issue that it's possible to construct a key path which the type system says returns a non-optional value, but actually returns nil at runtime via an optional chain.

1 Like
7

I agree that a single, unparenthesized, optional-chained key path expression should behave the same as a single, unparenthesized, optional-chained expression. The difference becomes clear when we break things over the course of a few assignments:

let bar = Foo().bar
let baz = bar?.baz
let coalesced = baz[default: 5]  // 5

let bar = \Foo.bar
let baz = bar.appending(path: \.?.baz)
let coalesced = baz.appending(path: \.[default: 5])
Foo()[keyPath: coalesced]  // 💥 (expected: 5)

I think that append is the parenthesizing operation. Parenthesizing a key path expression is even what gives you the ability to focus on the key path as a whole to append it:

(\Foo.bar?.baz).appending(path: \[default: 5])
// I expect to behave like:
(foo.bar?.baz)[default: 5]

// vs.

\Foo.bar?.baz[default: 5]
// I expect to behave like:
foo.bar?.baz[default: 5]

But I think what seals the behavior is the actual key path types:

\Foo.bar?.baz[default: 5]                       // KeyPath<Foo, Int?>
(\Foo.bar?.baz).appending(path: \[default: 5])  // KeyPath<Foo, Int>

To me this unquestionably communicates that one expression is optional-chained while the other is optional-coalesced, since one has a value type of optional Int? while the other has a value type of non-optional Int.

3 Likes
8

In addition to the resultant value types, the signature of the appending method itself I think admits only this interpretation. Things are a bit obscured in this example by the fact that Foo.baz is of type Int?, but if it were instead just Int IMO this aspect would be a bit clearer. The 'unparethesized' append operation could only accept a keypath of type KeyPath<Int, T> because the optionality is entirely external to the portion of the member chain 'below' the optional chain member. But the appending operation takes a keypath value which is rooted at the value type of the overall appended-to keypath. IOW, if baz were of type Int, the 'unparenthesized' interpretation of:

(\Foo.bar?.baz).appending(path: \.[default: 5])

would be nonsense, because baz does not have such a subscript member. Indeed, writing out the full keypath with such a setup fails to compile:

\Foo.bar?.baz[default: 5] // error: value of type 'Int' has no subscripts
9

thanks for going through that example a bit more explicitly – i think Stephen alluded to the same thing above but i didn't entirely follow the point being made. i'm convinced that the 'parenthesized' interpretation of the appending(path:) method is the appropriate, and perhaps only consistent one. so i think that means the reported behavior is just a bug in the handling of this configuration. it seems the runtime evaluation of keypaths treat the 'unwrapped a nil' case of optional chaining as a terminal step, when it should instead proceed to the next component that was composed with the optionally-chained expression, if any.