Pointers to Never

Using Swift
,
1

Consider some code like the following:

import Foundation

if let ptr = calloc(1, MemoryLayout<Never>.stride)?.bindMemory(to: Never.self, capacity: 1) {
  print("Dereferencing pointer to Never")
  ptr.pointee // Expression of type Never means control flow can't pass here
  print("wait what")
} else {
  print("ah, that makes sense")
}

Just like that, we’ve broken typical control flow rules without ever writing unsafe. I can imagine writing this into a generic function marked as returning its generic type arg, which gets inlined and breaks calling code.

Interestingly, if I throw ptr.pointee, it segfaults.

2

Even though you don't write the word unsafe, you're still using a type with unsafe in its name (calloc should return an UnsafeMutableRawPointer, I believe?).

After allocation, the memory is uninitialised. Even after binding the element type to Never, the memory is still uninitialised. So when you access pointee, you are accessing uninitialised memory, which breaks the API contract and is undefined behaviour.

The "fix" would be to initialise an element of type Never before you read from pointee -- which, of course, in the specific case of Never, you cannot do.

It's worth nothing that nothing about this is really unique to Never -- except that maybe some control-flow weirdness will happen when you have a value of it (because those values are not supposed to exist). But since that is after the UB, the compiler is allowed to do that control-flow weirdness.

3 Likes
3

Yeah. Actually, I think it’s UnsafeMutableRawPointer?, because it’s allowed to return NULL when you ask for too large of a block of memory.

See, that’s why I was using calloc instead of malloc. The former guarantees the memory will be zeroed out (which I recognize isn’t valid for all types, e.g. non-optional references), while the latter returns whatever garbage happened to be on the heap.

4

calloc doesn’t always create a valid initialized value in Swift, since all zeros is not a valid representation of all types. For instance you could have also calloc-ed memory for a non-Optional pointer type, and loading from that memory would give you an invalid value since the type is not nullable.

That said, even though it’s undefined behavior, it would be reasonable and safer for load and store operations for known-empty types to compile into traps, rather than proceeding as if nothing happened.

3 Likes
5

It turns out that it was considering the access to ptr.pointee an unused expression, and the optimizer removed everything except the print statements — even the call to calloc. It even considered the else branch dead code and removed it. I’m very surprised that it removes unused expressions of type Never.

6

That’s a consequence of it being undefined behavior. Having a value of Never “is impossible” so the optimizer may indiscriminately remove code that operates on value of type Never, as well as any code around it that may be affected by it, even if it’s reachable by normal control flow.

5 Likes
7

I’m curious why it segfaults if I throw the pointee. It seems to generate something like

let x = swift_allocError($ss5NeverON, $ss5NeverOs5ErrorsWP, 0, 0)
swift_willThrow()
swift_errorInMain(x)

$ss5NeverON is the type metadata for Never, and $ss5NeverOs5ErrorsWP is the the witness table for Never: Error. I’m not 100% sure what part of this is segfaulting, but if I had to guess, it’s allocating zero bytes for the Never instance and then attempting to dereference that (malloc & co are allowed to return an invalid pointer if you allocate zero bytes).

8

That sounds likely. The upshot of it being undefined behavior is that there are really no rules as to what happens when it occurs, since most parts of the compiler and runtime will assume it can't happen.

1 Like
9

Yeah, I can’t come up with any logic by which segfaulting is wrong. Like, if it didn’t do something to kill the whole program, you could write

do {
  try foo()
} catch let error as Never {
  // Something has gone horribly wrong.
  // It’s not even possible to do anything with `error` here.
}

So, throwing a Never has to at least kill the current task, the same way a normal uncaught error would.

On one hand, it’s possible to send errors across tasks with try await task.value. On the other hand, that doesn’t throw when the failure type is Never, so it has to return… what? There’s no value for it to return. The only logical thing that this could do is cause UB.

10

The most sensible thing to do IMO would be to have it so that an obvious attempt to dereference a pointer to Never traps immediately. UB means that "anything can happen", but that would be the safest "anything" possible. We could also probably harden the runtime so that known-uninhabited types like Never trap if they are dynamically instantiated, such as if one of the initialize* or assign* value witness functions are called, or if you attempt to call a swift_*[aA]lloc* function with such a type. Because of the aggressive deletion of UB code by the optimizer, though, it's hard to guarantee such a trap will happen, but we can do our best.

4 Likes
11

After some more messing around, I've found that when you extract it to a function, it matters what side of the function the .pointee is on.

func typedMalloc<T>(_: T.Type) -> UnsafeMutablePointer<T> {
  malloc(MemoryLayout<T>.size)!.bindMemory(to: T.self, capacity: 1)
}
typedMalloc(Never.self).pointee
print("Why am I still here")

func unsafeMakeUninitializedInstance<T>(_: T.Type) -> T {
  malloc(MemoryLayout<T>.size)!.bindMemory(to: T.self, capacity: 1).pointee
}
unsafeMakeUninitializedInstance(Never.self)
print("This is eliminated as dead code")

I have two unrelated points to make here.

  1. A weird corollary to this is that the former can't be used to make a function that explicitly returns Never unless you wrap it correctly:
func typedMallocAndDereference<T>(_: T.Type) -> T {
  typedMalloc(T.self).pointee
}

func thisDoesntEvenCompile() -> Never {
  typedMalloc(Never.self).pointee
}
func butThisDoes() -> Never {
  typedMallocAndDereference(Never.self)
}

I realize that typedMallocAndDereference here (as well as unsafeMakeUninitializedInstance above) is just asking for nasal demons, but this is a very weird inconsistency to me. It shouldn't matter whether the Never comes from a function call or a property access, right?

  1. Under -O, unsafeMakeUninitializedInstance is inlined in the SIL, which reflects it as this:
  %59 = apply %4(%3) : $@convention(c) (Int) -> Optional<UnsafeMutableRawPointer>, loc "/app/example.swift":10:5, scope 28 // user: %60
  switch_enum %59 : $Optional<UnsafeMutableRawPointer>, case #Optional.some!enumelt: bb3, case #Optional.none!enumelt: bb4, loc "/app/example.swift":10:33, scope 28 // id: %60

bb3(%61 : $UnsafeMutableRawPointer):              // Preds: bb2
  %62 = struct_extract %61 : $UnsafeMutableRawPointer, #UnsafeMutableRawPointer._rawValue, loc "/app/example.swift":10:35, scope 28 // user: %63
  %63 = bind_memory %62 : $Builtin.RawPointer, %12 : $Builtin.Word to $*Never, loc "/app/example.swift":10:35, scope 28
  unreachable , loc "/app/example.swift":12:1, scope 25 // id: %64

bb4:                                              // Preds: bb2
  %65 = integer_literal $Builtin.Int1, -1, loc "/app/example.swift":7:7, scope 6 // user: %66
  cond_fail %65 : $Builtin.Int1, "Unexpectedly found nil while unwrapping an Optional value", loc "/app/example.swift":10:33, scope 28 // id: %66
  unreachable , loc "/app/example.swift":10:33, scope 28 // id: %67

It actually calls malloc and bind_memory, but then is immediately followed by an unreachable. I think it would be a viable solution to emit a trap instead of unreachable here, but I wouldn't know how to check that.

12

And all of this is only true with malloc/calloc, since LLVM knows about those and DCE's them. If I use UnsafeMutablePointer<Never>.allocate(capacity:), it actually emits a call to swift_slowAlloc.

13

Ultimately, you cannot expect consistency where undefined behaviour is concerned. Whatever happens to occur should be considered pure coincidence.

6 Likes