[Pitch 2] Safe loading of values from `RawSpan`

I like the direction of this pitch; it's good to reduce the need for unsafe code.


I don't think Never should conform to FullyInhabited. In theory, it should always be okay to perform an unsafeBitCast if the destination type conforms to FullyInhabited and has the same size as the source type (and if both types are BitwiseCopyable). But unsafeBitCast(Void(), to: Never.self) is always undefined behavior because it constructs a value of type Never, despite the fact that Never and Void have the same size.


The proposal draft mentions a storeBytes method, bound to FullyInhabited & FixedWidthInteger. But isn't always safe to store a value that conforms to FullyInhabited (even if it also conforms to FixedWidthInteger), because the value may contain padding bytes. That would require another protocol; let's call it HasNoPadding. The protocol would be orthogonal to FullyInhabited; for example, the Bool type would conform to HasNoPadding, even though it doesn't conform to FullyInhabited.

A subtlety with HasNoPadding is that even if a value has no padding bytes, an inline array of values could have padding bytes. For example, Void has no padding bytes, but [1 of Void] does (this is because the "size" and "stride" of Void are different). To solve the problem with full generality, we would need a protocol like ArrayHasNoPadding. Another solution is to just make it so HasNoPadding requires arrays to have no padding, so Void wouldn't conform to HasNoPadding.


While the names FullyInhabited and HasNoPadding are unambiguous, they're also quite technical. We tried to avoid confusing names for BitwiseCopyable, which originally could've been called TriviallyCopyable (which is the name of the concept in C++ jargon). Thinking about these protocols in terms of what people would do with them, maybe FullyInhabited could be called BitwiseLoadable, and HasNoPadding could be called BitwiseStorable. (And ArrayHasNoPadding could be called something like BitwiseArrayStorable).


There is some prior art for these ideas in Rust: the bytemuck, zerocopy, and safe_transmute libraries, and the work done by the safe transmute project group, primarily the Safer Transmute RFC. These efforts are trying to use the type system to allow users to safely do more and more kinds of bit manipulation. As a result, they use some very complex protocol hierarchies and other type system tricks. For example, the zerocopy library has a FromZeros protocol, for types whose values can be safely "zeroed out".

All of that makes me think even if supporting safe bit manipulation for integer types is simple enough, we should probably draw a line on how much we want to generalize it (at least for now), because going too far could easily introduce a lot of complexity. Personally, I think restricting it to integer and floating-point types, and inline arrays of integer types, as the pitch currently does, would be good enough for now.

I also think maybe it would be good to discourage users from relying on layout details that aren't guaranteed by the language. The benefit of restricting the FullyInhabited protocol to integer and floating-point types, and inline arrays of these types, is that those types all have a predictable memory layout.


Maybe the functions with a FixedWidthInteger requirement should drop the FullyInhabited requirement. Then those functions would need to manually do bit manipulations (possibly using the BinaryInteger.words property, maybe with an optimization for the built-in integer types) instead of just directly writing the bit pattern of the value. But the benefit is that those functions would be interoperable with custom types that conform to FixedWidthInteger, even if they don't conform to FullyInhabited. Maybe they could even generalize to BinaryInteger, relying on the per-value BinaryInteger.bitWidth property.

5 Likes