Right, which is why leaking memory is actually not a particularly compelling example (in that case, the code storing the secret data there should clean it up, rather than hoping that whatever uses that memory next is safe).
UB possibly resulting from compiler-visible undefined loads violating program invariants is much more compelling.
I believe @ellie20 is suggesting that the result of the load would be built up via byte/word loads and arithmetic (which the compiler would be free to optimize), which would support the case you describe.
Could the Duration and EmptyCollection types also conform?
In the "Proposed solution" section, I think the second where clause is incorrect.
really? reading from uninitialized bytes is undefined behavior. safe code shouldnât have to assume other parts of the program exhibit undefined behavior and defend against it, right?
seeing this and Possible unsoundness with RawSpan and MutableRawSpan - #8 by ellie20 makes me concerned for the safety and soundness of these APIs, which are intended to be entirely safe alternatives to using raw pointers.
i really like @ellie20's suggestion of BitwiseLoadable and BitwiseStorable protocols, for fully inhabited types with and without padding respectively. perhaps later proposals could add something like a BitwiseLoadableButWithAValidationStep for types like Range.
ouu! and iâd throw in CollectionOfOne conditionally while weâre at it!
Yes. If you want to avoid leaking secret data, you have to live with the current ground truth that some parts of (almost) any existing system are not safe code. Safe code can avoid doing uninitialized loads, but that doesnât prevent leaks in the presence of unsafe code.
Yes, this would be a workable design.
If there was a guarantee that S initialiser zeroes padding bytes would that be a solution to this type of memory safety issue? Could it be done or is it deeply in Swift DNA to not do this (e.g. for performance reasons)?
You can always do this manually by inserting private var _: Int8 = 0 or whatever where there would be a padding byte, so itâs definitely doable. (The gotcha is that âwhere there would be a padding byteâ is target-specific).
That's looking too "C", "unswifty" and "unsafe by default".. Any change compiler could do this for us, invisibly and automatically?
I am also unsure why Never would conform to FullyInhabited. There are no valid values of Never, so it does not hold that every bit-pattern of its size represents a valid value, because no such bit-pattern does. That is true even though its size is zero.
We also clearly do not want to allow Never to be safely loadable, because then we would have safely derived a valid value of the type, which is impossible because there are no values of the type.
I do think I've convinced myself that there's no sense in which FullyInhabited could be useful for a non-BitwiseCopyable type, though, so I agree with that part of the proposal.
Iâd note that load(as: Never.self) is never-returning and traps. I included it for the same reason as we made Never conform to BitwiseCopyable. Since we are moving towards excluding types with padding bytes, that would exclude Never as well.
Interestingly similar to unsafeBitCast I'll be able using load to make new things even if that's prohibited in the current context:
struct S: FullyInhabited {
let value: Int
fileprivate init(value: Int) {...}
}
// another file:
let s: S = span.load(as: S.self)
It's a bit worrying but probably alright. Slightly more worrying is bypassing the initialiser / or static factory methods to make things bypassing any side-effects it might have. Example:
struct S: FullyInhabited {
static private(set) var count = 0
let value: Int
int(value: Int) {
self.value = value
Self.count += 1 // side effect
}
}
OTOH, this is probably a violation of the rule:
There are no semantic constraints on the values of its stored properties.
so not a good idea of marking this type as FullyInhabited to begin with.
Yeah, a type with invariants should never be FullyInhabited.
Without comment on this pitch per-se, I had to "solve" a similar problem in swift-mmio's API for projecting types from register bitfields. See Creating BitFieldProjectable Types.
Why is this necessary? Couldnât you evolve the type later while staying constrained to only have FullyInhabited properties and no semantic constraints on its stored properties?
Should conforming to the protocol require @unchecked then, like Sendable conformances the compiler canât currently validate?
Is an inline array of a fully-inhabited type also fully inhabited, even if the element type has padding bits?
Perhaps we shouldnât go so far. The goal of the pitch is to allow safe load/use, and FullyInhabited makes that possible.
But there are all sorts of invariants and semantics involved even with integer types. Just a counter can establish before and after semantics. A Ranging<> type could say if the end is before the beginning, the intended range is exclusive.
It seems perfectly valid for a type to both conform to FullyInhabited and to say that if clients get values other than via static initialization or copy of existing values, their program will misbehave. That type might be abusable, but safe loading doesnât make that any worse.
I think itâs sufficient to say that all stored bit patterns must be valid values, and that this pitch guarantees they can be loaded and used safely. Such types can add whatever invariants and semantics they aim to encode, with FullyInhabited as a lower bound, not an upper bound.
Iâve adjusted the definition of FullyInhabited to exclude padding bytes altogether. Alongside adjustments elsewhere in the document, itâs now described as follows in the âproposed solutionâ section:
FullyInhabitedWe propose a new layout constraint, FullyInhabited, to refine BitwiseCopyable. A FullyInhabited type is a safe type with a valid value for every bit pattern that can fit in its stride. This means that a FullyInhabited type's size equals its stride, and has no internal padding bytes.
By conforming to FullyInhabited, a type declares that it has the following characteristics:
It has one or more stored properties.
The types of its stored properties all themselves conform to FullyInhabited.
Its stored properties are stored contiguously in memory, with no padding.
It is frozen if its containing module is resilient.
There are no semantic constraints on the values of its stored properties.
The standard library's FixedWidthInteger and BinaryFloatingPoint types will conform to FullyInhabited.
For example, a type representing two-dimensional Cartesian coordinates, such as struct Point { var x, y: Int } could conform to FullyInhabited. Its stored properties are Int, which is FullyInhabited. There are no semantic constraints between the x and y properties: any combination of Int values can represent a valid Point.
In contrast, Range<Int> could not conform to FullyInhabited, even though on the surface it has the same composition as Point. There is a semantic constraint between two two stored properties of Range: lowerBound must be less than or equal to upperBound. This makes it unable to conform to FullyInhabited.
Other examples of types that cannot conform to FullyInhabited are UnicodeScalar (some bit patterns are invalid,) a hypothetical UTF8-encoded SmallString (the sequencing of the constituent bytes matters for validity,) and UnsafeRawPointer (it is marked with @unsafe.) UnsafeRawPointer is also an example of a type where semantic validity is unknown until runtime, since the runtime environment determines the actual range of valid values.
In the initial release of FullyInhabited, the compiler will not validate conformances to it. Validation of FullyInhabiteds non-semantic requirements will be implemented in a later version of Swift.
IIRC, we avoided the term "layout constraint" in the discussion of BitwiseCopyable because it wasn't a phrase that had really been established outside of some compiler/stdlib parlance. The final proposal for BitwiseCopyable just calls it a "limited protocol" and defines those limitations explicitly (e.g., it cannot be extended, and it cannot be tested using a dynamic cast).
Since FullyInhabited is a refinement of BitwiseCopyable, is it correct to assume that those same limitations apply to the former?
No padding bytes requirement could be too restrictive.
BTW, does that effectively mean that the type must be frozen (and public)? If I understand correctly, a non-frozen type that happens to have no padding today is not guaranteed to remain padding-free in future Swift versions, so won't this break app compilation?