We're looking into how best to give Swift the equivalent functionality of fixed-size arrays in C or other systems languages. Here are some directions I've been considering, and I'd love to hear the community's ideas and reactions:
Tuples as fixed-size arrays
This was the approach outlined in our most recent pitch on the subject: we already use homogeneous tuples to represent imported fixed-size array fields from C, and library developers sometimes reach for them manually (usually under duress) to reserve fixed-size space within a data type. So we could embrace this hack, and give the language nice sugar for declaring (n * T) instead of (T, T, T, ...), make such tuples conform to collection protocols so they can be indexed, sliced, and transformed with standard collection APIs, and so on. The main benefit of this approach is compatibility, particularly:
- we don't need to break source compatibility with the C importer's existing behavior
- we don't need to worry about back deploying a new kind of type metadata to older OS runtimes
On the other hand, there are drawbacks, including:
- Swift does not normally reserve tail padding for aggregates, including tuples; for example, the size of
(Int32, Int8)would be 5 bytes, even though the stride (the size rounded up to alignment, representing the distance between contiguous values in memory) would still most likely be 8. And by extension, the size of(2 * (Int32, Int8))would be 8+5 = 13 bytes, rather than the full 16 bytes. If this becomes the recommended tool for fixed-size arrays, then generic code will have to be mindful that the final element may not occupy the entire stride of the type. - Swift's calling convention currently always eagerly destructures tuples. A function taking a
(1024 * Int8)argument would therefore take 1,024Int8arguments at the machine code level, which is obviously not great. Thankfully such argument types are currently rare, so we could probably get away with changing the default ABI for very large tuples, but as with any ABI break we'd still need to do work to keep compatibility with the original convention just in case. - Tuples have a wide range of other special case behavior, such as elementwise implicit conversions, labeled/unlabeled conversions, exploding into closure arguments, and so on, much of which is not very scalable. Making it easy to define very large tuples will exacerbate these problems if we don't mitigate them.
A new kind of type
We could introduce a new kind of builtin type to represent fixed-size arrays. Starting from a clean slate, we can keep them detangled from the complexities of tuples. Conversely, if we take this route, we have to consider how a new fixed-size array type interacts with backward compatibility with the existing ecosystem:
- A new kind of type comes with new runtime metadata formats which won't be fully supported by the Swift runtime in already-shipped OSes. Such types would thus either become unavailable when deploying to those OSes, or be available with limited functionality, unless we're able to back-deploy full runtime support for them.
- We would want to migrate the C importer's behavior to use this new kind of type when importing array fields of C aggregates, which would create source breaks. We could manage this by some combination of:
- importing C array fields twice, once under their own
fieldNameas the existing homogeneous tuple representation, and again asfieldNameArrayor something similar as a fixed size array, - conditionalizing the behavior on language version mode, so that Swift 6 code sees the imported field in its array form.
- importing C array fields twice, once under their own
A nominal type using new generics features
With some new type system features, particularly integer generic arguments, we could theoretically express fixed-size arrays as a regular library type:
struct FixedArray<n: Int, T> {
// something to represent the layout of the array
}
Any such addition to the language would more than likely have the same back deployment issues for handling new type metadata as a builtin type would. We would also still need some kind of primitive type, perhaps available only to the standard library, to represent the underlying layout of the FixedArray type, so as far as workload and tradeoffs, this approach seems like a superset of the "new builtin type" approach.
Views over fixed-size storage
Alternatively, instead of directly exposing fixed-size arrays as first class types, we could provide the ability for type declarations to allocate fixed-size space within themselves, with API to expose the fixed-size buffer as an UnsafeBufferPointer (or, using our planned extensions to allow for move-only and nonescaping types, a safe BufferView type). We could have something that behaves similarly to a property wrapper, allocating storage for N elements in the containing type, and presenting the wrapped property API as a BufferView
of the storage:
struct MyFloat80 {
var exponent: Int16 = 0
// Allocate space within MyFloat80 for four contiguous UInt16 values
@FixedArray(count: 4, initialValue: 0)
var significand: BufferView<UInt16>
}
MemoryLayout<Float80>.size // => 10
Float80().significand[0] // => 0
A big benefit of this approach is that it avoids having to expand the type system in backward-incompatible ways—we'd need a mechanism to expand the capabilities of property wrappers so that the FixedArray annotation can allocate the right amount of storage in the containing type, but that's a compile-time transformation, and once that's done, the resulting type representation with fixed storage should fit within our existing runtime type system. This view-over-storage approach could also be a potential design direction for allowing systems programmers to allocate raw memory within classes and move-only structs, for use with atomics, locks, and other low-level primitives that cannot be stored in normal Swift properties, allowing one to write something like:
class LockHolder {
// Store a lock inline in the object without a separate allocation
// or dirty hacks
@RawMemory(of: os_unfair_lock.self, count: 1)
var lock: UnsafePointer<os_unfair_lock>
}
or even, when we have move-only types:
@moveonly
struct UnfairLock {
@RawMemory(of: os_unfair_lock.self, count: 1)
private var lock: UnsafePointer<os_unfair_lock>
init() { lock.initialize(with: OS_UNFAIR_LOCK_INIT) }
}
class LockHolder {
var lock = UnfairLock()
}
On the flip side, implementing this design safely relies on a chain of other new language features: we need move only or nonescaping types, as well as the new BufferView family of move-only view types. We are actively working on these features, but there is nonetheless a risk they may not be completed. A BufferView also loses some possibly useful static information about the underlying fixed-size storage
My thinking
I was originally in favor of the homogeneous tuples approach; despite the shortcomings and legacy issues with tuples, it seemed worthwhile to avoid any question of whether something as fundamental as fixed size arrays could be back deployed. Since then, we've gained some practical experience handling back deployment of expansions to the type system, with varying degrees of functionality:
- Concurrency added a bunch of new features to the runtime type system, including new builtin types and new function type attributes. We were able to successfully back deploy this functionality to older OSes using a dynamic library; however, doing so was a nontrivial amount of engineering work.
- Swift 5.7 adds new forms of existential types like
any Protocol<AssociatedType>, and extends the runtime metadata representation to support more complex constraints. These newly supported cases aren't fully supported when deploying to older OSes; however, they can be used in limited circumstances when we know the new metadata forms will not be necessary at runtime, such as in struct or class fields, or as nongeneric function arguments.
Particularly, our experience with expanding existential types is promising for the prospects of introducing a new kind of fixed size array type; it would be very useful to be able to use fixed-size arrays as fields and function arguments, even if they aren't fully expressive when targeting older OS versions. One wrinkle with taking this approach for fixed-size arrays is that they should be Collections, and get most of their API by virtue of being Collections. Calling an unspecialized generic method on Collection would need the type's metadata, and that would not be possible for fixed-size arrays on older OSes if we take this approach. We could conceivably do some forced specialization when possible, for collection methods in the standard library that are inlinable, and that may get us pretty far.
All that said, I see a lot of appeal in the BufferView approach. Oftentimes, fixed-size arrays aren't directly useful for specifically being an array of their fixed size, but are used as a tool to provide what's really an efficient fixed-capacity collection or buffer for some variable amount of information. That leads me to wonder whether fixed-size arrays are more a means to that end, and that focusing on the more general case of being able to allocate a buffer inside the layout of a type is the better problem to solve, and is an approach that can also potentially address the actual fixed-size array case.
