Hello Evolution,
A year ago, we added the InlineArray type to the standard library, but we didn't adopt it for imported C arrays because there were some technical issues we needed to work through. I now have a working implementation of the feature, so here's a matching pitch.
You can read the (not quite complete) draft proposal here: NNNN-modern-c-arrays.md · GitHub
For code that might build under both regimes, is there a recommended way to convert either projection to e.g. an UnsafeBufferPointer or Span?
Are InlineArray and homogenous Tuples of the same length ABI compatible? (i assume they are, just curious)
Was there some exploration of using a typealias-based approach + magic to mitigate the source-breaking effects?
I'm thinking of how we have CChar, CInt, etc.: can we have CArray<CInt, 3> that is an alias for (CInt, CInt CInt) for earlier platforms and [3 of CInt] for modern ones? It would be nice to be able to declare in Swift that a function takes an argument of "whatever type a C array is imported as."
I'm also thinking of how we have some magic to allow arrays to be magically passed as pointers at the interop boundary: could we have similar magic that permits passing either [3 of CInt] or (CInt, CInt, CInt) as arguments at the interop boundary?
The answer to both of these may well be that it's feasible but way too costly, but as there's precedent I wonder if there's some mileage to be had here.
(For context, I've been tinkering with the Foundation.Decimal type, which is declared in C for Darwin framework builds but declared in Swift elsewhere, with the mantissa currently of type (UInt16, UInt16, UInt16...) either via bridging or by explicit declaration, respectively. It's already irritating to code against two distinct declarations, but if the C version then comes into Swift as one of two distinct types depending on flags or compiler version without additional affordances, I worry it's going to get even messier....)
They store elements at the same offsets, have the same stride, and usually have the same size. However, if the element type has a size smaller than its stride (i.e. it's a size that doesn't align well), InlineArray’s size will include padding at the end of the last element, while a tuple will not. That doesn’t matter for this proposal because C arrays are laid out like InlineArray and are thus compatible with both, but it does mean that, for instance, you cannot always use unsafeBitCast(_:to:) to convert between a tuple and InlineArray.
InlineArray has a span property, and from there you can go through withUnsafeBufferPointer to get the buffer’s baseAddress. Tuples are harder—I don’t think there’s a way to do it without changing the pointer’s Pointee—but that’s no worse than what you already have to do.
There are a bunch of affordances I’d like to build but can’t because there’s no mechanism in the generic system that can convert a count and Element to a count-tuple of Elements.
I’ve thought about doing this, but I know @Slava_Pestov at least doesn’t want to add more unsafe implicit conversions, and since tuples don’t have this conversion it isn’t a regression.
We almost have the features necessary to provide a function to get a span over a tuple in the cases where it is safe to do so; I think the only thing missing is same-element constraints on packs:
@_lifetime(borrow tuple)
func span<T: ConvertibleToBytes, each Elt>(over tuple: borrowing @_addressable (repeat each Elt)) -> Span<T>
where each Elt == T
The ConvertibleToBytes constraint would ensure that T has no padding bytes to worry about (though it might be a bit over restrictive since it's tail padding in particular we're worried about here), and the @_addressable flag and borrow tuple dependency on the parameter would ensure that a stable address for the the tuple is passed down for the span to reference. The implementation (left as an exercise for the reader) would take the address of the tuple parameter and form a Span with a base address and count corresponding to the stride of the tuple divided by the stride of the element type.
A run-time check that MemoryLayout<T>.size == MemoryLayout<T>.stride would probably be sufficient. It'd be nice to check that at compile time, but I'm not sure it's worth blocking having a standard implementation for it. …that's probably a separate Pitch though!
Sorry, I meant a single expression that might compile against either. I can think of a few ways to spell it involving pointer shenanigans, of course.
Making it easy to write code that compiles in both modes isn’t really a goal of the design. Instead, I’ve put effort into ensuring that modules can change their setting independently of their dependencies.
(I should probably articulate that in the proposal.)
This is great to see! Soooo much nicer than tuples.
One small nit: ModernImportedCArrays/ModernImportedCArraysOnly may eventually become misleading names, if we someday make imported C arrays even more modern. Perhaps ImportCArraysAsInlineArrays/ImportCArraysAsInlineArraysOnly to make it more specific?
Yeah, totally get that. And, yes, it's true that it's not a regression for tuples per se. But it's still a speed bump for an end user who can pass (42, 42) to a function today but not in some future language version.
You mentioned importing projections under different names as a rejected alternative because it would lead to an inferior shape for the API. I wonder if it's worth considering overloading the same name with disfavored tuple-using functions, forgoing implicit conversions but aiming at the same source compatibility goal:
// Yes, I checked that this works.
// It's gross to write, but that's not really so terrible
// when generated under the hood in service of a nice API.
struct Float2x2: __Float2x2 {
@_disfavoredOverload
var elements: ((Float, Float), (Float, Float))
@_disfavoredOverload
init(elements: ((Float, Float), (Float, Float))) {
self.elements = elements
}
}
protocol __Float2x2 {
var elements: [2 of [2 of Float]] { get set }
init(elements: [2 of [2 of Float]])
}
@available(anyAppleOS 26, *)
extension __Float2x2 where Self == Float2x2 {
var elements: [2 of [2 of Float]] {
get {
unsafeBitCast(
elements as ((Float, Float), (Float, Float)),
to: [2 of [2 of Float]].self)
}
set {
self.elements = unsafeBitCast(
newValue, to: ((Float, Float), (Float, Float)).self)
}
}
init(elements: [2 of [2 of Float]]) {
self = .init(elements: unsafeBitCast(
elements, to: ((Float, Float), (Float, Float)).self))
}
}
var x: Float2x2 = .init(elements: [[42, 42], [42, 42]])
x.elements = [[1, 2], [3, 4]]
print(x.elements) // Preferred overload is of type `InlineArray`.
var y: Float2x2 = .init(elements: ((42, 42), (42, 42)))
y.elements = ((1, 2), (3, 4))
print(y.elements as ((Float, Float), (Float, Float)))
Great to see the limit of 4096 elements to be lifted to arbitrary sized C arrays!
However, while InlineArray doesn't have the same compile time problems, it does today have runtime performance problems. Using the subscript to get an element out of an InlineArray does copy the whole array first. Do that for each element and you easily end up with quadratic time complexity that should have been a linear algorithm. I don't think trading compile time performance for quadric runtime performance is the right call here. Unless we can make the InlineArray subscript O(1) and otherwise linear loops over it O(n) I don't think we should adopt InlineArray for C arrays just yet.
Otherwise I think this will be a great direct and makes interacting with e.g. types that are shared with a Metal shader a lot more convenient once we have acceptable runtime performance that allows it be used in practice.
FWIW, the status quo today of using withUnsafePointer(to:) to dynamically subscript a homogeneous tuple isn't necessarily any better in that regard.
Is this issue of subscripting InlineArray just a bug fix, or are there significant ABI issues or missing language features blocking improvements here?
I believe this is fixable with the judicious use of different subscript accessors (though I don't know which ones). All else fails, I wouldn't want to block this ergonomic improvement on what amounts to a compiler bug.
We should treat unnecessary copies of InlineArray values as bugs to be fixed, and not make design decisions based on that behavior, IMO. Until we fix those bugs, you can often avoid unnecessary copies of the entire array by indexing the span of the InlineArray rather than the array directly.
I notice that the memcpy call goes away if I edit the code example to make the array inout. That makes me think this is probably just an optimizer bug, not a fundamental flaw in InlineArray’s design.
I've been experimenting a little and I think we could provide these functions for conversions:
/// Returns the elements of `tuple` as an `InlineArray` of the same size and
/// element type.
///
/// This function works only for tuples of size `count` where every element is
/// of type `ArrayElement`. The compiler cannot check that you are following
/// these rules, but they are enforced by preconditions at runtime.
///
/// - Parameters:
/// - tuple: A tuple of `count` elements, each of type `ArrayElement`.
/// - arrayType: The type to cast `tuple` to. This is always an `InlineArray`
/// of the same count and element type as `tuple`.
public func tupleCast<let count: Int, ArrayElement, each TupleElement>(
_ tuple: (repeat each TupleElement),
to arrayType: [count of ArrayElement].Type
) -> [count of ArrayElement]
/// Returns the elements of `array` as a tuple of the same size and element
/// type.
///
/// This function works only for tuple types of size `count` where every element
/// is of type `ArrayElement`. The compiler cannot check that you are following
/// these rules, but they are enforced by preconditions at runtime.
///
/// - Parameters:
/// - array: An array of `count` elements, each of type `ArrayElement`.
/// - tupleType: The type to cast `array` to. This is always tuple of
/// the same count and element type as `array`.
public func tupleCast<let count: Int, ArrayElement, each TupleElement>(
_ array: [count of ArrayElement],
to tupleType: (repeat each TupleElement).Type
) -> (repeat each TupleElement)
They generate atrocious code at -Onone thanks to the use of parameter packs, but as long as they're inlined, -O cleans it all up pretty well.
Just a thought, how about a new language feature for this? In this sample, merge generates a static tuple type by iterating over the indexes given in the loop and putting together the elements. The merge loop in the body of the convertToTuple()function does the same thing at the level of terms, here with the elements of the inline array. The reverse direction initialises the array and expects that the input tuple matches the required form given by the typealias and assembles the InlineArray by going through each element with a new subscript for tuples.
extension InlineArray {
typealias TupleForm = merge for _ in 0..<count { Element }
// equivalent to (Element,...) count times
func convertToTuple() -> TupleForm {
merge for i in 0..<count { self[i] }
}
init(from tuple: TupleForm){
self.init { index in
tuple[index]
}
}
}