Another fixed-sized scoped-storage array manifesto spitball session

Evolution Discussion
,
1

I made (yet another) fixed-sized array paper, now more like a manifesto, on my Gist, at revision 2. I already hinted at this in another post on the design.

On some discussion about constepxr, someone mentioned that the MemoryLayout constants can't be used in a potential constexpr context; they are set around layout time, between the late-SIL and IR-Gen stages, while constexpr values need to be evaluated during the AST and early-SIL stages. So I thought: how could the design of FSA change if we threw out the constexpr requirement for the extents? The values, for the most part, still need to be known by IR-Gen time for layout.

Since we don't have constexpr right now anyway, not much changes. But it did give me a chance to put in other ideas I never put here before. To summarize:

// One dimensional arrays tweak the Standard array syntax to a reverse of Rust's FSA syntax
let myArray1: [5; Int]

// Unlike the rest of the C family, multiple dimensions can be defined inline
let myArray2: [2, 2; Int]
// A pack of extents can be a single one-dimensional Int array, for meta-programming purposes.  This has the exact same type as above.
let myArray2a: [[2, 2]; Int]

// Zero-dimensional arrays are supported, because why not?
let myArray3: [; Int]

But there are two new kinds of extents

// A private extent means we don't have to repeat ourselves.
// Here, we don't have to update a number every time the number of terms changes.
let myArray4: [private; Int] = [1, 2, 3, 4, 5]

// The power to imply a private extent even works for multi-dimensional arrays
// (The elements are filled in row-major storage order.)
let myArray5: [2, private; Int] = [1, 2, 3, 4, 5, 7]

// Unlike a private extent, an open extent can vary along initialization paths (but not after initialization).
let myArray6: [open; Int]
if myFlag {
    // If a private or open extent is used, the variable must have its shape fully defined before initializing its elements.
    // (Not needed if the FSA had an initializer in its declaration.)
    reify myArray6 as [2; Int]
    // Besides the traditional subscript, tuple-number syntax can be used for dereference.
    myArray6.0 = 5
    myArray6.1 = 7
} else {
    reify myArray6 as [100; Int]
    // The for-inout loop allows definitive initialization of FSA elements
    for x inout myArray6 {
        // The #indexOf primary expression gets the current element's index.
        let i = #indexOf(x)
        x = Int.random(0..<100) + i[0]
    }
}

Besides globals and type-level properties, FSAs with open extents are limited to code blocks, instance-level properties of classes, and function arguments.

There are several existential, protocol-like forms to use as generic constraints:

// Takes any FSA type
func myFunc1<T: [...; some Any]>(_ a: T)

// Returns any FSA type that doesn't have an open extent, including recursive checking if the element type itself is a FSA.
func myFunc2<T: locked [...; some Any]>() -> T

// Takes a one-dimensional array of a one-dimensional numeric array
func myFunc3<T: [_; some [_; some Numeric]]>(_ a: T)

// Takes an at-least two dimensional array, with a fixed second extent
func myFunc4<T: [_, 5, ...; some Any]>(_ a: T)

Although all the examples were existential on both the extent side and element side, existential type that are such on only one side are allowed. I added the locked specifier because without it, an array's extents and element-count can only be found out at run-time per instance. With the specifier, we get a subtype that we can attach type-level extent and size properties to. When constexpr support is added, FSA extent and count values should be constexpr whenever possible, to help with further meta-programming.

The reason for posting about the manifesto is to get comments on how to make it complete, and to find out any errors in the prose or interface or design. Since this is a new primitive type, the implementation will be at least as hard as the language-level changes being done now to support Apple's SwiftUI. This may be beyond me to implement, so I want to have as much as a complete & feasible a plan as possible before discussing who's actually going to do it.

Example helpful comments (assuming my current plan isn't perfect :upside_down_face:):

  • Here's what to read up on to figure out a design for part X.
  • Want to have a design for part X? There's paths A, B, and C, each with trade-offs; which one do you think we should prefer?
  • Want advice on part X?! You're completely ignorant about part W, which you need to do before even considering a design for X.

Even if I'm completely wrong about back-end details, I hope at least the design for the user-facing end is OK.

Some of my older related posts (using the forum's search function on my own ID):

2 Likes