InLineArray issue with big arrays

Using Swift
1

Not an expert here.

this code fails to build in Xcode without specifying any error. I was expecting to have more info on the Type itself. why is that?

@available(macOS 26.0, *)
let a: InlineArray<9_223_372_036_854_775_807_000_000_000_000_000_000_000_000_000_000_000_000, Int> = InlineArray(repeating: 0)
2

That number is larger than Int.max (you’d need a 256 bit architecture to hold an int of that size)

I would guess that the type checker is unable to infer what type your literal is meant to be.

I only have my phone to hand so can only check the error via a web based swift compiler which emits the following error:

exit status 1

Main.swift:5:20: error: expected type let a: InlineArray<9_223_372_036_854_775_807_000_000_000_000_000_000_000_000_000_000_000_000, Int> = InlineArray(repeating: 0) ^

error: fatalError

You can see Int’s error if you use the non-shorthand initialiser:

let tooBigInt = Int(9_223_372_036_854_775_807_000_000_000_000_000_000_000_000_000_000_000_000)

exit status 1
Main.swift:5:21: error: integer literal '9223372036854775807000000000000000000000000000000000000' overflows when stored into 'Int' let tooBigInt = Int(9_223_372_036_854_775_807_000_000_000_000_000_000_000_000_000_000_000_000) ^

If you want to deal with Integers that wide, you may need a special type. e.g.

2 Likes
3

Hmm, you're right, Xcode seems to report failure of compilation, but there's no report in the issue navigator or console.

I'd suggest filing a radar/feedback report however [bitter rant about Apple's attitude to bug reporting self-censored...]

4

I don’t need a number that big. I was messing with the InLineArray and expecting to be yelled at by Xcode. Instead I found -3 is acceptable as count, likelet a: InlineArray<-3, Int> = InlineArray(repeating: 0).

never filed a bug, I don't know where to start anyway

5

You can open an issue here: Issues · swiftlang/swift · GitHub

1 Like
6 
struct InlineArray<let count : Int, Element> where Element : ~Copyable

count is an Int, so can be any value in the range Int.min to Int.max (on an Apple Silicon Mac that's +/- 9223372036854775807)

Obviously it doesn't make sense to have a negative count. However there are reasons why Int is used instead of Unsigned Int.

A consistent use of Int for integer values aids code interoperability, avoids the need to convert between different number types, and matches integer type inference

2 Likes
7

Looks like that’s more of a compromise than a bug. The only thing that remains uncovered is maybe if a more specific error should be raised on build. However, I guess if this is about Swift or Xcode related. And, speaking about real world example, I wonder if computations on a bidimensional matrix of Int.max size are feasible.

8

On the snapshot and nightly builds, this no longer compiles. However, it still does not produce a diagnostic and instead causes a compiler crash: Godbolt

3 Likes
9

And when the number is big (e.g. 1000_000_000) the app crashes at runtime.
All of these issues are not ideal of course, but far from pressing.

10

In a playground, using a negative value for count, I got Swift/UnsafeBufferPointer.swift:74: Fatal error: UnsafeMutableBufferPointer with negative count and that's fair.

Instead, using 1_000_000_000 the playground managed to allocate near 4 gb of memory before crashing with

Playground execution failed:



error: Expression execution was interrupted: EXC_BAD_ACCESS (code=1, address=0xffffffff926d75a0).

The process has been left at the point where it was interrupted, use "thread return -x" to return to the state before expression evaluation.



* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xffffffff926d75a0)

  * frame #0: 0x000000010b526fc0 PlaygroundLogger`playground_logger_initialize

    frame #1: 0x000000010b478794 $__lldb_expr46`main at playground45-380736..cpp:8:1

    frame #2: 0x00000001009c3bf4 com.apple.dt.Xcode.PlaygroundStub-macosx`playgroundExecutionWillFinish + 4

it took a bunch of minutes on an apple silicon, so looks like the whole point it's not very productive. Likely, an InlineArray is going to fill a lot of memory and is killed. I had a similar behavior on Godbolt, and the accepted magnitude was more than what was acceptable on my Mac.

11

@sabino make sure you're putting your large InlineArray on the heap, not the stack (the stack is only a few MB, and you're asking for many GB of storage; that's never going to work). Also make sure that the size of your InlineArray is reasonable given the amount of RAM you have (eg. your billion Ints is already 8GB).

For this much data, there's probably no reason to actually use an InlineArray anyway, though; a regular ContiguousArray will be fine since you need a heap allocation anyway.

That said, I still found issues; I've filed #88549 (InlineArray takes compilation time proportional to number of elements) and #88550 (Large InlineArray crashes) based on this thread.

5 Likes
12

I find the existence of #88549 disturbing. I thought the main selling point of InlineArray over the pre-existing homogeneous tuples was that it didn't have this O(N) time type checking issue.

2 Likes
13

Thanks, I’m going to observe and learn from your bug reports. And of course I’ll dive deeper in the ContiguousArray. This could be a better storage for the Matrix struct I’m studying.

14

The main selling point in my case is that I can build 2 Matrices with Integer Generic Parameters with different count and get 2 different types, and that's huge in my code because I can safely conform Matrix type to AdditiveArithmetic. I was only hoping to use a variable as parameter but that's not allowed. Maybe it's a slippery slope, and it will be added in future...

15

I don't recall "builds fast" ever being brought up. I don't see why InlineArray<4, String> would be any faster to compile than [String] (given the same number of elements). It still has to separately type check each element of the literal, and there have been no specific improvements to literal checking I recall. It seems like there could be (if I check a string literal once, and the collection is all string literals, I shouldn't need to check any more) but I don't think the collection type impacts this at all.

16

It's not actually a type checker issue. I sampled it and the problem is in the LoadableByAddress SIL pass, which must be doing some linear (or worse?) work to determine if an InlineArray<1_000_000_000, _> can fit in a register. (Spoiler: It can't)

7 Likes
17

It was brought up as one of the reasons why it was an improvement over the current c array importing, and why changing the importer from using tuples for c arrays to InlineArray was listed as a future direction.

But, if it's not actually a type checking issue here, then my point was irrelevant anyway.

18

The culprit here looks like it’s the optimizer, not the type checker.

4 Likes
19

You can use integer generics on your own type, and still use ContiguousArray as the storage:

struct Matrix<let columns: Int, let rows: Int, Element: BinaryFloatingPoint> {
    var storage: ContiguousArray<Element>
    init() {
        storage = ContiguousArray(repeating: .zero, count: columns * rows)
    }
}

or similar

20

Yes, that’s exactly what I’m doing. But I can’t do this:

let n = 1

let v: InlineArray<n, Int>…