# SE-0229 — SIMD Vectors

(Chris Lattner) #64

There are a few reasons I bring this up, beyond the lack of collection-like capabilities and confusion in the integer domain. Consider the precedent this sets of tensor-like operations:

``````  func f(x : Tensor2D<Float>) {
// what is this a count of?  Is this a reduction of some sort?
let a = x.count
// oh, obviously this is the total element count of the tensor.
let b = x.elementCount
// this is the count of elements in the first dimension.
let c = x.shape[0]
}
``````

Additionally, SIMD Vectors are not necessarily 1D. Among other things, I'm helping to drive a hardware project that uses large architecturally two-dimensional vector registers. Without going into details of that project, consider a less aggressive design where you have something like `Vector4x4<Float16>` (like in Volta tensor cores). It could be surprising for `.count` to return 16 here. Naming this `.elementCount` seems like it would clarify the situation.

-Chris

(Richard Wei) #65

It's not obvious in the tensor example. One could argue that an `element` means a sub-dimensional tensor in `self`. So `elementCount` would equal the outer dimension (`self.shape[0]`).

In the SIMD vector case, I think `count` is a simple, good name because a vector is always 1-dimensional and an "element" is well-defined.

(Steve Canon) #66

I would call that a `Matrix4x4`, not a `Vector`, and I would expect it to conform to a different set of protocols.

(Chris Lattner) #67

The target independent vector types Steve is proposing are all 1D, but target dependent vectors are not necessarily so. We should think about this and be prepared to have protocols that unify them (also with tensors, since there are a lot of common algorithms that can be generic across the bunch).

TPU vectors are not square, and not matrix like (e.g. don't support matmul). They have lane vs sublane affinity issues, and support all the usual element-wise operations. You'd want your permute syntax/semantics to apply to the outer dimension. They fit very nicely with the model as you've described it so far.

In any case, I don't see what justifies `count` as a name here. The rationale for `Collection.count` is clear but doesn't apply here. What is bad about `elementCount` ?

(Richard Wei) #68

Yes, I agree that it's important to keep the commonality in mind when designing these APIs.

However, to make the imaginary protocol generalize tensors, it's even more important to address that fact that "element" can mean a sub-dimensional tensor. `elementCount` would not be a clear name in that case. This is why I chose the name `scalarCount` for `Tensor` because "scalar" is well-defined.

(Chris Lattner) #69

I'm fine with `.scalarCount` or other ideas, I'm just arguing for something more specific and with more clarity than `.count`.

(Thomas Roughton) #70

If we want to go generic, I personally feel that aiming for `Vector<Float, 4>` is far more appealing than `Vector4<Float>`, even though it would be the first instance of that in the language. It feels strange to go to a ‘false’ generic type - false in that it doesn’t really reflect the actual implementation of the type - without going all the way and also genericising the size.

If that’s not something that’s yet achievable, I still feel that `Type.VectorN` with possible user typealiases is the appropriate model. Eventually, when the language supports it, those nested types could be typealiased to the new `Vector<Type, Size>` to keep source compatibility.

(David Sweeris) #71

I apologize if this has already been brought up... I skimmed the proposal and didn't see it mentioned.

I prefer `Vector4<Int>` (and the hypothetical `Vector<Int, 4>` Torust mentioned) over `Vector4.Int` and `Int.Vector4`, but I wonder what that does to people who already have `Vector` types in the mathematical sense as opposed to the SIMD sense? Could/Should we be calling this `SIMD` or `SIMDVector` instead of just `Vector`? As in, `SIMD4<Int>` or `Int.SIMD4`?

And even `Vector<E, N>` would imho be a "false generic type" in the sense that `Vector<E, N>` would exist for only a very specific set of `<E, N>` pairs, ie
`<Float, 16>` would exist but `<Double, 16>` would not,
and there would be no `<E, 5>`, `<E, 6>` or `<E, 7>` even though there would be some `<E, 3>` and so on.

(Karl) #73

Okay, I had some time to go through the proposal in more depth:

• Your comment in the previous thread alludes to possible "machine-width" vectors. I don't see this in the proposal - why not? Is it coming in a follow-up proposal?

• I find the name `SIMDVector` too similar to the concrete `VectorN` types. Have you considered a name like `VectorProtocol` (and `IntegerVectorProtocol`, `FloatingPointVectorProtocol`)?

• I really, desperately wish we could embed protocols inside of (non-generic) types. Then we could create a simple `SIMD` caseless enum to contain the huge number of protocols this proposal would introduce. There are open questions about nesting inside of generic types, but we don't need to tackle those right now. I can't say how much work that would be (@Douglas_Gregor?), but if it were possible for Swift 5.1 (which I just made up), I think it's worth considering delaying until we can do that. This has a huge surface area.

• It would be nice if `SIMDVector.init(_: Array<Element>)` was generic. This would allow us to use custom collections and slices:

``````let vec = Vector4(myCollection.prefix(4))
let vec2 = Vector4(myCollection.suffix(4))
``````
• The proposal includes a way to get numbers from a collection/sequence in to a vector, but it isn't clear to me how I would get my results out of the vector back in to another collection/sequence. Some kind of `Sequence` or `Collection` view of the vector's elements is necessary for that (as Dave mentioned):

``````let vec = Vector4([1, 2, 3, 4])
/* do some processing */
myArray.append(contentsOf: vec.elements) // requires Sequence.
return Array(vec.elements) // requires Sequence
``````
• Have you considered a strongly-typed `Index` for `SIMDVector.subscript(_: Int)`, instead of raw integers and specially-named getters and setters? We could make the Index `ExpressibleByIntegerLiteral` for integer subscripting. This might help avoid runtime failures - since these are fixed-size types, we only need to check bounds when creating an Index, not every time we use one. I'm not sure if the proposed subscript could be `@compilerEvaluable` (when we have that), but bounds-checking for Index creation certainly could be:

``````protocol SIMDVector {
associatedtype Index: ExpressibleByIntegerLiteral // maybe Equatable, Hashable, Comparable, too?
subscript(_: Index) -> Element
}
extension Vector4 {
enum Index { case x, y, z, w }
}
extension Vector4.Index: ExpressibleByIntegerLiteral {
// future: @compilerEvaluable
init(integerLiteral: Int8) {
switch integerLiteral {
case 0: self = .x
case 1: self = .y
// ...etc
default: fatalError("Out of bounds") // future: compiler-evaluated `#assert`
}
}
}
let vec = Vector4([9, 8, 7, 6])
vec[.y] *= -1
vec[3] = 42
``````

Large vectors could simply wrap an integer, with no special names:

``````extension Vector64 {
struct Index: ExpressibleByIntegerLiteral {
private var _value: Int8
// future: @compilerEvaluable
init(integerLiteral: Int8) {
guard integerLiteral >= 0, integerLiteral < 64 else { /* future: compiler-evaluated `#assert` */ }
self._value = integerLiteral
}
}
}

``````
• The Mask's `all()` and `any()` functions do not match the names from `Collection`, and IMO are not clear enough about what they do. Collection calls these predicates `allSatisfy` and `contains`. I wonder if it is possible to bring these APIs closer together. I think it reads better:

``````// okay, the word "satisfy" isn't great here.
func allSatisfy(_ element: Bool) -> Bool {
guard element == true else { return !_any() }
return _all()
}
func contains(_ element: Bool) -> Bool {
guard element == true else { return !_all() }
return _any()
}

guard mask.allSatisfy(true) else { /* ... */ }
if mask.contains(false) { /* ... */ }
``````
• I think the name `SIMDIntegerVector.init(bitMaskFrom: Mask)` is awkward. Can we drop the word `From`?

• I would like to echo these points from other reviewers. The proposal needs more details.

(Steve Canon) #74

What is the `Protocol` suffix adding here? The default would be `Vector`, `IntegerVector`, and `FloatingPointVector`--what ambiguity do we need to resolve by deviating from it?

Sure. `Array<Element>` is the 99.9% case, but this is easy to add.

Sequence conformance is something that we could consider adding in a follow-on proposal.

Subscripting is a necessary escape valve for writing some code, but generally not the bread-and-butter of the SIMD programming model. Re-using an index is especially rare. The primary use is either a literal index (which could be checked at compile time as-is), or iterating through the elements in order. Being able to do arithmetic on indices, however, is a very nice convenience, so we'd end up with at least `ExpressibleByIntegerLiteral & Numeric`, which is basically ... integers. So this feels like a bunch of machinery for not a whole lot of gain.

`any` and `all` are by far the most common operations that you do on SIMD masks. Because they're used all the time, using shorter names makes sense. `allSatisfy` is a much less commonly-used operation on generic `Collections`. `any` and `all` are also the names used for these in every compute language, so there's a huge amount of precedent for them.

Note that people tend to use these directly on comparison results, rather than assigning them to variables first. So your example would become:

``````guard (x .>= 0).allSatisfy(true) else { /* ... */ }
``````

vs.

``````guard all(x .>= 0) else { /* ... */ }
``````

The latter reads far more naturally to me. This is partially a matter of taste, but I doubt that I'm the only who finds this to be the case. I should note that, while we want to conform to the norms of Swift, a huge selling point of the simd module on Apple platforms in C and C++ has been that it hews closely to compute language conventions. This makes it much easier to write similar code for CPU and GPU, and marshal data structures. This proposal backs away from that position somewhat in the name of conforming to the norms of the Swift language, but there's a lot of value in not throwing it away entirely, especially w.r.t. the naming of core free functions like these.

(Lukas Stabe 🙃) #75

I don't really think that's a problem or a "false generic type". We already have types that only accept certain other types as generic parameters. Some only accept types conforming to a protocol, some (although not in the stdlib afaik) only allow a fixed set of types.

Either way, they are parameters that alter a certain aspect of the structure/behavior of the parametrized type. That is certainly the case here, and only allowing those combinations of generic parameters that are allowed/make sense sounds reasonable imo.

I'd even say that writing `Vector<Float, 5>` and getting an error like "Generic parameters <Float, 5> are not allowed on `Vector`. Available combinations: (insert quite a long list here)" is a better experience than "Type Float.Vector5 not found".

(Chris Lattner) #76

Agreed. `Vector<E, N>` implies that any N can work, which is not the case here. In the future, if the type system and compiler got extensive extensions, it is possible that we could support something like this. Until then, we shouldn't mislead people.

Writing `Float.Vector` would provide the user (if in an IDE that supports code completion) with a list of all `.VectorN` that exist on `Float`.

(Karl) #78

What I'm suggesting is to remove the `SIMD` prefix from the protocols and turns it in to the `Protocol` suffix. I think it makes it clearer which is which:

``````extension SIMDVector { ... }
extension Vector4 { ... }
``````
``````extension VectorProtocol { ... }
extension Vector4 { ... }
``````

Hmm, yeah that's a good point. Although I guess you only really need `+` and `-`.

I do actually think the first one reads better as Swift code. It makes it more clear that `.>` returns a mask, not a `Bool`.

I'm not a fan of adding `any` and `all` free functions to the standard library. If we're going to do that, I'd prefer it was a separate module you had to import.

• We actually discussed and rejected these exact names when discussing the `Collection` analogues.
• Why have these be top-level free functions for such esoteric types that most Swift programmers will never use, while the `Collection` versions are instance methods? Especially if it reads so much better?

(David Hart) #79

I still don't see those on the `SIMDIntegerVector` protocol. Is that normal?

(Jordan Rose) #80

I'm staying out of most of the discussions, but the proposal is missing a discussion of how this will affect imported C code. At the moment, the answer is "not at all" (it's not in the implementation PR either), but I think that's really quite important, to the point where I'm not sure I'd want to have this proposal formally accepted without a second proposal that addresses this. We've had paired proposals in the past (Codable's SE-0166 and SE-0167).

(<simd/simd.h> isn't exactly part of the Swift Open Source project, but it's still relevant, and Clang's `ext_vector_type` even more so.)

(Steve Canon) #81

Right; I had been considering that to be a follow-on proposal, so mostly glossed over it here. The types are useful without the corresponding importer changes, but they make them more useful. I can certainly add a section to this proposal to discuss it, however.

(Xiaodi Wu) #82

`x.allSatisfy(true)` reads like `x.allSatisfy { \$0 == true }`, which would in fact suggest that we're looking at a collection of elements of type `Bool`, which `.>` does not return. In any case, I think most can agree that the example reads atrociously from a fluency standpoint.

I think key here is that it's not a perfect analog of `Collection` methods. Moreover, free functions have different naming conventions from members of types ("ed/ing" doesn't apply, for one--there's no `self` to mutate).

Nor does it matter that the type is "esoteric" to some; it's going to be a part of the standard library that's deliberately very small in the first place. And we're using a strongly typed language where users won't be accidentally invoking the free function with a bogus argument.

At base, it sounds like you're arguing that SIMD types shouldn't be a part of the standard library.

(Brent Royal-Gordon) #83

The way in which `any(_:)` and `all(_:)` read better is specific to vectors because they are called with/on the comparison, not the aggregate as `contains(_:)`/`allSatisfy(_:)` are.