SE-0229 — SIMD Vectors

[Chris_Lattner3 (Chris Lattner)]

@scanon

I would push back lightly on that and say that even though they aren't collections, when you are using count , you are using them in ways very similar to how you would use Collection ; it's the established name in Swift for "the number of elements in a thing", and we should stick to that, rather than making up a new term for each domain. If we defined this number on Tuples (I don't see any reason to, but ...) I expect we would call it count as well.

There are a few reasons I bring this up, beyond the lack of collection-like capabilities and confusion in the integer domain. Consider the precedent this sets of tensor-like operations:

  func f(x : Tensor2D<Float>) {
     // what is this a count of?  Is this a reduction of some sort?
     let a = x.count
     // oh, obviously this is the total element count of the tensor.
     let b = x.elementCount
     // this is the count of elements in the first dimension.
     let c = x.shape[0]
  }

Additionally, SIMD Vectors are not necessarily 1D. Among other things, I'm helping to drive a hardware project that uses large architecturally two-dimensional vector registers. Without going into details of that project, consider a less aggressive design where you have something like Vector4x4<Float16> (like in Volta tensor cores). It could be surprising for .count to return 16 here. Naming this .elementCount seems like it would clarify the situation.

-Chris

[rxwei (Richard Wei)]

@Chris_Lattner3

// oh, obviously this is the total element count of the tensor.
let b = x.elementCount

It's not obvious in the tensor example. One could argue that an element means a sub-dimensional tensor in self. So elementCount would equal the outer dimension (self.shape[0]).

In the SIMD vector case, I think count is a simple, good name because a vector is always 1-dimensional and an "element" is well-defined.

[scanon (Steve Canon)]

@Chris_Lattner3

Without going into details of that project, consider a less aggressive design where you have something like Vector4x4<Float16> (like in Volta tensor cores). It could be surprising for .count to return 16 here. Naming this .elementCount seems like it would clarify the situation.

I would call that a Matrix4x4, not a Vector, and I would expect it to conform to a different set of protocols.

[Chris_Lattner3 (Chris Lattner)]

The target independent vector types Steve is proposing are all 1D, but target dependent vectors are not necessarily so. We should think about this and be prepared to have protocols that unify them (also with tensors, since there are a lot of common algorithms that can be generic across the bunch).

TPU vectors are not square, and not matrix like (e.g. don't support matmul). They have lane vs sublane affinity issues, and support all the usual element-wise operations. You'd want your permute syntax/semantics to apply to the outer dimension. They fit very nicely with the model as you've described it so far.

In any case, I don't see what justifies count as a name here. The rationale for Collection.count is clear but doesn't apply here. What is bad about elementCount ?

[rxwei (Richard Wei)]

@Chris_Lattner3

We should think about this and be prepared to have protocols that unify them (also with tensors, since there are a lot of common algorithms that can be generic across the bunch).

Yes, I agree that it's important to keep the commonality in mind when designing these APIs.

However, to make the imaginary protocol generalize tensors, it's even more important to address that fact that "element" can mean a sub-dimensional tensor. elementCount would not be a clear name in that case. This is why I chose the name scalarCount for Tensor because "scalar" is well-defined.

[Chris_Lattner3 (Chris Lattner)]

I'm fine with .scalarCount or other ideas, I'm just arguing for something more specific and with more clarity than .count.

[Torust (Thomas Roughton)]

If we want to go generic, I personally feel that aiming for Vector<Float, 4> is far more appealing than Vector4<Float>, even though it would be the first instance of that in the language. It feels strange to go to a ‘false’ generic type - false in that it doesn’t really reflect the actual implementation of the type - without going all the way and also genericising the size.

If that’s not something that’s yet achievable, I still feel that Type.VectorN with possible user typealiases is the appropriate model. Eventually, when the language supports it, those nested types could be typealiased to the new Vector<Type, Size> to keep source compatibility.

[David_Sweeris]

I apologize if this has already been brought up... I skimmed the proposal and didn't see it mentioned.

I prefer Vector4<Int> (and the hypothetical Vector<Int, 4> Torust mentioned) over Vector4.Int and Int.Vector4, but I wonder what that does to people who already have Vector types in the mathematical sense as opposed to the SIMD sense? Could/Should we be calling this SIMD or SIMDVector instead of just Vector? As in, SIMD4<Int> or Int.SIMD4?

[Jens]

@Torust

If we want to go generic, I personally feel that aiming for Vector<Float, 4> is far more appealing than Vector4<Float> , even though it would be the first instance of that in the language. It feels strange to go to a ‘false’ generic type - false in that it doesn’t really reflect the actual implementation of the type - without going all the way and also genericising the size.

And even Vector<E, N> would imho be a "false generic type" in the sense that Vector<E, N> would exist for only a very specific set of <E, N> pairs, ie
<Float, 16> would exist but <Double, 16> would not,
and there would be no <E, 5>, <E, 6> or <E, 7> even though there would be some <E, 3> and so on.

[Karl]

Okay, I had some time to go through the proposal in more depth:

• Your comment in the previous thread alludes to possible "machine-width" vectors. I don't see this in the proposal - why not? Is it coming in a follow-up proposal?

• I find the name SIMDVector too similar to the concrete VectorN types. Have you considered a name like VectorProtocol (and IntegerVectorProtocol, FloatingPointVectorProtocol)?

• I really, desperately wish we could embed protocols inside of (non-generic) types. Then we could create a simple SIMD caseless enum to contain the huge number of protocols this proposal would introduce. There are open questions about nesting inside of generic types, but we don't need to tackle those right now. I can't say how much work that would be (@Douglas_Gregor?), but if it were possible for Swift 5.1 (which I just made up), I think it's worth considering delaying until we can do that. This has a huge surface area.

• It would be nice if SIMDVector.init(_: Array<Element>) was generic. This would allow us to use custom collections and slices:

  let vec = Vector4(myCollection.prefix(4))
  let vec2 = Vector4(myCollection.suffix(4))
  

• The proposal includes a way to get numbers from a collection/sequence in to a vector, but it isn't clear to me how I would get my results out of the vector back in to another collection/sequence. Some kind of Sequence or Collection view of the vector's elements is necessary for that (as Dave mentioned):

  let vec = Vector4([1, 2, 3, 4])
  /* do some processing */
  myArray.append(contentsOf: vec.elements) // requires Sequence.
  return Array(vec.elements) // requires Sequence
  

• Have you considered a strongly-typed Index for SIMDVector.subscript(_: Int), instead of raw integers and specially-named getters and setters? We could make the Index ExpressibleByIntegerLiteral for integer subscripting. This might help avoid runtime failures - since these are fixed-size types, we only need to check bounds when creating an Index, not every time we use one. I'm not sure if the proposed subscript could be @compilerEvaluable (when we have that), but bounds-checking for Index creation certainly could be:

  protocol SIMDVector {
   associatedtype Index: ExpressibleByIntegerLiteral // maybe Equatable, Hashable, Comparable, too?
   subscript(_: Index) -> Element
  }
  extension Vector4 {
    enum Index { case x, y, z, w }
  }
  extension Vector4.Index: ExpressibleByIntegerLiteral {
    // future: @compilerEvaluable
    init(integerLiteral: Int8) {
      switch integerLiteral {
        case 0: self = .x
        case 1: self = .y
        // ...etc
        default: fatalError("Out of bounds") // future: compiler-evaluated `#assert`
      }
    }
  }
  let vec = Vector4([9, 8, 7, 6])
  vec[.y] *= -1
  vec[3] = 42
  

  Large vectors could simply wrap an integer, with no special names:

  extension Vector64 {
    struct Index: ExpressibleByIntegerLiteral {
      private var _value: Int8
      // future: @compilerEvaluable
      init(integerLiteral: Int8) {
        guard integerLiteral >= 0, integerLiteral < 64 else { /* future: compiler-evaluated `#assert` */ }
        self._value = integerLiteral
      }
    }
  }
  
  

• The Mask's all() and any() functions do not match the names from Collection, and IMO are not clear enough about what they do. Collection calls these predicates allSatisfy and contains. I wonder if it is possible to bring these APIs closer together. I think it reads better:

  // okay, the word "satisfy" isn't great here.
  func allSatisfy(_ element: Bool) -> Bool {
      guard element == true else { return !_any() }
      return _all()
  }
  func contains(_ element: Bool) -> Bool {
      guard element == true else { return !_all() }
      return _any()
  }
  
  guard mask.allSatisfy(true) else { /* ... */ }
  if mask.contains(false) { /* ... */ }
  

• I think the name SIMDIntegerVector.init(bitMaskFrom: Mask) is awkward. Can we drop the word From?

• I would like to echo these points from other reviewers. The proposal needs more details.

  @Chris_Lattner3

  "There is also a set of protocols defining the vector sizes:" - if these are going to be public, then please provide more details in the proposal.

  @hisekaldma

  However, I’m missing a more detailed description of all the types proposed. The proposal only lists the concrete types briefly and describes the most important protocols, saying that the rest is self-explanatory. That makes it hard to understand exactly what is proposed. You either have to read through the PR (which is heavily gybbed), or build a toolchain yourself to try it out.

[scanon (Steve Canon)]

@Karl

I find the name SIMDVector too similar to the concrete VectorN types. Have you considered a name like VectorProtocol (and IntegerVectorProtocol , FloatingPointVectorProtocol )?

What is the Protocol suffix adding here? The default would be Vector, IntegerVector, and FloatingPointVector--what ambiguity do we need to resolve by deviating from it?

It would be nice if SIMDVector.init(_: Array<Element>) was generic. This would allow us to use custom collections and slices:

Sure. Array<Element> is the 99.9% case, but this is easy to add.

it isn't clear to me how I would get my results out of the vector back in to another collection/sequence.

Sequence conformance is something that we could consider adding in a follow-on proposal.

we only need to check bounds when creating an Index, not every time we use one.

Subscripting is a necessary escape valve for writing some code, but generally not the bread-and-butter of the SIMD programming model. Re-using an index is especially rare. The primary use is either a literal index (which could be checked at compile time as-is), or iterating through the elements in order. Being able to do arithmetic on indices, however, is a very nice convenience, so we'd end up with at least ExpressibleByIntegerLiteral & Numeric, which is basically ... integers. So this feels like a bunch of machinery for not a whole lot of gain.

any and all are by far the most common operations that you do on SIMD masks. Because they're used all the time, using shorter names makes sense. allSatisfy is a much less commonly-used operation on generic Collections. any and all are also the names used for these in every compute language, so there's a huge amount of precedent for them.

Note that people tend to use these directly on comparison results, rather than assigning them to variables first. So your example would become:

guard (x .>= 0).allSatisfy(true) else { /* ... */ }

vs.

guard all(x .>= 0) else { /* ... */ }

The latter reads far more naturally to me. This is partially a matter of taste, but I doubt that I'm the only who finds this to be the case. I should note that, while we want to conform to the norms of Swift, a huge selling point of the simd module on Apple platforms in C and C++ has been that it hews closely to compute language conventions. This makes it much easier to write similar code for CPU and GPU, and marshal data structures. This proposal backs away from that position somewhat in the name of conforming to the norms of the Swift language, but there's a lot of value in not throwing it away entirely, especially w.r.t. the naming of core free functions like these.

[ahti (Lukas Stabe 🙃)]

@Jens

And even Vector<E, N> would imho be a "false generic type" in the sense that Vector<E, N> would exist for only a very specific set of <E, N> pairs, ie
<Float, 16> would exist but <Double, 16> would not,
and there would be no <E, 5> , <E, 6> or <E, 7> even though there would be some <E, 3> and so on.

I don't really think that's a problem or a "false generic type". We already have types that only accept certain other types as generic parameters. Some only accept types conforming to a protocol, some (although not in the stdlib afaik) only allow a fixed set of types.

Either way, they are parameters that alter a certain aspect of the structure/behavior of the parametrized type. That is certainly the case here, and only allowing those combinations of generic parameters that are allowed/make sense sounds reasonable imo.

I'd even say that writing Vector<Float, 5> and getting an error like "Generic parameters <Float, 5> are not allowed on Vector. Available combinations: (insert quite a long list here)" is a better experience than "Type Float.Vector5 not found".

[Chris_Lattner3 (Chris Lattner)]

@Jens

If we want to go generic, I personally feel that aiming for Vector<Float, 4> is far more appealing than Vector4<Float> , even though it would be the first instance of that in the language. It feels strange to go to a ‘false’ generic type - false in that it doesn’t really reflect the actual implementation of the type - without going all the way and also genericising the size.

And even Vector<E, N> would imho be a "false generic type" in the sense that Vector<E, N> would exist for only a very specific set of <E, N> pairs, ie
<Float, 16> would exist but <Double, 16> would not,
and there would be no <E, 5> , <E, 6> or <E, 7> even though there would be some <E, 3> and so on.

Agreed. Vector<E, N> implies that any N can work, which is not the case here. In the future, if the type system and compiler got extensive extensions, it is possible that we could support something like this. Until then, we shouldn't mislead people.

[Jens]

@ahti

I'd even say that writing Vector<Float, 5> and getting an error like "Generic parameters <Float, 5> are not allowed on Vector . Available combinations: (insert quite a long list here)" is a better experience than "Type Float.Vector5 not found" .

Writing Float.Vector would provide the user (if in an IDE that supports code completion) with a list of all .VectorN that exist on Float.

[Karl]

@scanon

What is the Protocol suffix adding here? The default would be Vector , IntegerVector , and FloatingPointVector --what ambiguity do we need to resolve by deviating from it?

What I'm suggesting is to remove the SIMD prefix from the protocols and turns it in to the Protocol suffix. I think it makes it clearer which is which:

extension SIMDVector { ... }
extension Vector4 { ... }

extension VectorProtocol { ... }
extension Vector4 { ... }

@scanon

Being able to do arithmetic on indices, however, is a very nice convenience

Hmm, yeah that's a good point. Although I guess you only really need + and -.

@scanon

This proposal backs away from that position somewhat in the name of conforming to the norms of the Swift language, but there's a lot of value in not throwing it away entirely, especially w.r.t. the naming of core free functions like these.

I do actually think the first one reads better as Swift code. It makes it more clear that .> returns a mask, not a Bool.

I'm not a fan of adding any and all free functions to the standard library. If we're going to do that, I'd prefer it was a separate module you had to import.

• We actually discussed and rejected these exact names when discussing the Collection analogues.
• Why have these be top-level free functions for such esoteric types that most Swift programmers will never use, while the Collection versions are instance methods? Especially if it reads so much better?

[hartbit (David Hart)]

@scanon

TL;DR: adopts .< , .<= , .> , .>= , drops .* and .&* , reverts too & and | instead of && and || for masks.

I still don't see those on the SIMDIntegerVector protocol. Is that normal?

[jrose (Jordan Rose)]

I'm staying out of most of the discussions, but the proposal is missing a discussion of how this will affect imported C code. At the moment, the answer is "not at all" (it's not in the implementation PR either), but I think that's really quite important, to the point where I'm not sure I'd want to have this proposal formally accepted without a second proposal that addresses this. We've had paired proposals in the past (Codable's SE-0166 and SE-0167).

(<simd/simd.h> isn't exactly part of the Swift Open Source project, but it's still relevant, and Clang's ext_vector_type even more so.)

[scanon (Steve Canon)]

Right; I had been considering that to be a follow-on proposal, so mostly glossed over it here. The types are useful without the corresponding importer changes, but they make them more useful. I can certainly add a section to this proposal to discuss it, however.

[xwu (Xiaodi Wu)]

@Karl

I do actually think the first one reads better as Swift code. It makes it more clear that .> returns a mask, not a Bool .

x.allSatisfy(true) reads like x.allSatisfy { $0 == true }, which would in fact suggest that we're looking at a collection of elements of type Bool, which .> does not return. In any case, I think most can agree that the example reads atrociously from a fluency standpoint.

@Karl

I'm not a fan of adding any and all free functions to the standard library. If we're going to do that, I'd prefer it was a separate module you had to import.

• We actually discussed and rejected these exact names when discussing the Collection analogues.
• Why have these be top-level free functions for such esoteric types that most Swift programmers will never use, while the Collection versions are instance methods? Especially if it reads so much better?

I think key here is that it's not a perfect analog of Collection methods. Moreover, free functions have different naming conventions from members of types ("ed/ing" doesn't apply, for one--there's no self to mutate).

Nor does it matter that the type is "esoteric" to some; it's going to be a part of the standard library that's deliberately very small in the first place. And we're using a strongly typed language where users won't be accidentally invoking the free function with a bogus argument.

At base, it sounds like you're arguing that SIMD types shouldn't be a part of the standard library.

[beccadax (Becca Royal-Gordon)]

@Karl

Why have these be top-level free functions for such esoteric types that most Swift programmers will never use, while the Collection versions are instance methods? Especially if it reads so much better?

The way in which any(_:) and all(_:) read better is specific to vectors because they are called with/on the comparison, not the aggregate as contains(_:)/allSatisfy(_:) are.