SE-0229 — SIMD Vectors

Evolution Proposal Reviews
24

Cool. My observation is that there are two very different design choices here, with different tradeoffs (and I don't have an informed opinion about which is best). I mostly care about the concrete type situation, not generic code. Consider:

func f(a: Vector4<Float>, b: Vector4<Float>) {
  let cond = a .== b
}

There are at least three choices on the type of "cond". It could either be:

  1. A type like Vector4<Int32>, which would be target independent. This constrains the implementation for certain hardware, and you have to decide what the contents of the integers are (this is similar to the OpenCL design)

  2. A type like Vector4<Int32>, which would be target dependent. This allows hardware to use predicate registers or whatever else they want, but means that people will probably accidentally write non-portable code, because they'd end up depending on the details of whatever target they are building for. E.g. if you refactored the code above into: let cond: Vector4<Int32> ; cond = a .== b then it would build on your system, but fail on others.

  3. A type like VectorMask4 or VectorMask4<Float> that is target independent but has a target specific size and that models a vector of bools. This allows providing a vector of bools semantics, but allow targets to hold those booleans in the natural form returned by their vector comparison operators. To make this work well, the interface would have to be narrow (similar to that of Bool): standard integer operations would not be available. To do integer operations, you'd be forced to get a mask, then convert it to (e.g.) Vector4<Int32> using a labeled init method that indicates how you want the logical booleans projected onto the integers (e.g. zero or sign extending). The disadvantage of this is that it makes bitbanging on condition results a bit more awkward.

I don't have a strong sense for what is right here, but #3 is appealing to me because we could get target-independent vector code much more reliably than the other two options.

Have you thought at all about this issue?

-Chris

3 Likes
25

So the actual type in your concrete example would be SIMD.Mask32.Vector4. This type is potentially target dependent, though on all targets with current first-class Swift support, it just wraps Int32.Vector4.

The interface is narrow, and standard integer operations are not available.To do integer operations, you are forced to convert to Int32.Vector4 via init(bitMaskFrom:) (name possibly suboptimal). You get to skip this step for the &, |, ^ operations because they're part of the narrow API, but for more general operations you need to convert, just like you can't add two times a Bool to an Int in Swift.

So basically, #3, except that platforms can use different size mask types for different element sizes, which is a little bit noisy but it's very hard to get the codegen you really want otherwise. In my ideal world we would model these things as Builtin.VecNxInt1 and have a single mask type for each vector length, but that requires a quantity of hacking on LLVM guts that's probably outside the scope of a Swift stdlib change. There are, of course, conversions between vectors of the same length of all Mask types, so this works out OK in practice; changing lanes widths on masks turns out to be a relatively niche operation, so it's not too painful to require an explicit conversion.

On a platform with packed predication masks, you would still have separate concrete types for each element size, they would just be backed by the same underlying builtin, and the implementation of conversions between them would be NOPs.

1 Like
26

Hi all --

I have updated the proposal with some minor uncontroversial changes that have been discussed here. Here's the changelog and the diff.

TL;DR: adopts .<, .<=, .>, .>=, drops .* and .&*, reverts too & and | instead of && and || for masks.

5 Likes
27

Couldn't / shouldn't this be mentioned briefly in a "Future Directions" section in the proposal?

28

As a general review of the proposal, having followed the pitch thread, I very much like the overall design and will be very happy to have these types in the standard library.

More specifically, I mostly agree with the current set of operator overloads – having no prefix for most (T, T) -> T operators but using a dot prefix for anything that interacts with Bool makes sense. I’d personally prefer using .& and .| rather than & and | as the mask logic operators for consistency.

Given that direction, I’d also suggest considering an element-wise select operator (Mask) .? Vector : Vector, where the fact that it’s doesn’t short-circuit is somewhat implied by the dot prefix.

2 Likes
29

+1

yes

I agree with others that replacing(with:where:) is a bad name. The label where: might read more smoothly but we also risk un-teaching a standard library convention (“where: means predicate function”) we’ve worked very hard to teach users and it just doesn’t seem like a win to me. I’d rather just keep it simple with replacing(with:mask:).

Regarding the logic operators, .& and .| are a must for me. Using the existing &, |, or &&, || just has too many drawbacks for me to be okay with overloading them for masks. It’d also be nice for consistency if ‘everything’ related to vector logic followed the dot convention, because it’s just so fundamentally different from scalar logic.

it’s pretty good

Followed it since the beginning. It seems to me actually a little under-ambitious, as i’d really like to see more vector library support, i.e. prod(_:_:), dot(_:_:), cross(_:_:), etc, but it’s always better to be under-ambitious than over-ambitious.

Not sure what happened to the highHalf, lowHalf, etc properties, but i would really like to see Vector4.xyz and Vector3.xy swizzles or something equivalent as it’s a pretty common thing to want to de-homogenize a vector.

2 Likes
30

This name is more ambiguous to me, because it's not immediately clear how the mask is being used (i.e. I can think of plausible interpretations where it means the opposite operation). I understood the “where” version immediately.

1 Like
31

This is great, thanks Steve! Is this described in the proposal though? One of my concerns (and which has been raised by others) is that the proposal writeup is not very complete - there are a lot of aspects of the API design that are not described.

32

Awesome, this addresses most of my concerns, thank you!

The only remaining suggestions that I have are:

  • Consider changing user-exposed syntax from Int32.Vector4 -> Vector4<Int32> to align better with other container-y types (conceptual consistency matters even though vectors don't conform to collection).

  • Improve the proposal to be more self contained and complete, describing the core operations and behaviors of these types. The description of Vector2 and friends needs to be expanded substantially.

One additional clarifying question: if I understand your proposal correctly, you are defining &+ but not + on vectors (for lots of good reasons!). Is this correct? An alternate approach would be to define + with 2's complement wrapping semantics. This would be inconsistent with the rest of swift but would reduce syntactic noise in integer vector code. On average I agree with your existing design (as I understand it) but it might be worth mentioning this explicitly, because I missed it on the first pass and many other people probably did too. Will this meaningfully affect generic code that wants to use a + operator on both integer and float vectors?

Thanks again for driving this forward, it is great work!

-Chris

2 Likes
33

I'm still quite on the fence about this. I get the attraction, but it basically requires a layer of indirection "magic" on everything, which makes it harder for users to understand what's actually going on. That's not the end of the world, because users don't normally have to worry about how it works, but it's not particularly elegant. Also it takes up a bunch of names in the top-level namespace.

The inconsistency would be pretty jarring, and this would make it more complex to move code between vector and scalar implementations; it's nice to be able to easily "scalarize" vector code for debugging purposes. Also, especially when we start talking about wider vector types like Int8.Vector64, we can absolutely efficiently-vectorize checked arithmetic (it'll be slower than wrapping, but much faster than scalar). So I'd like to keep the "normal" operators available for that purpose.

4 Likes
34

what about

replacing(with:selector:)

35

I think selector is a rather unfortunate choice for a language tightly coupled with Objective-C...
May using something like "merge" or "combine" could help to avoid confusion?

2 Likes
36

We might also consider flipping the parameters around, and that could add some new opportunities to consider. Something like replacing(lanes: Mask, with: Self)

2 Likes
37

What about replacingElements(_ mask: Mask, with: Self)? That would solve both this issue, and the issue @rxwei pointed out in the pitch thread that the method is missing an object:

let v1: Int8.Vector3 = [1, 2, 3]
let v2: Int8.Vector3 = [4, 5, 6]
v1.replacingElements(v1 .< 3, with: v2)

What are we replacing? The elements that are less than 3. What are we replacing them with? The corresponding elements in v2.

5 Likes
38

+1

Yes. It's the first important step to standardize vector libraries in Swift.

Generally yes, but I have the following concerns:

  • It seems more fitting for SIMD to be its own module.
  • replacing(with:where:) does not have an explicit object, and its object can be "self" or "elements", which would result into different meanings unlike sort. Moreover, where: suggests a trailing closure while the argument type is a mask vector. I suggest the following alternatives:
    • replacingElements(with:selectedBy:)
    • replacing(with:selectedBy:). In this alternative, dropping "Element" seems to be okay because "selectedBy" suggests only elements can be selected.
    • I like the word "select" because
      • This API represents what "select" means in most vector libraries. It's familiar.
      • The word "select" naturally maps onto common sense "1 = select, 0 = not select". Neither "where" or "mask" maps onto this common sense easily.
  • I prefer generic types such as Vector4<Int8> for the same reasons listed by @Chris_Lattner3.
    • I think mapping combinations of vector types and generic parameters onto certain LLVM intrinsics is an implementation detail, which shouldn't interfere with API design.
    • Separating low-level SIMD from general vector API is a good argument for Int8.Vector4, but I think it's better to just put all VectorN<T> types into a separate SIMD module.
  • Existing free functions in stdlib are general enough to be top-level, but all and any are very much domain-specific and will likely be confusing in code completion. This is another argument for SIMD to be its own module.
  • Would it make sense to add named methods for each arithmetic function, and have each operator call that method?
    • This will increase discoverability of element-wise operators. People that are not familiar with .== will try to look up code completion.
    • This will work well in the future with improved operator lookup, I think.

I've used Accelerate, and many high-level numerics / ML libraries such as NumPy, TensorFlow and PyTorch. Compared to those ML libraries, the proposed SIMD API is more low-level. There's enough precedent to justify the choice of element-wise operators.

A quick read over APIs. Participated in the pitch thread. Some of my concerns in the original pitch thread have been addressed. Once we make decisions about operators based on this proposal, I'd like to apply necessary changes to the TensorFlow Swift library.

7 Likes
39

I will admit up front that I have little experience with SIMD-heavy code and I'm really just coming at this as someone with a sense for Swift aesthetics. There may well be compelling reasons why some of my suggestions are wrong.

  • What is your evaluation of the proposal?

I think this is a good addition to the language, and the proposal does a good job identifying the scope and solving many thorny questions. However, I have many quibbles with the specific design selected.

  • Is the problem being addressed significant enough to warrant a change to Swift?

Yes. We want some good solutions to these problems, and SIMD instructions are so low-level that they really need to be at least partially in the standard library.

  • Does this proposal fit well with the feel and direction of Swift?

This is where I disagree with the proposal. Except at the broadest, most conceptual levels, I think it doesn't really match the feel and direction of Swift.

Mixed metaphors

Vectors are in some ways collection-like—they have several elements of identical type, they are subscriptable, they can be initialized from an array literal, etc. And yet they don't conform to Collection, apparently out of concern that users may accidentally write un-vectorized algorithms with that conformance.

Vectors are in other ways tuple-like—they have a size fixed at compile time, the elements don't (typically) have the same semantic meaning, and they aren't supposed to be processed by loops. And yet their elements are accessed through a dynamic, potentially computed subscript instead of tuple.0-style properties and they can't be initialized by tuple literals; as a result, the compiler can't statically diagnose invalid literals or accesses.

I would like to see a vector design which knows what it's trying to be and more thoroughly embraces one or the other of these precedents. If some parts of its design can't be achieved in Swift 5, the remaining plans should at least be outlined in a "future directions" section.

That's not to say we should blindly copy everything about collections or everything about tuples—a vector isn't an ordinary collection and it isn't an ordinary tuple—but we should consciously try to move the design closer to one of them.

Non-generic

I strongly prefer the VectorN<Element> design for several reasons.

The first is simply that I believe it will feel more natural. I can't think of another case where we generate types in bulk rather than using a generic representation that would be easily achievable in our current type system. Generics exist specifically to allow types to be composed; why not use them?

The second is surface area. The way the proposal is written obscures the fact that it introduces something like 92 new public types. 53 of those types could be collapsed to something like 28 (assuming matching VectorN<Element> structs and VectorizableN, VectorizableIntegerN, and VectorizableFloatingPointN protocols) by adopting generics.

The third is extensibility. It would be nice to be able to say things like Vector3<CGFloat>, but the current design makes that very difficult. I think we could design some sort of forwarding system into the Vectorizable protocols so that types outside the standard library could be elements of vectors, so long as they can be losslessly converted into an underlying vectorizable type.

The fourth is the inherent GYB-biness of the proposal. By generating so many unrelated types sprinkled throughout the type system, we lose the opportunity to improve the implementation of the SIMD system over time. For instance, it's not inconceivable that a future version of Swift with generic integer literal parameters could rework builtins to support generic parameters—think Builtin.fadd<Builtin.Float32, 4>() instead of Builtin.fadd_Vec4xFloat32(). This would allow the code to be massively—perhaps even completely—de-GYBbed, but if we're generating tons of types all over the place, our gains from this will be pretty small.

In previous discussions, a common counter-argument has been that the implementations of these protocols will require just as much generated code. While this is probably true, I think it's irrelevant. What matters more is the public surface area of the feature, and that will most likely be reduced by this change.

Namespacing

I'm not entirely convinced that these types should be in the core standard library (i.e. libswiftCore), rather than in the simd module or something similar. They are somewhat specialized, and although I wouldn't expect anyone to use the name "vector" for a resizable array (since we use Array for that meaning), I could see someone wanting to use "vector" for the mathematical object without wanting the semantics of our implementations.

I don't understand why some types are prefixed with SIMD and others are not. The choice of naming seems somewhat random. If these types are included in the core standard library, I think they should all be prefixed with SIMD; if they're in a separate simd module, I think few or none of them should be.

The SIMD.Mask types

If I understand correctly, VectorN<Bool> can't really work for predicates/masks because the underlying representation of these is machine-dependent and isn't always compatible with a one-bit boolean. However, I don't understand why we can't model these as, say, VectorN<VectorBool> or something similar. I also have a sneaking suspicion (though I haven't tried it) that a good generic design could adjust for the fact that Bool looks different when it's inside a vector.

What's good

This is a long list of complaints, so I want to also stress some of what this proposal gets right:

  • The decision to use distinct VectorN types, and the sizes chosen.

  • The operators available, and particularly the final decisions on comparisons and logical operators.

  • The decision to break precedent with the naming of replacing(mask:with:) (or whatever it is eventually called).

  • The extremely low-overhead design.

I hate to suggest this many revisions during review, but the previous stage of discussion seemed somewhat abbreviated.

  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

As I said at the beginning, I have little personal experience with SIMD code.

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Mildly in-depth? I have prototyped some of the changes I discuss, although the prototype was never completed.

9 Likes
40

Brent's observation about the tuple-like aspects of vectors is interesting. It raises (for me) the question of whether would want to add language sugar to enable v.0 like a tuple. If we did that, it could argue for v.xy and other niceties from other languages.

In any case, these thoughts are not core to the proposal: they could be handled with a follow on.

1 Like
41

Splatting?

V.xyzw = v1.xxzz... mmmh...

1 Like
42

Generally, I am all for a Swift implementation of SIMD, but I feel the proposal does not completely embrace a "Swift feel" that has been so prevalent in other proposals.

Yes, current convention involves "rolling your own vector library" or leveraging awkward C-like syntax from <SIMD/SIMD.h>.

In some ways yes, and in some ways no. As mentioned previously on this thread and on the discussion thread, the naming convention does not feel very much like Swift. The author's argument for not going with Vector4, for example, is not very strong and he even concedes that there "[...] should not [be] any performance consequences [...]". Though I agree that this implementation may induce some added complexity, I think it is worth it to maintain the Swift API feel for users--most users are not going to care what the API looks like under the hood, but they are going to expect the language to maintain Swift naming conventions. So, I strongly echo the sentiments written above by previous reviewers.

Despite some talk on the discussion thread regarding keeping SIMD vectors apart from collections, there is still some awkward collection functionality remaining such as subscripting and count. If the previously mentioned naming convention were used, you wouldn't need a count method as you already known how many values are in the vector.

I really like the . syntax for the "lane-wise" operations. Very nice, but I would definitely make sure to use it throughout to maintain consistency so one knows . means we're working on a lane and the vector otherwise.

I've only used <SIMD/SIMD.h> but currently, the proposal as is does not seem make to make great strides in being that much more Swift. I really like the overall idea though.

I reviewed all of the discussion thread, proposal thread, and proposal. I compared current convention with the proposed changes and would consider my review at least a moderately in-depth study. Respectfully, I would like to echo the sentiment that the proposal does not feel complete as is. The proposal had a couple typos which caused significant confusion, and the sections involving implementation details and examples of use feel limited. I also feel the discussion phase of this proposal was too short, and though I am grateful a expert in SIMD is spearheading this project I feel like the discussion has been rather closed to most potential changes that would benefit the user level. I very respectfully suggest taking the time re-evaluate this proposal in discussion allowing time to flesh out the details of "why" design choices are being made (there are just too many choices that a reader is forced to take for face value in the current state). I am very excited about the idea of having SIMD in Swift, but I think this proposal just needs a little more time to cook.

5 Likes
43

-1 on this proposal.

SIMD types are important and should be part of the language, but I think this pitch has moved far too quickly from initial pitch to review. We have had much longer discussions about much smaller additions to the language.

The proposal has been substantially revised at least once, there are disagreements over how the types should be spelled, what the operators should look like, etc. It's just not ready.

On the spelling issue, one of the big benefits of using a generic type would be type inference for the element type. For example:

func doSomething(_ values: [Int32]) {
  let vec = Vector4(values) // Element type inferred from Array type - Vector4<Int32>.
}

Without generics, we would have to be explicit every time:

func doSomething(_ values: [Int32]) {
  let vec = Int32.Vector4(values) // Repetition of 'Int32'.
}
9 Likes