 `VectorNumeric` Type: In General & `Complex` Numbers

Hey @scanon,

I have a follow up question to my previous question: Introducing Complex Numbers Update.

I have developed a prototype version of complex numbers as part of the Swift for TensorFlow project (can be found in the PR here). We have a `VectorNumeric` protocol as well which defines any object that can exist in a vector space (found here). With complex number vectors, the dot product behaves differently in that it's something like this:

``````func dot(lhs: Vector<Complex>, rhs: Vector<Complex>) -> Complex {
var result = Complex(real: 0, imaginary: 0)
for i in lhs.length {
resultVector = resultVector + (lhs[i].complexConjugate() * rhs[i])
}
return result
}
``````

As such, we were debating on where `complexConjugate` should exist, and also about whether a `VectorNumeric` protocol is necessary.

1. In general, do you think a `VectorNumeric` protocol is necessary? Or could we use something else or maybe a different protocol. As it seems like a protocol would be needed for the `SIMD` vectors we have, the `Tensor` type the S4TF team is adding, and any custom types users could add.

2. Secondly, specifically regarding complex numbers, should `Numeric` refine `VectorNumeric`? The reason why is that complex numbers can be thought of as both a scalar value and a vector:

...complex numbers with the complex plane...makes their structure as a real 2-dimensional vector space evident. (wiki link)

Similarly, it could be argued that a real number is also a vector, except just a length 1 vector. Thus, could the `Numeric` protocol refine `VectorNumeric`? However, the biggest issue I see is ABI stability, which may prevent this.

1. Lastly, also specific to complex numbers, should `VectorNumeric` define `complexConjugate`? This follows from the second question in that complex conjugate seems to only be defined on complex numbers, and in other cases it would just return `self`. However, depending on the answer for 2, since complex numbers are like a vector of sorts, then `VectorNumeric` could define it.
3 Likes

Because this is public discussion I will add my opinions:

1. Do you think a `VectorNumeric` protocol is necessary?
Yes, though perhaps not for the stdlib.
2. Should `Numeric` refine `VectorNumeric`?
Perhaps, if `VectorNumeric` is defined as it is in `S4TF`, but I do not think that protocol deserves the name "VectorNumeric".
3. Should `VectorNumeric` define `complexConjugate`?
Probably not, `complexConjugate` is an implementation detail, `VectorNumeric` should define `dot(Self, Self) -> Scalar`.

What I imagine for vector protocols is:

``````// what is currently in S4TF as `VectorNumeric`
public protocol Vectorial : AdditiveArithmetic {
/// The type of scalars in the vector space.

static func * (lhs: Scalar, rhs: Self) -> Self
static func *= (lhs: inout Self, rhs: Scalar)
}

// if we had a protocol between `Numeric` and `FloatingPoint`
// that included failable divisibility (i.e. `divide(Self, Self) -> Self?`)
// (Namely the mathematical `Field` protocol)
// I would add that constraint to `Scalar` here, as that would enable
// implementations of `[sin|cos|tan]²AngleBetween(Self, Self) -> Scalar`
public protocol VectorNumeric : Vectorial {
static func dot(_ lhs: Self, _ rhs: Self) -> Scalar
}

extension VectorNumeric {
func isPerpendicular(to other: Self) -> Bool {
return dot(self, other) == .zero
}

/// because we cannot here guarantee that `Scalar: FloatingPoint`
/// so `length` is not defaultable.
var lengthSquared: Scalar {
return dot(self, self)
}
}

extension VectorNumeric where Scalar: FloatingPoint {
var length: Scalar {
return lengthSquared.squareRoot()
}
}
``````
3 Likes

The "complex numbers", when considered just as a real vector space, are really not the same thing as the actual complex numbers: they're isomorphic as groups, but the complex numbers are a field with an associated multiplicative operation that is not defined on R^2. In fact, a well-behaved multiplication doesn't even exist for many vector spaces (that's why the quaternions are only a division ring, not a field). So it's IMHO totally wrong to equate the two in any sort of way: yes, you can (and usually will) represent complex numbers as R^2, but you do different things on them and you do usually want to keep track of R^2 vectors and complex numbers separately.

The same goes for real numbers as 1-d real vectors; again you can do things to real numbers (e.g. dividing) that you don't want to do to vectors.

As for whether `VectorNumeric` should be a protocol ... the way I understand your definition it's just a vector space endowed with an inner product. Mathematically this is called an inner product space (or sometimes pre-Hilbert space, but that name is probably quite off-putting). I would be fine with having a type called e.g. `DotProductSpace`, but `VectorNumeric` sounds too much like `Numeric`, which is sort of like a ring, whereas a vector space with an inner product categorically isn't ("multiplying" two vectors gives you a scalar instead of a vector).

1. Possibly, but not under the name `VectorNumeric`
2. No, that would be mathematically very misleading
3. Same. And also, IMHO you really don't want to have an operation on a protocol that only makes sense for a very limited subset of implementers of that protocol. Complex conjugates belong to complex numbers, there's no real reason why this should be defined outside of that type.
5 Likes

Because this is public discussion I will add my opinions:

Thanks for the feedback! Since this will be proposed in Swift Evolution at a later date, we do appreciate it! I wanted to specifically reach out to @scanon as Steven will be working on the implementation of complex numbers.

Probably not, `complexConjugate` is an implementation detail, `VectorNumeric` should define `dot(Self, Self) -> Scalar` .

I do see what you mean about not including `complexConjugate` in `VectorNumeric`. To make sure, do you think `complexConjugate` would live in in the `Complex` number struct?

but `VectorNumeric` sounds too much like `Numeric` , which is sort of like a ring, whereas a vector space with an inner product categorically isn't

I agree, but currently I'm unsure about a name like `DotProductSpace`, since it's very restrictive (like you said, just a first thought example!), and we may add additional functionality (not 100% at this very moment though). However, I do see the name may not be the best/clearest given the mathematical context you gave.

Thanks for your other feedback as well!

1 Like

Mathematically, a lot of other concepts can be derived from inner products: length, orthogonality, angles, etc. (see @Dante-Broggi's extensions).* I do understand though that `DotProductSpace` doesn't convey this information for non-mathematicians, so maybe a better name can be found. In the case of real vectors, "euclidean space" is often used, but not really in the complex case.

(*) In order for these concepts to work as expected, the inner product needs to satisfy some laws (which are always satisfied by the standard euclidean component-wise dot product over real number component vectors).

1 Like

Certainly; Or possibly in a protocol that `Complex` inherits from.

Just wanted to ping you again @scanon , no worries if you are busy right now with WWDC next week! Totally understand.