Those are marked @_transparent and should never appear in a backtrace. Also, if the compiler picked that getter to implement multiplication, it would effectively be undoing the SIMD entirely by multiplying each element individually.
The compiler can fail to specialize if the call above is not able to specialize either, and that is dependent on how exactly mix is called. If the call happens through an unspecialized generic or an existential, then we'll hit this fully-unspecialized path.
This is a consequence of the fact that protocol witness tables are never specialized in Swift (or, more accurately, they're specialized as far as necessary to ensure that they work with all possible values of their type parameters). If you have to call through that witness table, you'll hit the unspecialized path.
This is a source of some pain in SwiftNIO, which calls through existentials on a regular basis.
Can you show the context of the call site to Color.mix? Are you ever calling this through a generic over InterpolatableData or through an any InterpolatableData?
These existentials look like the culprit. _prepareImage will be called with the witness table obtained by opening the any ColorSpace, which will have a pointer to the fully-generic implementation of mix.
To get optimisation, you need your code to be generic all the way up to a level where the concrete type is known. In your case here, you could try replacing the existentials with enums, then switch over the enum and call the function with the appropriate types. That'll allow the compiler to emit specialised calls.
While we're here, I recommend dropping the @_specialize annotations. They aren't doing what you want done: they force the compiler to generate specialisations, but they don't force the compiler to call them. For your use-case they don't matter: the compiler will specialise when it can.