Differentiable programming for gradient-based machine learning

+/+= doesn't have the right meaning, though (or is at least ambiguous[1]). It also introduces another non-homogeneous overload of +/+= with type constraints, and I'm not sure what the type checker implications of that change would be (@hborla, @xedin, any guesses where we are on this now?)

You're absolutely right that the scenario is precisely analogous to Stridable (I would argue that we shouldn't have used + there either, for basically the same reasons, but clearly that ship has sailed).

[1] Consider e.g. the manifold of rotation matrices of dimension n; the tangent space at any point is also a nxn matrix, but it's a different subspace, and move(along:) is not the normal matrix addition--move(along:) has to keep you on the manifold, but adding (in the normal matrix addition sense) a non-zero element of the tangent space always takes you off the manifold.

4 Likes