SE-0567: Relaxed floating-point operators

Hello, Swift community. The review of SE-0567: Relaxed Floating-Point Operators begins now and runs through December 1st, 2026.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to me as the review manager by email or DM. When contacting me directly as the review manager, please put [SE-0567] at the start of the subject line.

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

  • What is your evaluation of the proposal?
  • Is the problem being addressed significant enough to warrant a change to Swift?
  • Does this proposal fit well with the feel and direction of Swift?
  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available on the evolution website.

Thank you for contributing to Swift!

John McCall
Review Manager

10 Likes

I feel like I understand &+ etc. on integers as "truncate/wrap, don't crash". Reusing the names here for "maybe reassociate, don't worry about the sign of zero, give different results on different hardware" is sufficiently dissimilar to be worth a different spelling.

The lack of &/ is interesting — I presume this reflects some kind of LLVM limitation, but could LLVM not optimize division-addition pairs into the creation of a reciprocal, followed by fmas? And would &/ not be exactly the operator that could license such a transformation?

function f(q: Float, a: Float, b: Float, c: Float, d: Float) -> Float {
    let x = a &/ q + b
    let y = c &/ q + d
    return x + y
}

--->

function f(q: Float, a: Float, b: Float, c: Float, d: Float) -> Float {
    let r = 1.0 / q
    let x = fma(a, r, b)
    let y = fma(c, r, d)
    return x + y
}
5 Likes

No, there's no LLVM limitation here. Just that:

  • reciprocal formation is generally less valuable of an optimization than FMA and reassociation, since you can easily write it out yourself (and division is just much less common in perf-critical contexts).¹
  • reciprocal formation, unlike FMA formation or reassociation, tends to make computed results less accurate.
  • there isn't a pre-existing &/ operator defined by the stdlib.

We could certainly add it at some future point, but it seemed reasonable to subset out for these reasons.


¹ In particular, for high-speed dense linear algebra, you almost always want to license arbitrary FMA and reassociation on a per-target basis, but pretty much always want reciprocal formation to be precise and not set per-target. The nice thing about operators--as opposed to something like "fast math"--is you can do this on an operation by operation basis, which makes tossing it in less harmful, but it's also just less necessary.

3 Likes

I don't know why this just occurred to me, but given that users will often want to do this on a whole swath of code--yes, it's cool that a dedicated operator would enable a lot of specificity, but would it be more ergonomic to have some sort of @attribute for this to turn it on/off per scope?

One could contemplate a default @attribute as per SE-0478 on a file-level basis as well. Could that be a more suitable level of granularity for FMA and reassociation?

4 Likes

I think it is a nice addition, as this gives developers control to choose performance over precision.

I’m also wondering if you have considered replacing the current infix operators like ‘*’ directly, since we have just changed the definition of pi.

Don’t worry, you thought about it during the pitch thread too. :)

4 Likes

:person_facepalming:
I guess I'm consistent?

2 Likes

I think &/ should be defined – would help when numerator happens to be Int{8/etc}.min and denominator = -1.

IMHO the & prefix doesn't seem appropriate for indicating relaxed floating-point operations. It would make more sense for Float.max &+ Float.max to produce a value other than .infinity, or for signaling floating-point operations (if we had them) to not signal on overflow.

2 Likes

I expect we’ll explore alternatives during this review. Do you have a suggestion?

I'd suggest keeping this as a compile-time option – at least initially – similar to -ffast-math. This would let users opt in to the relaxed behaviour at their preferred scope: project › target › individual file level. Like an MVP that leaves the door open for a narrower scope in the future – if needed, though maybe ... it won't be needed (YAGNI).

I don't think &+ is a good spelling either, but Swift doesn't support per-file compilation flags. (I'd prefer either the function-attribute or macro-block approaches offered in the pitch thread.)

3 Likes

Would it make sense to extend these relaxed floating-point operators to SIMD types as well?

SIMD already has wrapping operators like &+ and &* for integer vectors. It seems natural to support the relaxed floating-point semantics for SIMD vectors with floating-point elements, allowing optimizations such as reassociation and FMA contraction.

I like the operator, it makes the decision to apply a relaxed operation clear without having to scan the surrounding scope. Reusing the symbol from integer addition works well as I can read it in both the integer and floating point case as “less strict, more performant alternative”.

2 Likes

Good to know! (and Xcode UI should probably disable setting those flags on a per-swift-file basis, otherwise it's misleading).

1 Like

similar to -ffast-math.

I would be strongly opposed to anything that licenses these transforms via compile flag or at file scope, because it means that copying a function from one file to another is likely to result in changes that may be catastrophic numerically or for performance, but which are very unlikely to be found by simple behavioral tests.

For instance, if I have:

default @fastMath

//... 3000 lines later ...

func euclideanNormSquared(_ a: Span<Float>) -> Float {
  a.indices.reduce(into: 0) { $0 += a[$1] * a[$1] }
}

and someone on my team does some refactoring and moves that function without a fast math default, it will suddenly be more than an order of magnitude slower. But the results that it produces will still be "fine" and this regression can easily slip through if I do not have dedicated performance tests, only to have catastrophic behavior when it encounters a large data set.

Conversely, if someone vendors Augmented.sum from Numerics:

public func twoSum(
  large a: Double, small b: Double
) -> (head: Double, tail: Double) {
  let head = a + b
  let tail = a - head + b
  return (head, tail)
}

by copying it into a file that happens to have @fastMath, instead of returning a correct head-tail sum, it will just return (a+b, 0). If they add tests specific to twoSum, they'll probably find this bug, but if twoSum is part of a larger algorithm, and is only necessary in certain cases, then they easily might miss this bug.

For these reasons, it is a significant advantage to have these operations be visibly different in the source. They have different semantics, and should be treated as different operations. We should not use file- or module-level switches to control the behavior (there have been innumerable bugs introduced to C and C++ programs in exactly this manner).

Using function- or block-scope modifiers would be more palatable, since people tend to copy whole units around, so they are more likely to be preserved. They are also more likely to be "sufficiently nearby" that a reader of the program will see them. But actually using distinct operators is by far the safest option (and makes them available so that people can experiment with other approaches).

14 Likes

This is worth a separate thread (but, note that x86 traps on Int.min/-1, so &/ would have to handle that case explicitly; it can't just "do what the hardware does" like &+ and &* can).

Yes. Reassociation has less value for SIMD (it's already vectorized, so licensing autovectorization usually doesn't buy you much), but opportunistic FMA formation has value.

Would one ever want to express “allow FMA but still trap on overflow?”

"trap on overflow" is not a thing that we express for floating-point (and no mainstream ISA makes it available operation-by-operation; it's either not supported at all or only supported via thread-local state. So there's no value in tying it to an operator).

Even in a world where we had bindings for it and hardware supported it, it wouldn't make sense, because the possibility of trapping (an observable side-effect) would itself prevent FMA formation.

1 Like

Sorry, I meant whatever distinguishes SIMD<T>.+ from SIMD<T>.&+. Is there a chance a programmer might want to allow FMA synthesis without opting into that behavior difference, or vice versa?