Add Float16

The last decade has seen a dramatic increase in the use of floating-point types smaller than (32-bit) Float . The most widely implemented is Float16 , which is used extensively on mobile GPUs for computation, as a pixel format for HDR images, and as a compressed format for weights in ML applications.

Introducing the type to Swift is especially important for interoperability with shader-language programs; users frequently need to set up data structures on the CPU to pass to their GPU programs. Without the type available in Swift, they are forced to use unsafe mechanisms to create these structures.

In addition, C APIs that use these types simply cannot be imported, making them unavailable in Swift.

To remedy this, I would like to add Float16 to the standard library:

45 Likes

A: FloatN is the more consistent pattern. Swift already has Float16 , Float32 , Float64 and Float80 (with Float and Double as alternative spellings of Float32 and Float64 , respectively). At some future point we will add Float128 . Plus, the C language type that this will bridge to is named _Float16 .

  • Maybe there's a typo? Swift doesn't "already have Float16".

  • Float32 and Float64 are currently alternative spellings (type aliases) of Float and Double, not the other way around. While I like Float16 better than Half, IMO what would be most consistent with today's naming scheme is to define struct Half and typealias Float16 = Half. Maybe it's worth clarifying this in this paragraph.

4 Likes

I think Float16 is a much better name than the unintuitive Half alternative.

19 Likes

Maybe there's a typo? Swift doesn't "already have Float16 ".

Yes, fixed.

Float32 and Float64 are currently alternative spellings (type aliases) of Float and Double , not the other way around.

They're both alternative spellings of each other; two names for the same thing. Which is the base type and which is the alias is just an implementation detail, because these types have custom mangling anyway.

While I like Float16 better than Half , IMO what would be most consistent with today's naming scheme is to define struct Half and typealias Float16 = Half .

Consistency is not a good goal to pursue on its own; it's only good when it improves clarity or ease of use (which it often, but not always, does). If we like the name Float16 better, we should use Float16. We already have the ambiguity for 32 and 64-bit types, but there's no reason to introduce it for 16 or 80 or 128 bit types just to match previous decisions.

I do think we could consider adding something like:

@available(*, unavailable, renamed: "Float16")
struct Half { }

to direct people who try that name to the thing that they're actually looking for.

12 Likes

+1 for this. I'm currently working on compact binary format, and was very surprised that swift doesn't have that already.

Btw, how hard it would be to add availability attributes only where it is actually needed? Because it is looks like almost every operation will just compile into cpu instructions, not to stdlib calls. Waiting for another 4-5 years is sad :(

1 Like

@scanon is it possible to add the type in a standalone swift package?

This is related to my question:

I doubt it, because the implementation would us LLVM intrinsics, just like current FloatN types do.

Maybe we should see if anything breaks by making Float/Double type-aliases of Float32/Float64. It's a tiny bit of house-cleaning, but apparently it may be source and ABI-compatible to do that.

The reason for the status quo is strictly due to diagnostics, and this is mentioned in the standard library source code:

// FIXME: it should be the other way round, Float = Float32, Double = Float64,
// but the type checker loses sugar currently, and ends up displaying 'FloatXX'
// in diagnostics.
/// A 32-bit floating point type.
public typealias Float32 = Float
/// A 64-bit floating point type.
public typealias Float64 = Double

I'd guess that's probably an old bug and doesn't apply any more.

It is technically possible to make a stand-alone package that avoids using the builtins. It’s something that could be considered, but would be essentially an orthogonal effort (there would be very little overlap in implementation). We could consider such a thing for Swift Numerics, but it probably wouldn’t go in preview (due to being an entirely separate implementation).

I.e. it’s possible, but should be treated as a separate thing from this proposal IMO.

1 Like

I said it before, and I'll say it again: Let's move to review! :construction_worker_man:

We should really have more proposals like that:

But still, imho some bikeshedding is actually the only thing that's missing, and list of existing languages/libraries and their naming choice(s) (Float16 vs Half) would round out the proposal (there are some entries in the other thread).

For anyone who has nothing to add but his personal favorite, imho this should be the easiest way to collect opinions:

What name do you prefer (choose up to four)?

  • Float16
  • Half
  • Something Else
  • I don't care much - just hurry

0 voters

Would you like to have an official typealias (choose one)?

  • Yes, two names are better than one
  • I don't care as long as I have Half
  • I don't care as long as I have Float16
  • I don't care
  • No, it has to be one name only

0 voters

3 Likes

I'm for the Float16 name and not adding a typealias for Half.

It also seems like potential overkill to make Half an unavailable struct just for the purpose of directing people to the correct type. It seems like a developer would only need to find this out once and that it could be discovered very quickly via a web search or documentation.

Also, the section The Basics in The Swift Programming Language could probably use a little more detail on floating point types. The integer section mentions (but does not emphasize) all sized integer types. The floating point section only mentions Float and Double with no mention of Float80 (or Float32 and Float64). Especially with the addition of Float16 it would probably make sense to add a few more sentences.

11 Likes

If we're going to go all the way and say this alternate spelling is common enough that it deserves a mention in the standard library at all, we might as well just add the typealias. I don't think our goal is to force developers to use any particular style -- if they really want to write Half, let them write Half :man_shrugging:

The way I see this is parallel to Int8/6/32/64: the explicit bitwidths are the true fundamental types, similar to how pure mathematics is as close as we can get to universal truths. And then there are types like Int, whose meaning depends very much on your environment. As a curious aside - the C data model evolved from the opposite direction, without fixed-width integers, before they were added in C99. Since SE-0191, we have a nice definition for what Int means in our data model: it's the maximum size of a Collection. No matter what kind of fancy indexes you create, the distance between them (and hence the Collection's count) cannot exceed Int.max (not UInt64.max, even though the count cannot be negative).

Now for Float. The official names in IEEE-754 are binary16/binary32/binary64. There's a brief reference to single and double precision in the opening line, but otherwise these names are used literally everywhere else. The names float/double are not universal, either: FORTRAN calls single-precision a REAL, Delphi and MATLAB call it a single, and Python/Ruby/PHP use float to refer to double-precision! We've decided to use the C names, but supposedly the C data model doesn't exactly define those sizes, either (kind of like int):

The actual size and behavior of floating-point types also vary by implementation. The only guarantee is that long double is not smaller than double , which is not smaller than float .

So what I'm getting it is: the names Float/Double are not at all universal, and Float32/Float64 are the more fundamental types. Conceptually, they should be aliases. Lots of things are not conceptually perfect, though. It's like an itch - you can understand the compulsion to scratch it, but once you start, you'll start seeing flaws everywhere and it'll never stop. Your only hope is to kind-of ignore it and hope it goes away.

EDIT: FWIW, I've actually seen Swift graphics code from people involved with Swift4Tensorflow that uses Float everywhere, when typically you'd use Double for graphics. I suspect they may be coming from Python and might not have been aware that Double exists.

6 Likes

Half is not completely outrageous as an alias, but we shouldn't add aliases unless there's a really compelling reason.

Would a compelling reason be that Metal uses half for it’s 16-bit floating point type? Especially with this part of the proposal in mind:

Introducing the type to Swift is especially important for interoperability with shader-language programs; users frequently need to set up data structures on the CPU to pass to their GPU programs.

Edit: I say this because when I first tried to pass a 16-bit float into a metal shader, my first instinct was to try Half in Swift, knowing that metal called it a half.

2 Likes

On top of the not-using-builtins aspect, there's lower value from a package version because one of the drivers of introducing this type is so it can be vended by ABI-stable frameworks, and they will require the ABI-stable version, not a packaged version.

2 Likes

In that case, typealias MTLHalf = Float16 could be added to the Metal framework.

1 Like

I know this is off-topic, but how do you add a vote in a post?

In the reply form, click the gear icon and “Build Poll”.

Interestingly, the “Number Rating” poll style appears to use string comparison for its “Min” and “Max” fields.

At least for me, if I set “Min: 2” and “Max: 10”, it shows in red “Minimum value must be smaller than the maximum value.”

But if I set “Min: 10” and “Max: 2”, there is no such warning. Let’s see what happens if I try that…okay, it let me put it in the comment box, let’s try posting it…

Dang it wouldn’t let me post an empty poll, showing a separate dialog with:

Looks like it’s just the red text in the poll builder form that uses the wrong type of comparison.

2 Likes

Side note: In general, I'm very sceptical towards polls, and hope that I didn't create a precedent for proliferation: Votes could not only create a misperception how proposals are decided. More importantly, it can be surprisingly tricky to get the questions right (bias!), and easy to misinterpret the results.
Therefor, I think polls are not qualified for really important questions (edit: in the context of Swift Evolution ;-).
In this case, the result I consider the most valuable is actually not which choice is more popular, but that in a group of 80 people, there has been no one who would not be happy with either Half or Float16.

3 Likes

My personal experience as a graphics programmer is that the overwhelming majority of 3D graphics code (at least in the game/GPU graphics world) uses C/HLSL/GLSL float (or even lower precision).

1 Like