[Pitch #3] Swift Compile-Time Values

Something that Swift Testing needs, and which may be a general need for macros, is a way to constrain some value to literals only (so that our @Test and @Suite macros can inspect the values directly.)

We currently use _const for this purpose, but if it is modified to behave the same as the proposed @const or @constInitialized, then a value could be passed that is constant, but not a literal, which would break our macros.

Is there room in this proposal for something like @literal, keeping in mind that macros only get the AST and no type information?

4 Likes

Read the whole pitch for the first time, great to see this moving forward again. :smiley: I wondered why Artem wasn't as active on github late last year, guess this explains it. Thanks for working on this detailed proposal.

Is any of this implemented yet? For such a large pitch, would be good to try things out with a working implementation, even if not fully baked yet.

One issue I've seen with compile-time implementations in other languages is how to deal with initializing floating-point constants when cross-compiling. Say you're initializing some 128-bit floating-point constant for AArch64 with a compile-time calculation, but the native hardware only supports 64-bit floating point: you then have to push everything to some soft-float library to get the same full precision as you would on native hardware, ie when compiling on AArch64 itself (I guess Float128 isn't supported by Swift yet, just expecting it will be, and the point stands for 80-bit also).

I don't have much else to say about this necessary @const feature, but I'd like to link this recent blog post comparing compile-time programming in Rust, Zig, and some other languages, including their syntax for such compile-time constants. That @embedFile feature would be useful in Swift too, so we could import and parse arbitrary data at compile-time. Also, I started watching this Jon Blow interview, the developer of the upcoming Jai language, and he talks a lot about his thoughts on macros and compile-time programming in the first hour.

While this @const computation would be a great feature to have, what I'm really looking forward to is being able to manipulate code using compile-time reflection, whereas this pitch is really about the first step of what I called compile-time code execution a couple years ago. You can see some of that compile-time manipulation in other languages in Renato's blog post, but I was trying to come up with a simple example that really demonstrates it, and this is what I got.

We all know data layout really impacts performance, as Andrew Kelley of Zig talked about a couple years ago. So if we start packing structs like he shows there, but need to do it differently for each platform's types and alignments, we might write it out like this today:

// long comment explaining our calculations of why
// we chose this data layout for each platform
struct Foo {
#if os(Android) && (arch(arm64) || arch(x86_64)
var goo: UInt
var moo: CLongDouble
var boo: Float
var noo: Double
#elseif os(Android) && arch(arm)
var goo: UInt
var boo: Float
var noo: Double
var moo: CLongDouble
#elseif os(Linux) && (arch(i386) || arch(x86_64))
...
// Lot of other platforms
...
#endif
}

This is verbose and depends on not always provided doc to explain the algorithm used for layout.

Conversely, if we move this to compile-time, it might look like this:

// compile-time code to calculate the right ordering layout
@const layout : String = calculate_layout()
public func calculate_layout() @const -> String {
  if (CLongDouble.size ... // compile-time logic to get the right layout
}
struct Foo {
#if layout == "CLDLast"
var goo: UInt
var boo: Float
var noo: Double
var moo: CLongDouble
#elseif ...
}

Nobody has to read doc to figure out the layout algorithm used, it is explicitly shown and run on each build. Of course, this won't work without such platform properties being available to reflect on at compile-time. In theory, this is all doable with macros too, but looking at both Swift's macro code and those other languages' alternatives shown above, that would just bury the compile-time algorithm in a bunch of obscure macro code.

Obviously, this is a simple example to demonstrate the value of such compile-time code manipulation, one can come up with much more. No doubt this is currently not possible without first figuring out the layering issues Artem mentions above.

Even if Swift never is able to do this, this is the direction other languages are heading in and the Swift language group should be contemplating.

3 Likes

A portion of this proposal, with some more limitations and usability issues (e.g. compiler diagnostics) is implemented for the @_section experimental attribute; so that's one way to try out some of the compile time concepts and behaviors. It doesn't (yet) use any of the new syntax proposed here, so the experience doesn't match the proposal, but some of the basic and even more complex use cases (e.g. user defined structs) do actually enforce compile-time-ness and constant folding. Needless to say, the implementation is experimental and probably full of bugs. But see e.g.: Compiler Explorer

4 Likes

I think this would be reasonable.

As-is, @const methods are allowed to be called at runtime, according to this proposal, as the annotation has no impact on code-generation. A few other folks have mentioned moving the attribute to before the method declaration, which I think is reasonable.

I believe that in the majority of cases, the compiler will be able to optimize uses of @const values across module boundaries to a direct use of an immediate value; however, we still have a need to allow @const values to be referenced across module boundaries and have storage, as @tshortli has mentioned, so they will still have a corresponding symbol which is accessed as described in the ABI and Memory Placement and Runtime Initialization section.

Macro inputs are a key use-case the pitch singles out as something it would be great to have a solution for, and also that it is one we do not have due to fundamental compiler layering. It is something we will very likely need to find a solution for in the future. In the meantime, as a workaround for this layering problem, I am not sure about adding @literal to this proposal as it could muddy the waters a bit. But I can at the very least leave the existing _const implementation alone, for the time being...

Not quite yet. Implementation is WIP over the coming weeks, before we gear up for a full-on evolution proposal with the feedback gathered here. Stay tuned and keep an eye out for PRs.
And thank you for sharing your experience about wide floating point semantics and for sharing some reference materials about other compile-time programming paradigms, it is very interesting food for thought.

While the layering constraints you point out prevent the kind of code-manipulation you describe, and even if Swift is never able to get this capability, Macros may still be a path forward toward some similar use-cases (as you also acknowledge), were we to design a solution for @const Macro inputs. And perhaps other tooling approaches could then also obviate the "bury the compile-time algorithm in a bunch of obscure macro code" problem.

1 Like

I suppose @_literal could be an experimental attribute if it came to it—we'd just be trading one experimental keyword for another really.

Please make sure to give this some discussion in the Future Directions section. :slight_smile: