[Pitch #3] Swift Compile-Time Values

Evolution Pitches
21

Something that Swift Testing needs, and which may be a general need for macros, is a way to constrain some value to literals only (so that our @Test and @Suite macros can inspect the values directly.)

We currently use _const for this purpose, but if it is modified to behave the same as the proposed @const or @constInitialized, then a value could be passed that is constant, but not a literal, which would break our macros.

Is there room in this proposal for something like @literal, keeping in mind that macros only get the AST and no type information?

4 Likes
22

Read the whole pitch for the first time, great to see this moving forward again. :smiley: I wondered why Artem wasn't as active on github late last year, guess this explains it. Thanks for working on this detailed proposal.

Is any of this implemented yet? For such a large pitch, would be good to try things out with a working implementation, even if not fully baked yet.

One issue I've seen with compile-time implementations in other languages is how to deal with initializing floating-point constants when cross-compiling. Say you're initializing some 128-bit floating-point constant for AArch64 with a compile-time calculation, but the native hardware only supports 64-bit floating point: you then have to push everything to some soft-float library to get the same full precision as you would on native hardware, ie when compiling on AArch64 itself (I guess Float128 isn't supported by Swift yet, just expecting it will be, and the point stands for 80-bit also).

I don't have much else to say about this necessary @const feature, but I'd like to link this recent blog post comparing compile-time programming in Rust, Zig, and some other languages, including their syntax for such compile-time constants. That @embedFile feature would be useful in Swift too, so we could import and parse arbitrary data at compile-time. Also, I started watching this Jon Blow interview, the developer of the upcoming Jai language, and he talks a lot about his thoughts on macros and compile-time programming in the first hour.

While this @const computation would be a great feature to have, what I'm really looking forward to is being able to manipulate code using compile-time reflection, whereas this pitch is really about the first step of what I called compile-time code execution a couple years ago. You can see some of that compile-time manipulation in other languages in Renato's blog post, but I was trying to come up with a simple example that really demonstrates it, and this is what I got.

We all know data layout really impacts performance, as Andrew Kelley of Zig talked about a couple years ago. So if we start packing structs like he shows there, but need to do it differently for each platform's types and alignments, we might write it out like this today:

// long comment explaining our calculations of why
// we chose this data layout for each platform
struct Foo {
#if os(Android) && (arch(arm64) || arch(x86_64)
var goo: UInt
var moo: CLongDouble
var boo: Float
var noo: Double
#elseif os(Android) && arch(arm)
var goo: UInt
var boo: Float
var noo: Double
var moo: CLongDouble
#elseif os(Linux) && (arch(i386) || arch(x86_64))
...
// Lot of other platforms
...
#endif
}

This is verbose and depends on not always provided doc to explain the algorithm used for layout.

Conversely, if we move this to compile-time, it might look like this:

// compile-time code to calculate the right ordering layout
@const layout : String = calculate_layout()
public func calculate_layout() @const -> String {
  if (CLongDouble.size ... // compile-time logic to get the right layout
}
struct Foo {
#if layout == "CLDLast"
var goo: UInt
var boo: Float
var noo: Double
var moo: CLongDouble
#elseif ...
}

Nobody has to read doc to figure out the layout algorithm used, it is explicitly shown and run on each build. Of course, this won't work without such platform properties being available to reflect on at compile-time. In theory, this is all doable with macros too, but looking at both Swift's macro code and those other languages' alternatives shown above, that would just bury the compile-time algorithm in a bunch of obscure macro code.

Obviously, this is a simple example to demonstrate the value of such compile-time code manipulation, one can come up with much more. No doubt this is currently not possible without first figuring out the layering issues Artem mentions above.

Even if Swift never is able to do this, this is the direction other languages are heading in and the Swift language group should be contemplating.

3 Likes
23

A portion of this proposal, with some more limitations and usability issues (e.g. compiler diagnostics) is implemented for the @_section experimental attribute; so that's one way to try out some of the compile time concepts and behaviors. It doesn't (yet) use any of the new syntax proposed here, so the experience doesn't match the proposal, but some of the basic and even more complex use cases (e.g. user defined structs) do actually enforce compile-time-ness and constant folding. Needless to say, the implementation is experimental and probably full of bugs. But see e.g.: Compiler Explorer

4 Likes
24

I think this would be reasonable.

As-is, @const methods are allowed to be called at runtime, according to this proposal, as the annotation has no impact on code-generation. A few other folks have mentioned moving the attribute to before the method declaration, which I think is reasonable.

I believe that in the majority of cases, the compiler will be able to optimize uses of @const values across module boundaries to a direct use of an immediate value; however, we still have a need to allow @const values to be referenced across module boundaries and have storage, as @tshortli has mentioned, so they will still have a corresponding symbol which is accessed as described in the ABI and Memory Placement and Runtime Initialization section.

Macro inputs are a key use-case the pitch singles out as something it would be great to have a solution for, and also that it is one we do not have due to fundamental compiler layering. It is something we will very likely need to find a solution for in the future. In the meantime, as a workaround for this layering problem, I am not sure about adding @literal to this proposal as it could muddy the waters a bit. But I can at the very least leave the existing _const implementation alone, for the time being...

Not quite yet. Implementation is WIP over the coming weeks, before we gear up for a full-on evolution proposal with the feedback gathered here. Stay tuned and keep an eye out for PRs.
And thank you for sharing your experience about wide floating point semantics and for sharing some reference materials about other compile-time programming paradigms, it is very interesting food for thought.

While the layering constraints you point out prevent the kind of code-manipulation you describe, and even if Swift is never able to get this capability, Macros may still be a path forward toward some similar use-cases (as you also acknowledge), were we to design a solution for @const Macro inputs. And perhaps other tooling approaches could then also obviate the "bury the compile-time algorithm in a bunch of obscure macro code" problem.

1 Like
25

I suppose @_literal could be an experimental attribute if it came to it—we'd just be trading one experimental keyword for another really.

Please make sure to give this some discussion in the Future Directions section. :slight_smile:

26

Does the current proposal imply that @const in a contravariant position requires that the value was produced by compile time logic but in a covariant position only indicates that the value is pure?

What is the distinction in the type signature between a function that is pure and can be called either at compiler or run time and a closure which was constructed at build time and has only compile-time captured context?

Should @pure and @const be distinct?

Related to literals, something I'm looking for is having recursive structures that only need to be written out statically and so effectively are always constructed as a literal.

Is there any consideration from having const as a qualification on a by-value type declaration like enum, requiring any value of that type to be constructed at compile time?

For example:

const enum Path {
    case empty
    case element(head: StaticString, tail: Path)
}

let path = Path.element(
    head: "Foo",
    tail: .element(
        head: "bar",
        tail: .empty
    )
)

This could potentially mean that StaticString and BigIntLiteral can be described in such terms too:

const struct StaticString {
    let utf8bytes: StaticArray<UInt8>
}

const struct BigIntLiteral {
    let sign: Bool
    let words: StaticArray<UInt>
}

This might potentially also mean that those types can be constructed from logic in a const function but that would require that function only ever be evaluated at build time (as opposed to simply being pure).

(I think in the above cases literal might better describe the desired effect rather than const)

27

What are the actual reasons we want to have compile time values and should they be considered distinct properties?

In performance cases, we sometimes want the guarantee that some expression isn't computed lazily. In this case we'd want to do something like annotate a binding to require that its value be constructed at compile time and that all sub-expressions of the assigned expression are pure (and any contained function calls are pure).

Where we have compile-time dependencies on values (such as layout properties) we need the guarantee that the value is knowable by the time layout is determined (which isn't necessarily at compile time). This is equivalent to saying the value needs to be knowable at the time type information is consumed (and so would need to be a valid type-level value). For this it is probably sufficient to require the expression appear and bind at the type level as a const generic or similar.

In memory, concurrency, or allocation sensitive cases, we want the guarantee that the complex value is stored in statically allocated memory. In some cases there is a desire for that value to be mutable at runtime but in many its desirable for that memory to be immutable and for duplicates to be coalesced so that a reference to that memory can be passed around in place of copies of that value into potentially allocated storage. StaticString and BigIntLiteral might indicate that it's more appropriate to the type of the value itself (by means of its declaration) guarantee that accessing it or any of its components doesn't incur a dynamic memory or allocation cost.

28

One the reason is to make integer generic parameters more flexible.

    static let someIntegerConstant = 42
    let d: IntParam<someIntegerConstant> // Error, not an Int generic parameter

With compile time constant, this kind of construct would be possible.