SE-0422: Expression macro as caller-side default argument

Douglas_Gregor · 2024-01-30T19:32:37.377Z

Hello Swift community,

The review of SE-0422 "Expression macro as caller-side default argument" begins now and runs through February 13th, 2024.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to me as the review manager via the forum messaging feature or email. When contacting the review manager directly, please put "SE-0422" in the subject line.

Try it out

Toolchains with this feature implemented (where each API name has a leading underscore) are available:

Linux (Ubuntu 20.04): https://download.swift.org/tmp/pull-request/70602/730/ubuntu2004/PR-ubuntu2004.tar.gz
macOS: https://ci.swift.org/job/swift-PR-toolchain-macos/1062/artifact/branch-main/swift-PR-70602-1062-osx.tar.gz
Windows: https://ci-external.swift.org/view/Pull%20Request/job/swift-PR-build-toolchain-windows/1142/artifact/build/artifacts/installer.exe

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available at

swift-evolution/process.md at main · apple/swift-evolution · GitHub

Thank you,
Doug Gregor
Review Manager

allevato · 2024-01-30T20:28:55.232Z

This feature looks like a good addition, and would be necessary to create macros that can reproduce the behavior of builtins like #file and #line. There are some build system interactions I'd like to understand better, though.

My understanding right now regarding macros and dependencies is the following. Imagine you have the following:

module M declares a macro implemented in MP
module A imports M and uses the macro
module B imports A, but not M

The plugin MP must be passed to the compiler when building module M and module A, for obvious reasons. The plugin does not need to be passed when building module B, because A's serialized module contains the result of already expanding the macro. As far as B is concerned, that macro may as well have never been there.

This appears to be true when emitting the textual .swiftinterface file for a module that uses a macro as well: the macro is expanded in the output. (I did observe a bug involving macros and inlinable functions, so I'm not sure what the correct behavior is supposed to be in that case.)

Expand for example source and swiftinterface snippets

If I write this source code:

import Observation

@Observable public class Foo {
  public var x: Int = 0
  public init() {}
}

then my textual interface shows the expansion:

public class Foo {
  public var x: Swift.Int {
    get
    set
  }
  public init()
  @objc deinit
}
extension lib.Foo : Observation.Observable {
}

This property is helpful for pruning dependency graphs, because it means we don't end up in a situation where all the macro plugins used anywhere in your dependency graph must also be passed to every compiler invocation higher up in the graph. The build system only needs to pass the plugins that belong to the libraries you directly depend on.

The feature proposed here, by necessity, breaks this property. Since a default argument expression must be expanded at each caller, then the serialized module or textual interface can only contain a reference to the macro. This means that it's not sufficient to only pass the macro plugin to the compiler when compiling that module; you must also pass it to the compiler when compiling any module that imports it. If you take the example from the top of the post and change A's usage of the macro to a default argument expression, then the plugin must also be passed when compiling module B.

So what I'd like to understand is, is it sufficient that the macro plugin only needs to be passed to the module declaring the macro, then one using it as a default argument, and then any module that directly imports that module? Or are there situations where this could recursively expand further?

I don't think any of this is cause to hold up the feature, because I don't think it's a flaw (I can't imagine how the feature would work any other way); I just want to better understand the consequences for build systems that will eventually need to support this. It's opening a small leak in what has so far been a relatively simple build model.

ensan-hcl · 2024-02-03T15:00:18.577Z

if they are used as sub-expressions of default arguments, they’ll be expanded at where they are written

I comprehend that this is the current functionality, but I fail to discern the rationale behind it. I had considered it to be a bug until I read this proposal. I had even posted about it.
In any case, I would like to inquire about the reasoning behind this behavior.

ApolloZhu · 2024-02-07T21:08:54.535Z

allevato:

are there situations where this could recursively expand further?

module M declares a macro #MyMacro
module A imports M and uses the macro in foo
module B imports A and uses foo

Since the macro will be expanded when B uses foo, any module importing B would not need to depend on the macro any more, so it should not recursively expand further

ApolloZhu · 2024-02-07T21:17:23.235Z

ensan-hcl:

reasoning behind this behavior

Changing the existing behavior would be source breaking (as any code relying on the implementation will behave differently) and should be discussed in another proposal, however, this proposal does enable you to write macros that expand to what you want at caller side, so we don't have to change how they work today:

func bar(caller: (String, Int) = #fileAndLine) {
    print("caller", caller)
}

bar(/*caller: (#file, #line)*/)

ensan-hcl · 2024-02-08T16:53:42.722Z

Thanks! and that's great!

But this answer is only half of what I want to know: what is the motivation to have different behavior in subexpressions? This may be a rude question, but if the specification described in this proposal only explicitly describes the current behavior as is, I suspect it actually describes a bug rather than desired behavior. If you know of a motive or any source of this specification, I would like to know

jrose · 2024-02-08T17:18:23.269Z

It’s mostly an implementation thing. Default argument expressions in Swift aren’t treated as bits of syntax that get copied to the call site; they’re more like little autoclosures that get called before calling the base function. The hardcoded expansions are the exception to this; we want #file to take on its value at the call site. So the compiler keeps track of whether a default argument was an expression, or one of the hardcoded expansions; with this proposal, I assume it keeps track of whether the argument was an expression or a macro. That’s much simpler than being able to compile arbitrary expressions into the caller.

This was actually a matter of correctness in early versions of Swift, where default argument expressions were not inlinable so that the library author could change them later without clients having to recompile. We realized that wasn’t worth it in the end because the library author would have to document the default behavior anyway, and then it was weird and too-subtle that someone could explicitly provide an argument that was the “default” and not get the same behavior in future versions.

At this point, apart from whether or not it’s a good idea to change to expression cloning, there’s one main design question I can think of: if a default argument contains a function call or macro that itself has one of these context-dependent default arguments, should that be attributed to the top-level caller rather than the intermediate library? A proposal to change behavior here should include explicit discussion of that, and deserves its own thread and write-up.

Douglas_Gregor · 2024-02-15T00:26:02.399Z

Thank you, everyone! The Language Steering Group has opted to accept this proposal.

John_McCall · 2024-02-15T00:30:26.212Z