[SE-0382] Default arguments for macros and macro implementations

Evolution Pitches
1

Hello,

I've been finalizing the implementation of SE-0382 "Expression Macros", and hit on an interesting design question related to default arguments.

The language semantics of default arguments are the same as for functions. So let's say we have a macro m1 with one parameter that has a default:

@freestanding(expression)
macro m1(bestNumber: Int = 17)

Now, we go ahead and use this macro… maybe without arguments, because we can:

#m1

Now, how should we communicate the use of the default argument to the implementation of the macro, in the expansion(of:in) call? One possibility is that we “rewrite” the expansion syntax node that we pass along to the macro implementation, so it sees:

#m1(bestNumber: 17)

I’ve been assuming that’s the right answer, because then the macro implementation itself can ignore the possibility of default arguments. That’s cool, but it’s also weird, because doing it correctly (so that we have valid code) potentially has to do a bunch of restructuring. Consider this macro declaration and use:

@freestanding(expression) 
macro m2(body: () -> Void, count: Int = 1) 

#m2 { print("here") }

To pass along the default argument, #m2 now has to be restructured into

#m2(body: { print("here") }, count: 1)

However, now it feels like we’re erasing information from the call site that the macro might care about, because the user clearly wrote a trailing closure and we are not passing it along as a trailing closure. That seems to be at odds with our general notion that we pass unmodified syntax trees to macros, and might have unfortunate effects on source-location computations that the macro might do.

To me, this implies that we should leave the macro expansion syntax as written, without trying to wedge in the default arguments. By itself, this means that the macro implementation would essentially have to re-implement the default arguments. That's doable, but seems unfortunate.

I suggest that we could extend MacroExpansionContext with an operation that provides the default argument based on a given parameter to the macro. For example, it might look like this:

protocol MacroExpansionContext {
  // ...

  /// Retrieve the default argument for the argument with the label `named` within the given macro expansion node.
  func defaultArgument(
    in macro: some FreestandingMacroExpansionSyntax,
    named: String
  ) async -> ExprSyntax?
}

In our m1 example, the macro implementation could use

guard let count = context.defaultArgument(in: node, named: "count") else { ... }

I don't love this API for a couple of reasons:

  1. Its signature is insufficiently general, because we should also be able to get the default argument from an attached macro. We could provide an overload that takes an AttributeSyntax, but it feels odd.
  2. Its true generalization promises more semantic information that we can provide right now. If we really generalized this, it would also support queries on function calls and subscript expressions that occur within the arguments. We don't have a type to express that in swift-syntax, nor the machinery in the compiler to answer this full query.
  3. The by-name query for arguments (named: "count") feels imprecise. If we were in a compiler, and had access to both the macro declaration (macro m1(...)) and its matching call, we would use the parameter declarations as the key. But macro implementations don't have access to the macro declaration. The by-name query also makes it impossible to query default arguments for unlabeled arguments (macro m3(_: Int = 17)).

What do y'all think?

Doug

2 Likes
2

You could go the other direction: pass the normalized macro call, with default arguments filled in and trailing closures eliminated, and extend MacroExpansionContext with a way to get the original, unnormalized macro syntax.

2 Likes
3

It's not only imprecise, but ambiguous if it's querying by the label, no? Argument labels aren't guaranteed to be unique.

Yeah, this was what came to mind for me as well, though in the other direction at first—by default supply the 'filled in' version but provide a way to get the original unmodified version as well. Or perhaps FreestandingMacroExpansionSyntax could provide an argumentListWithDefaultedArguments property?

4

The overarching decision made here by the language workgroup after reviewing the expression macro proposal is that we do not want to be in the business of determining a “normalized” form of the macro invocation. There would be arbitrarily many decisions to make about what Swift considers “normal” and, after all, a macro may have some legitimate business (for example, diagnostics) distinguishing between otherwise equivalent expressions.

So I would agree that what hews most closely to that philosophy is as outlined by @Douglas_Gregor. Perhaps the API to be added here could be an array of serialized arguments with any defaults inserted as well as any other implicit information?

1 Like
5

Yeah this is sort of what I was picturing from the argumentListWithDefaultedArguments property but I don't know that it makes sense as a property on the syntax node. It could also work as a separate API on MacroExpansionContext, but I agree with you that this feels like the right shape for the API rather than individual queries by label. If needed we could surface this as a list of label-argument pairs rather than just the arguments.

6

We actually reject attempts to declare two arguments with the same label for functions, and the same logic carried over to macros, so it's not ambiguous:

t.swift:1:21: error: invalid redeclaration of 'a'
macro m(a: Int = 17, a: Int = 18) =
                    ^
t.swift:1:8: note: 'a' previously declared here
macro m(a: Int = 17, a: Int = 18) =
       ^

[EDIT: The above is completely wrong. We do allow repeated labels, even though it's silly]

Syntax nodes are supposed to be 1:1 with their textual representation, so we don't get to have properties with extra information that isn't in the source code text.

This is an interesting point. There are other reasons why it could be valuable to have a more semantic representation of all of the arguments. We have this in the compiler, where there is an explicit mapping from each parameter to the set of argument(s) that are passed to that parameter, and it's used everywhere beyond the part of the type checker that forms it. It simplifies a bunch of things:

  1. For variadic parameters (and eventually parameter packs), you get all of the corresponding arguments together, rather than having to dig them out.
  2. For defaulted arguments, they're right there in the same place as arguments passed explicitly
  3. You never have to find an argument for a parameter; you can look it up by index if we know what the declaration looks like

Doug

2 Likes
7

Hmm but this doesn't apply when the label and parameter name are specified separately, right? Are macros not allowed to do that?

8

Oh, wow, silly me. WHY DIDN'T WE BAN THIS. Thank you for the correction, I've made a note of it in my incorrect post above.

Doug

4 Likes
9

It has long felt a little weird to me that you could write a macro which had different behavior based on whether an argument was provided as a trailing closure or not. We may be too far down this road already, but it feels to me like it would be great if there were certain things in Swift that I could depend on being equivalent, and that macros wouldn’t break this mental model. I guess we could say that writing a macro that has subtly different behavior based on something like whether or not a closure is trailing is a bad macro, but I’d prefer if this was just impossible. That way we reduce mental overhead of parsing the syntax, which will likely increase as macros are used more broadly.
I think for this to work we would need a pre-macro “canonicalization” step that presents a simplified interface to the macro and ensures the macro can’t have different behavior based on things that shouldn’t cause different behavior.

1 Like
10

It was required for ObjC compat, but also we repeat the empty label all the time (and it’s nice to not make that a special case).

9 Likes
11

We had a similar discussion about trivia---should we normalize whitespace and comments, because it would be a bad macro that depended on it? We eventually decided not to do any normalization, allowing macros to look at the precise source text and make their own decisions. I think that logic holds here as well.

What do you think about the idea of supplying the "semantic" macro argument list to the macro implementation that's been discussed here? Macros could rely on that if they want a more semantic view, and don't care quite so much about the explicit syntax written by the user.

Doug

1 Like
12

While I didn't follow the trivia discussion, and it still feels like a bad complexity/capability tradeoff to make it possible for macros to do whatever they want, something like a semantic view makes sense to me.

How difficult would it be to have the ability to declare that your macro only utilizes the semantic view? This way you can at least see quickly if a macro can do something weird.

1 Like
13

What features or scenarios would be made more difficult by only supplying the semantic view?

14

My first idea was to also pass the syntax of the macro declaration

@freestanding(expression) 
macro m2(body: () -> Void, count: Int = 1)

to the macro implementation, similar to how the macro invocation syntax is passed.

I have the feeling that this could make other things simpler, too, but I am not sure what exactly you mean by "semantic argument list". How would this look like in practice?