SE-0382: Expression Macros

[Douglas_Gregor]

@s-k

Accessing the nearest definition would help. However, if #printArguments also wanted to print the type name of self in case of a method, this would not be enough.
[snip]
There are some solutions I can think of. I am not sure if any of them are feasible:

• Macros could declare the need to access the parent nodes, disabling incremental builds.
• Maybe the compiler could provide some sort of declaration hierarchy to the macro, such as name, signature and attributes of the enclosing function, then attributes, type (class, struct, enum, etc.), name and inheritance of the type containing the function, maybe even the type containing this type, etc. This would solve example 3 only in part.
• Maybe the whole syntax tree could be cached and the relevant parts be updated during incremental compilation.

I don't think the third option is really feasible, because it means that macro implementations would become much more deeply entwined in incremental compilation. And I worry that the first option is an anti-feature: one errant macro could blow up your incremental build times.

I suspect the middle bullet might lead toward the right answer. From an incremental build standpoint, if any if your parents changes directly---say, you're in a function that adds a new parameter, or an extension that gets a new conditional requirement---you're going to have to get recompiled no matter what. So what if we kept the "spine" on the parent nodes in tact, but removed all child nodes that could be independent? So if we started with this code:

struct A<T> { }
extension A where T: Addable {
  func f() { }
  func g(a: Int, b: Int) {
    let a = 1
    let b = #myMacro + a
  }
}

we would give the implementation of myMacro something like this:

extension A where T: Addable {
  func g(a: Int, b: Int) {
    #myMacro
  }
}

this would let you ask syntactic questions about the parents of your macro expansion, without exposing parts of the source file that should be able to evolve independently of the code seen by the macro implementation. Now, this isn't going to solve all of the problems you enumerated. For example, you won't know whether A is a class, struct, protocol, or whatever, and you won't know whether self is accessible at all, but it does provide quite a bit of context.

@s-k

It would be unfortunate if expression macros would miss out on this information only because they were proposed and accepted before the other macro types.

I went ahead and write up a pitch for declaration macros so we can explore further down the path and, hopefully, learn more. That said, the Language Working Group (and in the past, the Core Team) has been clear that for large features that are split across several different proposals (like this one), it's absolutely willing to go back and review amendments to accepted proposals if we learn more while considering later proposals. We did this with concurrency two years back, and regular expressions last year, and will do the same for macros.

@j-f1

Something that I’m noticing in all of the example macros is a pervasive use of (macro-)runtime verification of arguments. I feel like while doing all this checking explicitly in the macro body allows for maximum flexibility in macro interfaces, there could be some benefit to offering an alternative, more-strongly-typed API.

Yes, I think we can automate away a lot of the boilerplate checking of macro arguments. However, it won't be done by the compiler, because the compiler isn't going to be able to form calls to arbitrary functions like this when invoking the macro. Instead, I would view this as some improvements we can make to the SwiftSyntaxMacros library itself. For example:

protocol SingleArgumentExpressionMacro: ExpressionMacro {
  associatedtype ArgumentSyntax: ExprSyntaxProtocol
  static func expansion(of node: MacroExpansionExprSyntax, in context: inout MacroExpansionContext,
                                     argument: ArgumentType) throws -> ExprSyntax
}

extension ExpressionMacro where Self: SingleArgumentExpressionMacro {
  static func expansion(of node: MacroExpansionExprSyntax, in context: inout MacroExpansionContext)
      throws -> ExprSyntax {
    guard let arg = node.argumentList.first?.element?.as(ArgumentSyntax.self) else {
      throw BadArgumentError()
    }
    return try expansion(of: node, in: context, argument: arg)
  }
}

If we had associated type packs, we could make this handle an arbitrary number of arguments, which would be... amazing. Anyway, I think this is a good direction, and it's something we should pursue within SwiftSyntaxMacros. However, I don't think these improvements should be part of this proposal, or even part of the evolution process... we should treat it like any other API improvements and extensions to swift-syntax, through pull requests and discussion there.

@soumyamahunt

But what about scenario where I might want to construct type based on the input provided, i.e. building a type from json string:

macro jsonObj<T>(_: String) -> T

let obj = #jsonObj("""
{
"one": 1,
"two": 2
}
""")

I suppose it depends on what you want to do with the type. You could declare some local types, form a result from that, and have jsonObj return an any <some protocol> to hide the returned type. If you want to actually create types that are usable elsewhere and aren't erased, you'll need something more like my follow-up pitch for declaration macros.

@soumyamahunt

Or registering routes in a server framework:

app.get("hello/:name") { req in
    // req could be inferred from the provided string as
    // SomeGenericStruct<(name: String)>
    let name = req.parameters.name
    return "Hello, \(name)!"
}

This, again, might be something that would need declaration macros, if the goal is to have the macro parse the string and produce new type information for it. Alternatively, and I suppose I could cover this as a future direction, I could imagine an annotation for a macro parameter that says "arguments to this parameter are not type checked at all". Arguments to such parameters would lose all of the benefits discussed in the section on type-checking arguments, but it would let you form more-specific type information for use in checking those arguments later. I'm not sure how I feel about such a feature, but it is an extension that could fit into the model.

@Max_Desiatov

Here's one use case I'd like to highlight. With the example repository I've been able to implement a compile-time conversion from string literals to arrays of bytes.

Neat! Thank you.

@Douglas_Gregor

@Snowy1803

Is there a reason to not have this: all expression macros use the function-like syntax in their declaration, and when invoking the macro, the argument clauses and trailing closures can be omitted when no arguments are specified.
This would be similar to how property wrappers work today.

I certainly appreciate that this gets rid of the -> vs : distinction in the grammar, so it feels simpler. It means that a no-parameter macro can be expanded with or without the (): but it's not ambiguous (as it is with functions), that's not going to change, and we've already done this now with property wrappers, which are similar in nature to macros.

Alright, I think I like it! Thank you.

I'm going to backtrack a bit on my earlier enthusiasm for this change. Macros are so much more expression-like that allowing one to omit the () for macros, when it has a different behavior for functions, that introducing an inconsistency here feels worse than having macro authors decide between : and ->.

Doug