SE-0382: Expression Macros

[Douglas_Gregor]

@s-k

I finally came around to working some more on the result builder macro.

Very cool!

@s-k

Is there any chance that macros could return an opaque result type? I think this feature would be very valuable.

I can certainly see how it would be valuable. Opaque result types as they exist today for functions/properties/subscripts have a notion of identity that is tied to the function/property/subscript returning them, where the identity of the opaque type is determined by that function/property/subscript and any generic arguments to it. That notion of identity doesn't work for macros, because macros get the source code of the arguments and everything that you can access via the macro expansion context, so there's no way to establish identity. That's the logic I followed in banning opaque result types for macros.

However, we could say that every use of a macro that has an opaque result type produces a completely unique opaque type, similar to what happens when we open an existential. From a compiler perspective, we know the opaque result type after macro expansion.

That's all a very long-winded way to say that I think we can lift the restriction, and allow opaque result types for macros. @Joe_Groff does the above seem plausible to you?

@s-k

1. Making type inference smart enough to infer the return type of the closure if the if false { ... } is dropped.

I think this is my preferred solution, which we've talked about at various times as being a "join" of the types of each of the returns.

@s-k

If a user of the macro makes a mistake, the error produced is not so great. One example would be the error Static method 'buildBlock' requires that 'Int' conform to 'View'. The message can be helpful if the user knows a little about result builders. However, with the current implementation the error does not point to any code location. So the user must look through every line of code to find the error. I think the compiler could be improved to point to the location of the macro. However, it could not know, which subexpression is at fault.

The compiler model here does have some of this information, but it's not being surfaced well to users. When a macro is expanded, the compiler creates an internal buffer containing the source code produced by the macro implementation. If something goes wrong when type-checking that code, the compiler will print an error message pointing into that buffer, with a follow-up note showing the location where the macro expansion was triggered. The result looks something like this (extracted from a compiler test case):

macro:addBlocker:/swift/test/Macros/macro_expand.swift:81:7-81:27:1:4: error: binary operator '-' cannot be applied to two 'OnlyAdds' operands
oa - oa
~~ ^ ~~
/swift/test/Macros/macro_expand.swift:81:7: note: in expansion of macro 'addBlocker' here
  _ = #addBlocker(oa + oa)
      ^~~~~~~~~~~~~~~~~~~~

The addBlocker macro is my silly example which replaces + with -, so the oa - oa code is the result of macro expansion of #addBlocker(oa + oa). That weird macro:addBlocker: file name is the name of the internal buffer, showing that the macro-expanded source code is there in a compiler, but it's only showing one line of it. We can do better here, for example by leveraging the nifty formatter in swift-syntax to show more of the macro expansion buffer in the compiler's output. (This idea just came to me; I haven't tried it yet)

If you're using Xcode, you'll have to look into the build log to see the compiler output. Clang and Swift use a different serialized format for diagnostics when talking to IDEs, so all of this information is lost right now. I have an RFC on the LLVM Discourse to extend that serialized format to include macro-expansion buffers. Swift's compiler already implements it, but IDEs would need to pick up the new APIs to provide a good experience here.

@s-k

The current implementation produces an error if a macro wants to return a closure whose return type depends on a generic argument:

That looks like a bug in the compiler implementation, rather than a limitation of the design.

@s-k

Is this the expected behavior or just a limitation of the current implementation? I would expect overloading macros to work just like overloading functions.

It's a limitation of the current implementation, thanks!

@s-k

All in all, I think the general directions laid out in this proposal is great and I am surprised that I came this far with the proposed feature.

Thank you for diving deep into result builders-via-macros! It's great to see what can be achieved with macros, what is still out of reach, and where the pain points are.

@s-k

@Douglas_Gregor I would appreciate it if you could respond to the questions in my previous post before the review deadline. I would especially want to know if the problems I encountered were bugs or features. Then we will be able to discuss the implications of that. Thank you.

Sorry 'bout the delay. Answers above.

@s-k

And there is one more concern I want to mention: Up until recently, if you wanted to use SwiftSyntax in a library (e.g. for code generation in a swift package plugin), you had to fulfill one of these requirements:

1. Depend on the exact SwiftSyntax version matching the toolchain of the library user. This is not feasible because different users use different versions of Xcode, etc.
2. Supply a dylib for every toolchain-platform combination with the library.

Is this still a requirement?

Nope! With the advent of the new Swift parser, the swift-syntax package is all-Swift and fully standalone.

Doug