Is this something that fits into the vision for macros? Expression syntax is a subset of pattern syntax in Swift (because every expression is considered a match-by-operator pattern if nothing else), so it wouldn’t have to be at odds with “you should be able to parse a macro without knowing the declaration for it”. Of course any particular macro would have to opt into it somehow. The immediate use case that made me think of it was Proposal draft for `is case` (pattern-match boolean expressions), which neither I nor anyone else have taken any further in the last several years, but surely there are other interesting ways to use this as well (does anybody have any?).
// Straw syntax
@freestanding(expression)
public macro is<T>(_ value: T, case pattern: @pattern T) -> Bool = …
print(#is(URL("https://swift.org"), case: .some(_)))
(I am not proposing this #is(_:case:) right now, I’m asking about macro expansion.)
Somewhat related: on numerous occasions I wanted to be able to pass syntax elements like identifiers directly into a macro, so that I wouldn't force the user to wrap what is supposed to be swift syntax into string literals, just so I can parse it back.
If macros have no access to type information and are stuck trying to deduce type relationships from syntax alone, then adding to it the limitation that the macro definition and invocation must be composed exclusively of fully typed expressions is further limiting what little freedom macros had to work with.
On the opposite end of the spectrum, if deterministic type relationships are desired, then I think macros should have access to type semantics on top of the bare syntax.
One example that comes to mind is recreating a completely compile-time version of SwiftUI's DynamicPropery lookup: I'd like to iterate over all properties of a struct whose type conforms to a protocol.
Another example: Iterate over the list of superclasses of a class and register them for some dynamic lookup mechanism.
Hmm… do you have the option today of using a macro expansion to iterate over all properties and then dynamically query through the runtime to determine if each type conforms to a protocol? Or are you looking for a way to leverage what the compiler knows about these types to keep things down in the macro level using compile-time information as much as possible without going through the runtime?
Option 2
Of course, I could just collect all properties into the body of a generated method and emit dynamic type casting to filter the unused ones, but this has significant runtime overhead, which is completely unjustified, given that the compiler knows the types at compile-time.
Another problem is that this runtime trickery can only be done on things that you know in advance (like a yes or no question, which a dynamic type cast is). What if I want to implement something like the StringInterpolation type or the DistributedActorTransport protocol, which can't be fully represented using protocol witness tables and have to check whether a method on a type is defined or not? I'd have to be able have my macro check if the type of the property has a specific method defined.
I was thinking along the lines of performing a dynamic lookup during the macro expansion… not in the output of the macro expansion. So the cost of the dynamic lookup could be paid at compile time… but the actual code generated from your macro expansion is correctly scoped to only the correct properties that should be allowed.
Another strategy along these lines I see shipping soon is to produce a "family" of functions with generic overloads consistent with what you want to filter down through your macro. At runtime you can pass your instance variables to this function and let the compiler choose the correct overload. Here is an example I see for Observation:
But how would you get to the semantic information during macro evaluation? Are you thinking of attempting to circumvent the limitations of the macro evaluation context by invoking external tools to dump the type information of all the involved modules and try to parse that from the macro?
Ooooooh, so the idea is (as I understand) to define something like this:
public func handleEachMember() {
/* Relies on @_transparent to ensure that there won't be any runtime overhead compared to compile-time type filtering. */
handleMember(\.one)
handleMember(\.two)
}
@_transparent private func $handleMember<Value>(
_ keyPath: <Self, Value>
) { /* ignore */ }
@_transparent private func $handleMember<Value: DynamicProperty>(
_ keyPath: <Self, Value>
) { /* handle it appropriately */ }
... right?
This is certainly very helpful in trying to mitigate some of the limitations, but it's a use-case specific workaround. I was wondering if there's a conceptual reason why compile-time type information cannot be made available to macros.
@jrose , I wonder if the whole concept of passing syntactic nodes into macros instead of typed expressions is something that fits within the vision for macros. There are a lot of instances of macro-like syntax in the compiler that take purely syntactic parameters.
My first thought was assuming a "happy path" where the types this macro might operate on are well defined types that we control at the level of this macro: types from standard library or types we defined in modules that can be imported without a circular dependency back to swift-syntax. But in a general case where this macro might operate on any type… then yeah I'm not sure I have a great answer for you other than the workaround from Observable.
For the most part… I think an idea like this might get you most of the way there. But the first gotcha I would want to keep in mind is situations where the code that calls your code comes from a context without the generic specialization to choose the correct overload. I saw something along these lines when trying to overload DynamicProperty.update with generic constraints… no matter what I tried the SwiftUI infra kept calling my unspecialized overload because SwiftUI didn't know what type it was. So I ended up with just one update with dynamic logic inside to switch over what would have otherwise been my "static" logic.
Today’s macros are syntactic, they do not get types, but the types are still useful because at the use site they inform completion and diagnostics.
I don’t want this thread to get sidetracked into “macros should have access to type information”, that would be a massive redesign that has been discussed plenty elsewhere. I do think macros that can use arbitrary syntax is an interesting idea, and agree that “pass your arbitrary syntax as a string” is pretty unsatisfying. But I also think that “macros accepting patterns” is a much less controversial idea because it keeps those diagnostic and completion benefits while still allowing macros to do more, and it also (as said originally) means a formatter or syntactic linter like swift-format can act on the code without having to assume anything about it, simply by always parsing macro arguments as patterns. That’s why I’d like to stay grounded here, though we could discuss both kinds of further expansion in other threads.
In the particular case of patterns I guess your strawman @pattern T syntax means "some pattern which could (refutably) match a value of type T, and doesn't bind any variables"?
It seems reasonable to me, and I definitely think it'd be useful — we have a hack in our testing utilities to implement XCTAssertCase(value, pattern) which would be much more cleanly done in this way, and I could definitely see swift-testing adopting it.
But like the others in this thread, my mind immediately goes to the future:
What if you did allow binding variables, that might be fine too, and the macro could do even more fun things as a result. Maybe the macro should be able to specify whether it wants to allow patterns that bind variables, so maybe the syntax needs to be extended to take some options?
What other bits of syntax does Swift have that are "close enough" to being able to be typed? Function declarations come to mind; maybe I should be able to write #forward(func foo(x: Int, y: String) throws(CancellationError) -> Double) where the macro looks like
@freestanding(expression)
public macro forward<each Arg, Failure: Error, Result>(
_ decl: @function_declaration (repeat each Arg) throws(Failure) -> Result
) = …
Which of course makes me wonder about suggesting @pattern specifically, because it seems like a pretty open-ended list of grammar rules that might be useful with such an extension to macro capabilities.
I agree that this would be a monumental change. It would make macros insanely powerful, but it would come at an equally insane price. I'd love to learn more about the pros and cons of this. If it truly isn't worth doing, then maybe there are alternatives that I could reach for instead.
This one always seemed more realistic and more inline with what macros are today. I understand the symbol lookup optimization and code completion requirements, but ultimately, being able to truly take over the parsing of an AST would make macros so much more powerful in the exact opposite direction from the aforementioned static type info scenario.
I'm a bit confused as to what exactly an @pattern is. It seems to be somewhere between a type and a syntax. Wouldn't it be more sensible to represent a pattern as a first-party type? If a hypothetical Pattern<T> existed in Swift, it would also demystify a lot of compiler magic. For instance, a switch statement would essentially be a highly optimizable syntax sugar over Pattern operations. Admittedly, I have a somewhat limited understanding of how type information is represented within the compiler, so what I'm saying could very well be complete nonsense. I'd love to hear a more educated take on the possibility of promoting language constructs like patterns to the type system level. Perhaps it could even serve as the pioneering effort into a possible future of fully syntactic macros.
macro expansion.)