A Possible Vision for Macros in Swift

The idea of AST to AST transformation by macro is excellent. It has very great potential. I want to show one practical example. I have created a proof of concept for the Power Assert library using currently publicly available code.

If you write a macro like the following:

let a = 4
let b = 7
let c = 12

#powerAssert(max(a, b) == c)
#powerAssert(a + b > c)

let john = Person(name: "John", age: 42)
let mike = Person(name: "Mike", age: 13)

#powerAssert(john.isTeenager)
#powerAssert(mike.isTeenager && john.age < mike.age)

It will generate the following code:

let a = 4
let b = 7
let c = 12

PowerAssert.Assertion(#"#powerAssert(max(a, b) == c)"#, line: 14).assert(max(a, b) == c).capture(expression: max(a, b), column: 13).capture(expression: a.self, column: 17).capture(expression: b.self, column: 20).capture(expression: max(a, b) == c, column: 23).capture(expression: c.self, column: 26).render()
PowerAssert.Assertion(#"#powerAssert(a + b > c)"#, line: 15).assert(a + b > c).capture(expression: a.self, column: 13).capture(expression: b.self, column: 17).capture(expression: a + b > c, column: 19).capture(expression: c.self, column: 21).render()

let john = Person(name: "John", age: 42)
let mike = Person(name: "Mike", age: 13)

PowerAssert.Assertion(#"#powerAssert(john.isTeenager)"#, line: 29).assert(john.isTeenager).capture(expression: john.isTeenager.self, column: 18).capture(expression: john.self, column: 13).render()
PowerAssert.Assertion(#"#powerAssert(mike.isTeenager && john.age < mike.age)"#, line: 30).assert(mike.isTeenager && john.age < mike.age).capture(expression: mike.isTeenager.self, column: 18).capture(expression: mike.self, column: 13).capture(expression: mike.isTeenager && john.age < mike.age, column: 29).capture(expression: john.age.self, column: 37).capture(expression: john.self, column: 32).capture(expression: mike.isTeenager && john.age < mike.age, column: 41).capture(expression: mike.age.self, column: 48).capture(expression: mike.self, column: 43).render()

Run this code produces the following output:

#powerAssert(max(a, b) == c)
             |   |  |  |  |
             7   4  7  |  12
                       false
#powerAssert(a + b > c)
             |   | | |
             4   7 | 12
                   false
#powerAssert(john.isTeenager)
             |    |
             |    false
             Person(name: "John", age: 42)
#powerAssert(mike.isTeenager && john.age < mike.age)
             |    |          |  |    |   | |    |
             |    false      |  |    42  | |    13
             |               |  |        | Person(name: "Mike", age: 13)
             |               |  |        false
             |               |  Person(name: "John", age: 42)
             |               false
             Person(name: "Mike", age: 13)

An executable project is available here.

Wouldn't it be great to use such rich assertions in the Swift testing framework? I look forward to a future where this kind of AST manipulation is in Swift.

Unfortunately, SwiftSyntax does not provide complete type information that does not appear in the source code, so there are some cases where code generation fails.

For example, the value cannot be captured if a type is omitted due to dot syntax or an anonymous closure argument is used.

It would be great if Swift macros could be passed the type information checked by the compiler as well as the source code structure to achieve this use case.

42 Likes

This is amazing! I would love to have these rich assertions for all of the tests I write. Thank you!

Yes, that's a great point. Since we are type-checking the arguments to a macro, it would be possible to annotate the various subexpressions with type information that could be provided to the macro. The big trick, as ever, is that we need to have a representation of the Swift type system that we can pass through.

Doug

17 Likes

+1 for the idea of figuring out how to get types of subexpressions over to macros.

However…. for a usecase like power asserts, maybe you could instead offer a way to get a fully-qualified name for references to a declaration? That seems like it would be easy to pass over a text-based protocol. No need to come up with a new format either, since it should all be spellable in Swift code.

I know we don’t really have fully-qualified names, but we can usually get pretty close to it in standard Swift.

This seems like a pretty useful feature for macros in general, regardless of type info.

Right, knowing which declaration a name refers to is another interesting bit of information that's determined by type checking and could be provided to macros.

Doug

Just following up, but with a few minor fixes to the compiler and pulling this into my local version of swift-syntax, it works as advertised:

% swiftc-dev -enable-experimental-feature Macros t.swift -sdk `xcrun --show-sdk-path -sdk macosx` && ./t
#powerAssert("hello".hasPrefix("h") && "goodbye".hasSuffix("y"))
             |       |         |    |  |         |         |
             "hello" true      "h"  |  "goodbye" false     "y"
                                    false

I did, however, change the signature and generic signature of the macro to the following:

  static var genericSignature: GenericParameterClauseSyntax? = nil
  static var signature: TypeSyntax = "(expression: Bool) ->   

In the near future, it should become possible to integrate these macros into a development compiler without having to rebuild the whole compiler, so folks can experiment more.

Doug

14 Likes

Hey all, I've pitched expression macros in another thread, the first proposal that's part of this vision.

Doug

9 Likes

I'd be interested in seeing more information provided in the MacroExpansionContext. When implementing a delegate method, I frequently find myself doing this:

func delegator(_ delegator: Delegator, willBegin operation: SomeOperation) {
    print(#function, delegator, operation)
}

func delegator(_ delegator: Delegator, didFinish operation: SomeOperation) {
    print(#function, delegator, operation)
}

This is a quick way for me to try out a new piece of code and quickly get a sense of the control flow between the delegator and the delegate. I was thinking that this would potentially be a nice macro:

func delegator(_ delegator: Delegator, willBegin operation: SomeOperation) {
    #printArguments
}

However, when I looked at the MacroExpansionContext, I didn't see a way I could implement this. Is there a way to get the containing scope and, if it's an argument-taking thing, get the list of arguments so I can splat out a print(...) call?

4 Likes

You can walk to the parent nodes of the MacroExpansionExprSyntax (just follow the “parent” property of each syntax node), and you’ll be able to get context information like that.

Doug

6 Likes

Ah, excellent! Sounds like that's sufficient already then :smile:

1 Like

If I were a wisenheimer, I'd say I want to write macros that generate macros.

Seriously, in my Swift programming, I often have to look at the build log to get enough information to just debug type issues. The compiler tries to output information, but Xcode doesn't reflect it back nicely. (And even more information would be better.) Point is, I would be very worried about the debug/error experience of macros unless there were enough resources available on the tooling side for really good support when things go wrong. The tooling effort is likely non-trivial.

7 Likes

Perhaps I missed it in the proposal or the thread, but would it be possible to use this system to capture the names of variables as they are declared? Or, to put it another way, to take some input in a macro and use it to declare the name of a let or var? I'm thinking of a macro I used in Obj-C in the past to declare views with accessibility identifiers to make debugging easier.

AXIDView(profileContainer);

This would be equivalent to:

UIView *profileContainer = [UIView new];
profileContainer.accessibilityIdentifier = @"profileContainer";

Here, I always want the accessibility identifier to be the same as the variable name so I can find it easily in the view debugger, layout constraint errors, or logs. I haven't thought much about it in SwiftUI, so I don't know if I would need an equivalent, but I believe I've wanted to be able to know the name of a variable at run-time in non-UIKit Swift code before.

3 Likes

In the system that I've described, all of the arguments to a macro need to be well-typed expressions before the macro will be expanded. That means profileContainer will need to already exist, so you won't be able to introduce it for the first time in AXIDView.

Now, it's possible that in the future we'll allow certain macro arguments to opt out of type checking, such they only need to be syntactically well-formed. Even so, we will want to be careful about allowing macros to declare new names, because that can have an impact on compile-time performance for incremental builds: specifically, we'd have to expand macros to figure out what names they declare, and those names could interfere with other non-macro code.

Doug

1 Like

Thanks. That makes sense. And one could presumably use the proposed macros to write something like autoAccessibilityID(someView), which is still an improvement.

This has probably been considered somewhere, but it just occurred to me that macros offer some interesting opportunities for interoperability with C++ templates. It's always nice when a C++ template can be imported as a first-class constrained generic, but without hints from the programmer, in general that can be arbitrarily hard… and likewise, writing those hints could be arbitrarily hard. Obviously using a template whose parameter is dependent on a Swift generic parameter would still erase type information, but it's something at least.

I know that folks who have been working with C++ templates from Swift have found that need to introduce a bit of boilerplate for each instantiation, and were hoping that macros could help. @ktoso might have some insight here.

Doug

Could you elaborate a bit? At least for function templates I don't see what macros get us that generic functions don't (maybe specialization/performance, but that can already be done with attributes).

More generally, I think most of the boilerplate and attributes are on the C++ side, not the Swift side.

What I'm hitting and hoping to use macros for is on the Swift side, but more because of needing to generate boilerplate "bridges" between concurrency models. I.e. some C++ code using their own concurrency things and needing to call into Swift async functions; so I want to generate a small (and very repetetive) "bridge from C++ futures, to async function" method declarations. Not really much to do with C++ templates.

Usually it looks like this:

@_expose(Cxx)
actor Something { 
  func doIt() -> X { X() }

  nonisolated public func doIt(promise: PromiseX) {
    Task { promise.completeWith(await self.doIt) }
  }
}

There is also annoying boilerplate on the C++ side because templates... since we can't use Promise<X> and have to do this:

using PromiseX = Promise<X>;

in order for Swift to be able to use this specialized template... I don't know if macros could help get rid of this annoyance, probably not.

We were thinking about “universal references” over in Val land, and examples like this one seemed particularly difficult. How are we going to represent g's function call operator to Val? If we had macros, and were to import all C++ templates into Val as macros with macro expansion defined to instantiate the template, then at least in concrete code, uses of g would do what is clearly the right thing, and the template would be represented to Val not as a foreign kind of entity (imported C++ template) but as a Val macro that, at least from the outside, obeys the usual language rules.

Getting g to do what a C++ programmer expects from inside a Val generic, where the argument to g is (dependent on) a generic parameter, is a harder problem that would require monomorphization of Val generics and delayed macro expansion. So I'm not saying mapping templates to macros is a slam dunk, but it's worth investigating

Anyway, it seems like all the same reasoning applies to Swift. I guess what I'm saying is that templates are in many ways more like macros than they are like Swift generics, and, if you're going to have macros, trying to squeeze them into the mold of generics might be a losing proposition.

(As discussed in previous threads, I'll use the term monomorphization rather than specialization here to prevent ambiguity even though specialization is often the more common term when discussing C++ templates.)

I may be missing what you are proposing here, but I generally disagree. While I do like to (mostly facetiously) question the difference between templates and C++ macros, I don't see how Swift macros will be helpful in the import of templates.

There are a number of problems when importing templates into Swift, and I think monomorphization is near the bottom of the list. More importantly, Swift already has the infrastructure in place to monomorphize any imported C++ template (or Swift generic for that matter). The problem here is a more fundamental one: the places where templates cannot be monomorphized arise at module boundaries where we decide not to include templates definitions in the module (a mostly artificial limitation that would be relatively easy to overcome), and potentially deep in generic code where, again, we impose an artificial limitation to prevent substitution failures (and other unfortunate, template-related diagnostics) from leaking into Swift.

To put this a different way, macros work at a syntactic level (hence swift syntax), the issues we run into with importing templates are almost exclusively semantic. I could see creating a macro to assist in explicit monomorphization of templates (applying a substitution map to a call), but that is not a problem that needs solving. Macros are helpful when you need custom syntax but cannot modify the compiler. C++ interop is part of the compiler, so we don't need to define a macro for our monomorphization syntax (we can just modify the compiler). Swift already has a tool for constructing and applying substitution maps, and we can leverage this existing infrastructure which will result a more native experience anyway.

The semantic challenges with importing C++ templates are real, though, and we've discussed them at great length over the forums and in the C++ interop work group. If you see a way for macros to assist in type checking, etc. I'd be very interested to hear.

1 Like

To address this concrete suggestion, Swift macros still need to be type checked, so all the problems with type checking that arise when wrapping a C++ template in a Swift generic apply here as well (at least as far as I understand, but maybe I'm mistaken).