[Solved] assertMacroExpansion() reports my macro has invalid syntax but it works in Xcode

Using Swift
1

I write a public memberwise init macro of my own version (it's similar to those availbale on the net). It works well in Xcode, but when I add a test for it, assertMacroExpansion() reports a syntax error (not a diff error). Below are the details and my analysis.

The test:

func testMemberwiseInit() {
        assertMacroExpansion(
            """
            @mInit
            public struct Foo {
                var x: Int
                var y: String
            }
            """,
            expandedSource:
            """
            public struct Foo {
                var x: Int
                var y: String
                public init(x: Int, y: String) {
                    self.x = x
                    self.y = y
                }
            }
            """,
            macros: testMacros
        )
    }

The error message:

failed - Expanded source should not contain any syntax errors, but contains:
3 │ var x: Int
4 │ var y: Stringpublic init(x: Int, y: String) {self.x = x
5 │ self.y = y}
│ ╰─ error: unexpected code 'self.y = y' in initializer
6 │ }

The error message also includes the expanded syntax tree, which shows there is only one statement in init body and hence the syntax error (see unexpectedAfterElements node at the end of the log).

Expanded syntax tree

Expanded syntax tree was:
SourceFileSyntax
├─statements: CodeBlockItemListSyntax
│ ╰─[0]: CodeBlockItemSyntax
│ ╰─item: StructDeclSyntax
│ ├─modifiers: ModifierListSyntax
│ │ ╰─[0]: DeclModifierSyntax
│ │ ╰─name: keyword(SwiftSyntax.Keyword.public)
│ ├─structKeyword: keyword(SwiftSyntax.Keyword.struct)
│ ├─identifier: identifier("Foo")
│ ╰─memberBlock: MemberDeclBlockSyntax
│ ├─leftBrace: leftBrace
│ ├─members: MemberDeclListSyntax
│ │ ├─[0]: MemberDeclListItemSyntax
│ │ │ ╰─decl: VariableDeclSyntax
│ │ │ ├─bindingKeyword: keyword(SwiftSyntax.Keyword.var)
│ │ │ ╰─bindings: PatternBindingListSyntax
│ │ │ ╰─[0]: PatternBindingSyntax
│ │ │ ├─pattern: IdentifierPatternSyntax
│ │ │ │ ╰─identifier: identifier("x")
│ │ │ ╰─typeAnnotation: TypeAnnotationSyntax
│ │ │ ├─colon: colon
│ │ │ ╰─name: identifier("Int")
│ │ ├─[1]: MemberDeclListItemSyntax
│ │ │ ╰─decl: VariableDeclSyntax
│ │ │ ├─bindingKeyword: keyword(SwiftSyntax.Keyword.var)
│ │ │ ╰─bindings: PatternBindingListSyntax
│ │ │ ╰─[0]: PatternBindingSyntax
│ │ │ ├─pattern: IdentifierPatternSyntax
│ │ │ │ ╰─identifier: identifier("y")
│ │ │ ╰─typeAnnotation: TypeAnnotationSyntax
│ │ │ ├─colon: colon
│ │ │ ╰─type: SimpleTypeIdentifierSyntax
│ │ │ ╰─name: identifier("String")
│ │ ╰─[2]: MemberDeclListItemSyntax
│ │ ╰─decl: InitializerDeclSyntax
│ │ ├─modifiers: ModifierListSyntax
│ │ │ ╰─[0]: DeclModifierSyntax
│ │ │ ╰─name: keyword(SwiftSyntax.Keyword.public)
│ │ ├─initKeyword: keyword(SwiftSyntax.Keyword.init)
│ │ ├─signature: FunctionSignatureSyntax
│ │ │ ╰─input: ParameterClauseSyntax
│ │ │ ├─leftParen: leftParen
│ │ │ ├─parameterList: FunctionParameterListSyntax
│ │ │ │ ├─[0]: FunctionParameterSyntax
│ │ │ │ │ ├─firstName: identifier("x")
│ │ │ │ │ ├─colon: colon
│ │ │ │ │ ├─type: SimpleTypeIdentifierSyntax
│ │ │ │ │ │ ╰─name: identifier("Int")
│ │ │ │ │ ╰─trailingComma: comma
│ │ │ │ ╰─[1]: FunctionParameterSyntax
│ │ │ │ ├─firstName: identifier("y")
│ │ │ │ ├─colon: colon
│ │ │ │ ╰─type: SimpleTypeIdentifierSyntax
│ │ │ │ ╰─name: identifier("String")
│ │ │ ╰─rightParen: rightParen
│ │ ╰─body: CodeBlockSyntax
│ │ ├─leftBrace: leftBrace
│ │ ├─statements: CodeBlockItemListSyntax
│ │ │ ╰─[0]: CodeBlockItemSyntax
│ │ │ ╰─item: SequenceExprSyntax
│ │ │ ├─elements: ExprListSyntax
│ │ │ │ ├─[0]: MemberAccessExprSyntax
│ │ │ │ │ ├─base: IdentifierExprSyntax
│ │ │ │ │ │ ╰─identifier: keyword(SwiftSyntax.Keyword.self)
│ │ │ │ │ ├─dot: period
│ │ │ │ │ ╰─name: identifier("x")
│ │ │ │ ├─[1]: AssignmentExprSyntax
│ │ │ │ │ ╰─assignToken: equal
│ │ │ │ ╰─[2]: IdentifierExprSyntax
│ │ │ │ ╰─identifier: identifier("x")
│ │ │ ╰─unexpectedAfterElements: UnexpectedNodesSyntax
│ │ │ ├─[0]: keyword(SwiftSyntax.Keyword.self)
│ │ │ ├─[1]: period
│ │ │ ├─[2]: identifier("y")
│ │ │ ├─[3]: equal
│ │ │ ╰─[4]: identifier("y")
│ │ ╰─rightBrace: rightBrace
│ ╰─rightBrace: rightBrace
╰─eofToken: eof

But I don't understand how that could occur. My macro implementation does adds a \n character when generating the code. And if it was a bug in my code, the diff check in assertMacroExpansion() would fail in the first place, because I define the expandedSource param in my test as below:

            """
            public struct Foo {
                var x: Int
                var y: String
                public init(x: Int, y: String) {
                    self.x = x
                    self.y = y
                }
            }
            """,

Since the diff check passes, how can the expanded syntax tree in the log is like the above?

So I suspect it's not an issue with my code. However, I did an experiment by downloading other people's implementation on the net and added the test to it. The test passed. So it seemed to suggest the odd issue is specific to my code.

I wonder if anyone have a possible explanation on this odd behavior? I googled but can't find any report of the issue. My environment: macOS 13.4, Xcode 15.0 beta (the first beta release). Thanks.

2

How are you generating the body of your initializer? It's hard to see where the unexpected nodes are coming from without seeing that code. One guess is that a CodeBlockItem can only have a single statement in it, so if you're trying to generate multiple statements in a single code block item, that could end up giving you everything after the first statement as unexpected nodes. Instead, each assignment you generate needs to be its own CodeBlockItem.

The reason this "works" in Xcode is because all the compiler sees is the final source text, which the macro plugin renders from your source tree. But even unexpected nodes maintain any whitespace they had surrounding them when they were initially parsed via the string interpolation, so in this case it just happens to render as something that looks like valid code, and that's what the compiler processes. But it's still surfacing a bug in the macro implementation that needs to be addressed.

3

Thanks. I'm still digesting your explanation. The following is my code (sorry I didn't post it because I thought it's unlikely a code issue). I use InitializerDeclSyntax to generate init body and don't deal with CodeBlockItem explicitly.

public struct MInitMacro: MemberMacro {
    public static func expansion(
      of node: AttributeSyntax,
      providingMembersOf declaration: some DeclGroupSyntax,
      in context: some MacroExpansionContext
    ) throws -> [DeclSyntax] {
        let storedProps = declaration.as(ClassDeclSyntax.self)?.storedProps() ??
        declaration.as(StructDeclSyntax.self)?.storedProps()
        
        guard let storedProps else { return [] }
        
        let paramTuples = storedProps.compactMap { property -> (name: String, type: String)? in
            // This doesn't work for "var x, y: Int". 
            guard let patternBinding = property.bindings.first?.as(PatternBindingSyntax.self) else { return nil
            }
            
            // This doesn't work for implicit type.
            guard let name = patternBinding.pattern.as(IdentifierPatternSyntax.self)?.identifier,
                  let type = patternBinding.typeAnnotation?.as(TypeAnnotationSyntax.self)?.type else {
                return nil
            }
            
            return (name: name.text, type: type.description)
        }

        let _paramLiteral = paramTuples.map { "\($0.name): \($0.type)" }.joined(separator: ", ")
        let paramLiteral = "public init(\(_paramLiteral))"
        
        let _bodyLiteral = paramTuples.map { "self.\($0.name) = \($0.name)" }.joined(separator: "\n")
        let bodyLiteral: ExprSyntax = "\(raw: _bodyLiteral)"
        
        guard let initDeclSyntax = try? InitializerDeclSyntax(
                    PartialSyntaxNodeString(stringLiteral: paramLiteral),
                    bodyBuilder: { bodyLiteral }),
              let finalDeclaration = DeclSyntax(initDeclSyntax) else {
            return []
        }
        
        return [finalDeclaration]
    }
}
4

I'm afraid that I have some difficulty in understanding what you said. I think assertMacroExpansion() works this way:

  • It first expands my macro and gets the following code:

    public struct Foo {
    var x: Int
    var y: String
    public init(x: Int, y: String) {
        self.x = x
        self.y = y
    }
}

  • Then it checks if the code is vallid by parsing the code to generate syntax tree.

The fact that step 1 succeeded (otherwise it should report a diff error, instead of a syntax error) means my macro works correctly? I guess I must be misunderstanding something, but not sure what it is.

EDIT: Please ignore the above part. I figure out where my understanding is incorrect. My macro code calls swiftsyntax API to generate code. So assertMacroExpansion() (swiftsyntax actually) has syntax tree first, and generate the code text based on that tree. So the above code text is output, instead of input. Based on this understanding your explanation makes sense indeed.

There are two further questions, however:

  1. How is the syntax error reported by assertMacroExpansion() generated? Is it by some SwiftSyntax built-in API or by assertMacroExpansion()? If it's by SwiftSyntax built-in API, shouldn't it fails in Xcode also?

  2. I still can't see what's the issue in my code. I'll take a look at how other people does it.

5

I believe this is the problem; you're trying to create a single ExprSyntax from multiple independent statements. Instead, each self.SOME_PROPERTY = SOME_PROPERTY assignment should be its own ExprSyntax.

6

Thanks! That makes sense. I modified the code to generate init() using a more straightforward approach. And the test works now.

The diff:

         let _paramLiteral = paramTuples.map { "\($0.name): \($0.type)" }.joined(separator: ", ")
-        let paramLiteral = "public init(\(_paramLiteral))"
         
         let _bodyLiteral = paramTuples.map { "self.\($0.name) = \($0.name)" }.joined(separator: "\n")
-        let bodyLiteral: ExprSyntax = "\(raw: _bodyLiteral)"
         
-        guard let initDeclSyntax = try? InitializerDeclSyntax(
-                    PartialSyntaxNodeString(stringLiteral: paramLiteral),
-                    bodyBuilder: { bodyLiteral }),
-              let finalDeclaration = DeclSyntax(initDeclSyntax) else {
-            return []
+        let initDeclSyntax: DeclSyntax = """
+        public init(\(raw: _paramLiteral)) {
+        \(raw: _bodyLiteral)
         }
         
-        return [finalDeclaration]
+        return [initDeclSyntax]
1 Like