Swift-Testing Support for Macros?

Development Macros
1

Hi! I'm seeing what looks like false-positive results for assertMacroExpansion in swift-testing:

@Test func example() async throws {
#if canImport(MyMacroMacros)
assertMacroExpansion(
    """
    #stringify(a + c)
    """,
    expandedSource: """
    (c + d, "c + d")
    """,
    macros: testMacros
)
#else
throw XCTSkip("macros are only supported when running tests for the host platform")
#endif
}

Unless there is something wrong about my test code… this looks incorrect. This test "passes" from swift-testing and (correctly) fails with XCTest. Is this expected behavior (for now)? Does anyone know if we are expecting assertMacroExpansion to support swift-testing by the time 6.0 goes live in Q3?

2

assertMacroExpansion() currently requires XCTest. @ahoppen is working on adding Swift Testing support.

We are looking at potentially providing an overload of #expect() that you can use instead of this function, which might look something like:

#expect {
  #stringify(a + c)
} expandsTo: {
  (c + d, "c + d")
}

This may require compiler changes to allow macros to expand without first type-checking their arguments. @Douglas_Gregor would know more about what is or is not currently possible here.

2 Likes
3

It is exactly the problem I was trying to highlight in this thread: Challenges with `XCTAssert` in `swift-testing` and 3rd Party Libraries. It's hard to notice that the assertions you are invoking in your test methods are not being evaluated.

I wonder how to indicate that this method—assertMacroExpansion—has to be used only in XCTest land, and this one—#expect—only in swift-testing?
Shouldn't we go in the direction that one method will work in both lands?

4

There are arguments to be made in both directions. At this time, we aren't proposing anything specific.

5

The swift-syntax 600.0.0 prereleases contain a SwiftSyntaxMacrosGenericTestSupport modules that can be used to test macros using swift-testing.

Copying from the swift-syntax pending 600.0.0 release notes

  • SwiftSyntaxMacrosGenericTestSupport
    • A version of the SwiftSyntaxMacrosTestSupport module that doesn't depend on Foundation or XCTest and can thus be used to write macro tests using swift-testing . Since swift-syntax can't depend on swift-testing (which would incur a circular dependency since swift-testing depends on swift-syntax), users need to manually specify a failure handler like the following, that fails the swift-testing test: Issue.record("\($0.message)", fileID: $0.location.fileID, filePath: $0.location.filePath, line: $0.location.line, column: $0.location.column)
    • Pull request: #2647
5 Likes
6

We have no plans to add such a feature to macros at this time. It would be a bit of a philosophical shift for macros, so it's not a change we would make lightly.

Doug

5 Likes