Swift-Testing Support for Macros?

Development Macros
1

Hi! I'm seeing what looks like false-positive results for assertMacroExpansion in swift-testing:

@Test func example() async throws {
#if canImport(MyMacroMacros)
assertMacroExpansion(
    """
    #stringify(a + c)
    """,
    expandedSource: """
    (c + d, "c + d")
    """,
    macros: testMacros
)
#else
throw XCTSkip("macros are only supported when running tests for the host platform")
#endif
}

Unless there is something wrong about my test code… this looks incorrect. This test "passes" from swift-testing and (correctly) fails with XCTest. Is this expected behavior (for now)? Does anyone know if we are expecting assertMacroExpansion to support swift-testing by the time 6.0 goes live in Q3?

2

assertMacroExpansion() currently requires XCTest. @ahoppen is working on adding Swift Testing support.

We are looking at potentially providing an overload of #expect() that you can use instead of this function, which might look something like:

#expect {
  #stringify(a + c)
} expandsTo: {
  (c + d, "c + d")
}

This may require compiler changes to allow macros to expand without first type-checking their arguments. @Douglas_Gregor would know more about what is or is not currently possible here.

3 Likes
3

It is exactly the problem I was trying to highlight in this thread: Challenges with `XCTAssert` in `swift-testing` and 3rd Party Libraries. It's hard to notice that the assertions you are invoking in your test methods are not being evaluated.

I wonder how to indicate that this method—assertMacroExpansion—has to be used only in XCTest land, and this one—#expect—only in swift-testing?
Shouldn't we go in the direction that one method will work in both lands?

4

There are arguments to be made in both directions. At this time, we aren't proposing anything specific.

5

The swift-syntax 600.0.0 prereleases contain a SwiftSyntaxMacrosGenericTestSupport modules that can be used to test macros using swift-testing.

Copying from the swift-syntax pending 600.0.0 release notes

  • SwiftSyntaxMacrosGenericTestSupport
    • A version of the SwiftSyntaxMacrosTestSupport module that doesn't depend on Foundation or XCTest and can thus be used to write macro tests using swift-testing . Since swift-syntax can't depend on swift-testing (which would incur a circular dependency since swift-testing depends on swift-syntax), users need to manually specify a failure handler like the following, that fails the swift-testing test: Issue.record("\($0.message)", fileID: $0.location.fileID, filePath: $0.location.filePath, line: $0.location.line, column: $0.location.column)
    • Pull request: #2647
5 Likes
Make SwiftPM run XCTest and Testing tests in parallel by default?
6

We have no plans to add such a feature to macros at this time. It would be a bit of a philosophical shift for macros, so it's not a change we would make lightly.

Doug

5 Likes
7

Does swift testing support Macros at this point? Can’t seem to find any documentation on it

8

There haven't been any changes here since the thread dropped off. API for testing macro expansions would need to be part of the swift-syntax package (which already provides some testing-library-agnostic building blocks for you.)

1 Like
9

I think it depends on what you mean by "support Macros".

If you want to use SwiftSyntaxMacrosGenericTestSupport today to test macros from swift-testing this is supported. Here is an example of what this might look like as a template from SPM:

You can also find SwiftSyntaxMacrosGenericTestSupport examples in the Equatable package from Ordo One:

As discussed previously there is a circular dependency that means you have to "roll your own" failure handler… but you can do this in just one place and your module that depends on swift-testing and swift-syntax can define that for all your tests to import.

10

There is also a full featured macro testing tool we have built here: GitHub - pointfreeco/swift-macro-testing: Magical testing tools for Swift macros.. It works in XCTest and Swift Testing, and It automatically expands macros for you inline:

It also supports diagnostics (warnings, errors and fixits):

5 Likes