There is a correlation between macro invocations and the multiple. For example, if you write a macro that takes no parameters and immediately returns an extremely simple output (#emptyString -> ""), then invoke it 2000 times in one file, you can get up to a 13X slowdown on an M1 Max.
@chiragramani has already covered this, but yes, if you build in release, you can roughly halve this overhead to a 6X slowdown.
This is about the simplest test I can think of of the raw overhead of macros; they are doing just about the minimum amount of work in the process itself that it's possible to do. If you or anybody else has a more minimal test they'd like to see feel free to describe it and we can give it a shot.
Also worth noting that we tried what was at the time the latest Swift 5.10 compiler shipping with the last Xcode 15.3 before the one that dropped yesterday, and there was no change.
I acknowledge that the design of Swift Macros has some inherent slowdowns in the name of security and compiler stability, and that those are at least good and worthy goals, perhaps even strictly necessary ones. But I think it is fair to say that when the cost is measured in the dozens of seconds, there is a project size threshold past which it's simply not feasible to adopt them, and we are hoping to see that threshold rise into the millions of lines of code.