If your budget is high enough for it to matter, the new Mac Pro is supposed to come out next month. Or at least the web configurator will go live then... I don't recall whether it's actually shipping in December. Either way, though, if you're interested in working on a mac, you might want to wait and see how that machine benchmarks.
In general, building Swift scales very nicely with more cores and RAM but there are bottlenecks that may or may not matter to you. In particular, the standard library does not build concurrently. Maybe someday it will, but not today. If all you want to hack on is the standard library, then clock speed (and L3 cache size) matters more than cores for incremental rebuild performance.
I/O is really important. A full debug build of LLVM, Clang and Swift is > 50GB, and the executable outputs (like the compiler) are like 8GB or something all together. If you regularly change things in the compiler, many of those executables need to be linked every time (which is highly disk-bound, uses basically no CPU).
My old laptop from 2014 does a clean debug build in ~2.5 hours IIRC and incremental builds normally take about a minute (mostly linking), maybe 2 or 3 minutes tops.