Without undermining your statement, I do think this is a localized problem.
For example, right now I'm working with a relatively large SwiftUI (heavy type inference) codebase and the 5.7 toolchain. I really can't complain about the compile time. Even when compiling from 0, due to the numerous Xcode problems with reporting warnings/errors that often require a clean, performance is good, under a minute on a M1 machine.
You might be facing other issues, perhaps even unrelated to the compiler, if you can see the performance degrade when adding one single file.
Nuke the whole thing from orbit. I understand that it was built to handle the 0.001% edge cases in Unicode, but it made the 99.999% of normal cases stupidly painful.
Pointers
Swift successfully made pointers as difficult as every first-semester CS student always thought they were. There’s 987 different kinds, the specific kind of pointer you need to work with whatever C library you’re trying to call is NEVER the kind you have, and thanks to Satan’s type checker, you can’t just drop down to raw, stupid pointers.
Again, I understand the reasoning: Swift wants to solve buffer overflows. I get it. But there are days I would LAUNCH CHRIS LATTNER’S CAR INTO THE SUN if I could just have my simple * operator back.
OK I think this thread has run its course. It's been open for a year and gets bumped to the top of the forum list every few days, but the comments aren't bringing anything new and some of them are getting unpleasant (thank you for community members who are flagging CoC violations btw).
New threads on specific areas that can be improved are welcome, especially with constructive thoughts on how.