Wow, thinking about performance seems like quite a luxury; I just hope we can get to correctness sometime soon.
The biggest problem SPM has right now, as far as I can tell, is that it's severely under-resourced. Just looking at the still-open issues I've filed, it looks like approximately one bug a week since mid-September, and several of the enhancement requests there are of the form “give me feature X so I can work around bug Y.” The impression I've gotten is that the team is too busy with… something else; I don't know what… to do more than speculate about possible causes for these problems. Actual diagnosis, to say nothing of fixing problems, appears to be out of reach. I do get the sense there are some serious architectural problems that prevent SPM from tracking dependencies properly, from working properly on Windows, etc., but those could all be addressed if the team were given the resources to do so.
Yeah, more developer control is crucial. The current SPM design approach makes it extremely difficult, and sometimes impossible, to work around a bug when you find one. For example, it appears -whole-module-optimization is broken but there's no supported way to turn it off in SPM and even though I know the sequence of swiftc commands that would be required to build my project in release, turning off WMO in the right place, there's no way to get SPM to issue those commands. Right now, I'm stuck. The ability to get in and adjust how these things are handled would save me.