After trying various things, I gave up. The ultimate goal would be to port more code over to Swift it seems. In our case, that is very unlikely for now.
In the end it seems I was left with two options:
Return the Mac Studio
Stick with it, hope for improvements in Xcode / macOS and spend more time optimizing current build times
Turning those frameworks into pre-compiled xcframeworks
Replacing SPMs in Xcode with xcframeworks (since Xcode would frequently re-resolve packages, re-compile them for no reason and whatnot - making matters worse, especially when switching between projects / builds / configurations etc)
By doing this, we were able to get our clean builds to finish under ~130 seconds compared to ~230 seconds (100 seconds reduction). Our incremental builds went from 35 seconds to around 15-20 seconds. Both the Studio and the MBP build in the same time (still).
I can only imagine how this would only get better over time if the Mac Studio eventually does what it's capable of doing.
I'm seeing the same thing. xcodebuild and swift build are consistently slightly more than twice as fast on my m1 ultra studio as on my baseline m1 laptop (i.e. the Ultra is about where a Max should be) on a variety of projects. A Cmake+Ninja project OTOH shows about a 3x difference between the two, and that's with a long non-parallelizable link at the end that's shrinking the difference.
Sadly neither xcodebuild nor spm support dry run so you can't (trivially) just invoke the relevant build commands via a non-broken scheduler. Maybe this is a reason to jump on the bazel train.
On the topic of EnableSwiftBuildSystemIntegration alone: as the name suggests, the benefits of it are in better build scalability for Swift projects/targets. So with this breakdown of source-code, it is unlikely to be of much significance. Even for pure Swift projects, its effect is heavily dependent on the shape of your target dependency graph. Some projects are already structured in a way where no additional parallelism can be extracted via better scheduling.