`swift test` tries to build all targets instead of just those needed for testing

[wes1 (Wes)]

@Geordie_J

We can’t put those flags into Package.swift specific to those targets, because they require unsafe flags, which from my understanding would make our package unconsumable from other packages

I haven’t checked, but wasn’t this fixed in Swift 6.2?

• discussion https://forums.swift.org/t/pitch-disable-checks-for-unsafe-flags-in-swiftpm/80698
• PR: https://github.com/swiftlang/swift-package-manager/pull/8951