I think the pitch makes sense at a high-level.
I think this will need a lot more concrete design, potentially with different merging strategies based on the type of settings. For example, Xcode has a pretty sophisticated model here, with multi- and single-value settings, $(inherited) etc -- a lot of the cases that solves will likely come up in a multi-layer build settings model for SwiftPM as well.
There's also the question of how to address potential ambiguity with swiftLanguageVersion (see SE-0435) since that would also be part of SwiftSetting now, but there's an existing package level property for this already.