You still seem to be missing the point. None of us want to support older versions of Swift. We have to support older versions of Swift because Apple doesn't support older macOS versions. If macOS had the same level of support as Linux, I think virtually everyone would be willing to upgrade almost immediately, baring any show stopping bugs (which Apple's release cadence also exacerbates). Few people are refusing to upgrade their tooling, they simply can't.
You also really don't need to keep explaining how SPM works here. Not only do we know, but it's also a narrow view of the Swift community, given the continued use of CocoaPods and Carthage. Those tools exist, are important, and still fill niches SPM just can't yet.
In the end, the community's concern here is that, while we love new Swift features, especially features with such high user value, there are really no affordances for libraries to adopt the features while still maintaining their user base. This includes things like source compatibility, upgrade automation, OS compatibility, and release cadence.