Discussion continued in SwiftPM slack and it quickly became apparent that this inability to easily install packages is a symptom of a much bigger problem with the Package.swift manifest.
I created a proposal that I think will solve the problem presented here by @calebkleveter and many other problems: Spm static dependencies