Thanks for a tip! 
I've observed Binary packages quite a lot, even attempting to do some on-the-fly switching between source and binary packages possibility, still having one quirk there in Xcode 12.5 hopefully to be addressed in the next release. 
Yet I'd prefer a solution without SPM involved, like – having an option to disable that check would be completely okay and in sync with classic frameworks so I don't have to look for hacky or wrapping workarounds, moreover in a completely Swift-free project.