@compnerd raised this and other linux SDK layout issues a couple years ago, and I think some of those have since been fixed but not this one. I agree that we should place the Swift libraries in architecture-specific directories on all ELF platforms, since they don't have fat libraries.
I don't know if the Swift compiler already links against that architecture-specific path at compile-time as you say, but it certainly doesn't add that architecture-specific path to the RUNPATH invoked at runtime, so that will need to be changed too. This layout issue is the main reason I distribute three separate architecture-specific Android SDKs, rather than rolling them into one.
@3405691582, do you agree for OpenBSD too?