Yes, that's exactly the problem we have if we try to use -gmodules today; since the paths in the breadcrumbs don't match because each machine has a module cache in a different location, we end up with a bunch of warnings about the modules not being present at link time, IIRC.
This is precisely the problem I'm attempting to solve. Since it turns out that the version of clang linked into swiftc isn't always the same version as the clang binary distributed in the same Xcode toolchain (there's usually a slight revision number drift), and matching up the exact invocation with ClangImporter internally would be difficult anyway, my goal is to implement a new swift driver mode that I can invoke and have it emit a PCM using the same clang invocation that it would later use to import it. I posted about it in some more detail a while back here—I need to update that thread since I've made slightly more progress recently.
So, my hope was that if I have a dependency graph of mixed Swift and Objective-C code, and my PCMs are explicit dependencies, then those breadcrumbs are now paths that are guaranteed to be valid (and remapped via -debug-prefix-map?) and LLDB would have enough information to reconstruct types without having to rely on knowing the full set of include paths and other flags that were present during the build. Since many of our projects are broken up into hundreds of modules, that set of flags becomes rather large.