You may be interested to read this: [cfe-dev] Apple's investment into Clangd and refactoring tooling
It sounds like Apple agrees with you, and is looking at using some variation of the Language Server Protocol in a future release of Xcode, so chances it will get a lot easier for other IDEs/editors such as Microsoft's Visual Studio Code to more fully support Swift.