How coupled is SourceKit-lsp to Swift Package Manager?

I've worked on a variety of projects (like Swift itself!) that, for various reasons, can't be built with Swift Package Manager, but where it would be super-nice to have LSP support. Doesn't seem so hard in principle, but you never know. So before I start crawling through the source code, I thought I'd ask here:

  • What services is SPM providing to SourceKit-lsp?
  • Would maintainers be open to patches that open SourceKit-lsp up to other build systems?
  • Any suggestions about where to start?

TIA,
Dave

4 Likes

Hey Dave,

In addition to swiftpm, we currently have two other methods to communicate with a build system.

  1. Clang's json compilation database format (compile_commands.json). This is what I use for working with C++ code in cmake projects like swift/llvm. Cmake generates the file, and I symlink it into my source directory so the tool can find it. Compilation databases can also be used for Swift code (e.g. we use it for our tests), but I don't know of any tools that will generate it automatically right now for Swift code. We talked to compnerd about doing this for the Swift support in cmake, but never came to a design that everyone was fully happy with and it hasn't been on top of anyone's priorities yet.

  2. There has been some work towards supporting a build system API based on the build server protocol. We discussed this here, but I'm not sure what the latest status is, or if anyone is using this in production yet. Maybe @richardh or @DavidGoldman could say how far along things are.

  • What services is SPM providing to SourceKit-lsp?

The primary thing is looking up compiler arguments for a file, including the include paths that reference your swiftpm build directory so that we can find compiled modules. SwiftPM also performs indexing while building, and provides us with the location of that index data. In the future we would like to automatically watch for changes to compiler arguments so we can update without reopening a project, and we could also imagine adding support for automatically building modules (rather than relying on them already being built).

  • Would maintainers be open to patches that open SourceKit-lsp up to other build systems?
  • Any suggestions about where to start?

I would suggest looking at our build server API and see if that would work for you. Hopefully Richard or David can say more about the current status as well. I'd be happy to improve this API as necessary
to support additional build systems.

1 Like

Hi. We are just returning to this project after a brief hiatus, it is usable in its current state although missing support for some of the API. Per file flags for diagnostics is working, along with update notifications when build dependencies change. Still missing is indexing support to limit the index based on currently selected scheme / destination, something I was planning on getting to in the upcoming weeks.

2 Likes

I've got sourcekit-lsp working with an internal build system and vscode. The easiest method was to feed it the compilation database that @blangmuir referenced.

I've forgotten the exact reasons why but I ended up having to writing a build server. I'm pretty sure it was just so that all the paths to the indexstore/db and compilation database could be fiddled with.

It's not complex, and this might set you on the right track: BuildServer.swift · GitHub

I requires a forked sourcekit-lsp to expose BuildServerProtocol and some other bits and pieces that are useful.

1 Like

Neat!

I requires a forked sourcekit-lsp to expose BuildServerProtocol and some other bits and pieces that are useful.

To clarify, were these changes so you could use sourcekit-lsp's internal libraries to write your server, or did you need to change sourcekit-lsp itself in order to use your server? Would you be interested in sending us a PR? We don't promise a stable API right now, but we could at least expose the modules you needed.

To clarify, were these changes so you could use sourcekit-lsp's internal libraries to write your server, or did you need to change sourcekit-lsp itself in order to use your server? Would you be interested in sending us a PR? We don't promise a stable API right now, but we could at least expose the modules you needed.

I'd have to go back and look at it again to say for sure and what might be PR worthy! I definitely think all the protocols, structs and transport infrastructure you have in sourcekit-lsp would be beneficial for anyone writing a new build server.

I started off knowing nothing about lsp or buildservers so I can't say I took the shortest path. I recall the vscode plugin wouldn't allow arguments to be supplied to sourcekit-lsp. That led me down the buildserver route (just because it starts up automagically) and then after I sunk a bunch of time into that, it just felt like that was going to get me a working system. TBH if the vscode plugin could pass command line args to sourcekit, and sourcekit had flags to set indexdb/cdb paths then that might've worked just as well.

1 Like

Sounds reasonable, would you mind filing a bug (bugs.swift.org) with any specific details of what you wanted to configure?

I took a look at all this again today. It was great to see a ton of improvements to discovery of the index store paths and so on.

From what I see, as long as compile_commands.json exists in the root of the project directory, sourcekit-lsp parses the file to discover locations of the index store etc. So I've been able to use it without a build server at all today. Just one or two problems with vscode that I created another thread about.

Awesome stuff!

1 Like