SourceKit-LSP and CMake

In this video attached to this post, @ktoso seems to show sourcekit-lsp working with a CMake project (exact time in video). In FDB's repo, there is this doc which seems to imply the only thing you need is to specify the correct path to the Swift binaries, but I was unable to get this to work with a sample project.

I think the main issue is that SourceKit can't understand the project structure, as I didn't provide it with a path to compile_command.json (which CMake outputs weird, see command the screenshot below), or other build artifacts.

Is there a good resource for getting sourcekit-lsp set up with a CMake project (and perhaps having it autocomplete C++ sources as well?)

One follow up question: Swift ships with an ancient clang (it seems like clang-13 as of Swift 5.9). If we want sourcekit-lsp to work with C++ files (i.e. to be able to autocomplete C++ API in Swift and vice versa) does our project have to compile with clang-13, or is there some way to tell sourcekit to hang out with a newer clangd?

1 Like

The foundationdb built is a bit more complex than the usual build so we did this dance there: https://github.com/apple/foundationdb/blob/main/SWIFT_IDE_SETUP.md

We could put together a sample how to make it work in a simple cmake project I think.

@Alex_L could answer the c++ question probably

Hello, so normally passing -DCMAKE_EXPORT_COMPILE_COMMANDS=YES to your CMake invocation will generate the compile_commands into the build directory automatically. You'd want to symlink it from the build directory into the project root if you're building out-of-source so that clangd and sourcekit-lsp can find it and use it as the project "root". Unfortunately, that still doesn't quite work for Swift yet due to how CMake models a Swift build. The : in the command position is effectively a more universal /usr/bin/true so things don't "crash", but it certainly isn't enough for LSP to do anything meaningful. I have a branch here that re-models Swift in CMake, and one of the things it does is hook up the emission of the compile-commands correctly so we do get LSP: Files · ewilde/swiftly-remodelling · Evan Wilde / CMake · GitLab

The branch isn't quite ready for prime-time which is why it's not in a PR yet; WMO doesn't work at all and a few other dependency edges are missing, but it is somewhat livable for little experiments (or just generating compile_commands and then using real CMake) if you want to try it out. In my early testing, it does actually have autocomplete and go to definition mostly works, which is kind of fun (though it usually jumps into the bridging header when asking for the definition of the Swift function in C++).

As per the clang version, I think that's just because someone didn't update the Clang version. It's from the swift/release/5.9 branch, which came from stable/20221013 in GitHub - apple/llvm-project: The LLVM Project is a collection of modular and reusable compiler and toolchain technologies. This fork is used to manage Apple’s stable releases of Clang as well as support the Swift project., so the clang is about a year more recent than clang-13.

1 Like

I have a branch here that re-models Swift in CMake, and one of the things it does is hook up the emission of the compile-commands correctly so we do get LSP

Thanks for the info, but I'm not sure I 100% understand. @ktoso is saying they have this working for FoundationDB, but you seem to be saying that CMake doesn't actually yet support exporting the correct compile commands. Are the FBD folks using a patched CMake, or is there a different mechanism by which they are giving the project information to sourcekit-lsp?

The foundationdb built is a bit more complex than the usual build so we did this dance there

Looking at this doc it seems it's mostly to do with cross-compilation, rather than source kit, unless I'm missing something (I think the only source kit specific change is adding the path to sourcekit-lsp).

FDB has a custom Cmake target that uses a python script to generate a CDB for their C++ code. They need to do it because they use source generation for their C++ dialect, so Cmake's support for CDB doesn't work for them. We extended that script to also generate the relevant entries for Swift files in the generated CDB. So basically it's a very custom solution that fits into the existing FDB workflow.

This is the script that generates the CDB: https://github.com/apple/foundationdb/blob/main/contrib/gen_compile_db.py

Thanks for the pointer, that is what I was looking for!

So just to make sure I understand what's going on here, by "CDB" you mean "compile_commands.json" (I gathered that from here, though for a moment I was worried that there was yet another "how do I compile this file" format :sweat_smile:). And it looks like what's happening here is that you fix compile_commands.json "in post" and sourcekit-lsp picks it up from a well-known location (may be nice to add a "compile-commands-dir" setting like in clangd)

Having something like compile-commands-dir makes sense to me. Could you file an issue at Sign in to GitHub · GitHub? We can discuss how to design it there.

Yeah, compile_commands.json is the usual naming convention for the "compilation database". Details on them are listed here JSON Compilation Database Format Specification — Clang 18.0.0git documentation.

Since I recently got a bit of extra time on my hands, I'll do you one better (though happy to discuss the design as it is not 1:1 with compile-commands-dir): Add support for a compile database file not at the root of a workspace by GeorgeLyon · Pull Request #915 · apple/sourcekit-lsp · GitHub

2 Likes

Thanks, everyone, I now have this working in my project! Its not exactly a "just the basics" CMake project, as it builds as part of the LLVM build flow, but if it is interesting to anyone a link is here (the project itself has very little functionality at the moment). Might be useful as another example outside of FDB's support (which has a CMake build target you need run to rewrite compile_commands.json)

I was able to get it working with sourcekit-lsp main after my PR merged, without needing comile_commands.json at the root of the project! That said, being a random main hash it wasn't exactly stable, so instead I'm just symlinking the compile_commands.json to the root of my repo.

The Swift commands are correct because I'm building CMake from source with @etcwilde's changes he linked above, thanks for those btw!

1 Like

Glad that is working for you. It is a very experimental CMake, so there will be bugs. If you see anything glaring, let me know and I can try to figure something out.