Hey! I wanted to express my interest in joining GSoC with Swift this year. I'm particularly interested in the topic of improving the display of documentation during code completion in SourceKit-LSP with @ahoppen
I have some experience working with Gradle LSP and BSP implementations, so I am familiar with how LSP works in general. However, I haven't delved into the finer details of how a language backend provides a meaningful developer experience. That’s something I’m looking forward to learning here.
I'm fairly new to Swift, but I have experience with languages like Rust and C++, so in that regard, this environment feels familiar and like a place where I can thrive. I have reviewed the project description and set up a local development environment for SourceKit-LSP and the Swift language to begin testing. I'm currently playing with a development instance of sourcekit-lsp to see how it works currently.
I have a few questions regarding the project proposal that I’d like to clarify to get me going:
How exactly does SourceKit-LSP and the Swift backend communicate with each other to get the existing code intelligence functionality?
How does CodeCompletion.cpp interact with other elements of Swift lang to provide code completion features? Navigating and understanding such a large codebase is quite intimidating for me, so having a clear starting point along with an explanation of how the existing implementation works would be extremely helpful.
What changes would be required in Swift's codebase to implement the LSP signature help request in order to provide code completion results? Would all of these changes be limited to CodeCompletion.cpp, or would modifications be needed outside of CodeCompletion.cpp as well to allow these changes to work? In other words, does Swift already have a built-in suggestions API that just needs to be integrated with LSP, or does this functionality need to be implemented from scratch?
The gist is that code completion starts Swift’s type checker with a special CodeCompletionExpr inserted at the location at which code completion should be performed. The constraint system will then produce possible solutions for the expression (multiple in case there is ambiguity, eg. in the case of overloads) and we inspect those solutions to (among others) infer the type that we are completing on and then use the compiler’s name lookup logic to find members on that type to suggest. There are a few types conforming to TypeCheckCompletionCallback in lib/IDE that provide most of the code completion functionality. For example ArgumentCompletion provides the completion when inside a call. sawSolutionImpl infers the information we need during completion and collectResults produces the actual results based on that information.
You can see some of this in action when building the compiler locally and running swift-ide-test -code-completion -code-completion-token COMPLETE -source-filename /path/to/file.swift -sdk $(xcrun --show-sdk-path --sdk macosx) -debug-constraints on a file that contains #^COMPLETE^# at the location where you want to perform completion. Reading and understanding that output is non-trivial though.
This would require new functionality to be implemented in ArgumentCompletion.cpp and to be exposed through a new sourcekitd request. How exactly that should work would be part of the GSoC project.
as the completion results, I can see the documentation and return type, but not the information about about the function parameters or the return type.
As per the project description,
Every code completion item can have documentation associated with it and while completing a function signature, the editor can display the available overloads, parameter names and their documentation through signature help. Currently, SourceKit-LSP only displays the first line of an item’s documentation in the code completion results and does not provide any signature help.
This project would implement functionality to return the entire documentation for all code completion items and also implement the LSP signature help request.
textDocument/hover already has this functionality of being able to read the documentation with respect to the overload being in use as seen here
I assume that by looking over how textDocument/hover works, and similarly implementing the same functionality with textDocument/signatureHelp, we can achieve the desired outcome for this proposal. Is this the only deliverable? Is this what constitutes signatureHelp? I feel like I might be missing something here, as this doesn’t seem to make sense as an intermediate project in that case.
I don’t expect major challenges with the display of the full documentation for code completion items either. The challenging part for signature help will be to figure out which parameter you are currently entering. We currently have logic like this only for argument completions, eg. with the following, the cursor being at |.
func test(a: Int, b: Int = 1, c: Int = 2, d: Int) {}
test(a: 1, |
This currently suggests both b, c and d as argument labels. We need similar behavior for signature help (though we might be able to re-use significant parts here). And just getting everything to build and passing the information through different layers will, in my experience, make this an intermediate project
I'll be then looking more into how hoverRequest request works under the hood and try to get a prototype working with signatureHelp with at least some of the hoverRequest features working, it should also be a good way to learn how the information goes through the layers, which I think should be a good point to start. I get some of the layers, but I feel like there are a lot which could take some time to understand. So in the process I tried to build a local copy of sourcekit-lsp and ran it on my own vscode for debugging. The screenshot I attached earlier uses the local build of sourcekit-lsp which I was building using swift build. But recently, after the latest snapshot I run into this error when using swift build
When building alongside swift with --sourcekit-lsp and --install-sourcekit-lsp flags, it still works, but the problem is that it builds the release version of sourcekit-lsp and I cannot see an option to specify to create a debug variant of sourcekit-lsp specifically, and I cannot use lldb to debug with it. I tried changing SOURCEKIT_TOOLCHAIN_PATH, but had no luck there. Could you provide a lead on how to debug this, and sourcekit-lsp itself more efficiently?
I'm currently debugging sourcekit-lsp with a local build directed through the VSCode Swift extension, which I believe is the best approach. However, I find debugging SourceKit and sourcekitd quite challenging. While I can use lldb alongside the sourcekit-lsp codebase and navigate through it relatively easily, debugging sourcekitd or anything related to the Swift side of the codebase has been much tougher.
I've tried multiple approaches to debugging SourceKit. When I open VSCode, it spawns sourcekit-lsp as a process, which in turn starts SourceKitService. SourceKitService is responsible for handling XPC requests with the Swift backend to provide code intelligence functionality. Understanding the sourcekit-lsp codebase has been relatively smooth, but getting more insights into the swiftlang/swift codebase would be extremely helpful.
For instance, when I attach lldb to the sourcekit-lsp process, I can see the sourcekitd image and related code, but I can't access SourceKit files or key components like swift/lib/IDE/CodeCompletion.cpp. lldb simply doesn't seem to locate those files.
How can I debug SourceKit and the Swift codebase more effectively? Any help would be greatly appreciated.
I read the SourceKit README and found that using libsourcekitdInProc.dylib might solve my issue. I also tried disabling XPC by setting set(SWIFT_SOURCEKIT_USE_INPROC_LIBRARY TRUE) in SourceKit/CMakeLists.txt, but that didn’t help.
I'm unsure how to properly build SourceKit with these flags or whether that would achieve what I'm trying to acheive. Any guidance on this would be greatly appreciated!
Yes, debugging sourcekitd is typically a little harder. What I usually do, is to break down the behavior I want to debug into something that can be run using sourcekitd-test (like the tests in swift/test/SourceKit at main · swiftlang/swift · GitHub) and then debug the sourcekitd-test executable. Setting --extra-cmake-options='-DSWIFT_SOURCEKIT_USE_INPROC_LIBRARY:BOOL=TRUE' in your build-script invocation really helps here because you no longer need to deal with XPC services that way.
As a third alternative, you could try building a full toolchain locally, but I don’t have good instructions for that at hand and it requires more compute resources than the suggestions above and iterating is also slower.
