Introducing SourceKit-LSP


(Ben Langmuir) #1

As recently announced by @akyrtzi, we have been working on a new language service for Swift and C-family languages based on the Language Server Protocol (LSP). I am excited to say that the source code for SourceKit-LSP is now available on GitHub! While the new language service is still in early development, we are making it available today to continue its development in the open. Check it out at https://github.com/apple/sourcekit-lsp!

SourceKit-LSP provides code-completion, jump-to-definition and the other semantic editor features from the LSP for Swift and C-family language projects. It enables first class support for Swift and cross-language projects in editors such as Visual Studio Code and Sublime Text. SourceKit-LSP is built on top of sourcekitd and clangd for high-fidelity language support, and provides a powerful source code index as well as cross-language support. It natively understands Swift Packages and is designed to be cross platform, currently supporting macOS and Linux.

Instructions for trying out the new service are at https://github.com/apple/sourcekit-lsp#getting-started. Right now the best way to try it out is to build it from source. Part of the project plan is to explore good ways to package the service for users to try out. Another thing to be aware of: in this early stage of development, we recommend trying out SourceKit-LSP with the swift-DEVELOPMENT-SNAPSHOT-2018-11-01-a toolchain snapshot from https://swift.org/download/#snapshots under “Older Toolchains”. There is an issue with the current 2018-11-13 toolchain that we hope to have resolved soon. We intend to support Swift-5 and future release toolchains going forward.

There is still lots of work to do, but SourceKit-LSP already supports many important editing features. The following table, taken from https://github.com/apple/sourcekit-lsp#status gives an overview of the current status. If you are using Linux, be aware that index-based features such as jump-to-definition depend on some changes (swift-clang#219) that are not in a toolchain snapshot as of swift-DEVELOPMENT-SNAPSHOT-2018-11-01-a, but should be available soon.

Feature Status Notes
Swift :white_check_mark:
C/C++/ObjC :x: clangd is not available in the recommended toolchain. You can try out C/C++/ObjC support by building clangd from source and putting it in PATH.
Code completion :white_check_mark:
Quick Help (Hover) :white_check_mark:
Diagnostics :white_check_mark:
Fix-its :x:
Jump to Definition :white_check_mark:
Find References :white_check_mark:
Background Indexing :x: Build project to update the index using Indexing While Building
Open Quickly :x:
Refactoring :x:
Formating :x:
Folding :x:
Syntax Highlighting :x: Not currently part of LSP.

We welcome contributions to this new project! We have the same policies for contributing as the rest of the Swift project. One great way to get involved is to file bugs for any problems you run into at https://bugs.swift.org/ in the “SourceKit-LSP” component and engage with us in the forums at https://forums.swift.org/c/development/sourcekit-lsp. If you would like to submit a patch, SourceKit-LSP itself is written in Swift, and the core indexing support, IndexStoreDB, is written primarily in C++. We are adding PR testing support very soon.


Can this project eliminate the famous 'tabs vs. spaces' problem (at least in Swift projects)?
(Sam Deane) #2

From the point of view of tools wishing to use the service, I'd have thought that the only sensible way to distribute it would be alongside swift itself as part of the toolchain.

Any tool that wants to work with the service is likely to also be working with swift and/or spm itself, thus having to locate them in some way.

Being able to rely on the sourcekit-lsp binary being in a known location relative to the swift binary would simplify things and avoid having to ask the user for multiple locations.


(Ben Langmuir) #3

I agree this would be the most convenient choice right now. There would still be practical questions to figure out, like how to support swift-5 toolchains. The new service is not going to converge in the same way as the other parts of the toolchain in the swift-5 timeframe, so I don't want to put the lsp server in the normal toolchain building infrastructure in the swift-5 branch.

Right, this is both easier for users to understand if they already know where the toolchain was installed, and simpler for sourcekit to find the corresponding swift/sourcekitd/etc. at runtime.


(Sam Deane) #4

Could you expand a bit more on this.

Are you saying that it'll not build with swift-5 for now? I can see why that would be -- too many goalposts moving at one time -- but it begs the question of whether it will understand the swift 5 syntax, even if it doesn't actually build under it for a little while.

I can foresee it being a bit messy for users (and editor support/plugin authors) if it's possible for the sourcekit-lsp and the toolchain to be out of sync.

Possibly that wasn't what you meant at all, however :slight_smile:


(Ben Langmuir) #5

Sorry for any confusion! What I'm talking about for swift-5.0 is logistics. We build the swift-5.0 compiler, stdlib, corelibs, etc. together in CI and for packaging. The straightforward way to add new projects to a toolchain is to add it to the same infrastructure and build it all together. I don't think LSP should do that in swift-5.0. We still want to fully support the swift-5.0 language and libraries, but we need to work out a different way to test it in CI, etc. without impacting the the swift-5 release.


(Adrian Zubarev) #6
I moved this question into an own thread, please see bellow. You can still unfold this spoiler to view my previous post. I have a few questions I want to ask. However I'm not sure if it's the right place for that. I have to admit that I have very little to none knowledge related to the LSP technology, so please bare with me.

Judging from the table in the OP there are or will be features that are source code related that editors and IDE's can start sharing. By 'sharing' I mean in the context of shared API on how to handle similar things in different editors. In particular I'm speaking about Folding and Formatting features. By this day I've seen tons of of Swift libraries and various code styles. However there are and always will be inconsistencies between different libraries due to different coding styles.

For instance I would like to contribute two multiple projects, but I nearly always end up screwing up the formatting, just because the projects follow a different source style that I do or my IDE is set up to follow. By that I only refer to one single thing: tabs vs spaces (2, 4, etc.).

  • Wouldn't be nice if we can generalize at least a few major cases with this project, starting with that problem?!
  • In the long term it would be great to have a shared mechanism that each editor / IDE can automatically make use of for such fundamental things.

I Imagine something along the following line:
Project A uses tabs, IDE automatically adjust itself, stdlib uses 2 spaces, the IDE will adjust itself again (locally not globally).


Can this project eliminate the famous 'tabs vs. spaces' problem (at least in Swift projects)?
(Ladislas de Toldi) #7

You might want to try EditorConfig https://editorconfig.org/ :)


(Adrian Zubarev) #8

I had a quick glance on that project and it looked similar to a linter except that it's sharing settings for editors which does not provide any support for latest Xcode since 3rd party plugins are no longer officially supported. That said, looking at the LSP features above like Folding does require some source code analysis for the feature to work. So something like a determination of tabs vs. spaces could be also provided by the project itself? In case of spaces one only need to figure out how determine the required width. (The weirdest part are probably "smart spaces" which is a mix of tabs and spaces some editors can create for code alignment between multiple lines.)


(Argyrios Kyrtzidis) #9

Hey Adrian, we have a dedicated forum category at https://forums.swift.org/c/development/sourcekit-lsp I'd recommend to start a thread there.


(Adrian Zubarev) #10

Moved, thank you. ;)