RFC: Split up SourceKit-LSP’s SwiftPM Support

[owenv (Owen Voorhees)]

RFC: Split up SourceKit-LSP’s SwiftPM Support

Recently, I’ve been investigating how best to organize SourceKit-LSP’s support for requesting build settings and preparation from SwiftPM as we integrate the new Swift Build backend. Currently, SourceKit-LSP directly links libSwiftPM and contains an implementation of the BuiltinBuildSystem protocol for Swift packages. I propose moving to a model where SwiftPM instead provides a build server in the toolchain which implements this functionality. SourceKit-LSP could then eliminate its build time dependency on SwiftPM, instead launching the build server at runtime and interacting with it using the existing BSP (Build Server Protocol) support. This division of responsibilities has a couple major advantages:

1. SwiftPM’s support for background indexing preparation, vending compiler arguments, etc. can be colocated with the rest of its build system support, improving overall maintainability and reducing the risk of unintended behavior differences across different clients.
2. It enforces a clearer separation of responsibilities between SourceKit-LSP and build system implementations.
3. It allows vending a standalone implementation of core LSP and BSP types to better support other client and server implementations.

Proposed Changes:

I propose accomplishing the above by:

1. Splitting SourceKit-LSP’s implementation of core LSP/BSP model types and transport into a new swift-language-server-protocol package.
2. Adopt swift-language-server-protocol as a dependency of SwiftPM and introduce a new swift package bsp build server implementation in the toolchain.
3. Update SourceKit-LSP to adopt swift-language-server-protocol, remove the SwiftPM dependency, and delegate to SwiftPM’s BSP instead.

Splitting up SourceKit-LSP

The new swift-language-server-protocol package would include the LanguageServerProtocol and BuildServerProtocol targets mostly unchanged. These modules vend the core LSP and BSP models with no dependencies other than Foundation and the stdlib.

I propose that the new package should also include the LanguageServerProtocolJSONRPC module, renamed to LanguageServerProtocolTransport. This module contains support for managing a connection to a LSP/BSP client or server. It will also grow to include the LocalConnection and QueueBasedMessageHandler types from LanguageServerProtocolExtensions in SourceKit-LSP.

A new module, BuildServerProtocolTransport will include BuildSystemMessageDependencyTracker from BuildSystemIntegration in SourceKit-LSP, which is generally useful to concrete BSP implementations.

LanguageServerProtocolTransport brings in a few additional dependencies which complicate the package split, namely SwiftExtensions , CAtomics , swift-crypto , and SKLogging . SwiftExtensions and SKLogging are used throughout various components of SourceKit-LSP, and build module-aliased versions for inclusion in the SourceKit plugin. swift-crypto is in the logging subsystem to hash redacted freeways strings, and CAtomics is used as a replacement for Swift’s builtin atomics when deploying to older OS versions.

SwiftExtensions consists of various smaller utilities which are both used internally to LanguageServerProtocolTransport and appear in public API. This module will move to the new package, and types which were previously package-accessible will become SPI for use by SourceKit-LSP.

SKLogging contains SourceKit-LSP’s relatively specialized logging implementation. Due to its use of OSLog on Darwin platforms, it’s difficult to hide behind an abstraction barrier without defeating its mechanisms for privacy-preserving logging. It also currently hardcodes a default logging subsystem. I propose:

• Moving the sources into the new package.
• Extending the global LogConfig type to require configuring a default subsystem at process launch before the first messages are emitted
• Exposing currently package-accessible types as SPI for use by SourceKit-LSP

The CAtomics dependency can be removed by raising the deployment targets of the new package, SourceKit-LSP, and SwiftPM to macOS 15.0.

The swift-crypto dependency will be removed as it’s used in a single location to redact logged string arguments which hasn’t turned out to be too useful in practice compared to replacing them with a constant.

Adopting swift-language-server-protocol in SwiftPM

SwiftPM will add a new subcommand, swift package bsp . This command will use SwiftPM’s built in build support along with the new package to implement a build server for Swift packages which provides compiler arguments and build preparation support.

Adopting swift-language-server-protocol in SourceKit-LSP

Adopting the new package in sourcekit-lsp should be largely mechanical and mostly consist of updating imports and dependency specifications, and removing duplicated code. Some, mostly minor, updates will be needed to lookup the new BSP server and treat it as an eligible source of build system functionality for Swift packages.

cc @ahoppen @bnbarham

[Max_Desiatov]

@owenv

Splitting SourceKit-LSP’s implementation of core LSP/BSP model types and transport into a new swift-language-server-protocol package.

Yes please! This has been a requested feature for long time.

Overall the proposed changes make perfect sense to me, but separate packages for LSP and BSP protocols are especially appreciated and would be a great boost for the overall tooling ecosystem.

[Lancelotbronner (Christophe Bronner)]

Yes please! I'd love this to both help third-party tooling support Swift and allow Swift projects to support third-party languages & build servers!

[rauhul (Rauhul Varma)]

• Will other protocols be added to this package e.g. DAP, etc.?
• Does it make sense to split each of them out into individual packages?
• If not, should the package be more generally named, like "swift-ide-protocols"?

[etcwilde (Evan Wilde)]

Overall, sounds good to me. Does this have any implication on the compile-commands support or will that stay the same?

[owenv (Owen Voorhees)]

@rauhul

Will other protocols be added to this package e.g. DAP, etc.?

I don't intend to propose any additional functionality/protocols support, I'm really just focused on extracting what's already in SourceKit-LSP in reusable form.

@rauhul

• Does it make sense to split each of them out into individual packages?

While there's an argument to be made that particular client/server implementations will only care about one of either LSP or BSP, I don't think it makes sense to split them up. From an organizational perspective, there are enough shared bits of the transport implementation and base protocols to make it convenient to keep them together in one package. Also, both have very minimal dependencies so splitting them up doesn't simplify consumer's package graphs at all.

@rauhul

If not, should the package be more generally named, like "swift-ide-protocols"?

I'm not opposed to some bikeshedding. I wrote swift-language-server-protocol primarily because it comprises the bulk of the functionality today and is a more recognizable name. I'm not a huge fan of having IDE in the name because there are other types of tools which might implement LSP/BSP

[owenv (Owen Voorhees)]

@etcwilde

Overall, sounds good to me. Does this have any implication on the compile-commands support or will that stay the same?

I don't think it does, that could remain one of SourceKit-LSP's BuiltinBuildSystems. While it could be implemented as a separate BSP executable using the new package, I don't see a really compelling reason to do so.