Running SourceKit-LSP tests with local build of SourceKit

Steffeeen · March 7, 2026, 1:52pm

I want to run some tests in SourceKit-LSP while using a local build of SourceKit. I have already followed the Getting Started Guide and have a local build. However, I'm not sure what the proper way is to get SourceKit-LSP to use this local build of SourceKit.

I tried the SWIFTCI_USE_LOCAL_DEPS environment variable, but that's primarily intended for CI as far as I can see, it also didn't seem to work for me. The test ran normally, but it didn't seem to use my local SourceKit build.

I have also tried the SOURCEKIT_TOOLCHAIN_PATH environment variable, but couldn't get it to work. I set it to /Users/steffen/dev/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64. This causes the test to fail with one of two errors:

Error Domain=NSCocoaErrorDomain Code=3840 "JSON text did not start with array or object and option to allow fragments not set. around line 1, column 0." UserInfo={NSDebugDescription=JSON text did not start with array or object and option to allow fragments not set. around line 1, column 0., NSJSONSerializationErrorIndex=0}

or

ResponseError(code: LanguageServerProtocol.ErrorCode(rawValue: -32001), message: "Build server failed to initialize", data: nil)

I also see this output from SourceKit in the terminal:

sourcekit: [1:connection-event-handler:11015: 0.0000] Connection interrupt
sourcekit: [1:updateSemanticEditorDelay:11015: 0.0003] disabling semantic editor for 10 seconds
sourcekit: [1:pingService:11015: 0.0003] pinging service
sourcekit: [1:sourcekitd_send_request:12035: 0.0026] request dropped while restoring service
sourcekit: [1:pingService:11015: 0.0246] pinging service
sourcekit: [1:connection-event-handler:4355: 0.0247] Connection interrupt
sourcekit: [1:updateSemanticEditorDelay:4355: 0.0247] disabling semantic editor for 10 seconds
sourcekit: [1:pingService:4355: 0.0247] pinging service
sourcekit: [1:pingService:14851: 0.0482] pinging service
sourcekit: [1:pingService:15367: 0.0482] pinging service
sourcekit: [1:connection-event-handler:4355: 0.0483] Connection interrupt
sourcekit: [1:updateSemanticEditorDelay:4355: 0.0483] disabling semantic editor for 10 seconds
sourcekit: [1:pingService:4355: 0.0483] pinging service

ahoppen · March 9, 2026, 11:37am

Could you try adding the following to your build-script invocation?

--extra-cmake-options='-DSWIFT_SOURCEKIT_USE_INPROC_LIBRARY:BOOL=TRUE'

That way sourcekitd will be launched in-process, which usually makes debugging easier and which should give you a stack trace how sourcekitd crashed. Alternatively, you can open Console.app and look for SourceKitService crash logs.

Steffeeen · March 9, 2026, 4:28pm

~~I tried this build-script invocation:~~

utils/build-script --skip-build-benchmarks \ --swift-darwin-supported-archs "$(uname -m)" \ --release-debuginfo --swift-disable-dead-stripping \ --bootstrapping=hosttools --sccache --extra-cmake-options='-DSWIFT_SOURCEKIT_USE_INPROC_LIBRARY:BOOL=TRUE'

However, ninja does not rebuild anything. Do I need to do a full rebuild?

I guess this is the relevant CMakeLists.txt where the option decides what gets build? When I run the build-script invocation above SWIFT_SOURCEKIT_USE_INPROC_LIBRARY is always undefined. I verified this by adding

message(STATUS "The value of USE_INPROC is: ${SWIFT_SOURCEKIT_USE_INPROC_LIBRARY}")

~~before the if(). It also seems like the if() and elseif() should be the other way around? Otherwise, XPC is always used, if available, even if the SWIFT_SOURCEKIT_USE_INPROC_LIBRARY is set.~~

Update: I needed to add --reconfigure to the build-script call. However, I needed switch around the if() and elseif() in this CMakeLists.txt. I also needed to adjust the check in this CMakeLists.txt. However, the tests still fail with an error:

Could not find .framework directory relative to 'file:///Users/steffen/dev/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/lib/libsourcekitdInProc.dylib

I have also looked into the crashes when not using InProc. I mostly have crashes for xctest where the relevant stack trace looks like this:

Thread 1 Crashed:
0   libswiftCore.dylib            	       0x1976a11f8 _assertionFailure(_:_:file:line:flags:) + 176
1   libSwiftSourceKitClientPlugin.dylib	       0x106cf9144 sourcekitd_plugin_initialize_2(_:_:) + 620 (ClientPlugin.swift:41)
2   libSwiftSourceKitClientPlugin.dylib	       0x106cf8ed0 sourcekitd_plugin_initialize_2 + 12
3   sourcekitd                    	       0x105f9c8ac loadPlugin(llvm::StringRef, sourcekitd::PluginInitParams&) + 292
4   sourcekitd                    	       0x105f9c768 sourcekitd::loadPlugins(llvm::ArrayRef<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>>, sourcekitd::PluginInitParams&) + 244
5   sourcekitd                    	       0x105f74204 void std::__1::__call_once_proxy[abi:nn200100]<std::__1::tuple<sourcekitd_load_client_plugins::$_0&&>>(void*) + 92
6   libc++.1.dylib                	       0x185dbd3b8 std::__1::__call_once(unsigned long volatile&, void*, void (*)(void*)) + 196
7   sourcekitd                    	       0x105f72c28 sourcekitd_initialize + 1168
8   SourceKitLSPPackageTests      	       0x113183144 SourceKitD.init(dlhandle:path:pluginPaths:initialize:) + 2216 (SourceKitD.swift:217)
9   SourceKitLSPPackageTests      	       0x113182854 SourceKitD.__allocating_init(dlhandle:path:pluginPaths:initialize:) + 92
10  SourceKitLSPPackageTests      	       0x1131826b4 SourceKitD.__allocating_init(dylib:pluginPaths:initialize:) + 584 (SourceKitD.swift:188)

Sometimes, I also get SourceKitService crashes which look like this:

Thread 3 Crashed::  Dispatch queue: message-handling
0   libswiftCore.dylib            	       0x1976a11f8 _assertionFailure(_:_:file:line:flags:) + 176
1   libswiftCore.dylib            	       0x19770f378 swift_unexpectedError + 656
2   libSwiftSourceKitPlugin.dylib 	       0x1154e37e4 SourceKitD.servicePluginApi.getter + 248 (SourceKitD.swift:124)
3   libSwiftSourceKitPlugin.dylib 	       0x11556d054 RequestHandler.init(params:completionResultsBufferKind:sourcekitd:) + 124 (Plugin.swift:44)
4   libSwiftSourceKitPlugin.dylib 	       0x11556cfc8 RequestHandler.__allocating_init(params:completionResultsBufferKind:sourcekitd:) + 72
5   libSwiftSourceKitPlugin.dylib 	       0x115574290 sourcekitd_plugin_initialize_2(_:_:) + 1952 (Plugin.swift:262)
6   libSwiftSourceKitPlugin.dylib 	       0x115573ae8 sourcekitd_plugin_initialize_2 + 12
7   SourceKitService              	       0x10292e47c loadPlugin(llvm::StringRef, sourcekitd::PluginInitParams&)::$_0::operator()(char const*) const + 8 (sourcekitdAPI-Common.cpp:318) [inlined]

As they both seem related to plugin loading, I also tried adding SOURCEKIT_LSP_TEST_PLUGIN_PATHS=RELATIVE_TO_SOURCEKITD. But this caused another error:

Could not find SourceKit plugin. Ensure that you build the entire SourceKit-LSP package before running tests.

Searching for plugin relative to relative-to-sourcekitd://

I have verified that everything is built by running swift build.

ahoppen · March 9, 2026, 8:13pm

Sorry that I forgot to mention the --reconfigure bit, completely forgot about that.

OK, that is indeed interesting what’s going on here. Would you be able to change the precondition in ClientPlugin.swift:41 to a fatalError and include the plugin paths in the failure message? And also instead of try! in SourceKitD.swift:124 maybe wrap it in a do {} and instead fatalError with the error message. That would hopefully help us understand what’s going wrong. And both of these would be good changes to make regardless to help diagnose issues like this in the future.

And you shouldn’t need to set SOURCEKIT_LSP_TEST_PLUGIN_PATHS=RELATIVE_TO_SOURCEKITD, that’s just needed for some kind of CI setups.

Steffeeen · March 10, 2026, 10:30am

I added a message to the precondition(). To me, it seems like sourcekitd gets loaded twice. When the precondition fails SourceKitD.forPlugin.path is

file:///Users/steffen/dev/swift-project/build/Ninja-RelWithDebInfoAssert/swift-macosx-arm64/lib/sourcekitd.framework/Versions/A/sourcekitd

while pluginPath is

file:///Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/sourcekitd.framework/Versions/A/sourcekitd

Update: I investigated this further. If I run swift test --filter InlayHintTests the precondition always fails for the testInlayHintResolveCrossModule test. Specifically while sending the InlayHintRequest. However, if I run just this single test, the test fails with XCTFail("Expected type hint for MyType"), but the precondition does not fail. Together with Codex I managed to create this minimal reproducer which reliably triggers this crash.

@_spi(SourceKitLSP) import LanguageServerProtocol
import SKTestSupport
import XCTest

final class PreconditionReproducer: SourceKitLSPTestCase {

  func testA() async throws {
    let testClient = try await TestSourceKitLSPClient()
    let uri = DocumentURI(for: .swift)
    testClient.openDocument("", uri: uri)
    _ = try await testClient.send(InlayHintRequest(textDocument: TextDocumentIdentifier(uri)))
  }

  func testB() async throws {
    let project = try await SwiftPMTestProject(files: ["a.swift": "let x = 1"])
    let (uri, _) = try project.openDocument("a.swift")
    _ = try await project.testClient.send(InlayHintRequest(textDocument: TextDocumentIdentifier(uri), range: nil))
  }
}

When switching the names of the two tests around to force a different execution order the two mismatching paths also switch around. It seems like SwiftPMTestProject always chooses the system toolchain.

I cannot reproduce the SourceKitService crash currently, but I have added the do {} block as you suggested to SourceKitD.swift to hopefully get more information if I get the crash again.

Steffeeen · March 12, 2026, 4:17pm

I looked into this further. As testB uses a SwiftPMTestProject the sourcekitd path is obtained from the build target for the project. The SwiftPMBuildServer queries the toolchain registry for the available toolchains. However, as my custom toolchain does not have clang and clangd, the toolchain registry returns the system toolchain. In testA the toolchain only needs to support sourcekitd, swift and swiftc and thus my custom toolchain is chosen by the toolchain registry.

Update: my toolchain has clang and clangd in its bin directory which are symlinks to ../llvm-macosx-arm64/bin/clangd. They are however broken, because they should be pointing to ../../llvm-macosx-arm64/bin/clangd (notice the two ..). I also tried a completely fresh build of swift, this build does not even have the clang and clangd symlinks.

As a workaround, I can just create symbolic links for clang and clangd in my toolchain that just point to the system toolchain. However, I think this issue should be fixed properly.

One approach I can think of is a hierarchy of toolchains. If a toolchain does not have all tools it can delegate these tools to its parent toolchain. The system toolchain can always be the root toolchain. So if the toolchain specified by SOURCEKIT_TOOLCHAIN_PATH does not have clang, for example, we check the parent toolchain to see if it has clang. As I'm not really familiar with the code around toolchains I'm not sure how much effort implementing this would be.

ahoppen · March 15, 2026, 11:20am

Great find, thanks for the detailed investigation, @Steffeeen.

We don’t want to mix tools from different toolchains. While it may be fine to mix clangd from a different toolchain, mixing swiftc and sourcekitd from different toolchains is just bound to cause trouble elsewhere (speaking from experience here).

Instead, what I would suggest to do is: If SOURCEKIT_TOOLCHAIN_PATH is set, ToolchainRegistry should only ever return that toolchain and never any other. Now, if we call preferredToolchain(containing: [\.clangd, …]) and the toolchain does not contain clangd, we should log a fault and return nil. While that does mean that all clangd tests will fail in such a toolchain setup, at least it’s obvious why it is failing and you are able to fix it by either not running those tests, if they are not needed, or by ensuring your toolchain contains clangd through symlinks (like you added) or by ensuring we build it. What do you think? If you agree, would you be able to open a PR for it?

Steffeeen · March 15, 2026, 4:35pm

That seems like a good approach to me. I will work on a PR.