Error in sourcekit-lsp?

Using Swift
,
1

I'm trying to use sourcekit-lsp for semantic highlighting of Swift source code in my custom code editor (Windows platform, Swift 5.10.1). The textDocument/semanticTokens/full request is succesfully sent to the language server, and response is successfully received. Token types (described at sourcekit-lsp/Sources/LanguageServerProtocol/SupportTypes/SemanticTokenTypes.swift at main · swiftlang/sourcekit-lsp · GitHub) in the response mostly correspond to my test source code, except for one token - the function name sayHello is described as a token of type "Bracket" for some reason.

Here is my test code:

// greeting
func sayHello() {
    print("Hello, World!")
}

And here is the log of the language server response:

Tokens of the test code

deltaLine = 0
deltaStart = 0
length = 11
type = Comment
modifiers = 0

deltaLine = 1
deltaStart = 0
length = 4
type = Keyword
modifiers = 0

deltaLine = 0
deltaStart = 5
length = 8
type = Bracket
modifiers = 0

deltaLine = 1
deltaStart = 4
length = 5
type = Function
modifiers = 512

deltaLine = 0
deltaStart = 6
length = 15
type = String
modifiers = 0

The first token is Comment, then in the next line there is a Keyword (func), then in the same line there is a Bracket (?) (sayHello), then in the next line there is a Function (print), etc.

Why the function declaration sayHello is described by the language server as Bracket? Is it an error in sourcekit-lsp, or am I misunderstanding something?

2

CC: @ahoppen

3

I think I've found the answer.

I tried to get token types from master branch SemanticTokenTypes.swift, which was wrong for Swift 5.10.1. It has token types that correspond to Microsoft Specification, up to decorator token type, and add several additional token types:

// The following are LSP extensions from clangd
  public static let bracket = Self("bracket")
  public static let label = Self("label")
  public static let concept = Self("concept")
  public static let unknown = Self("unknown")

  /// An identifier that hasn't been further classified
  ///
  /// **(LSP Extension)**
  public static let identifier = Self("identifier")

I've downloaded release version 5.10.1 sourcekit-lsp source code, and in the SyntaxHighlightingToken.swift file found that after the decorator token type (which is the last type in Microsoft specification) there is only one additional type:

/// **(LSP Extension)**
    case identifier

So actually sayHello in my test code has token type identifier (as it should be), and not bracket, (bracket is absent in Swift 5.10.1).

4

Hmm, things are a bit more complicated regarding compliance with Microsoft specification of token types: Swift master branch which I initially tried to use, correspond to the spec:

namespace
type
class
enum
interface
struct
...

but v5.10.1 release source code is not:

namespace
type
actor
class
enum
interface
struct
...

5

I assume that you are not respecting the SemanticTokensLegend that is returned by SourceKit-LSP as part of the initialization options. To be correct, you need to use that legend to map the token indices to the token kinds.

6

I couldn't figure out which request returns the legend. The initialization request returns server capabilities, but the legend is absent there.

Here's the response of the initialization request (formatted for readability, was one line) - no legend is seen here:

Response to Initialization request 
{
  "result": {
    "capabilities": {
      "referencesProvider": true,
      "declarationProvider": true,
      "workspace": {
        "workspaceFolders": {
          "changeNotifications": true,
          "supported": true
        }
      },
      "documentHighlightProvider": true,
      "callHierarchyProvider": true,
      "foldingRangeProvider": true,
      "completionProvider": {
        "resolveProvider": false,
        "triggerCharacters": [
          "."
        ]
      },
      "hoverProvider": true,
      "workspaceSymbolProvider": true,
      "definitionProvider": true,
      "executeCommandProvider": {
        "commands": [
          "semantic.refactor.command"
        ]
      },
      "codeActionProvider": true,
      "textDocumentSync": {
        "willSaveWaitUntil": false,
        "willSave": true,
        "openClose": true,
        "change": 2,
        "save": {
          "includeText": false
        }
      },
      "typeHierarchyProvider": true,
      "implementationProvider": true,
      "documentSymbolProvider": true,
      "colorProvider": true
    }
  },
  "jsonrpc": "2.0",
  "id": 1
}
7

Ah, I see. I suspect that your editor does not support dynamic capability registration for semantic tokens (clientCapabilities.textDocument.semanticTokens.dynamicRegistration)?

If it did, you should get a client/registerCapability request from SourceKit-LSP that contains the legend as registerOptions.legend after opening the first Swift file. If you don’t support dynamic registration, you should have the legend in the initialize response as capabilities.semanticTokensProvider.legend.

If your editor doesn’t support dynamic capability registration, we don’t return the legend in 5.10, which should be fixed in the 6.0 toolchains you can download from Swift.org - Install Swift and which are included in the Xcode 16 betas.

1 Like
8

Indeed, my editor uses the bare minimum of LSP requests, no any client capability requests (notifications?) are implemented. I will consider adding clientCapabilities.textDocument.semanticTokens.dynamicRegistration.

That's very good news!

Thank you for guiding me in the right direction.

9

One problem solved

My editor now includes clientCapabilities.textDocument.semanticTokens.dynamicRegistration parameter in the initialize request, and successfully receives the legend in a client/registerCapability request from SourceKit-LSP.

One more problem

I'm still learning how SourceKit-LSP works, and this is the first time I encountered a request from server to client, and not vice versa. The Microsoft Specification says

Every processed request must send a response back to the sender of the request.

So my client should send a response to every processed request it receives from the LSP server. However I receive textDocument/publishDiagnostics requests from SourceKit-LSP that are missing the id field. A response to any request should indicate which request it correspond to via the id field. Sending a response without the id is pointless. Moreover, the Microsoft Specification indicates that the id field in any response is required (non-optional).

Is this a bug in SourceKit-LSP v5.10.1?

Update: I have realized that no id means notification, not a request.

10

Yes, textDocument/publishDiagnostics is a notification, not a request.