Question on Background Indexing

Development SourceKit-LSP
1

I'm still working on Noctule (Swift for JetBrains IDEs) and have questions on background indexing.

  1. I created a new SPM project (swift package init --type executable) and opened it in an LSP-enabled editor (tested in VSCode + Swift, too)
  2. I created a new class MyNewTypein Sources/MyNewType.swift.
  3. I opened Sources/main.swift and add a reference to the new class: let x = MyNewType(). Code completion is suggesting the new type.
  4. When I now switch to "MyNewType.swift" and search references on the class identifier, then nothing is found.
    1. Restarting sourcekit-lsp did not help
    2. Changing from backgroundIndexing: true to backgroundIndexing: build did not help
    3. Building with swift build on the command line fixed the problem

I assumed that background indexing is supposed to make this scenario work to avoid repeated swift build steps.

Questions:

  • Should such new types be automatically picked for "find references" if background indexing is enabled?
  • If "yes", what could be wrong with my setup? Would both backgrounIndexing: true and backgroundIndexing: build make this work or just build?
  • If "no", what would be a good way to make this work?

Coming from non-Swift development with JetBrains IDEs, executing swift build after trivial changes like adding a new class or renaming an existing type does not feel right to me. I hope the IDE is not supposed to run builds after such changes...

I think I understand the complexity involved, especially with compiled languages. But I still hope that it's somehow possible to make Swift work well in the editor...

Thank you!

Setup:

  • macOS
  • Swift 6.3.1 toolchain (via Swiftly)

.sourcekit-lsp/config.lsp:

{
  "swiftPM": {},
  "compilationDatabase": {},
  "fallbackBuildSystem": {},
  "defaultWorkspaceType": "swiftPM",
  "backgroundIndexing": true,
  "backgroundPreparationMode": "build"
}
1 Like
2

Yes, although background indexing only happens when you save the file because it operates with on-disk contents.

My bet would be that you either

  1. Don’t save the file
  2. Don’t send a workspace/didChangedWatchedFiles notification on the file save and thus SourceKit-LSP doesn’t know that the file has been modified.
  3. Background indexing is disabled

Also, swift build resolving the problem indicates that background indexing is disabled for some reason, so we use the index that’s generated during the actual build. You should be able to see how SourceKit-LSP resolves the initialization options and if something disables background indexing by enabling extended logging and running the following while SourceKit-LSP is launched

log stream --predicate 'subsystem CONTAINS "sourcekit-lsp"' --signpost --debug
3

Isn't it enough to send textDocument/didSave notification on file save?

4

Thank you for your reply!

JetBrains IDEs typically don't save the files. Saving it manually made indexing work.
I verified that the logs showed that background indexing was enabled for the workspace.

I didn't know that a save is needed to make certain operations work. I naively assumed that the LSP text sync and the other events would provide all needed data to SourceKit LSP.

I'm not yet sure how to handle this reliable, though.

For example, if a user invokes "Find references" and my client first saves the open files and then invokes the LSP request textDocument/references, how could I make sure that the request operates on the updated index and not on the incomplete index data?

I'm assuming that background indexing would be performed async.

Would workspace/synchronize be suitable for this or does it not sync on background indexing?

Thanks a lot!

update: I'm sorry, I only noticed the synchronize request but did not check the parameters. Parameter index seems suitable. Would sending synchronize be the right approach to ensure an updated index before a rename?

5

If you need to guarantee up-to-date information, sending a synchronize is the only and correct option that you have. However, depending on the changes, background indexing may take a while (handful of seconds is fairly common) if targets need to get re-prepared, so I’m not sure if you want to block editor functionality on it. In VS Code, we have decided that it’s OK to show users data that’s potentially slightly outdated.

Long term, it would be great to also support indexing of in-memory files but that would require a major redesign / rewrite of indexstore-db.

2 Likes