High Memory Usage of sourcekit-lsp in VSCode on Ubuntu 20.04

[martinlau]

I am experiencing an issue where after some time, sourcekit-lsp consumes almost all my memory when programming in VSCode on Ubuntu 20.04. I have tried both Swift 5.10 and Swift 6, but the problem persists. This high memory usage is causing significant inconvenience. Does anyone have suggestions on why this might be happening or how to resolve it?

[Genaro-Chris (Genaro Christian)]

Yeah, I noticed it since swift 5.5

[taylorswift]

same problem on Ubuntu 24.04. oftentimes, sourcekit-lsp is by far the biggest memory user on my system:

[ahoppen (Alex Hoppen)]

We have an issue tracking the problem (High memory usage · Issue #1541 · swiftlang/sourcekit-lsp · GitHub) but haven’t been able to gain any insight into what’s using the memory yet.

I am not super familiar with memory debugging on Linux, in case somebody has a way to capture what objects are using the memory (something similar to a memgraph on macOS), that would be tremendously helpful.

[martinlau]

@ahoppen

We have an issue tracking the problem (High memory usage · Issue #1541 · swiftlang/sourcekit-lsp · GitHub) but haven’t been able to gain any insight into what’s using the memory yet.

I am not super familiar with memory debugging on Linux, in case somebody has a way to capture what objects are using the memory (something similar to a memgraph on macOS), that would be tremendously helpful.

I'm not sure if heaptrack meets your requirements. After a preliminary analysis using heaptrack, I found that SourceKit::WorkQueue::DispatchData::callAndDelete() had the highest memory consumption. However, as I’m not very familiar with this tool, I cannot provide accurate results.

GitHub - KDE/heaptrack: A heap memory profiler for Linux

[ahoppen (Alex Hoppen)]

Thanks @martinlau, that’s quite helpful. I will try and find the time to look at it and update the issue I referenced above once I have news.

[ahoppen (Alex Hoppen)]

Thanks for the tip with heaptrack, it enabled me to track the issue down to Fix a memory leak in `DispatchData.withUnsafeBytes` by ahoppen · Pull Request #850 · swiftlang/swift-corelibs-libdispatch · GitHub.

One question: Did you run heaptrack on a toolchain you downloaded from swift.org or did you build a toolchain yourself? I wasn’t able to get any symbol information out of heaptrack without debug information for the binaries that I was testing. Wondering if there’s a trick.

[martinlau]

@ahoppen

Thanks for the tip with heaptrack, it enabled me to track the issue down to Fix a memory leak in DispatchData.withUnsafeBytes by ahoppen · Pull Request #850 · swiftlang/swift-corelibs-libdispatch · GitHub .

One question: Did you run heaptrack on a toolchain you downloaded from swift.org or did you build a toolchain yourself? I wasn’t able to get any symbol information out of heaptrack without debug information for the binaries that I was testing. Wondering if there’s a trick.

Glad to see the progress! For heaptrack to read symbol information, it might be necessary to build the toolchain yourself for testing. When I ran heaptrack on the toolchain downloaded from swift.org, I couldn’t see symbol information—only a general memory trace.

[ahoppen (Alex Hoppen)]

OK, I figured that you could see symbol information because you pointed to SourceKit::WorkQueue::DispatchData::callAndDelete().

[martinlau]

Hi @ahoppen,

Based on my recent heaptrack memory analysis, I haven't found high memory usage related to the symbol SourceKit::WorkQueue::DispatchData::callAndDelete() (currently using Swift version 6.0.2). Instead, the memory consumption appears to be primarily focused on _dispatch_operation_perform. This might be related to Fix a memory leak in DispatchData.withUnsafeBytes #850.
If needed, I can provide the gz files from my last two analyses.

[ahoppen (Alex Hoppen)]

Odd, but your new findings match the memory leak that I fixed in Fix a memory leak in `DispatchData.withUnsafeBytes` by ahoppen · Pull Request #850 · swiftlang/swift-corelibs-libdispatch · GitHub.