Concurrency-aware debugging with Embedded Swift platform runtimes

Platform Embedded
, ,
1

Hello all,

Following the earlier discussion with @felipepiovezan in the Multicore Concurrency thread and his follow-up on swiftlang/swift#91047, I wanted to split Embedded Swift debugger support into a dedicated discussion.

The original question was how LLDB can determine which Swift task is executing on each thread. Since then, the Embedded Concurrency implementation has changed in a way that makes the question more general.

Embedded now builds Concurrency against the Embedded Threading backend. The runtime’s reserved TLS entries, including the current ActiveTask, go through Swift’s ThreadLocalStorage abstraction and ultimately through the swift_tls* platform hooks. The concrete storage behind those hooks is selected by the EmbeddedPlatform implementation linked into the final program.

That implementation might use:

  • native pthread thread-specific storage
  • dynamically allocated pthread keys
  • RTOS-provided task-local storage
  • a fixed per-core table
  • a single global slot for a single-threaded system
    or a completely unknown platform-specific alternative.

This creates a mismatch with the current debug ABI. _swift_concurrency_debug_internal_layout_version includes a _concurrency_current_task_storage_kind, which tells LLDB how the runtime stores the current task. The enum already recognizes that there are several possible storage strategies, but the Embedded Concurrency library cannot fully determine that strategy when it is built. It only knows that it will call the platform abstraction; the concrete representation is selected later at link time.

I think it is useful to separate this into two problems:

What is an execution context?
On a hosted platform this is usually an OS thread already represented by LLDB. On embedded targets it might instead be an RTOS task, a CPU core, or the only execution context in a single-threaded program. Some platforms may therefore require debugger or debug-server support before Swift can even answer questions about tasks.

Given an execution context, where is its current Swift task pointer?
Once LLDB has an appropriate thread or execution-context representation, it needs a fast and passive way to locate the corresponding ActiveTask storage.

In our earlier discussion, I suggested letting LLDB call a runtime or platform accessor. Felipe pointed out an important constraint:

This mapping of Threads -> Task has to be fast. It happens on every single stop [...] This is particularly important when communication between the debugger and the target is slower than local.

That makes an inferior call through _swift_tls_get unattractive as the general solution. It would require executing target code in the context of each thread, could perturb target state, and may not be available on bare-metal or remote targets.

Pausing here, hope this provides enough context to start a conversation.

Cheers.

3 Likes
Exploring Multicore Concurrency for Embedded Swift
Exploring Multicore Concurrency for Embedded Swift
2

Thank you for the summary, Gonzalo!

It's a sad common theme that most swift features get merged without a debugging story, but here we are; let's figure out what's the best way of remedying this.

I would like to offer one further way of looking at the current challenge; after the related changes got merged, the concurrency library can sometimes describe the thread local storage implementation used: it can do so for non-embedded cases, but it cannot do so for embedded cases.

I think it is useful to separate this into two problems:

What is an execution context?

While this is a real problem, I don't believe it is related to the problem at hand. The mapping of system-level abstractions into the "thread" abstraction LLDB presents to users is a well known problem, and lives on a different layer of the debugger. I'd like to make the assumption LLDB can do this for whatever system is being debugged (if it can't, then it can't debug any process for that system), and focus on the problems that surface once the mapping is complete.

Given an execution context, where is its current Swift task pointer?

As you summarised, LLDB made the assumption that the concurrency library fully describes everything about concurrency, which is no longer true. I see two ways around this:

  • Change the non-embedded case, so that we have a uniform representation.
  • Keep the non-embedded case as is, and add a new level of indirection for embedded: set the _concurrency_current_task_storage_kind to a new value called definedInShim, and create a new debug contract for those shims: they must expose some global symbol defining their TLS implementation. LLDB would then know to query a different symbol.

I'm leaning towards the second approach.

To answer something you proposed on the other thread:

Would it make sense to start with a PR to add an unavailable storage kind?

I think this would make the problem worse. The shim libswiftEmbeddedPlatformMultiThreadedDarwin uses the exact same implementation as the non embedded case on Darwin, so we patched LLDB to just assume that shim until we have a better story. Using unavailable would regress that case.

1 Like
3

Also, please let me know if I misunderstood what you meant by execution context!

4

Thanks, Felipe! Hopefully I can help solve this as much as I helped exacerbate the problem :sweat_smile:

Also, please let me know if I misunderstood what you meant by execution context!

Let me start with this. I’d define a platform execution context as the independently executing platform state associated with context-local storage: an OS thread on hosted systems, a hardware core or hart on bare-metal systems, or a scheduler-managed context on an RTOS.

I would like to offer one further way of looking at the current challenge; after the related changes got merged, the concurrency library can sometimes describe the thread local storage implementation used: it can do so for non-embedded cases, but it cannot do so for embedded cases.

Yes, thanks for the clarification. I was scoping my wording to Embedded debuggability, but this is a more accurate characterization of the current situation.

What is an execution context?

While this is a real problem, I don't believe it is related to the problem at hand. The mapping of system-level abstractions into the "thread" abstraction LLDB presents to users is a well known problem, and lives on a different layer of the debugger.

I have a slightly different view here, although I think the difference may mostly be about layering.

The way I understand it, obtaining the current task pointer is a function of the debugger-visible execution context. The existing implementations work because LLDB knows how to map an LLDB Thread to either compiler TLS or Darwin TSD. On a bare-metal target, a debugger might expose each core as an LLDB Thread, while the PAL stores current-task pointers in a CPU-indexed table. Some layer still needs to define how that debugger thread maps to the corresponding CPU index.

I agree that this mapping does not necessarily belong in the Swift Concurrency plugin. We can assume that the platform integration has already exposed the relevant contexts as LLDB threads and focus this contract on locating the current task once that mapping exists. I just want to make the prerequisite explicit, because without it the Embedded debugging story might remain incomplete.

Keep the non-embedded case as is, and add a new level of indirection for embedded: set the _concurrency_current_task_storage_kind to a new value called definedInShim, and create a new debug contract for those shims: they must expose some global symbol defining their TLS implementation. LLDB would then know to query a different symbol.

I was exploring this same approach, and I think it fits well. Concurrency explicitly declares that the decision has been delegated, and only then does LLDB consult the shim-provided value. That avoids the ambiguity of looking opportunistically for an optional secondary symbol.

Along with that, I’ve been exploring a few lookup options that a shim could select:

  • A fixed indexed-storage fast path for platforms with a small, stable set of execution contexts, such as CPU cores. LLDB could read the task pointer directly from a known table once it has the context index.
  • A helper that returns the address of the current context’s task-pointer slot. LLDB would call it once per debugger-visible thread and cache the address, assuming that address remains stable for the lifetime of the thread.
  • A fully general helper that returns the current task pointer and must be called on each lookup. This would support platforms that cannot expose stable storage, with the inferior-call performance cost you described earlier.

I think these options could give platforms a useful range of contracts, from direct memory access to a fully procedural lookup, depending on what their execution model can support. Does that sound like a reasonable shape for the shim-side debug contract?

5

I think I agree with everything you said.

Would you (or myself, either way is fine) like to give it a try to expose the definedInShim enum value and have the "multiple threads" shim define its "static tls key" entry defined? It would be effectively an NFC patch that lays out the foundation for other shims.

2 Likes
6

Awesome!

Sure, I can take care of that. Let me send you a PR later today / tomorrow.

1 Like
7

Alright, sorry this took a bit longer.

I put up two options. I’m leaning towards the first, but it’s a broader change, and I don’t want to stretch the original storage-kind field’s meaning too much :)

Option 1: Deferring to the platform library becomes a flag, separate from the concrete storage kinds. LLDB follows that indirection once and rejects a platform value that tries to defer again. The storage-kind definitions also move into a shared C-compatible header so platform implementers can use named ABI values.

Option 2: A more conservative approach that adds a platform_defined storage kind to indicate the indirection. It keeps the definitions in the existing debug header, with the PAL publishing the corresponding numeric constant.

Here are the LLDB changes for Option 1. If Option 2 is a better fit, adapting the decoder should be a small change.

Open to thoughts and feedback!

8

Thank you for putting those together, @Gonzalo_Larralde ! I like option 1 as well, and left some minor points in the two PRs.

Next step is to change the destination branch for the LLDB code to stable/21.x and run testing on those. I can do some local testing as well

(sorry for the delay, I was on vacation!)

1 Like
9

Btw I've just run some tests locally and things seem to work!