How does the linker "know" to link the embedded runtime?

Hi @kubamracek @rauhul @Philippe_Hausler I hope you're well?

I'm trying to get embedded mode working on the S4A IDE and toolchain. The basics are there but programs crash when class instances go out of scope and get released by my cut down ARC in our cut down runtime. Debugging with avr-gdb, when the object is released, _swift_release_dealloc looks in the type metadata for the virtual destructor at the standard location, which is a negative offset from the type metadata in the class instance "isa" pointer. It looks like embedded swift class metadata is much simpler and there's no "full type metadata" before it, so this operation finds and jumps to a garbage pointer causing a crash.

The fix is pretty simple, make sure I build and link to the new, simple, pure swift runtime that embedded mode is supposed to use and it should all work. I didn't think much about it at first, then I thought, "how does this work?" ... the new runtime (which I think is basically just in the EmbeddedRuntime.swift file in the standard library core directory?) basically produces the same symbols as the standard C++ runtime (swift_retain, swift_release, swift_deletedMethodError, swift_once, swift_allocObject... etc.)

Why don't you get linker errors for duplicate symbols? How does the linker know to use the "embedded" ones? Sorry if this is a dumb question!

Also, is there any documentation of the metadata layout for the new class runtime?

Any advice or information is helpful, thanks!

Carl

Embedded doesn’t link against the actual Swift runtime, so there’s no preexisting symbols that the linker can complain about.

1 Like

The linker doesn't know anything about the Swift runtime. The linker just links in libraries that it's told to link in. The compiler simply doesn't tell it to link against the regular runtime in embedded mode.

1 Like

So the methods are found in the swift/stdlib/public/core/EmbeddedRuntime.swift at main · apple/swift · GitHub file (as you found already) not to be confused with the c++ runtime files (those are not used for embedded). Technically we could eventually one day have most of the runtime implemented in Swift itself. The embedded mode does a few things; one of which is avoid the linkage to the swift runtime code.

Your crash tells me that you are somehow linking the C++ code that comprises the Swift runtime. None of these files should be part of the embedded builds.

This should be the entry point at which a deallocating reference type should be hitting for embedded mode. That does not need to call out to metadata (since that is not part of the emissions of the layouts of reference types in embedded mode).

3 Likes

Thank you so much everyone.

Yes @Philippe_Hausler ... we have a heavily trimmed down and customised standard library and a very, very trimmed down C/C++ runtime on our platform. It's basically just ARC, HeapObject.cpp, KnownMetadata.cpp, RefCount.cpp and a few headers. (We also have super basic stubs for things like swift_once and swift_slowAlloc ... exclusivity is disabled on our platform as it's single threaded.)

Heavy runtime is an absolute no-no for us, as you know. For years we had nothing except a no-op stub for swift_once!

Embedded mode is opt-in for builds on our platform. I'm going to change embedded builds not to include our micro C++ runtime and use the new embedded one today. Exciting. It's very cool. :slight_smile:

I have a somewhat out of date embedded swift (from January) so the call site for swift_deallocObject in EmbeddedRuntime.swift on our version is older (it still has the C stub which I think was needed to effect calling convention voodoo... which we don't need anyway on AVR). I'll update swift to head soon, when I've got everything building with the January version. Baby steps.

Thanks so much for the clear and super helpful information as always. I'm guessing you guys are all getting ready for next week. I hope you all have a fantastic time and I look forward to seeing talks about embedded swift (either there or somewhere else). Wish I could come again some day (when $ and time permit). It's been 6 years since my last one!!

Cheers,
Carl

2 Likes

@carlos42421 Are you maybe running into Bug in Embedded/ARC/WebAssembly combination? / Bug in Embedded/ARC/WebAssembly combination? · Issue #73768 · apple/swift · GitHub ?

1 Like

I saw your post with interest @turbolent ... I think my case is a bit simpler really! It's not a compiler bug, I just need to only build (and link) the new embedded runtime when I'm doing builds with embedded mode enabled. It's taking a bit of massaging to make it all fit with my existing parts, but I'll get it working over the next few days I'm sure.

It's very nice to see people working actively with the embedded mode, especially in slightly unusual ways, as we are. I mentioned your forum post to the community on our Slack channels!

Carl

2 Likes

Do you happen to have a link to your slack instance? I know @parispittman is trying to collect some community links

1 Like

@pdshelley any ideas on this? You set up our community Slack, is there a way we can give a link so people can join without us having to email them an invitation? It would be great to get more people in with minimal friction I think?

So I think we've already figured out what the solution is ("don't link the standard runtime at all"), but let me clarify how does EmbeddedRuntime.swift get "automatically" linked: Since it's implemented in Swift, it's treated like any other piece of Swift code and it's part of the standard library (Swift.swiftmodule). Because any Swift code implicitly imports the standard library, the embedded runtime (when in embedded mode) gets imported too. And since the entire contents of the embedded standard library has the "emit into clients" behavior, the compiler (not the linker) will compile the embedded runtime into the object file when building the client code.

Also, is there any documentation of the metadata layout for the new class runtime?

Not really, sounds like a good idea to add such a doc. But for now, see this: swift/lib/IRGen/ClassMetadataVisitor.h at main · apple/swift · GitHub. The layout is essentially: (1) superclass metadata pointer, (2) destructor, (3) ivar destroyer, (4) individual vtable entries. There is no "negative offset" magic.

3 Likes

I believe this link should let people join the S4A Slack community:
Slack

2 Likes