Ladybird Browser and Swift - Garbage Collection

Using Swift
1

Hello!

I've been chatting about this with several Swift team members over the past few weeks/months, but I figured a post here on the forums was due.

At the Ladybird Browser project we've been experimenting with more and more Swift C++ Interop lately.

In general our goals with Swift fall into a few categories. The first one is that of course, we want to incrementally start writing new code in a memory-safe language. That's the current zeitgeist, and we don't want to be left behind! But how to achieve that is still a bit murky. The main idea is that the best places to add Swift to are places where we are processing untrusted input, and places where fearless concurrency could add some sizeable performance, ergonomics, and correctness gains. At its core a Web Engine is a big event loop with JavaScript only executing from a single Actor. The spec waves its hands and says "do this in parallel", but in practice we just throw everything on the single-threaded event loop anyway.

Prototypes

The first thing I looked at was converting some of our AppKit/Cocoa UI code for macOS into Swift, which was pretty straightforward. It required pulling in a lot of CMake special sauce from Apple's example repo, but hopefully over time more and more of this logic can be pulled into CMake itself.

The next thing I prototyped was rewriting our HTML Tokenizer in Swift. The current version is a straight port of the C++ code, for the most part. I'm sure there's a more idiomatic way to write it. As with all things in Ladybird though, the first priority with all code is to implement the relevant specification exactly as written, and only add optimizations when weird things start showing up in profiles :).

Rewriting existing code is cool and all, but how do we prevent a flood contributors from trying to rewrite the world while we're trying to meet our alpha deadline in two years? The ideal situation would be creating a framework and template for how to start writing new code and new features in Swift.

The Elephant in the Room

The main problem with writing new code is that for the most part, we've already isolated untrusted parsing of data into separate processes. Images are decoded in a special ImageDecoder process that is allowed, and even encouraged to crash and be restarted if any funny business happens when parsing raw image data bytes. All network traffic happens in our RequestServer process, which uses libcurl under the hood. There's still a few places like the HTML tokenizer, the JavaScript parser, font loading, and video decoding that could use some isolation, but in general those parts of the code are already written.

So what sorts of new code are we writing today? Well, for the most part it's going to be CSS parsing and tree-building, layout fixes, and fleshing out the missing Web APIs. LibJS, our custom JS engine, already passes a staggering 95% of test262, the official ECMAScript test suite. LibWasm, our custom WebAssembly interpreter, already passes 100% of the WebAssembly test suite as well.

The elephant in the room about all the areas under active and rapid development is that almost everything in LibWeb is garbage collected. We've placed the memory management of our DOM, our HTML APIs, and everything to support the HTML and CSS specifications (and any other w3c/whatwg specification) under the control of our LibJS (now split into LibGC) garbage collector. Our GC is quite naive, being a single-threaded stop-the-world conservative stack-scan based monolith. But it gets the job done, and we're not doing too bad on the Web Platform Tests these days, passing 1.6M out of 1.85M total tests.

Swift and GC?

I came into this thinking that we would have to do a lot of shenanigans to let Swift types interop with our garbage collector. But I think now that it might not be so bad.

The simplest case is when a C++ type wants to own a Swift type to defer some work to it. Thanks to the existing C++ interop capabilities in Swift 6.0, we don't have to do much of anything. We can just store a Swift type from the generated C++ bridging header as a data member in our C++ type, and presto. Memory managed. Whether it's a value type or an ARC type, the bindings hide that from us.

The more interesting case is when we want Swift types to participate in our GC. In an ideal world, we could add an annotation to our Swift types to say "You are blessed with GC, visit your edges when requested for the mark-and-sweep pass and all will be ok". Unfortunately, none of our contributors are swift-syntax or Macro gurus, so we need to do things the hard way first.

After chatting with the Interop team last week at their office hours, and with @grynspan on the Swift OSS Slack for far too much time this weekend (doesn't he have things to do on the weekend? :P ), we came up with a design that will allow us to import foreign types into our custom GC, and teach swift types how to allocate themselves into a "foreign cell" for ownership by someone more GC-aware than they are.

The PR is here if you'd like to take a look at the cursed FFI we cooked up.

This first cut plays games with void* and other type erasure to store both a this pointer and hold onto a reference for a Swift type's metadata pointer to make a JNI-like API for initializing, finalizing, destroying, and visiting edges of Swift-GC types. I'm looking forward to prototyping some mixed C++/Swift patches that use this new interface to try it out. I'm sure we'll learn along the way how to make it more ergonomic.

If anyone on the forums here has ideas on how to make this task easier, or how to make it look as snazzy as the swift-java work that's going on, I'd love to hear your thoughts! Either here on the forums, or in the # swift channel on the Ladybird discord.

Thanks,
Andrew Kaster

27 Likes
2

Neat!

Right. And this is a way to bend the curve toward memory safety without having to stop the world and rewrite things.

Interesting! I'm accustomed to WebKit's use of reference counting for the DOM et al, so this is a different challenge.

Presumably the idea here is that this is a member macro that synthesizes visit_edges to visit all of the stored properties? Anyway, I do think it's generally good to start with doing things the manual way until it's right, then layering the macros on top once you're ready.

I left some comments on the PR. The swift-java bindings are using the type metadata pointer (Any.Type on the Swift side) in a similar manner, so that the non-Swift language (in our case, Java) can allocate memory in which it can store an arbitrary Swift instance.

Doug

5 Likes
3

Interesting! I'm accustomed to WebKit's use of reference counting for the DOM et al, so this is a different challenge.

Yeah, the story goes that back in the day Andreas wanted to do something similar in WebKit, but he could never get the support for it. After moving from the pattern of RefCounted types + JS Wrappers to directly GC-allocating our DOM and Web APIs, we found that it was significantly easier to avoid leaks and cycles. It's still easy to end up leaking an entire document if you create a strong GC root via a Handle<T> somewhere. Thankfully those require less work than figuring out who is supposed to own who, and what links are supposed to be weak, etc.

Regarding the macros, for sure, we need to manually write out the shape a few times before trying to code-gen it.

Thanks for your feedback on the PR! As a heads up we use a fully rebase-based workflow on Ladybird, so I'll be fixup-squashing any feedback into the relevant commit so we can cherry-pick the whole stack of commits at once on top of our main branch.

Andrew

4 Likes
4

Really cool effort, cheers!

As for lessons wrt. GC and Swift... we're early on the Java interop adventure but some ideas I can share already are: for short lived calls where Swift objects are allocated in the other language but are not expected to outlive the scope of the calls we effectively reference count them on the Java side and "this was supposed to be destroyable by now, let's check and do so". This is nice because registering "let me know when this object is GC-ed" is expensive on the JVM.

But we do also offer a way to "decrement the reference count when this isn't retained in the JVM" anymore... The problem with that is that it's expensive to do this in the JVM because it adds new GC roots.

I'm wondering if this may be cheaper for you all if the GC is perhaps simpler? Overall you probably can rely on building that kind of "release when no longer used in GC world" and just rely on a single retain from GC world this way.

The other concern about this in JVM world is that if an object got promoted to some old generation it can be quite long between GC actually triggering for such old/tenured generation (who knows, with huge heaps minutes even), so there's a concern in JVM land about this leading to not freeing resources timely, because the JVM has no idea that it potentially is keeping a lot of native memory alive etc. That's why at least in Java/Swift we'll have both kinds of memory management - encapsulated in the SwiftArena type you can look up in the project (the releasing mode isn't implemented yet).

But we've not done much benchmarking and real world testing with this yet, but that's just some of the thoughts about marrying the GC and ARC worlds of the two languages.

Just some food for thoughts; Really looking forward to seeing your integration work out! Let us know if you bump into any trouble or have questions!

3 Likes
5

One simplification for our efforts over swift-java is that our GC is quite naive.

It's non-moving, non-compacting, and non-generational. I'm sure there's other GC literature terms that don't apply to it as well :slight_smile: .

I'd rather be very judicious with allowing folks to add strong GC roots. We've found that whenever there's a leak in LibWeb or LibJS, there's probably some GC-allocated object holding onto a strong GC root (e.g. a GC::Handle<T>) at the center of it.

If possible, forcing the lifetime of most objects to either be eternal or linked to a specific HTML Document is likely to keep our house in order. Of course, as we start trying to do more and more things asynchronously that becomes a bit more difficult. But in theory, every job or piece of async work has a well-defined lifetime that we can end by cancelling async work tied to a particular document or a particular navigation history entry. Or tied to a specific Origin (or Storage Key, as the storage standard defines them).

Not sure how well that will scale, as I've already encountered problems where our naive GC isn't compatible with common Swift-isms. Namely, swift-testing creates a thread pool to run jobs on, and our Heap object stashes the stack extents for the thread it was created on. So if you have more than one test in a process, you're likely to end up trying to collect() from a thread different than the one the Heap was created on. In this case, our conservative stack scan goes on an adventure through (potentially) un-mapped memory looking for the end of another thread's stack compared to the current stack pointer. Not great. It can be 'fixed' with a thread_local around the "global" heap, but our concepts of a single Heap and single JavaScript Agent (called VM in our implementation) per-process are going to make actually taking advantage of Swift Actors and Async interesting in the presence of GC'd Swift objects.

1 Like
6

If you're non-moving etc that indeed may make things much easier for the integration! Could more directly pass your objects to Swift without the heroics JNI goes through to make such happen :sweat_smile:

Also since you own the GC runtime you can do nicer tricks like attaching the lifetimes to some existing objects like you said -- that sounds very promising :slight_smile:

You're likely to hit this with any swift async code I think.

Technically, there is a way to hook the global executor enqueues like we did in foundationdb (talk about it here). You could use that to avoid Swift's global thread pools entirely and enqueue all the work to some threads you own -- you probably have some eventloop that you could execute the work on :thinking:

We are also thinking about making a proper Swift API for this capability, but nothing to share yet. It's been on our minds in order to allow end users customize the global concurrent and main actor executors.

Maybe this might get you out of jail with threading interoperability?

1 Like
7

Hmm, we could use this for Android too :) But maybe I'll wait for a public API...