Compilation caching integration with toolchain-cas-plugin

Development Compiler
1

In order to better support distributed compilation caching with Swift open source toolchains, @akyrtzi , @cachemeifyoucan and I have requested a new swiftlang repository, swiftlang/toolchain-cas-plugin. This repository will contain an implementation of the LLVM CAS plugin API to support cross-platform distributed caching in SwiftPM and other swift build systems via a shared library distributed as part of the toolchain. In order to approve the repository, the ecosystem steering group has requested:

  • A diagram that includes all components in a full setup including any remote component
  • Any IPC or service protocol that developers are expected to implement

This post attempts to address those requests.

The following components are involved when using distributed caching via a SwiftPM build:

Toolchain components:

  1. SwiftPM - The tool users directly interact with. Maps user-specified caching configuration options to the build request provided to the underlying build system. Users interact with SwiftPM to configure the path to a socket used to communicate with their preferred caching daemon, and to initiate the build. Developers of distributed caching backends can choose to automate the setup process by scripting the SwiftPM CLI.
  2. Swift Build - Implementation detail of the toolchain. Uses the build request provided by SwiftPM to configure Swift/Clang tasks appropriately to enable caching and coordinates the scheduling of frontend, cache query, and upload tasks. When using compilation caching, interaction with the CAS is delegated to the compilers/compiler drivers and may fallback to the builtin LLVM CAS implementation if no LLVM CAS plugin is available. When using generic task caching for other build tools, Swift Build requires and uses the provided plugin’s API directly. Whether or not a plugin is available, it is important for all cached tasks in a build to share a CAS backend, to ensure size limits and eviction policies are consistently applied.
  3. Clang/Swift compilers - Implement support for dependency scanning and performing cached compiles. When configured to use remote caching, they load toolchain-cas-plugin at runtime and interact with it via the LLVM CAS plugin API.
  4. toolchain-cas-plugin - Implementation detail of the toolchain whose source is in the new repository. A shared library which implements the LLVM CAS plugin API described by llvm-project/llvm/include/llvm-c/CAS at next · swiftlang/llvm-project · GitHub. The CAS plugin provides both a local on-disk CAS and the ability to connect with a local caching daemon via a GRPC-based IPC mechanism. It is loaded at runtime by the compilers and/or build system. This repository will contain the canonical copy of the gRPC API used to communicate with caching daemon implementations.

Interchangeable user-provided components:

  1. Caching daemon - Handles GRPC requests from toolchain-cas-plugin. The implementation is expected to vary depending on the type of remote caching service being used and the context builds are running in. For example, a daemon running inside a CI environment’s ephemeral VM may handle cache uploads differently from a long lived daemon being used across many desktop builds.
  2. Remote caching service - Receives requests from the caching daemon via some implementation-defined mechanism. To allow remote caching to support a wide variety of potential use cases, the toolchain does not prescribe any particular API or authentication mechanism used to communicate between the caching daemon and remote service.

caching
caching2621×379 70.9 KB

The gRPC protocol is defined by toolchain-cas-plugin and implemented by the caching daemon of a particular distributed caching implementation. The spec for this API is currently in swiftlang/llvm-project (llvm-project/llvm/lib/RemoteCachingService/RemoteCacheProto at next · swiftlang/llvm-project · GitHub) and is planned to move to swiftlang/toolchain-cas-plugin. Implementations may use any mechanism they choose to implement communication between a caching daemon and remote service.

5 Likes
2

Thanks for writing this up. It's great to see the plugin and the protocol moving into the open. That will help us a lot.

Some context on where we're coming from: at Tuist we run the "caching daemon" and "remote caching service" parts of this diagram in production for Xcode builds. We started on the current CASDBService / KeyValueDB protocol via COMPILATION_CACHE_REMOTE_SERVICE_PATH. We later moved to our own CAS plugin, loaded through COMPILATION_CACHE_ENABLE_PLUGIN and COMPILATION_CACHE_PLUGIN_PATH, which wraps the toolchain plugin and does the remote work itself. The reason was deep module graphs: on a 194-module fixture with the cache about 2ms away, a fully warm remote build took 325-445s against a 274s no-cache baseline, despite 4447/4449 cache hits. We ruled out our daemon and transport, and spindumps taken during the stalls showed every process idle while the plugin's gRPC client threads went unscheduled for 30-50s at a time. With the remote work moved into our plugin, the same build takes 113.6s, against a 105.5s floor for replaying from a warm local CAS. We couldn't dig further without the source, so having it open is very, very welcome.

While the protocol is still being defined, here's what we've learned and what we'd love to see going forward:

1. Fetching a result and everything it references in one round trip. Today a lookup returns a value, and each referenced object is a separate Get/Load. Because LLVM CAS results are graphs of small nodes, a client only discovers the next level after loading the current one, and latency per object dominates on deep graphs. We learned this the hard way: the first version of our own plugin fetched one key and one object at a time, and on a link with about 60ms of latency it took 701-727s against an 88s floor on a project with a deep graph. What fixed it was borrowing the model of Bazel's Remote Execution API, where GetActionResult returns the digest of every output up front, can inline small outputs, and leaves the rest to BatchReadBlobs. We map each node to a blob and return the whole closure with the lookup. On that same link, our plugin now builds in 134s from an empty local CAS, against 193-211s through the current remote service path. A batch Get, or a lookup variant that returns the closure, would let daemons do the same.

2. Clear rules for local-only and global lookups. Swift Build issues local-only queries from its scheduling thread and global ones from the cache query tasks. If a local-only query ever waits on the daemon, the whole build is serialized behind network round trips. It would help to document that local-only queries never reach the daemon, and that global ones may run concurrently.

3. Never serving a key whose objects aren't available. A reader should never get a hit whose value references objects it can't retrieve. That can happen through ordering (the key is written before its objects finish uploading) and, more commonly in our experience, through eviction (the remote drops objects but keeps the key). We'd suggest three things. First, the plugin's put path writes the key only after every referenced object has uploaded successfully. Second, the protocol says a daemon SHOULD only return a hit when the objects it references are available, the same way REAPI requires of GetActionResult. Today a daemon can't check that, because values are an opaque map; returning the closure with the lookup (point 1) would give it what it needs. Third, a hit whose objects turn out to be missing should degrade to a miss everywhere. Today Swift recompiles in that case, but clang fails the build, so one evicted object can break a build that would otherwise just be slower.

4. Versioning or a capabilities call. Daemons tend to outlive toolchain upgrades. Something like REAPI's GetCapabilities, which returns the supported API version range, limits, and whether writes are allowed, would let the plugin detect an incompatible daemon and fall back to local caching cleanly, instead of misreading responses.

5. Bounding the local store. We've seen the on-disk CAS grow well past COMPILATION_CACHE_LIMIT_SIZE (9x in one test) unless something calls llcas_cas_prune_ondisk_data after the last handle closes. Will the toolchain plugin prune on its own, or is that expected to be the client's job?

6. Remote caching for clang shouldn't depend on the remote service path. Swift Build decides whether a remote cache exists purely from remoteServicePath != nil (CASOptions.hasRemoteCache). That one check controls both uploading clang and module outputs and scheduling the tasks that fetch them back. With only a plugin configured, every C/ObjC/PCM/PCH compile stays local: on one app, 259 of 930 tasks were cached without it and 930 of 930 with it. Today we set COMPILATION_CACHE_REMOTE_SERVICE_PATH to our daemon's socket just to turn that on, and our plugin consumes the option so the toolchain plugin's own remote client never connects. It would be cleaner if the plugin could declare that it handles remote caching. Long term, we'd much rather drop our custom plugin and go back to providing just a remote socket, as originally intended. That would be simpler for our users and for us. Unfortunately, we can't do that with the performance hit we measured on the current remote service path.

Additionally, is the plan for Xcode to ship this same plugin, or is it scoped to open-source toolchains for now?

Once the repository exists, we're happy to reproduce our deep-graph case against the stock plugin with a minimal daemon, and contribute fixes or benchmarks upstream.

3

Preface this is my personal opinion and not a representation of the Ecosystem Steering Group.

Overall, I continue to be hugely in favor of solving compilation caching but I would like us to have a firm understanding of the holistic picture before we create repositories and decide on gRPC proto definitions. My main motivation behind understanding this more is that we are about to define abstractions that we expect the ecosystem to implement. This means those abstraction contracts need clear definitions and stability guarantees. This goes to @fortmarek 4th point about versioning.

I personally expect that any gRPC (or other protocol) service definition to:

This will lead to a flourishing and stable ecosystem building on top of those APIs.

Taking a step back and looking at the holistic picture. @owenv diagram ends the swiftlang provided story at the gRPC service definition that is spoken between the plugin and a user-provided daemon. I strongly believe that swiftlang should provide a common daemon that is shipped with the toolchain. This common daemon should define its own gRPC service. Furthermore, I believe that swiftlang should provide a self-deployable production-ready reference implementation of this gRPC service that can be backed by blob storage providers such as AWS S3 (similar to GitHub - buchgr/bazel-remote: A remote cache for Bazel · GitHub). This could be even something that the @server-workgroup builds. The goal of both of those would be to make the out-of-box experience for build caching that Swift provides as complete as possible.

I want to be clear though. I don't expect all of this to happen at once. This is clearly a large scope with many interconnected pieces. However, in my opinion it is very valuable to understand our target architecture since it will allow the community to build along with us.

4

If this is something the team would be willing to eventually land on, that'd be awesome. Maintaining the long-running daemon has taken a lot of iterations on our side to get right and its management makes life hard for any team/company wanting to roll their own implementation of remote caching.

Long-term, we'd love for the toolchain to allow us to specify a server URL and how to authenticate, and the toolchain would deal with the rest. Similar to Bazel (although we have some reservations around its credentials management). That way, teams could rely on stable foundations, regardless of the backend provider they may use for the remote cache.

1 Like
5

We have spent significant amount of time implementing Xcode Compilation Caching protocol (LLVM CAS).

To be brief, coming from Bazel/Gradle/Sbt/Buck/etc remote caching world LLVM CAS caching is the most complicated protocol to support yet it's the least efficient in actually saving build times.

Build caching with Swift / Xcode really needs to cache higher level build graph tasks rather than individual compiler calls and be configurable via xcodebuild / SwiftBuild by taking remote server URL, token and whichever other settings it needs.

It is unrealistic to expect all Xcode/Swift users to research 3rd party unix socket daemon implementations, have them running correctly as a system daemon, especially across Linux distros with no unified socket activated configuration like LaunchAgents on macOS. To add to this when you implement unix socket daemon it has no way of distinguishing multiple Xcode / Swift projects because none of the gRPC cache requests actually carry that information which complicates things tremendously and we have to run a daemon with unique unix socket per project just to distinguish them.

We strongly suggest considering caching higher level build graph actions, leaving enough flexibility in the future protocol for proper Test Caching and Index Caching and directly support remote cache URL + bearer token in the native build stack instead of local unix socket daemons.

We can go into more details on each item listed for pages and pages of forum threads and that's part of the problem to be frank, current protocol needs to be simplified both for end users and for companies/communities that try to provide its support.

Thank you for your time and for gathering the community feedback!

1 Like