Build times regression with Explicitly Built Modules

[chiragramani]

Hi,

We’ve been exploring the Explicitly Built Modules feature introduced in WWDC 2024 to understand its impact on build times.

Observation:
Enabling the feature resulted in a significant build time regression. Despite aligning macros, language versions, and settings as per the guidance, the regression persists.

Below are our benchmarks for reference:

Setup:

• Identical dependency graph and project setup.
• Unified configurations across all targets to prevent compilation of multiple module variants(e.g., UIKit) as highlighted in the WWDC session.
• XC Result bundles and reproducers are enclosed in this fb.
• Xcode 16.1, Tuist version: 4.20.0, WMO, Apple M3 Max, macOS 15.2, 64GB
• benchmark script and other setup details in the reproducers enclosed.
• Reproducer(without result bundles but they can be generated locally via ./benchmark.sh): GitHub - chiragramani/ExplicitlyBuildModulesRegressionRepro: Explicitly Build Modules Regression Repro

Benchmarks:

	Without Explicitly Built Modules	With Explicitly Built Modules	Regression
Run 1	149.1 seconds	2595 seconds	1639.4%
Run 2	150.7 seconds	2481 seconds	1545.8%
Run 3	151.6 seconds	2524 seconds	1565.1%
Avg			1576.8%

Based on the Xcode settings, it appears that this feature is still in an experimental stage. The objective of this thread is to understand any challenges others may have encountered with explicit module maps and to share/learn about potential workarounds. These insights will help us refine and optimize our setup effectively.

If we’ve missed any considerations for optimizing this setup, please let us know! Looking forward to insights from the community.

[ArtemC]

Thank you for reporting this and including a reproducer project @chiragramani!

While I can confirm that I see similar build performance characteristics on this synthetic workspace benchmark, and the scale of this sample is worth studying and optimizing for, I am curious how it was generated - is the overall structure of this reproducer meant to capture a real project that you are working on?

Given the massive scale in quantity of single-target subprojects, this workspace is not necessarily representative of real-world Application/Framework structure, so the build-time regression figures here are not the same as those developers will see at-desk in their projects.

We have been actively studying Explicitly Built Modules performance and making significant improvements across the various tools involved (recently, e.g. with changes like [Dependency Scanning] Parallelize Clang module queries by artemcm · Pull Request #76915 · swiftlang/swift · GitHub), and having extreme-scale examples like this is also helpful to that cause - stress-testing of the overall build system involved is a useful tool in measuring progress of our build time optimization work. That said, in the meantime, there's a simple improvement that can be made to your reproducer workspace:

Unified configurations across all targets to prevent compilation of multiple module variants(e.g., UIKit) as highlighted in the WWDC session.

Currently, module dependencies of targets of different projects in a workspace build are not yet able to be shared due to the fact that Xcode projects use a per-project working directory setting for the compiler. One thing you can do manually is overload the build setting COMPILER_WORKING_DIRECTORY for the workspace build to some value that is workspace-global and is consistent/shared across all of its sub-projects. While not bringing this synthetic workspace to parity with Implicitly Built modules yet, I am seeing over 3x improvement with this approach.

[chiragramani]

Thank you @ArtemC. We all really appreciate the effort going into optimizing Explicitly Built Modules performance - it's a significant win for the benefits it brings.

I am curious how it was generated - is the overall structure of this reproducer meant to capture a real project that you are working on?

Yes, the reproducer mimics the dependency graph of one of our heavily used app targets. However, it’s not entirely realistic as real modules typically consist of significantly more files, and few workspaces even have some of the modules unfocussed/pre-built. For simplicity, each target in the reproducer is limited to just two files. This setup was specifically designed to study the micro-level impact of the Explicitly Built Modules feature on a given dependency graph, when used at this scale.

One thing you can do manually is overload the build setting COMPILER_WORKING_DIRECTORY

Oh nice, appreciate the tip! I will try it out. Just to confirm, in our core projects, we do pass -working-directory in a consistent way. Does Xcode's COMPILER_WORKING_DIRECTORY resolve to that, or does it provide additional functionality beyond it?

[ArtemC]

@chiragramani

Just to confirm, in our core projects, we do pass -working-directory in a consistent way. Does Xcode's COMPILER_WORKING_DIRECTORY resolve to that, or does it provide additional functionality beyond it?

This should be largely equivalent, this build setting resolves to overloading -working-directory, yes.

[chiragramani]

I’m also exploring a slightly different approach. Since developers typically debug only a handful of focused modules - usually 3-4. I’m considering enabling explicitly built modules feature only for these targeted modules while keeping the rest of the project unchanged. (given the build performance regression observed when enabling it across the entire project at the scale demonstrated in the repro.)

In a sample repro project, enabling explicitly built modules feature reduced the time for the first po in LLDB from an average of 25 seconds (over 3 runs) to just 0.8 seconds. (clang module cache was cleaned and the lldb-rpc-server was terminated) This is a game changer for debugging performance, and once again, huge thanks to the team/contributors for making this feature possible .

I’d love to get your thoughts on the following partial implicit + explicit module approach:

1. What potential concerns might arise from enabling explicit module builds only for a subset of the project while leaving the rest on implicit builds? Are there any known pitfalls with mixing and implicit module builds?
2. Can PCM invalidation still occur if there are overrides in ~/.lldbinit? If so, what external factors should we be mindful of when rolling this out to ensure stability?

Note: This applies only to targets built by Xcode, not those built with Bazel.

[ArtemC]

@chiragramani

I’d love to get your thoughts on the following partial implicit + explicit module approach:

1. What potential concerns might arise from enabling explicit module builds only for a subset of the project while leaving the rest on implicit builds? Are there any known pitfalls with mixing and implicit module builds?

I am glad to hear about the vastly-improved debugging experience. We, of course, aim to get build performance to a point where a partial explicit+implicit configuration is not required. In the meantime, if debugging specific targets is a priority then this approach may work. In theory, intermixing implicitly-built and explicitly-built targets should be fine - though it does mean that you are losing out on some of the benefits that a consistent target build-graph would get, such as being able to share built module dependencies between targets, which I don't think can occur between explicit and implicit module targets. There may be some additional unknowns in this configuration, but I think it's worth a try.

@chiragramani

2. Can PCM invalidation still occur if there are overrides in ~/.lldbinit? If so, what external factors should we be mindful of when rolling this out to ensure stability?

I'm not as familiar with this, what kind of overrides may be contained in ~/.lldbinit and could they affect debugger's compiler configuration? I'll re-direct this one to @Adrian_Prantl or @Dave_Lee.