We’ve been exploring the Explicitly Built Modules feature introduced in WWDC 2024 to understand its impact on build times.
Observation:
Enabling the feature resulted in a significant build time regression. Despite aligning macros, language versions, and settings as per the guidance, the regression persists.
Below are our benchmarks for reference:
Setup:
Identical dependency graph and project setup.
Unified configurations across all targets to prevent compilation of multiple module variants(e.g., UIKit) as highlighted in the WWDC session.
XC Result bundles and reproducers are enclosed in this fb.
Based on the Xcode settings, it appears that this feature is still in an experimental stage. The objective of this thread is to understand any challenges others may have encountered with explicit module maps and to share/learn about potential workarounds. These insights will help us refine and optimize our setup effectively.
If we’ve missed any considerations for optimizing this setup, please let us know! Looking forward to insights from the community.
Thank you for reporting this and including a reproducer project @chiragramani!
While I can confirm that I see similar build performance characteristics on this synthetic workspace benchmark, and the scale of this sample is worth studying and optimizing for, I am curious how it was generated - is the overall structure of this reproducer meant to capture a real project that you are working on?
Given the massive scale in quantity of single-target subprojects, this workspace is not necessarily representative of real-world Application/Framework structure, so the build-time regression figures here are not the same as those developers will see at-desk in their projects.
We have been actively studying Explicitly Built Modules performance and making significant improvements across the various tools involved (recently, e.g. with changes like [Dependency Scanning] Parallelize Clang module queries by artemcm · Pull Request #76915 · swiftlang/swift · GitHub), and having extreme-scale examples like this is also helpful to that cause - stress-testing of the overall build system involved is a useful tool in measuring progress of our build time optimization work. That said, in the meantime, there's a simple improvement that can be made to your reproducer workspace:
Unified configurations across all targets to prevent compilation of multiple module variants(e.g., UIKit) as highlighted in the WWDC session.
Currently, module dependencies of targets of different projects in a workspace build are not yet able to be shared due to the fact that Xcode projects use a per-project working directory setting for the compiler. One thing you can do manually is overload the build setting COMPILER_WORKING_DIRECTORY for the workspace build to some value that is workspace-global and is consistent/shared across all of its sub-projects. While not bringing this synthetic workspace to parity with Implicitly Built modules yet, I am seeing over 3x improvement with this approach.
Thank you @ArtemC. We all really appreciate the effort going into optimizing Explicitly Built Modules performance - it's a significant win for the benefits it brings.
I am curious how it was generated - is the overall structure of this reproducer meant to capture a real project that you are working on?
Yes, the reproducer mimics the dependency graph of one of our heavily used app targets. However, it’s not entirely realistic as real modules typically consist of significantly more files, and few workspaces even have some of the modules unfocussed/pre-built. For simplicity, each target in the reproducer is limited to just two files. This setup was specifically designed to study the micro-level impact of the Explicitly Built Modules feature on a given dependency graph, when used at this scale.
One thing you can do manually is overload the build setting COMPILER_WORKING_DIRECTORY
Oh nice, appreciate the tip! I will try it out. Just to confirm, in our core projects, we do pass -working-directory in a consistent way. Does Xcode's COMPILER_WORKING_DIRECTORY resolve to that, or does it provide additional functionality beyond it?