Compilation speed: help test batch mode!

I believe, but haven't checked, that the compilers included in Xcode releases have assertions disabled also.

I'm not using the same exact code as before (neither the same machine) but here are my build times with the newer toolchain. All the errors reported before have been fixed. (I did have a problem at one point with bundling Swift support libraries with my WatchKit extension but I think it was a fluke with using the new build system, not this toolchain.)

  • swift-DEVELOPMENT-SNAPSHOT-NO_ASSERTIONS-LTO-2018-05-02-a, -Onone, UBSan enabled
  • iMac Pro (2.3 GHz, 18x2 cores)
  • 2,417 Swift files, ~328,673 lines of code (using cloc) in tested build target
  • Xcode 9.4 beta 2, new build system preview enabled
  • Build time with no changes (linking, packaging, signing, build scripts, etc.): 43s
Build Mode Scope of Changes Build Time
Single File Semi-clean build (excl. Obj-C) 403s
Single File Changed method body 60s
Batch Mode Semi-clean build (excl. Obj-C) 79s
Batch Mode Changed method body 61s
Whole Module Semi-clean build (excl. Obj-C) 175s
Whole Module Changed method body 162s

Correct. The toolchains in Xcode releases are LTO+no-asserts, like the batch-mode testing toolchain here. The speedups you're observing in single-file and whole-module modes relative to the toolchain in the last Xcode release are additional performance improvements; batch-mode is not the only compilation-performance work that has been done in the current release cycle.

1 Like

Finally got around to running another set of tests.

Structure

Here is our project structure and stats:

Target Files LOC
Module A 116 8,544
Module B 636 37,611
Module C 77 5,125
App 760 55,159

Module C depends on A and B, App depends on All.

Times

I tested both Clean builds and Incremental with all 3 compilation modes. For incremental I added/removed one line of code from within a method. I also noted “Failed Incremental” times, which I explained in my previous post. I tested both on Xcode 9.4b2 stock and then with the swift-DEVELOPMENT-SNAPSHOT-NO_ASSERTIONS-LTO-2018-05-02-a toolchain.

9.4b2 bundled toolchain

Single File

Clean: 303s

Target Incremental Failed Incremental
Module A - -
Module B 16s 148s
Module C - -
App 18s 57s

WMO

Clean: 125s

Target Incremental Failed Incremental
Module A 12s 42s
Module B 29s 63s
Module C 10s 43s
App 39s 40s

swift-DEVELOPMENT-SNAPSHOT-NO_ASSERTIONS-LTO-2018-05-02-a

Single File

Clean: 261s

Target Incremental Failed Incremental
Module A - -
Module B 16s 138s
Module C - -
App 14s 49s

WMO

Clean: 97s

Target Incremental Failed Incremental
Module A 12s 40s
Module B 28s 59s
Module C 10s 39s
App 36s 37s

Batch

Clean: 125s

Target Incremental Failed Incremental
Module A 9s 39s
Module B 16s 52s
Module C 9s 42s
App 14s 27s

Summary

  • Seeing some small gains in non-Batch compilation as well.
  • Clean Batch is slower than 9.4b2 WMO, but the same speed as stock WMO and a whole lot faster than Single File.
  • Incremental compilation is the same as Single File, which is great.
  • Failed Incremental are now about the same as if not slightly faster than WMO! :tada:

As it stands we would switch to Batch over WMO for our Debug builds, since it's currently the best of both worlds (outside of a full clean build).

We have a big mixed objc/swift project using a lot of pods. The main project has 61 kloc swift + 30 kloc objc, pods have in sum 37 kloc swift + 88 kloc objc.

Note that for tests we have activated WMO/batch mode for all targets, even for the Pods.

Results:

  • WMO -Onone: 3:25
  • batch mode: 5:32
  • single file: 8:09

I also counted the time for only the swift compilation phase of the main project. In batch mode it took around 2 minutes, and with WMO it took slightly less than 1 minute.

=> Batch mode is a good direction to go, but still not as fast as WMO. There might be cases where we would switch to batch mode, e.g. when working on a feature that takes a lot of recompilations. But when switching branches a lot, I'd still prefer WMO.