Optimizing and Debugging Incremental Build Time in Swift 5.5

If your incremental (i.e. Debug) builds seem to be too slow, here some things to try (in no particular order):

  • In Xcode, go into the Report Navigator and examine your build log to see what's going on and where the time might be going.

  • See if some stage of your build is always replacing an imported file,

  • In "Other Swift Flags", add -driver-show-incremental and -driver-show-job-lifecycle. Then expand the build log portions of the "Compile Swift Source Files" build log entry. These options will cause the compiler to emit "remarks" explaining why the compiler decided to compile files.

  • If you are making lots of changes to top-level types, the incremental system won't be able to be as selective as possible. If you wrap one or more top-level types in an enclosing type, protocol, or extension, the compiler can be more selective.

21 Likes

More things to check:

In the build log, click on the hamburger all the way on the right of an "CompileSwiftSources" line, to expose the command-line-invocation of the compiler. Copy and paste the long invocation line into a text editor, and look for any of the following:
-incremental (you want),
-enable-batch-mode (you want),
-disable-batch-mode (you don't want),
-wmo (you don't want), and
-whole-module-optimization (you don't want).

If you see any problems, there's an issue with your build settings.

1 Like

When looking at the build log, make sure you have "Recent" and "All Messages" selected, as shown below. (It seems obvious, but when tired, I have made this mistake myself.)

1 Like

Sorry for bumping this topic but I'm just trying these ideas, thanks for the post!
What kind of remarks? What should I be looking for? Can't see anything special with those flags.

I assume you read the above directions, so I'm wondering how/where you are looking?

I've tried these as well and don't see anything extra in the build logs when compiling Swift source. Even the exported raw build logs don't show any special output. What are these extra messages supposed to look like?

Just tried it in a test project. Here's what I got:

remark: Incremental compilation: Incremental compilation could not read build record at  '/Users/ungar/Library/Developer/Xcode/DerivedData/testOpts-cfzbfaldainanafpjlkdudcpepxc/Build/Intermediates.noindex/testOpts.build/Debug/testOpts.build/Objects-normal/arm64/testOpts-master.swiftdeps'
remark: Incremental compilation: Disabling incremental build: could not read build record
remark: Incremental compilation: Enabling incremental cross-module building
remark: Found 1 batchable job
remark: Forming into 10 batches
remark: Adding {compile: main.swift} to batch 0
remark: Forming batch job from 1 constituents: main.swift
remark: Starting Compiling main.swift
remark: Finished Compiling main.swift
remark: Incremental compilation: Reading dependencies from main.swiftdeps
remark: Incremental compilation: Scheduling all post-compile jobs because something was compiled
remark: Starting Merging module testOpts
remark: Finished Merging module testOpts

In Xcode, I had shown the build, selected All and All Messages, and clicked on the line: "Compile Swift Source files (arm64)" to expand the desired build log.

Are those supposed to appear before the rest of the content in the build log? In my builds I see various notes at the start but no remark lines.

Here's a log. As you can see, the flags are there in the driver invocations but there's no remark output.

An incremental Alamofire build log. ```

Showing Recent Messages

Prepare build
note: Using new build system
note: Planning
note: Build preparation complete
note: Building targets in dependency order
note: Swift build system integration enabled.

Analyze workspace

Build target Alamofire macOS of project Alamofire with configuration Debug

SwiftDriver Alamofire\ macOS normal x86_64 com.apple.xcode.tools.swift.compiler (in target 'Alamofire macOS' from project 'Alamofire')
cd /Users/jshier/Desktop/Code/Alamofire
export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
export SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk
builtin-SwiftDriver -o /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Objects-normal/x86_64/Alamofire\ Swift\ Driver\ Planning\ Finished -- /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -incremental -module-name Alamofire -Onone -enable-batch-mode -enforce-exclusivity=checked @/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Objects-normal/x86_64/Alamofire.SwiftFileList -driver-show-job-lifecycle -driver-show-incremental -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -target x86_64-apple-macos10.12 -g -module-cache-path /Users/jshier/Library/Developer/Xcode/DerivedData/ModuleCache.noindex -Xfrontend -serialize-debugging-options -application-extension -profile-coverage-mapping -profile-generate -enable-testing -index-store-path /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Index/DataStore -swift-version 5 -I /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Products/Debug -F /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Products/Debug -c -j20 -output-file-map /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Objects-normal/x86_64/Alamofire\ macOS-OutputFileMap.json -save-temps -no-color-diagnostics -serialize-diagnostics -emit-dependencies -emit-module -emit-module-path /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Objects-normal/x86_64/Alamofire.swiftmodule -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/swift-overrides.hmap -Xcc -iquote -Xcc /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Alamofire-generated-files.hmap -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Alamofire-own-target-headers.hmap -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Alamofire-all-target-headers.hmap -Xcc -iquote -Xcc /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Alamofire-project-headers.hmap -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Products/Debug/include -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/DerivedSources-normal/x86_64 -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/DerivedSources/x86_64 -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/DerivedSources -Xcc -DDEBUG=1 -emit-objc-header -emit-objc-header-path /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Objects-normal/x86_64/Alamofire-Swift.h -working-directory /Users/jshier/Desktop/Code/Alamofire -experimental-emit-module-separately

SwiftDriver\ Compilation\ Requirements Alamofire\ macOS normal x86_64 com.apple.xcode.tools.swift.compiler (in target 'Alamofire macOS' from project 'Alamofire')
cd /Users/jshier/Desktop/Code/Alamofire
export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
export SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk
builtin-Swift-Compilation-Requirements -- /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -incremental -module-name Alamofire -Onone -enable-batch-mode -enforce-exclusivity=checked @/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Objects-normal/x86_64/Alamofire.SwiftFileList -driver-show-job-lifecycle -driver-show-incremental -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -target x86_64-apple-macos10.12 -g -module-cache-path /Users/jshier/Library/Developer/Xcode/DerivedData/ModuleCache.noindex -Xfrontend -serialize-debugging-options -application-extension -profile-coverage-mapping -profile-generate -enable-testing -index-store-path /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Index/DataStore -swift-version 5 -I /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Products/Debug -F /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Products/Debug -c -j20 -output-file-map /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Objects-normal/x86_64/Alamofire\ macOS-OutputFileMap.json -save-temps -no-color-diagnostics -serialize-diagnostics -emit-dependencies -emit-module -emit-module-path /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Objects-normal/x86_64/Alamofire.swiftmodule -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/swift-overrides.hmap -Xcc -iquote -Xcc /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Alamofire-generated-files.hmap -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Alamofire-own-target-headers.hmap -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Alamofire-all-target-headers.hmap -Xcc -iquote -Xcc /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Alamofire-project-headers.hmap -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Products/Debug/include -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/DerivedSources-normal/x86_64 -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/DerivedSources/x86_64 -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/DerivedSources -Xcc -DDEBUG=1 -emit-objc-header -emit-objc-header-path /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Objects-normal/x86_64/Alamofire-Swift.h -working-directory /Users/jshier/Desktop/Code/Alamofire -experimental-emit-module-separately

SwiftDriver\ Compilation Alamofire\ macOS normal x86_64 com.apple.xcode.tools.swift.compiler (in target 'Alamofire macOS' from project 'Alamofire')
cd /Users/jshier/Desktop/Code/Alamofire
export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
export SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk
builtin-Swift-Compilation -- /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -incremental -module-name Alamofire -Onone -enable-batch-mode -enforce-exclusivity=checked @/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Objects-normal/x86_64/Alamofire.SwiftFileList -driver-show-job-lifecycle -driver-show-incremental -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk -target x86_64-apple-macos10.12 -g -module-cache-path /Users/jshier/Library/Developer/Xcode/DerivedData/ModuleCache.noindex -Xfrontend -serialize-debugging-options -application-extension -profile-coverage-mapping -profile-generate -enable-testing -index-store-path /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Index/DataStore -swift-version 5 -I /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Products/Debug -F /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Products/Debug -c -j20 -output-file-map /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Objects-normal/x86_64/Alamofire\ macOS-OutputFileMap.json -save-temps -no-color-diagnostics -serialize-diagnostics -emit-dependencies -emit-module -emit-module-path /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Objects-normal/x86_64/Alamofire.swiftmodule -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/swift-overrides.hmap -Xcc -iquote -Xcc /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Alamofire-generated-files.hmap -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Alamofire-own-target-headers.hmap -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Alamofire-all-target-headers.hmap -Xcc -iquote -Xcc /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Alamofire-project-headers.hmap -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Products/Debug/include -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/DerivedSources-normal/x86_64 -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/DerivedSources/x86_64 -Xcc -I/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/DerivedSources -Xcc -DDEBUG=1 -emit-objc-header -emit-objc-header-path /Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire\ macOS.build/Objects-normal/x86_64/Alamofire-Swift.h -working-directory /Users/jshier/Desktop/Code/Alamofire -experimental-emit-module-separately

Build succeeded 12/11/21, 7:41 PM 0.3 seconds

</details>

How about this? Try adding -gazorp to the other Swift flags. When I do that, I see:

error: unknown argument: '-gazorp'
Command CompileSwiftSources failed with a nonzero exit code

in the build log for the Swift compiler when I expand it in Xcode. What do you see?

It properly fails my build as well.

error: Unexpected error in driver invocation: unknown argument: '-gazorp' (in target 'Alamofire macOS' from project 'Alamofire')

It looks like the build log you're looking at has been processed. Either what Xcode is showing you is different--maybe someone after me has made a change somewhere--or you're not seeing the output of the swiftc command. Hmmm....

Here's another experiment: copy the swiftc invocation out of the log, remove the -parsable-output flag, and run it from a shell.

[Which version of Xcode are you using, and are you looking at the build log in Xcode? Can you post a screen shot?]

This is the Xcode 13.2RC, though I also tried this on 13.1 before.

Copying the swiftc command (though there's no -parsable-output to remove) does result in the proper remarks. For example:

remark: Incremental compilation: Incremental compilation has been disabled, because different arguments were passed to the compiler
remark: Incremental compilation: Disabling incremental build: different arguments were passed to the compiler
remark: Incremental compilation: Read dependency graph '/Users/jshier/Library/Developer/Xcode/DerivedData/Alamofire-gqexvpxfnitcltctvgfmsnsdspgw/Build/Intermediates.noindex/Alamofire.build/Debug/Alamofire macOS.build/Objects-normal/x86_64/Alamofire macOS-master.priors'

It does seem like Xcode is filtering that output.

Only difference I can see from my raw invocation is that Xcode prepends builtin-Swift-Compilation --, which I'd guess filters the output.

Could be; builtin-Swift-Compilation could well be something added later. I wonder if there's some option somewhere causing it to be set? I'll see if I can get the same version of Xcode you're using.

My Xcode is not prepending that "builtin" thing. I even tried an xcodebuild from the shell and got the remarks in the output. At this point, I'm afraid I don't know, sorry.

I may have found the workaround. The new default, EnableSwiftBuildSystemIntegration seems to make the remarks disappear from the build log when it is enabled. You can try defaults write com.apple.dt.XCBuild EnableSwiftBuildSystemIntegration 0 to get them back.

I tried that before, but I will again. I've heard that setting it to 0 may not deactivate it, so I'll try delete next time.

What worked for me was quitting Xcode, setting the pref to 0, and restarting Xcode, then rebuilding with the Other Swift Options as above, and then looking at the "Compile Swift Sources" entry in the log.
Good luck!

We have project setup where we have a few frameworks and we use CocoaPods. We have Framework A, and Framework B depends on Framework A, and Framework B depends on some Cocoapods.

As a simple test scenario, I'm just modifying the body of a method, not a type name or signature. I see that the incremental build in Framework A is working well and only rebuilding that one file.

Framework B always triggers a complete re-compile though. There are a number of repeated remarks:

remark: Incremental compilation: Changed: RxDataSources-umbrella.h -> implementation of source file StringLocalizable.swiftdeps in StringLocalizable.swiftdeps

This repeats for every file that has an import RxDataSources

remark: Incremental compilation: Changed: HOPUtilities.h -> implementation of source file RxInlineAnimatedCollectionViewDataSource.swiftdeps in RxInlineAnimatedCollectionViewDataSource.swiftdeps

This repeats for every every file that has import FrameworkA

remark: Incremental compilation: Changed: module.modulemap -> implementation of source file UIColor+extensions.swiftdeps in UIColor+extensions.swiftdeps

This repeats for about everything it appears. Some of the files only have an import UIKit and only have top level declarations involving UIKit types.

Seems like there's something incorrectly triggering the change, as the .h files and module.modulemap from what appears to be UIKit is triggering recompliation – or I'm interpreting this output incorrectly!

Is there anywhere that documents the expected incremental build behavior? I was assuming that only changes to public signatures or public inlined bodies would trigger re-compilation but realize there's a lot of nuance. Is it expected to rebuild every file that imports Framework A if Framework A is re-built for any reason? That certainly would change our framework strategy substantially.

Thanks for this debug flag, it's great to have some ways to see how things are behaving.

1 Like