Slow "Name binding" phase


(Dan Stenmark) #1

Over the past few days, compilation times for one of our Swift modules has seemingly doubled out of the blue, with many files taking upwards of five seconds to compile. When building with -debug-time-compilation, I'm finding that a significant amount of time is being spent in each file's Name Binding phase.

===-------------------------------------------------------------------------===
                               Swift compilation
===-------------------------------------------------------------------------===
  Total Execution Time: 4.2773 seconds (5.9876 wall clock)

   ---User Time---   --System Time--   --User+System--   ---Wall Time---  --- Name ---
   0.5624 ( 41.9%)   2.8354 ( 96.7%)   3.3979 ( 79.4%)   4.8494 ( 81.0%)  Name binding
   0.6536 ( 48.6%)   0.0870 (  3.0%)   0.7407 ( 17.3%)   0.9919 ( 16.6%)  Type checking / Semantic analysis
   0.0556 (  4.1%)   0.0015 (  0.1%)   0.0571 (  1.3%)   0.0578 (  1.0%)  Parsing
   0.0384 (  2.9%)   0.0068 (  0.2%)   0.0452 (  1.1%)   0.0511 (  0.9%)  IRGen
   0.0269 (  2.0%)   0.0020 (  0.1%)   0.0289 (  0.7%)   0.0297 (  0.5%)  SILGen
   0.0037 (  0.3%)   0.0003 (  0.0%)   0.0040 (  0.1%)   0.0041 (  0.1%)  SIL optimization
   0.0028 (  0.2%)   0.0005 (  0.0%)   0.0033 (  0.1%)   0.0033 (  0.1%)  Serialization, swiftmodule
   0.0002 (  0.0%)   0.0001 (  0.0%)   0.0003 (  0.0%)   0.0003 (  0.0%)  Serialization, swiftdoc
   0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)  SIL verification, pre-optimization
   0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)  AST verification
   0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)   0.0000 (  0.0%)  SIL verification, post-optimization
   1.3436 (100.0%)   2.9337 (100.0%)   4.2773 (100.0%)   5.9876 (100.0%)  Total

What could have brought this on, and what are the usual suspects I should be going after to bring our compile times back down?


(Jordan Rose) #2

"Name binding" is mostly "import resolution", so I'm wondering if either you've started importing a lot more than before, or if you're measuring a "clean module cache" time where Swift has to read in and cache (say) the UIKit headers.


(Dan Stenmark) #3

We've been able to correlate it to the recent addition of Firebase to our project (via CocoaPods). Interestingly, it was only being imported into a single file (inside one of our app's modules), and even with the import + setup code being removed, just simply having the Pod as part of the build target affected the build times of every single file in our module.

@jrose, could you elaborate a bit more on how swift-clang handles import resolution? Namely, how would the addition of a (admittedly larger-than-most) framework so drastically affect the Name Binding phase of every single file in the owning module?


(Jordan Rose) #4

Because all files in a module are visible to one another, imports get resolved in every file even if only one needs to be recompiled. (This is part of why Xcode 10's "batch mode" was such a big win for projects with many files.)

The design of modules on the Clang side is that the first time you import a framework, the compiler will do some extra work to make it efficient to load later and then cache that in the filesystem; where exactly it does that depends on whether you've set up Xcode to use a shared DerivedData folder or a project-relative one.

If you're not deleting the module cache folder between builds, one thing that could be affecting your build times here is that the compiler does need to check that none of the headers have changed since it did the caching. That's supposed to be a quick stat on the filesystem, but if your headers are on a remote filesystem or if you have some particularly intrusive antivirus software running, you might get noticeable slowdowns.

If you're up for a little spelunking into the compiler, the downloadable toolchains at https://swift.org/download/ have debug symbols available, which means you could profile a single frontend invocation in Instruments and see where the time is going.


(Anthony Latsis) #5

Sorry if this is an obvious question - do we have a similar optimization for Swift in relation to other Swift modules, i.e. don't re-resolve imports everywhere if none of the import declarations have changed?


(Jordan Rose) #6

Good question! Swift's binary "swiftmodule" format is already a lot faster and lazier than reading from source, so the need for an even more optimized format hasn't come up as much. The one thing that's using more indirection than the Clang serialized AST format is cross-module references, which are basically using a constrained form of lookup instead of some offset into the other module structure. This means they don't need to be rebuilt (in theory) if the dependency changes in some compatible way.

(On the other hand, no one has quite the volume of Swift code to compare to the amount of Objective-C that's processed when you do #import <UIKit/UIKit.h>, so maybe in the future we'll want to build additional "fast cross-module references".)

The parseable interfaces that serve as the stable module format do have a speed problem, and so we do compile those to a binary format (the existing one) and stash them in the same module cache location that Clang uses.