Needs some guidance in order to let this file compile faster

Using Swift
1

I wrote a syllable-composing module for an input method. This module consists of only one file.

Tekkon/TekkonSyllableComposer.swift at 06cdbbd2a1c77521e77cdd687bff9d2a9d4f41af · vChewing/Tekkon (github.com)

However, in Xcode timeline it looks like it takes pretty much time to compile.

Therefore, I would like to ask here for possible guidance in how to optimize this module for faster compilation.

// More than 20 seconds for compilation on mac mini 2018 (64GB Ram, 6 core) with Xcode 14.

See screenshot here: Needs some guidance in order to let this file compile faster. : swift (reddit.com)

P.S.: I tried to tear this file into certain parts. However, this only boosts the compilation time if I only compile this module only, as it utilizes multiple threads for individual files. If I use this as a swift package in an Xcode project, Xcode only compiles it using one thread per architecture. That's why I decided to duplicate this question from Reddit to here.

4

Without trying to compile it effectively, my intuition is that these [String: String] static dictionary literals are hard to validate. I have a related case where I had very large dictionaries of [Int: [String]], and it was taking a lot of time to compile. My dictionaries are now externalized as plists, and it builds much faster.
I would try to remove the content of all dictionary to check if it builds much faster. If it does, I would try to find another approach to bring their data into the app. You can try to serialize and lazy load plists, or even json. This is slower at runtime, but you can fallback to literals when shipping.
I would extract these mostly constant dictionaries from the file I'm working on to begin with, as it may already help.

1 Like
5

@tgrapperon is probably right about those large static values taking a lot of time. Before resorting to serialization, maybe try just declaring their types? So instead of

static let arrPhonaToHanyuPinyin = /* … */

try

static let arrPhonaToHanyuPinyin: [[String]] = /* … */

etc.

You can also try to get more fine-grained information about compile times with some compiler flags.

1 Like
6

Thank you for the idea. I just tested it (with module separation into 4 files) but the compile time doesn't change. It costs <5s in multi-core compilation and almost 30s in single-core compilation.

Before using @tgrapperon's approach, I would like to see how slow the speed of compiling such large dictionary literals in Objective-C.

7

Obj-C will be very fast since it doesn't do much compile-time verification of your values, nor does it do any type inference or generics. It will make you builds more complex though, and you'll have to vend the Obj-C target separately from the Swift one in SPM. Best to fix this in Swift if you can.

1 Like
8

Testing the linked file on an M1 Max MBP benchmarking swiftc file.swift in hyperfine (Xcode 14):

Original file: 1.943s
Give types to the two arrays: 1.507s (23% improvement).
Comment out the contents of two arrays: 1.209s (38% improvement).
Comment out all dictionaries as well: 0.361s (81% improvement).

1 Like
9

Oh. M1 Max CPU is really powerful.
Mine is mac mini 2018 (highest CPU customizable on order).

10

In a test of just compiling a large array of strings, Instruments says 80+% of the time in swift-frontend is spent in llvm::CSEMIRBuilder::getDominatingInstrForID(llvm::FoldingSetNodeID&, void*&). Next closest is (anonymous namespace)::TwoAddressInstructionPass::removeClobberedSrcRegMap(llvm::MachineInstr*) at 13%.

1 Like
11

Some updates:

The real problem for this case has been found: Massive usages of string literals where an ExpressibleByStringLiteral type exists.

This confuses the compiler, taking more time to check "whether a string literal is a string" for more than thousands of times (in this project).

I eliminated the usage of ExpressibleByStringLiteral, and the compilation time reduced from 8.5 sec to 3.3 sec.

12

I suggests long compilation is caused by the lack of explicit Type for static constants like static let arrHanyuPinyinTextbookStyleConversionTable and static let arrPhonaToHanyuPinyin. Specifying Types explicitly (like Array<Array<String>>) can often improve compilation time.

Try to add compiler flags and see the results:
-Xfrontend -warn-long-function-bodies=10
-Xfrontend -warn-long-expression-type-checking=10

1 Like
13

Thank you for stating that. I addressed it in the latest release of this package.

14

Perhaps you could restate that as static let arrHanyuPinyinTextbookStyleConversionTable: [(String, String)] as well, considering every element is a pair.

1 Like
15

For finding bottlenecks, you could also add this in your Package.swift:

swiftSettings: [
    .unsafeFlags([
    "-Xfrontend",
    "-warn-long-function-bodies=<#ms#>",
    "-Xfrontend",
    "-warn-long-expression-type-checking=<#ms#>"
    ])
]

where ms is the threshold where the compiler should warn that a function or expression is taking too long to parse.

1 Like
16

Thank you for stating that. I addressed it in the latest release of this package.
(I mentioned this in the reply right above yours. The #0 post is not editable anymore.)

17

That's strange, I still see static let arrHanyuPinyinTextbookStyleConversionTable: [[String]] there.

1 Like
18

The I must misunderstood your suggestion.
Are you suggesting me to make them a single array of ValueTuples?

e.g. Make it into [(String, String)] ?

19

Exactly.

1 Like
20

That makes sense, considering that accessing out-of-bound index of an array in Swift always crash an app instead of returning a nulled value. ValueTuple can avoid this.

I just didn't aware that this change can also affect compilation time. I'll try it later.

21

Again thank you for your recommendation.
This refactor reduced the overall compilation time from 3.3s to 1.7s.
Incredible.

22

Compile array init with large array is very slow:

I end up using json and decode at run time.

1 Like