After seeing a few large swift projects (2/3 man-year projects) completely collapse under compilation times, and yet see no hint of addressing those issues in swift evolution, i came to the idea that maybe there was something wrong in the way those projects were architectured / developped, and not with swift itself.
So my question:
Modules:
How big should a single modules be ? 10 average class/struct ? 100 ? 1000 ?
How many modules is it ok for a big iOS app to have ? 10s ? 100s ? 1000s ? (i remember a time a few years ago when modules increased launch time for the app, so more than 10 was out of question)
Type inference :
Am i supposed to write the types of every variable / closure ? like
let myA: A = myFuncCall()
Am i supposed to write the types of constant expressions ? (i'm remember a bug of dictionaries with heterogeneous types taking minutes to type-infer)
Generics:
Should i consider compilation times before starting to use generics in my code ? Or should it not interfer ?
All those issues have been hitting me in the past with regards to compilation times, but i thought it was a temporary issue. It's been a few years now, and it still doesn't feel like a top priority...
i'm using default xcode settings. I hope incremental compilation is ON by default, but i'll double check.
i've seen a very notable increase in compile time since 5.5, which made me really worried.
In particular, i've recently had to recompile an obj-c codebase of a very large project, and thought there was a bug because it compiled so fast.
i did, and this is how i came up with my list of questions. It's always the same kind of issues, mostly around type inference. Which made me wonder if my projects were correclty scoped in modules of the right size (among other things)
My question really isn't about a specific example, or help troubleshoot a particular problem on one function call.
This is an issue i've seen in many different projects, with different people working on it, and from what i've heard in the community around me, is actually a well-known issue with Swift language at the moment.
My question is more general, about the general best practices (max module sizes, etc.) for having correct compile time...
Edit: i don't want to sound rude, and appreciate your good will, but i wouldn't want the discussion to be about a specific call.
thanks, this look like a good start.
Is there any official benchmark on what we should expect compilation time to be for various project sizes and configuration ?
I don’t know of any. I mean, the obvious answer is “as long as it takes”, no? It depends on your system and tolerance for waiting.
I’d recommend using generics as much as possible (assuming they accurately describe what you want), since the compiler can (depending on the settings) specialize them and come up with the perfect balance of binary size and runtime performance. Besides, they’re way easier to work with if done properly.
In terms of compilation, the general consensus is that type inference is the biggest issue. I still recommend relying on it whenever you can, since it can make future changes much more straightforward, but it can be a nightmare if there’s too many potential interpretations.
Thankfully, that’s pretty easy to fix: if your profiler surfaces an expression that is taking ages to compile, just specify the type explicitly. It doesn’t even change the final product!
One more thing: for release builds, you can often get a significant runtime performance and binary size improvement at the cost of extremely long compilation time using things like whole-module optimization or even cross-module optimization.
Those options compile the entire module or every module at once, respectively. This allows the compiler to make insanely context-specific optimizations, like skipping code in a dependency that isn’t actually called. So don’t use the fastest compilation modes for everything.
At a high level, the scalability of Xcode and the various bits of compiler infrastructure can cause issues with overall compile times. While these aspects have been improving over the last few years, extremely large projects can still cause outsized compile times. So avoiding overly complex architectures like VIPER or Riblets, where every view controller equivalent is split into four or five files, can help with overall build times. Unfortunately there's no simple solution for massive projects beyond splitting into smaller modules that can be precompiled separately from the main project. The point at which you do that is up to you, as you have balance the complexity of such infrastructure with the build times you're seeing. Unfortunately Apple's tooling does almost nothing to help here, but there are community resources that can help get you set up.
At a low level, you have the performance impact of the compiler on individual properties and functions, usually as a result of Swift's extensive use of type inference. While also an active area of development, there are a lot of surprising edge cases. You can use the various compiler diagnostic flags (no single documentation source, but blogs like this outline how) to see if any of your functions or other expressions are particularly expensive to compile. You can then optimize their build time by simplifying or providing explicit type annotations. I haven't seen anything else, like splitting large files into smaller ones for better parallelism, explored as another solution.
This is mainly what i've been doing (and swift devs around me as well).
However, i'm currently not satisfied with what looks like a status quo in the way swift community deal with compilation times. Basically:
use all the fancy swift type inference and generics and libs
build a gigantic module
iterate for a year and then see compilation times reach absurd levels.
What i would like instead is the swift community (and most likely the core team) acknowledge that there are indeed limits to how much you can put in a single module if you want to keep compile times in check, and provide official guidelines. It could be code style, or general recommendations on modules size and projects structure, etc. They would help take preemptive measures, rather than let people crash into the wall.
Given such a gradual process it's hardly "crash[ing] into the wall." Instead, it's a balance of various factors, including a rather subjective tolerance for build times. Nothing you state here is inevitable, especially year over year as the compiler and other tooling improves. Some teams will never hit these limits. Others seem to go out of their way to find the edges of Apple's tooling. And if these standards or limits existed in any objective form you'd already see the community talking about them. So it doesn't seem likely anything useful could come out of this, aside from general guidelines that apply to every programming language. But feel free to gather this data and see if it's useful.
Xcode extensions are largely useless. Given the variable nature of "too long" it's unlikely there's a single limit that can be applied generally. What takes an old Intel laptop 500ms may take a future Apple Silicon Mac Pro a tiny fraction of the time. If you really want you can enable particular limits as a warning and fix them as you go. But it's usually fine as an occasional auditing tool rather than something you deal with day to day.
At the end of the day it's unlikely Apple will spend any time documenting limits that they're continuously working to overcome. While I share your frustration in how quickly they make progress and the priorities they have, I'd rather they spend time pushing limits rather than documenting them.
There's probably a threshold effect at which you start noticing the delay, and it becomes very painful.
I understand why the core team would like to spend all its effort on actually improving the situation. Are there any compilation speed benchmarks in the swift compilation test suits, observing the differences between releases ? Golang does that (by compiling parts of the stdlib), as compilation speed seem to have been a major objective since the inception of the language, i wonder if swift does the same.
The reason i'm becoming frustrated is that it looks to me as the problem has crippled the language since its first days, and yet i barely see the issue mentioned in swift evolution when evaluating new features. It's not even a talking point (although i'm not reading everything, so i may have missed something).
I’d hardly say the issue has crippled the language. It is not difficult to improve compilation times with profiling, and personally I think it is almost always worth accepting longer compilation time in exchange for improved readability, writability, runtime efficiency, binary size, or runtime performance.
As for official guidelines: focus on decoupling things. If you have a bunch of unrelated code in the same module, consider splitting it into different modules. This is hardly the main reason to do so, but it makes it far easier for the compiler to figure out what needs to be rebuilt.
Splitting into modules indeed seems the last option for my project (now reaching 10min build time on a project that's already split up into 2 modules, with incremental builds sometimes taking up minutes for no easy to understand reasons).
I already expected this to be the recommendation when i wrote my original post, hence my questions : what are the orders of magnitudes for a project regarding modules ?
If you are using incremental compilation, the compiler will try to determine which files are impacted by changes you have made since you last compiled everything. It errs on the side of compiling too many things rather than too few, as the latter could cause undefined behavior.
If you split things into modules, and only change files in one of them, the compiler can more easily determine whether files in other modules are impacted. However, this isn’t just about modules.
If you mark declaration as private or fileprivate, the compiler does not have to worry about them in other source files. If you use composition and nesting rather than massive monolithic types, the compiler can narrow down what your changes impact.
For instance, consider a structure. Swift is extremely good at compacting structures to use as little memory as possible, even with a substantial amount of nesting. However, this means that the way structures are laid out in memory can be impacted by the way its properties are laid out in memory. As a result, changing the contents of a structure that is used in many other structures can necessitate a lot of recompilation. If those structures are extremely complicated, that can more of an issue.
In practice, though, the main time sink is almost always type inference. So just use a profiler and specify the type of problematic expressions in advance.
I think you need to find the source of your slow builds before you attempt to take any action. There are specific recommendations, like @Saklad5 mentioned, which can help in certain areas, but they aren't all equally valuable.
First, of course, is to figure out where your clean builds are spending their time. You can build in Xcode with a timing summary to see which files or dependencies may be taking an abnormally long time to compile, and you have the various flags I, and others, have pointed out to see if type inference or other compiler steps are taking too long. You can investigate if there are any particular outliers in building your files or if you project just has so much code that taking 10 minutes to build is expected.
Second, you should investigate why your incremental compilation isn't working well. At its best it can be very accurate and compile only the files needed for any particular change. But it's also easy to break with things like custom build steps that touch a lot of files for no good reason, or build dependencies that don't properly calculate their input and output file sets. I believe there's a compiler flag that will output what the compiler considers changed during builds but I can't find it at the moment. If you can get that info you should be able to narrow down what may be causing your abnormal incremental build times.
Third, once you have the appropriate data, evaluate your biggest winners. Perhaps your endpoint security software needs to ignore your source directory. Perhaps you need to cache big dependencies. Or perhaps a few more modules might limit changes to only active parts of the code base and you can even vend binary versions that don't need to be built at all. There is no silver bullet.