There seems to be some degradation in the "emit module" phase of building between XCode 15.0 ( Swift 5.9.0.128.108 ) and XCode 15.1 Beta ( Swift 5.9.2.1.6 )
We have several different frameworks and apps in my XCWorkspace. In the 15.1 beta, most of them compile around the same speed, or slightly faster. However one framework we have seems to take over 10x as long as before. When looking into it in the Xcode timeline, almost all of the time is spent on the "Emitting Module" phase. It went from ~56 seconds to ~1440 seconds on a clean build. Also some of the files do seem to compile quite a bit slower, but this is a large bottle neck with nothing else really happening in the timeline besides "emitting module" at the time. If i watch Activity Monitor at the time, my CPU has a swift-frontend process pegged at 100% during that time. But otherwise nothing seems suspicious. Comparing the output the framework is almost identical in size and I don't see anything else really noticeable.
Any ideas on what would cause this huge discrepancy ?
I just identified and fixed a case where we saw this happening when a module has a very large number of Swift files in it. (It could also happen with a large number of macro expansions). For the project where we had seen the regression, the "Emitting Module" phase went from 300s down to 32s.
It's likely to be the same as what you're seeing. We can verify that with a spindump if you're able to capture one, or we can kick off a toolchain build if you'd like to try that.
[Edit: for those who are curious, the compiler had a linear-time algorithm to map from its internal representation of a source location to the source file in which that location occurs. The algorithm has been linear-time "forever", but a recent bug fix put it into a hot path. The fix is to turn it into a logarithmic algorithm with a single-element most-recently-used cache.]
Forgive me for my ignorance, but is it possible to test this out prior to it being included in the new xcode. I know swift can be distributed as a stand alone, but i'm not sure if it's possible to update the integration with XCode or if you just have to wait.
Is it known yet if this fix will be included in the 15.1 RC ?
We could potentially test out this fix by installing/running an OSS toolchain that has this fix. More context can be found here: Swift.org - Download Swift
[...] none of the test projects there have a huge number of Swift files in a single target.
Could someone please sketch an outline of how I might contribute such a test?
I work on a target with a "huge number of Swift files" every working day and it'd be great if CI could catch performance problems or regressions so I can update my Xcode without anxiety about how my productivity will be impacted.
Has there been any update to this? We have a project that takes 5-9 minutes longer to build on Xcode 15.2 with a specific SDK we have to implement and it is really affecting our developer workflow.
In the meantime, we have to do our development on a branch with this SDK not imported. We didn't have this problem before Xcode 15 either.
We're also facing this problem in Xcode 15.2 using Swift 5.9.2, and emitting our iOS target module takes around 50 seconds. Even though it happens in parallel with changed sources compilation, the module emission does slow the build down quite significantly.
"Emitting module" is a bit of a gray area for me, and trying to inspect it using methods others have listed in this thread didn't result in much.
At the risk of sounding stupid, why does the build emit a Swift module for the iOS target at all, if no other target imports it? I suppose it is a required part of the build - I've tried disabling debugging, testability, and toggling some build settings that relate to Swift modules, and the build was still emitting a module.
If emitting a module is required nonetheless, what would be good candidates to tinker with, in aim for making single module emission a faster process? Maybe there's some non-obvious things like having both SPM and CocoaPods dependencies, generics, lots of extensions scattered around the target's files, Objective-C exposure of some Swift classes and extensions, something else?
I think it's referring to the generation of system modules used to link your compiled artifacts. Xcode 16's explicit module build system, while slow, is useful to gain insight into how many variants of the system modules you're actually building, and then taking action to reduce that number. Can you try it out to see if it gives you anything? Once you run the build and remove the variants you don't need, builds without the explicit modules should still be more efficient without the overhead of the explicit build system.
This is good advice, and I do think anyone hitting these issues should investigate whether their project configuration is requiring extra module variants to be built. However, I want to clarify that the "emit module" task in the build log is not what turns system modules into precompiled modules for the compiler (if you enable explicit modules, you will see tasks that specifically say they are compiling various system modules). The emit module task is a compiler job that takes all of the sources of the target being built as input and produces various module outputs that unblock building downstream targets. For example, it emits artifacts like the binary .swiftmodule (a serialized representation of the module's AST), the textual .swiftinterface file if the module has library evaluation enabled, and the .tbd for linking.
I'm not sure what I would recommend tinkering with; I think this requires a more in-depth investigation. In general, the emit module task ought to be pretty fast since it gets to skip parsing and typechecking for most source code (non-inlinable function bodies get ignored entirely). You and others in this thread may be hitting some pathological cases that need to be root caused by a compiler engineer. The best way for us to investigate would be with a reproducing project, so if anyone has a project that they can share where this reproduces that would be really helpful. Short of that though, filing a feedback with Apple and attaching a build timeline and a system trace recorded while performing a build may provide some clues. If you file a feedback, let me know and I'll get it screened.
Now seeing this myself (likely was back then too) where the "Emit Swift module" stage for an app target takes 80 - 100s in Xcode 15.4 and 16, so I'd like to investigate more.
Since there are no downstream targets for an app target, which parts of this process should actually apply? In Xcode's build visualizer I can see that this process starts during the build of the app target (not at the beginning but close) and continues in parallel with compilation of the source files. It then continues so long it blocks completions of the overall process. From the looks of it, the emission is one giant command with all 1200+ app target files, so in addition to its own inefficiencies, it's likely subject to slowdown from system security software injecting itself in the file read process. Is there any way to parallelize this step? It's competing with the compiler frontend processes, the macro expansion processes, asset catalog compilation, and other things, so having it work on its own may actually be faster. I'd try running it on its own but Xcode cleans up a lot of the intermediate products it needs, so running the command on its own just fails.
It's not safe to assume that the emit module step is superfluous for an app target. The .swiftmodule emitted for an app target may be used by lldb when debugging the app. It can also be necessary as an input for compiling the tests for the app. If there were no tests and you could express your intent to not debug the built binary, then a sufficiently smart build system could skip it maybe, but that would be a fairly narrow optimization at that point. I think it would be more useful for this discussion to take it as a given that emitting a module is a necessary step; it's designed to be serial but fast, and it should always be faster than object file compilation for the same target.
Something surprising is happening. If I had to guess it's a regression where some computation snuck into the emit module job that is either entirely unnecessary or should be cached but isn't, and for whatever reason it is especially expensive when operating on the whole module, which is one of the aspects of the emit module task that can be a major difference vs. other tasks.
Sharing a time profile of the work done during the emit module task for your build would help root cause the slow down. Would you be able to do that via Feedback Assistant?
There are ways to ensure intermediates don't get cleaned up automatically but I'll have to do some digging to give you the right instructions for that.
My assumption was only that the full list of uses you provided may not be necessary for app targets, in order to formulate workarounds that may be effective. For instance, I tried turning off testability for the debug builds and found it had no effect on the emit module phase. To be more specific, are there any build flags that would affect this step? I'd like to find a workaround rather than just gathering evidence for a feedback. So far nothing I've tried has affected the step at all. Would the size of an asset catalog come into play here?
Actually, just to be sure, do you have Explicit Modules enabled already? If it isn't, I would enable it and try building again and re-examining the length of the build tasks. Without explicit modules enabled, the SDK module building work is billed to the various compilation tasks in unpredictable ways, so it's very helpful to remove that variability.
