Thanks @oscbyspro - I think you've pointed out the culprit, I did a diff of the build outputs from profile vs. optimised builds and saw the following diff (just an example, many of these):
I've tried to disable it thus (but still have a performance difference):
But strangely enough I get this diff now:
How exactly did you disable it?
I also saw a diff on -debug_variant but haven't been able to find any information on that.
Long story short, SPM dependencies are built in Xcode with code coverage flags for a release build. For this to happen there should be a Unit Testing Bundle within the project; the default test plan configuration has code coverage enabled (which is auto created once there is a test) and this option overrides the overall project level setting for code coverage... resulting in Xcode projects with 14.3(and maybe older) - 15.1 shipping with slower dependencies than need be, and probably affecting quite a few projects.
The workaround is to disable code coverage in the test plan configurations:
Minimal example can be found here https://github.com/ordo-one/external-reproducers/blob/xcode-testplan-coverage/xcode-code-coverage-slowdown
Reported under Feedback Assistant as:
Xcode release build compiles SPM dependencies with code coverage flags, ignoring project settings resulting in major performance slowdown #FB13431597
Ugh, if I understand you correctly - that shipped binaries are including code coverage instrumentation - then that really is a nasty bug. Hopefully Apple fix it promptly.
Yeah, we were fairly lucky catching it - thanks to @Sarunas for tracking down the real root cause.
Hope itâll be fixed soon, one wonders how many apps are shipped with performance regression as a resultâŚ
Hello I was unable to reproduce the issue from the minimal example you provided. I was able to build the application with and without test coverage (observing the mentioned compiler flags in the build logs). However the times for performing the calculations were the same in both cases. I tried to reproduce it on Xcode 15.0 and macOS 14.1.
I was also not able to observe the compiler flags from the minimal example in the build logs of our application which meets all the requirements that you have provided (have spm dependencies and testing targets defined, no test plan though).
We also observe a âweirdâ performance slowdown. Building our application in Release without âDebug executableâ enabled leads to immense performance benefits. Swipe killing the application and starting it again leads to the performance as if we built it with âDebug executableâ enabled (around 2 times worse performance on the tested devices). Starting the application from Instruments for cpu profiling also leads to the increased performance (regardless of how you built it). Downloading the app from the app store/installing a release binary built from CI and starting it manually leads to the worse performance. Our application is in background while doing the intensive work. Would be glad to know if the details I mentioned also apply to your case. Also any information on the topic is greatly appreciated.
Sorry, but where? I don't see an xcodebuild option to exclude architectures. Sorry if I missed the context needed to understand.
You can pass any build setting to xcodebuild, and it will override whatâs in the xcodeproj. In this case, @NeoNacho is suggesting to pass EXCLUDED_ARCHS=x86_64. An alternative would be passing ONLY_ACTIVE_ARCH=YES.
Correct, the time calculation here is the same since the sample calculation code is within Xcode project and does not include the code coverage flags. It is all projects from SPM that are build with code coverage and thus are slowed down.
What is important is that you have been able to observe compiler flag coverage flags in the logs.
Strange as the scheme should show an auto created test plan like:
Haven't observed slowdown on the application if it is downloaded from the CI artifact or built locally from Xcode for profiling.
I see. In that case our issue is most likely not the same you observed. Thank you for the response.
I haven't tried it yet, but this seems to have been fixed in Xcode 15.3. From the Xcode 15.3b2 release notes:
Fixed: Schemes provide a new âOverride Architecturesâ build option, which controls the set of architectures that will be built for all targets in the workspace, including Swift packages. The recommended option (and default for new schemes) is âMatch Run Destinationâ. Full details are available by clicking the information icon next to the âOverride Architecturesâ setting on the scheme build options sheet in Xcode. (66146584)
Just tried it! WOHOOOOO!
This saves SO much time, just very difficult to find the location they reference, so for anyone hunting for 5 minutes as I did, here it is:
Thanks @ole for the heads-up!
For reference, here's a screenshot of the info panel that's mentioned in the release notes:
Match Run Destination - Build only the most appropriate architecture for each target, based on the selected run destination. Use this option to reduce build times for iterative development.
Universal - Build all compatible architectures for each target. Use this option when you want to verify all architectures build successfully, at the cost of slower build times.
Use Target Settings - For targets with the ONLY_ACTIVE_ARCH build setting set to YES, build only the most appropriate compatible architecture based on the selected run destination. For targets with the ONLY_ACTIVE_ARCH build setting set to NO, build all compatible architectures.
When using a build-only run destination and/or performing an Archive build, each target always builds for all available architectures regardless of the selected option.
Unfortunately this setting doesn't seem to work for me, as I still see x86_64 items being built when targeting the iOS simulator when either building "Match Run Destination" or "Use Target Settings" which is set to always only build the active arch. swift-syntax and compiler plugins still build universally. And Xcode 15.3 still has the overall build performance regression I reported for beta 1. So I'm afraid it won't help at all.
It works for me for a macOS app, no longer building x86/64 shedding 30% build time at first look.
Odd, were you already building only for the active arch? What did you see building for Intel? In both cases I only ever saw macros and their dependencies building universally.
We have a macOS Apple-silicon only app that we (for reasons) primarily build in optimized mode - which so far has been building basically everything for x86/64 + arm. We have tried to get rid of the x86 builds as we donât even have such a machine (and none of our users do either), but up until now even though we tried all the settings for the Xcode project to only build for arm, it only worked for debug builds - optimized builds would always build universally. Just checking the build log and you could see all source files build for both (and flipping back the new setting would regress the behavior so itâd build for both again).
With the new setting each source file is only built for arm as expected, which is fantastic - not sure why it doesnât seem to work for you, can you see both archâs being built in the log?
Guess you might run into something else as you mention simulatorsâŚ/
