I was also waiting for WWDC24 hoping it would address compile times with macros. Basically we are holding off adoption of macros or libraries based on macros because of swift syntax. It is ok for local builds with incremental compilation but on build agents, where we do a clean clone + build, it destroys our build times. Please prioritize solving this or at least provide some roadmap so we can plan around the shortcomings until they are resolved.
9 Likes
Because a binary swift-syntax is something that works right now, rather than sometime in the indeterminate future where usage of locally included toolchain bits is possible. If they wanted, swift-syntax could ship a binary artifact (for Apple platforms at least) right now and fix this issue for the vast majority of users. They don't seem to want to do that until a fully general solution is possible for all platforms, so we're left waiting and trying to work around the problem.
3 Likes
Macros being precompiled has been tackled and solved IMO by a few folks (see CocoaPods).
I found this one Support Swift macros with CocoaPods | by Soumya Mahunt | Medium, I'll try to precompile a macro and link it with unsafeFlags in Package.swift later, but don't think it'll work for swift packages...
Exactly, and the lack of a clear answer as to why it's not being done other than the constant refrain of "we want to support Linux and Windows too" is really disappointing.
Wanting a cross-platform fix is admirable and I am all for it, but IMO it should not be a reason to delay an urgent fix for a burning problem for the majority of Swift macros' clients right now â Swift developers targeting Apple platforms.
2 Likes
WWDC24 is still ongoing and things are liable to change across the betas, I am praying that it becomes apparent that a lot of folks, just like you and me, aren't able to plan for what would be massive investments of time into refactoring our codebases given that we have zero clue on the timeframe/roadmap of a fix.
I would love to just know, where can I pitch in? This is an OSS effort, I don't want to just complain about the state of things, I'd love to pitch in. I don't know how to contribute here if proposed fixes are not getting a clear acknowledgment or reply and a roadmap (even a partial one) has not been shared.
3 Likes
I wouldn't be so sure that's the right path. It'd be all too easy to subdivide it into "fix for Apple platforms" and "fix for everyone else", and from Apple's perspective the latter is much harder to defend when the rubber meets the road in planning meetings and the like. Apple still provide the vast majority of staffing for Swift.
I'm as frustrated as anyone by macros' lack of practicality currently, but especially in light of Apple's announced swiftlang symbolic divestiture, it's important to hold the platform equality line.
It's also contrary to lots of major recent work, like the new Foundation to finally unify things across all platforms. We saw there how harmful it was to the community - and the feature in question, too - to have differences between platforms.
I for one wouldn't adopt macros in any of my open source Swift packages even if macros performed well [only] on Apple platforms, because I won't do that to my users.
6 Likes
Thatâs a pretty clear example of letting the perfect be the enemy of the good. That we could solve the issue for the vast majority of Swift users today, without harming other platforms, makes it the obvious short term win. There is no downside here. For Swift to block such a huge win waiting for a general solution across platforms which donât even have a stable distribution mechanism is a waste of everyoneâs time and money.
That youâd take an ideological stance against such a solution is irrelevant: youâd be blocking macros anyway, so nothing changes there either way.
4 Likes
It's an example of letting better be the enemy of best.
Your premise that it doesn't harm other platforms is, I fear, false, because without the pressure of the larger Swift community on the issue, it'll likely get less attention.
One can argue about whether it's worth it - whether it's the right trade-off - but I don't think it's accurate to portray either approach as having no downsides.
2 Likes
I'm sorry, are you actually suggesting that we don't solve the issue for most developers because it may, eventually, some day lead to improvements in the support for other platforms? Not only is history not on your side (Apple doesn't seem to take priority from the community well) but that seems like an incredibly toxic way to run a project. Most must suffer (if you'll excuse the hyperbole) so that the few may eventually benefit?
3 Likes
I was actually just typing this out when I saw Jon's comment, so forgive me if it sounds redundant.
I'm understanding this correctly, your argument is that there is a need to self-manufacture this artificial pressure, and we're aware that it's artificially manufactured? The pressure here being the pressure to care about other platforms.
I'm not sure it's the right path either! It's a path that to the best of my knowledge unblocks developers today, without harming in-progress/future development. I'd love to know why it is might be harmful from the folks who are charged with making these decisions.
I don't think it's unreasonable to expect some notion of a roadmap/timeline especially given the marketed push to adopt these features, and as I stated earlier - if this is a beta feature, the responsible thing to do here would be to disclose it as such.
2 Likes
While I want a general solution to binaries, it wouldn't hurt to propose a workaround until we achieve the true goal. At the end of the day, it was Apple that made a huge deal about macros in last year's WWDC and they were advertising it for the developers on their platforms, so it is in their favor (I hope) to meet the expectations they set.
1 Like
We are back to this in that the tone of discourse is 'it wouldn't hurt' rather than 'this is highly unusual and subpar disclosure'.
1 Like
Hi,
I work on a project with a pretty big codebase.
Our team has recently adopted macros on a single module in the project, and i would like to share our finding from this experience.
First of all, let me share some info on the module and the macros in question:
- the module is ~100k LOC
- there are ~700 macros applications in the module
- all macros are attached member/memberAttribute
- macros generate inits, getters, setters, nested classes, properties
-0- When adopting macros, we were already aware of the build time overhead that comes from building swift-syntax, and given that we use Bazel in the project, we were able to almost entirely mitigate that overhead by precompiling and caching our macros.
-1- When we applied the macros in the module, we saw the following changes in the build times:
Clean build time (mean, s):
- Before: 85.001
- After: 152.284
Incremental build time (mean, s):
- Before: 17.283
- After: 84.018
-2- Then we realised that we were building the macros for Debug, and after switching to Release, we saw the following changes in the build times:
Clean build time (mean, s):
- Before: 149.204
- After: 110.498
Incremental build time (mean, s):
- Before: 80.036
- After: 34.682
-3- Then we were curious, if all overhead was coming entirely from our implementation of the macros, so we converted them to essentially no-op, and we saw the following changes in the build times:
Clean build time (mean, s):
- Before: 110.453
- After: 86.511
Incremental build time (mean, s):
- Before: 43.259
- After: 18.090
So, firstly, as you can probably see, it's really important to be building macros for Release, as that apparently significantly impacts their performance and therefore build times.
Secondly, as you can probably see, we did not observe any overhead by the macros themselves, therefore it can entirely be attributed to our implementation of them. And, since we didn't yet invest in optimising them for performance, given these results, we are hopeful we can still improve.
And, finally, obviously, precompiling and caching macros can really help with not having to build swift-syntax again and again.
I hope our finding will be helpful. Thanks.
Notes:
- All measurements were taken with precompiled macros
- All measurements are given as mean value over 5 runs
- All measurements are given in seconds
- "After" indicates the measurements for the commit with mentioned changes
- "Before" indicates the measurements for the the immediately preceding commit
19 Likes
Can you test generating one of your macros using swift-syntax and without? It would be good to see if the bottleneck is swift-syntax or just generating any code at all.
Does anyone know if the Xcode 16 release includes anything that is relevant to this thread?
By just searching the keyword syntax in the release notes I couldn't find anything related.
Doesn't seem to.
- Projects must still build
swift-syntaxfrom source. swift-syntaxis still built in the active project configuration, so debug for debug builds.- The compiler still continuously launches the macro plugins as separate processes.
- Various compiler stages are blocked waiting for macro expansions, such as the emit module step discussed in another thread.
- Macro expansions seem to be uncached in many instances, so incremental builds can still pay a huge toll while unnecessary expansions are computed.
- Xcode 16's Swift Intelligence service (on Sequoia) really doesn't understand macros, so we get a lot of suggested completions that just don't work.
So yeah, lots of fun.
5 Likes
- Building in Release mode (say, to profile the app in Instruments) also builds swift-syntax in Release mode, even though debug mode was presumably already fast enough for local builds, and it isn't even going to be running in your final binary. This adds several minutes to the build time!
1 Like
Debug swift-syntax is actually pretty slow, so ideally we'd just have a single release optimized version and use that all the time. There are strategies that let you do that but they aren't generally applicable to using open source packages.