Compilation extremely slow since macros adoption

Hi everyone,

I've adopted macros to generate boilerplate code for my model classes (it's a bit like what Swift Data does) and first compilation now takes an horrible amount of time : it has gone from maybe 30 seconds to something like 5 minutes or more. Following compilations also take much longer even if I don't change anything in the macro implementation swift file or in the swift files using the macros.

Here is a screenshot of Xcode showing compile times :

I've made a unit test for the macro implementation and it doesn't take that long to generate the extra code : less than 1 second for Library.swift for example which is my biggest class.

Is there a way I can profile what is taking so much time ? Any tips on how to improve compilation times with macros ?
It's really hard to work with such long waiting times :frowning:

Thanks for your help !

12 Likes

To be a little more precise, the code is nearly identical to what it was before I created the macros, it's just that a lot of it is now generated by the macros but the final swift files are almost the same.

2 Likes

I had a similar experience trying macros for the first time yesterday. I figure it's because macros use the SwiftSyntax package, so you have to build that now too.

3 Likes

The difference is likely whether you are building in release or debug mode.

1 Like

Yes I'm building in debug mode, like I did before using macros. I don't think SwiftSyntax package is to blame because it doesn't much time to compile at the start of the build and it's only built once after a clean. It's really my model classes that have gone mad and are taking a huge amount of time compared to before.

The question may be whether building SwiftSyntax in Debug mode negatively affects its runtime performance in a way that building it in Release mode would not. If so, I'm not sure there's a way to always build it in Release mode with all other dependencies under your control. SPM still has little control here, especially through Xcode.

2 Likes

Increases build time too much.
We have a process to codeGen a large API that results in many endpoints with completion handlers and async variations; along with many request/response DTOs in a single large swift file.
The goal was to reduce that file size using macros.
The Debug clean build time for that generated file started at ~44 seconds.
Using a MemberwiseInit macro to create the DTO initializers slowed that down to 338 seconds.
Removing the MemberwiseInit macro and replacing for @AddAsync resulted in 287 second build time.
Was expecting some build time increase, but not that much.

1 Like

What are you building on that compiling a single class takes 44 seconds? How big is the class? If your build is already that slow than the overhead of compiling swift-syntax could be significant. On my M1 Ultra where a small app takes ~12s to build, adding swift-syntax doubles that to ~23s. But that build cost should be constant moving forward.

On a related note, are there any compiler flags that can tell us how long the macro expansion build phase is taking? There may be macros that are too expensive to run all the time and should be replaced with generated code outside the build process.

Macbook Pro M1.
It's a large app, total build time 163 seconds.
Sorry, not a single class, its a large single code generated swift file, with many structs (endpoints, DTOs) and enums.
Reducing the amount of up-front codeGen with macros reduced the file size by half, but increased clean build time by 3x.

*Edit. we were just evaluating macros. we're obviously not going to use them in the scenario above. just noting that they increased the build time much more than i expected

Generating multiple files (and using multiple files in general) should allow better build performance by parallelizing at least some of the process. But yeah, macros were never going to improve build performance.

If you want to go further and report issues to Apple I would suggest breaking out your generated code into a separate package and compare it directly to the macro version so that it's actionable.

Make sure you're isolating the impact of macros. You might be generating more code than you were before.

The pre-generated code is identical to expanded macro code (no more, no less).
codeGen init replaced by expanded @MemberwiseInit (on request/response structs)
codeGen async replaced by expanded @AddAsync (on endpoint func with completion)

How large of a file are we talking here? How many macro expansions?

large API = 2MB swift file
it does build in parallel with other packages, so it doesn't become a bottleneck until macros introduced.
~500 func completions with @AddAsync
~1K request/response structs with @MemberwiseInit

Out of curiosity, are the build times you're reporting here for Debug builds, or Release builds? I ask because my team has recently found itself in a similar situation (adding macros slows build times too much to allow adoption), but in our case, the issue is definitely just the compilation of SwiftSyntax and not the macro expansion itself: adding a dummy macro which does nothing added no overhead to Debug builds, but doubled build times for Release builds in CI (from ~15m to ~30m).

It might help to be sure you're separating those out in your timings.

long source files are a known compiler performance killer. for a long time i had no idea something as trivial as splitting a large file into smaller files could have such an outsize impact on compilation times.

You would likely see an immediately build performance improvement by splitting that into 8 or more files (really whatever batch count is most common for your team's machines, for an M1 Pro it would be 8 or 10 I think) so that at least some of the build work can be done in parallel for those files.

But that's huge and you should definitely file a report with Apple with it attached so they can use it as a performance test.

This is the same for me on M1 Ultra MacBook Pro.
Here is an example of one of my most simple model class before macro expansion :

//
//  LibraryObject.swift
//  LAFoundation
//
//  Created by Cyril Anger on 03/07/2019.
//  Copyright © 2019 LyricApps. All rights reserved.
//

import SwiftUI
import LAFoundation
import LAGraphics

public extension String.LocalizationTable {
	static let libraryObject = Self("LibraryObject", bundle: .module)
}

@DatabaseModel(localizationTable: .libraryObject)
open class LibraryObject: DatabaseObject {
	open class var defaultName: String {
		fatalError()
	}
	
    @Attribute([.validation], defaultValue: "Self.defaultName")
	open var name: String
    private func validateName(_ name: String) -> String {
        return name.validPathIdentifier
    }
    
	open var defaultIcon: LAImage? {
		return nil
	}
    
    @Attribute([.referenceComparison])
    @Image(maxSize: CGSize(width: 32, height: 32), compareMaxSize: CGSize(width: 24, height: 24))
	open var icon: LAImage?
	
	open var isHidden = false
	
	// MARK: Init
	
	override public init(databaseManager: DatabaseManager, uniqueID: String = UUID().uuidString) {
		name = Self.defaultName
		super.init(databaseManager: databaseManager, uniqueID: uniqueID)
	}
}

And after expansion :

//
//  LibraryObject.swift
//  LAFoundation
//
//  Created by Cyril Anger on 03/07/2019.
//  Copyright © 2019 LyricApps. All rights reserved.
//

import SwiftUI
import LAFoundation
import LAGraphics

public extension String.LocalizationTable {
	static let libraryObject = Self("LibraryObject", bundle: .module)
}

@DatabaseModel(localizationTable: .libraryObject)
open class LibraryObject: DatabaseObject {
	open class var defaultName: String {
		fatalError()
	}
	
    @Attribute([.validation], defaultValue: "Self.defaultName")
	open var name: String
	{
	    @storageRestrictions(initializes: _name)
	    init(initialValue) {
	        #warning("[Macro] name")
	        _name = initialValue
	    }
	    get {
	        observationRegistrar.access(self, keyPath: \.name)
	        return _name
	    }
	    set {
	        let oldValue = _name
	        _ = oldValue // Ignore warning
	        var newValue = newValue
	        if newValue.isEmpty != false {
	            newValue = Self.defaultName
	        }
	        newValue = validateName(newValue)
	        guard oldValue != newValue else {
	            return
	        }
	        observationRegistrar.withMutation(of: self, keyPath: \.name) {
	            notifyChange()
	            databaseManager.undoManager?.registerUndo(withTarget: self) {
	                $0.name = oldValue
	            }
	            databaseManager.undoManager?.setActionName(Self.nameUndoActionName)
	            _name = newValue
	            registerModification()
	        }
	    }
	}
    private func validateName(_ name: String) -> String {
        return name.validPathIdentifier
    }
    
	open var defaultIcon: LAImage? {
		return nil
	}
    
    @Attribute([.referenceComparison])
    @Image(maxSize: CGSize(width: 32, height: 32), compareMaxSize: CGSize(width: 24, height: 24))
	open var icon: LAImage?
	{
	    @storageRestrictions(initializes: _icon)
	    init(initialValue) {
	        #warning("[Macro] icon")
	        _icon = initialValue
	    }
	    get {
	        observationRegistrar.access(self, keyPath: \.icon)
	        return _icon
	    }
	    set {
	        let oldValue = _icon
	        _ = oldValue // Ignore warning
	        guard oldValue !== newValue else {
	            return
	        }
	        observationRegistrar.withMutation(of: self, keyPath: \.icon) {
	            notifyChange()
	            databaseManager.undoManager?.registerUndo(withTarget: self) {
	                $0.icon = oldValue
	            }
	            databaseManager.undoManager?.setActionName(Self.iconUndoActionName)
	            _icon = newValue?.resized(toFit: CGSize(width: 32, height: 32), scale: 3)?.withRenderingMode(.alwaysOriginal)
	            registerModification()
	        }
	    }
	}
	
	open var isHidden = false
	{
	    @storageRestrictions(initializes: _isHidden)
	    init(initialValue) {
	        #warning("[Macro] isHidden")
	        _isHidden = initialValue
	    }
	    get {
	        observationRegistrar.access(self, keyPath: \.isHidden)
	        return _isHidden
	    }
	    set {
	        let oldValue = _isHidden
	        _ = oldValue // Ignore warning
	        guard oldValue != newValue else {
	            return
	        }
	        observationRegistrar.withMutation(of: self, keyPath: \.isHidden) {
	            notifyChange()
	            databaseManager.undoManager?.registerUndo(withTarget: self) {
	                $0.isHidden = oldValue
	            }
	            databaseManager.undoManager?.setActionName(Self.isHiddenUndoActionName)
	            _isHidden = newValue
	            registerModification()
	        }
	    }
	}
	
	// MARK: Init
	
	override public init(databaseManager: DatabaseManager, uniqueID: String = UUID().uuidString) {
		name = Self.defaultName
		super.init(databaseManager: databaseManager, uniqueID: uniqueID)
	}
	
	public enum CodingKeys: String, CodingKey, CaseIterable {
	    case name, icon, iconScale, isHidden
	}

	required public init(from decoder: any Decoder) throws {
	    let databaseManager = try decoder.databaseManager
	    _ = databaseManager // Ignore warning
	    let container = try decoder.container(keyedBy: CodingKeys.self)
	    name = try container.decode(String.self, forKey: .name)
	    if _name.isEmpty != false {
	        _name = Self.defaultName
	    }
	    icon = try container.decodeIfPresent(LAImage.self, forKey: .icon)
	    isHidden = try container.decode(Bool.self, forKey: .isHidden)
	    try super.init(from: decoder)
	}

	override open func encode(to encoder: any Encoder) throws {
	    try super.encode(to: encoder)
	    var container = encoder.container(keyedBy: CodingKeys.self)
	    try container.encode(name, forKey: .name)
	    try container.encodeIfPresent(icon, forKey: .icon)
	    try container.encode(isHidden, forKey: .isHidden)
	}

	override open class var columns: [String] {
	    return super.columns + [CodingKeys.name, CodingKeys.icon, CodingKeys.iconScale, CodingKeys.isHidden].map(\.rawValue)
	}

	required public init(databaseManager: LAData.DatabaseManager, row: DatabaseManager.AnyResultSet) throws {
	    let row = row.keyedBy(CodingKeys.self)
	    name = try row.decode(String.self, forKey: .name)
	    if _name.isEmpty != false {
	        _name = Self.defaultName
	    }
	    icon = try row.decodeIfPresent(LAImage.self, forKey: .icon)
	    isHidden = try row.decode(Bool.self, forKey: .isHidden)
	    try super.init(databaseManager: databaseManager, row: row)
	}

	override open func createPrepare() -> ((DatabaseManager.AnyQuery) throws -> Void) {
	    let prepare = super.createPrepare()
	    let name = name
	    let icon = icon
	    let isHidden = isHidden
	    return { query in
	        try prepare(query)
	        let query = query.keyedBy(CodingKeys.self)
	        try query.encode(name, forKey: .name)
	        try query.encodeIfPresent(icon, forKey: .icon)
	        try query.encode(isHidden, forKey: .isHidden)
	    }
	}

	required public init(databaseManager: LAData.DatabaseManager, object: LAData.DatabaseObject, mode: CopyMode) throws {
	    let object = try object.cast(Self.self)
	    name = object.name
	    if _name.isEmpty != false {
	        _name = Self.defaultName
	    }
	    icon = object.icon
	    isHidden = object.isHidden
	    try super.init(databaseManager: databaseManager, object: object, mode: mode)
	}

	override open func compare(with otherObject: LAData.DatabaseObject) throws -> [SynchronizationChange] {
	    var changes = try super.compare(with: otherObject)
	    let otherObject = try otherObject.cast(Self.self)
	    changes.appendIfNotNull(SynchronizationChange.compare(value: name, of: self, with: otherObject.name, of: otherObject, key: CodingKeys.name.rawValue, table: .libraryObject, action: { object, name in
	                object.name = name
	            }))
	    changes.appendIfNotNull(SynchronizationChange.compare(value: icon, of: self, with: otherObject.icon, of: otherObject, key: CodingKeys.icon.rawValue, table: .libraryObject, action: { object, icon in
	                object.icon = icon
	            }, maxSize: CGSize(width: 24, height: 24)))
	    changes.appendIfNotNull(SynchronizationChange.compare(value: isHidden, of: self, with: otherObject.isHidden, of: otherObject, key: CodingKeys.isHidden.rawValue, table: .libraryObject, action: { object, isHidden in
	                object.isHidden = isHidden
	            }))
	    return changes
	}

	private static var nameUndoActionName: String {
	    return String(localized: "name", table: .libraryObject)
	}

	private static var iconUndoActionName: String {
	    return String(localized: "icon", table: .libraryObject)
	}

	private static var isHiddenUndoActionName: String {
	    return String(localized: "isHidden", table: .libraryObject)
	}

	override open func assertIsIdentical(to otherObject: LAData.DatabaseObject) throws {
	    try super.assertIsIdentical(to: otherObject)
	    guard let otherObject = try? otherObject.cast(Self.self) else {
	        throw DatabaseError.unsatisfiedConstraint("type is different")
	    }
	    _ = otherObject // Ignore warning
	    guard name == otherObject.name else {
	        throw DatabaseError.unsatisfiedConstraint("name is different")
	    }
	    guard isHidden == otherObject.isHidden else {
	        throw DatabaseError.unsatisfiedConstraint("isHidden is different")
	    }
	}
}

I have about 30 model classes like that in my project and most are more complicated than this one. I think my build time after a clean is like 10x it was before.

They are Debug builds.
The timings above are related to heavily adding macros to the single large file only.
We have adopted some other macros, which brought in swift-syntax, and that lighter adoption of macros did not effect either of our builds much (Debug or Release).

In addition to the absolute performance of the macro phase itself, Xcode is still a huge issue, as its SPM integration builds Release-mode packages universally, so you pay for an entire arch build you don't need and it runs it in parallel, so it creates more resource contention. With swift-syntax being so slow to build, this can really balloon the build time.

I've also seen this effect the Xcode Preview canvas, so I may need to file some more issues.

6 Likes