Have you considered publishing these as binary dependencies for Swift Package Manager that can be used directly without downloading the repo locally? Simplest option is to have a thin layer secondary repo with the same Package.swift file that you have. Or the static linker patch issue prevents that?
Now it can create Xcframework but here the linking is not correct as SwiftSyntax and SwiftSyntax509 are embedded:
➜ Objects xcodebuild -create-xcframework -library $PWD/SwiftBasicFormat.dylib -output /tmp/SwiftBasicFormat.xcframework
xcframework successfully written out to: /tmp/SwiftBasicFormat.xcframework
"Proxy/Local" binary frameworks are a potent idea that could be applied to many repos. I'm not familiar with SPM's repo but you could probably use an old version of Xcode 13 to generate a .xcodeproj from the Package.swift and it's largely RTFM from there. I singled out swift-syntax as it seemed to be causing huge discomfort in the macro community and I was looking at it recently for a PR. The static linking etc I had to get up to was only because macro plugins are sandboxed. It'd be far easier for other repos.
My take on why SPM coupling comes up specifically is theseissues where people are having to have a dependency on the sourcekit-lsp repo (which depends on SPM) when all they are after is the Codeable data model for the LSP protocol so they can write clients. That really should be separated out into its own package.
Edit: I used .xcodeproj path largely because I had experience with it.
xcodebuild archive -scheme SwiftBasicFormat -quiet -configuration Release -destination 'generic/platform=macOS' -archivePath /private/tmp/swift-syntax/SwiftBasicFormat-macos.xcarchive -derivedDataPath /private/tmp/swift-syntax/SwiftBasicFormat-macos.derived SKIP_INSTALL=NO BUILD_LIBRARY_FOR_DISTRIBUTION=YES SWIFT_SERIALIZE_DEBUGGING_OPTIONS=NO
xcodebuild -create-xcframework -framework /private/tmp/swift-syntax/SwiftBasicFormat-macos.xcarchive/Products/usr/local/lib/SwiftBasicFormat.framework -output /private/tmp/SwiftBasicFormat.xcframework
xcframework successfully written out to: /private/tmp/SwiftBasicFormat.xcframework
but the linking here is wrong, it doesn't create a link to SwiftSyntax but it is fully embedded:
otool -L /private/tmp/SwiftBasicFormat.xcframework/macos-arm64_x86_64/SwiftBasicFormat.framework/SwiftBasicFormat
/private/tmp/SwiftBasicFormat.xcframework/macos-arm64_x86_64/SwiftBasicFormat.framework/SwiftBasicFormat (architecture x86_64):
@rpath/SwiftBasicFormat.framework/Versions/A/SwiftBasicFormat (compatibility version 0.0.0, current version 0.0.0)
/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 2202.0.0)
/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.61.1)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1600.157.0)
/usr/lib/swift/libswiftCore.dylib (compatibility version 1.0.0, current version 5.9.2)
/usr/lib/swift/libswiftDarwin.dylib (compatibility version 1.0.0, current version 0.0.0, weak)
/private/tmp/SwiftBasicFormat.xcframework/macos-arm64_x86_64/SwiftBasicFormat.framework/SwiftBasicFormat (architecture arm64):
@rpath/SwiftBasicFormat.framework/Versions/A/SwiftBasicFormat (compatibility version 0.0.0, current version 0.0.0)
/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 2202.0.0)
/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.61.1)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1600.157.0)
/usr/lib/swift/libswiftCore.dylib (compatibility version 1.0.0, current version 5.9.2)
/usr/lib/swift/libswiftDarwin.dylib (compatibility version 1.0.0, current version 0.0.0, weak)
It seems like providing a pre-compiled version of Swift-Syntax is a pretty simple solution that Apple should provide to third-party macro creators.
Otherwise it really puts a wet blanket on the otherwise incredible capabilities that macros unlock. It would be awesome to see Apple support this directly.
@Sarunas, is it possible to have frameworks that are generated from Packages using SPM include the .swiftinterface files? Is there the equivalent of the Xcode setting "Build for distribution"?
Highlighting this for the folks coming in new to the thread since there's been quite a bit of activity, and I've had a couple of folks message me off-forums asking me about the team's response. This is my attempt at summarizing the key points (please correct me if I'm wrong).
TLDR:
The Language Steering Group is aware of the build-system issues causing pain for macro adopters.
No timeline for a fix as of Feb 20th, 2024.
My two cents for Swift developers considering adopting macros:
Be aware of the fact that you're adding ~38,000 LoC of Swift source code to your project. This cost is forwarded to anyone consuming your Swift packages.
Just adding SwiftSyntax can potentially add up to 12 minutes to your build time on Xcode Cloud as noted by @Ignacio_Soto.
It's peculiar enough that a core language feature is tied to a still-maturing package manager riddled with performance/build-systems issues, but what I find extremely disappointing is that these issues aren't mentioned anywhere in any official documentation/release notes.
While it's great that we can discuss this here on the forums, I don't think we should expect this to serve as a discoverable resource on the serious issues blocking folks on the adoption of macros.
If the LSG is aware of these issues as issues hindering adoption, one would expect that awareness to be reflected somewhere other than a forums thread. Perhaps an update/addendum to the Developer Experience section of the Swift 5.9 Released blog post is warranted?
If not, I'd love to know if there's an appropriate documentation article/resource that I can PR to!
will explicit modules help at all with the compilation time of swift-syntax?
it does mention that modules can be shared across targets so maybe that results in less swift-syntax rebuilds
I haven't watched the videos about it yet, so I'm not sure what my expectations should be, but the explicit module option actually regresses build performance on one of my small projects by almost 50% (23s to 33s on an M1 Ultra). Xcode 16b1 generally regresses 10 - 12% over Xcode 15.4 (23s to 26s).
AFAIK there's no upcoming solution to macro build performance.
Still don't understand why people push for precompiled swift-syntax standalone distribution when it would be much more convenient to have it as a part of the toolchain, yes, it has some drawbacks like locking swift-syntax updates delivery to swift updates, but you'll have problems with one of those anyway, at least the usage would be much more convenient
However I came up with another idea that should fix the following issues
compilation time
swift-syntax version updates
What if macros could be precompiled?
The problem with hidden malicious code could be solved with a combination of a few checksums, artifacts should be pretty small so they could be saved right in the repo. Client side won't even need to compile anything, just recalculate the checksum and use precompiled macros. And since they are precompiled, as macros dev you won't have to depend on .exactswift-syntax version to avoid conflicts on updates and as macros user you won't have to struggle from increased compilation times or care about fetching right swift-syntax version (which may cause conflicts btw), sounds like win-win to me
@Kyle-Ye has the (inferring here) decision to not utilize the support for xcframeworks to solve this for Apple platforms been discussed explicitly anywhere?
I repeatedly have asked this on forums and GitHub issues, and have not received a reply acknowledging linked community projects nor a straight answer to why it cannot be solved incrementally. There very well may be a good reason, but me along with others engaged in the discourse of this issue are left wondering.
I agree that new threads will not help accelerate the process, and can only reiterate my question hoping that we’ll receive some meaningful reply to at the very least glean insight on why xcframeworks cannot be used — even say, gated by an opt-in build flag, something that Package.swift is flexible enough to allow for without forcing it as a solution for every client of swift-syntax.
Macros being precompiled has been tackled and solved IMO by a few folks (see CocoaPods). The practical blocker is meaningful integration with Xcode, and the fact that it is married to SwiftPM.
Heck, I’d be able to solve this issue if Xcode allowed me to use my own fork of SwiftPM, but AFAIK it uses a private fork of SwiftPM and selecting a custom toolchain breaks a lot of essential IDE features that are needed for developing on Apple platforms.
So here we are left unable to solve the practically adopt a fix interim despite being motivated enough to solve it ourselves, and without a clear answer on as to why xcframeworks cannot be used (which again there might be a good reason for, I’d really love for it to be articulated).
