In SwiftProtobuf 1.31.0 a protoc product was made available via a SwiftPM artifact bundle. This caused a number of surprising build failures for users who don’t depend on the protoc product. To mitigate this we added an environment variable in SwiftProtobuf 1.31.1 so that users could opt-out of the protoc product.
One of the issues users face is transient build failures as a result of the download failing; this is particularly frustrating when you’re not using protoc. As such we are planning to make protoc opt-in instead of opt-out until we have a more stable solution for providing a protoc binary via SwiftPM.
While this may break some users we think the protoc product hasn't yet been widely adopted and that breaking a few users in order to un-break many is an appropriate trade-off.
If you are depending on the protoc product or have other concerns please let us know.
Thank you @georgebarnett, I fully support going ahead with this. This causes an enormous amount of trouble.
Making it opt-in sounds like exactly the right thing to do right now.
Over time I would recommend making it an .executableTarget(name: "protoc", ...) that's compiled with SwiftPM -- quite possibly in a separate package (or maybe even a Package.swift in the upstream repo?). That way folks can depend on it if they choose to. This will also allow supporting all platforms and build systems where we don't have or don't support binary dependencies.
This isn't all that straightforward since the upstream repo also has its own upstream dependencies (Abseil), so we'd be asking multiple teams to maintain SwiftPM manifests that they don't otherwise use and we'd need to make sure that any changes they make to their project structure were properly reflected before they cut any releases.
I think this really points to a missing SwiftPM feature—I should be able to describe targets that live in external repositories and have SwiftPM clone those and build them without having to add Package.swift files to the original repositories or fork them.
I think I’ve seen a few things work around this by having a new report (“swift-[other_things_name]”) that just contains a Packge.swift and then uses the other repo as a git submodule. Then each time they make a release the Swift “wrapper” updates the submodule reference and cuts a tag themselves. Not great, but works for some cases.
But it also runs into the whole problem of does Swift PM always need to fetch git submodules, and some projects have test only things as submodules, and so fetcher those is a drag on normal development.
This isn't all that straightforward since the upstream repo also has its own upstream dependencies (Abseil), so we'd be asking multiple teams to maintain SwiftPM manifests that they don't otherwise use and we'd need to make sure that any changes they make to their project structure were properly reflected before they cut any releases.
Yes, that's fair. I would also be happy with what @thomasvl describes here:
I think I’ve seen a few things work around this by having a new report (“swift-[other_things_name]”) that just contains a Packge.swift and then uses the other repo as a git submodule. Then each time they make a release the Swift “wrapper” updates the submodule reference and cuts a tag themselves. Not great, but works for some cases.
tl;dr either way (Package.swift in Abseil & protoc and whatever else) or a swift-protoc.
I think this really points to a missing SwiftPM feature—I should be able to describe targets that live in external repositories and have SwiftPM clone those and build them without having to add Package.swift files to the original repositories or fork them.
This is really interesting. Mind filing an issue for that?
I opened a PR to build protoc from source by including both protobuf and abseil-cpp as a submodule. I would propose that we move the build from source discussion over to the PR there.