One of the more frequently requested capabilities in SwiftPM is the ability to depend on multiple major versions of the same package within a single project. Today, if two parts of your dependency graph require incompatible major versions of the same package SwiftPM is unable to resolve the package dependency graph and fails out. Although there are existing workarounds for this, it causes folks to restructure projects in ways that feel unnatural.
I've been working through what it might look like for SwiftPM to support this. The work naturally breaks down into a few areas: evaluating how existing module aliasing could serve as a foundation, and augmenting the PubGrub solver to handle multiple majors. There's also some interest in extending prebuilts support to handle multiple versions of the same package, and further down the line seeing how SwiftPM can include C support in all this (though the support for this is not in the current scope for this investigation).
Before going further down a specific path, I'd love to hear from the community: Have you run into this problem, and what did your workarounds (if any) look like? Concrete use cases help validate which parts are most impactful to solve first. Are there aspects being underestimated or angles not being considered here?
Only workaround I've found is using a different repo and module name (Package vs. Package2). However, even that is very fragile if the dependencies of the two versions are misaligned at all, so I end up essentially never using it. For that reason, community practice around packages is usually to depend on nothing you don't own, which is pretty unhealthy for the overall ecosystem. Using different versions of a package is simply one part of SPM's general limitation that all packages are visible to the resolution of all other packages. Or, that there's no way to hide dependencies for particular packages. Ideally, all of my direct dependencies would be able to have separate dependency trees, with version reuse only an optimization, and no symbol collision unless my direct dependencies expose a dependent type in their public API. Without that, even allowing separate major versions will only work when both versions can resolve to the same dependency versions, which is a pretty big limitation.
Thanks for opening this thread. As you know, this is something that I think is very important to unblock the evolution of the ecosystem. We have also discussed this in the Ecosystem Steering Group and think that this is one of the key capabilities that we need in Swift PM to set ourselves up for some of the upcoming required migrations of existing packages to leverage the latest language features.
I think the approach of leveraging the existing aliasing functionalities of Swift PM and modifying the PubGrup algorithms makes sense to me. I do think that it is important that the aliasing is inverted so that module names by default include the package identity, the module name, and package version. This will ensure that module names are not clashing.
There's also some interest in extending prebuilts support to handle multiple versions of the same package
In my personal opinion, pre-builts are just an optimization for build time. They are capable of fulfilling a package/module requirement with a pre-built artifact. At best, they wouldn't even be modeled in SwiftPM but are pushed further down into swift-build. However, I think there is a closely related thing that some people refer to as pre-builts, which are SDKs. Right now, SDKs can fulfill package modules, e.g. the iOS SDK offers the Foundation module. I am very interested in seeing an integration of SDKs with packages so that SDKs contain metadata about what package at what version it includes. This would allow SwiftPM to include this information during resolution and would combat the problem where we have to constantly add modules to the non-Darwin platforms toolchain just because they are shipped in some platform SDK. Having said that, I think this should be split into a separate proposal.
I agree that C target support should be out of scope, and we already require packages that have C modules to prefix their symbols. The guidance here should change to prefix the symbols with both the package name and the package major version to avoid clashes.
i think you are maybe underselling the value prebuilts can provide in mitigating dependency hell. i’d say maybe 80 percent of dependency conflicts i encounter in day-to-day work have nothing to do with core library dependencies, but rather dependencies for peripheral components like linters, formatters, tests, documentation, etc. the root cause of this problem is the insistence on building all of these tools from source.
i am not sure if this is what OP meant by extending prebuilts support. it’s possible she is talking about xcframeworks and not just plain executable tools like i am. but right now it is just far too difficult to package, distribute, and consume prebuilt artifactbundles, which causes people to fall back on source compilation, which greatly inflames the SwiftPM dependency hell problem.
to provide a concrete data point, dollup is a code formatter tool i wrote a while back and use on all of my projects. a few months ago i switched to consuming dollup as a prebuilt artifactbundle and that decision has paid enormous dividends due to dollup itself having dependencies in common with projects that use dollup, dependencies that experience very frequent major semver bumps. the problem is this kind of distribution pipeline is extremely complex to set up, and it feels like there is a lot of room for this process to be streamlined.
In this case we are specifically talking about the swift-syntax prebuilts which may be a lot easier to get working since they get extracted to their own directory instead of the shared modules directory the build system has now.
That said, I am also thinking through how we could support general prebuilts. But that's outside the scope of the multi-major version issue.
When I already built the main executable target, building the executable target SwiftFormat, found in the dependency SwiftFormat-mirror, which has no connection to my version of Foundation, fails to build because it uses other APIs from Foundation too. (If I build this target first, it succeeds. It only fails if my version of Foundation is already built)
My current workaround is that I changed my format.sh wrapper to use --build-system swiftbuild and everything else uses the default build-system.
