We'd like to gather some feedback from the community on the changes to the dependency resolution behavior that would affect tools version 6.0 onward.
We are considering lifting a long-standing requirement that package-level dependencies are not allowed to form cycles in the package graph.
This new behavior would mean that packages can be inter-dependent as long as they don't form cycles between their targets (modules). For example, package A can depend on package B and B on A as long as their products/targets are not inter-dependent, meaning that a product/target from A cannot depend on a product from B if that product directly or indirectly depends on the same product from A.
We expect this to be a fairly niche use case. Consider eg. testing frameworks - these should be able to depend on other libraries while still allowing those libraries to use the same testing framework. Concretely, it's completely reasonable that swift-foundation should be able to use swift-testing to test andswift-testing use swift-foundation for underlying functionality.
This sounds terrific! Would it be possible to give more concrete examples of the kinds of layering that would be allowed, versus those that create build cycles? The example given (using a testing library that depends on Foundation to test Foundation) sounds to me like it would create such a cycle.
To simplify things, let's pretend that swift-foundation has only two targets, which we'll call foundation (the library) and test-foundation (the tests). swift-testing could contain arbitrary targets, so long as they only depend on foundation. We would currently have the following package-level dependency graph:
Thank you, @scanon ! Yes, this indeed means that we are pushing cycle detection down into the module level.
Another example of the similar pattern is swift-testing + swift-syntax. Currently it won't be possible for swift-syntax to depend on swift-testing because package-level dependency creates a cycle (swift-testing already depends on swift-syntax for macros) but with proposed changes as soon as there are no target (aka module) dependency cycles it should be possible to express that. For example, .testTargets of swift-syntax would be able to depend on Testing target from swift-testing
This seems like the obviously right thing to do; packages are first-and-foremost a distribution mechanism, and there shouldn't be any restrictions around cross-package dependencies once the package manager has downloaded all of the packages it determines that it needs. As far as the compiler is concerned, some modules just happen to be co-located in the same source tree, but it doesn't care beyond that.
i think it would be helpful (for things like cache invalidation on Swiftinit) to surface some sort of concept of a ‘weak dependency’ at the SwiftPM level. there should be restrictions on weak dependencies, e.g., they should only be available in executable/plugin targets and library targets that cannot possibly be depended-upon by external consumers.
One subtle point worth discussing: resolving dependencies at module scope means that adding a new intra-package dependency becomes potentially source-breaking, requiring (formally, at least) a semver major version bump.
E.g., if we have the following dependency graph:
+------------+ +------------+
| Package A | | Package X |
|+----------+| |+----------+|
|| Module B | ------> | Module Y ||
|+----------+| |+----------+|
| ^ | | |
| | | | |
|+----------+| |+----------+|
|| Module C | <------ | Module Z ||
|+----------+| |+----------+|
+------------+ +------------+
then making module Y depend on module Z (a change that was previously purely internal to Package X, with no observable difference to users) becomes source-breaking because it would introduce a cycle, and requires Package X bump its major version (or better, coordinate with Package A)
Note that this situation can only arise when Package X and Y are part of a package dependency cycle (which by definition does not exist at all today). In general, I expect it to be feasible for most packages to manually check all such cycles and ascertain that no module cycle is being introduced before adding such a dependency, avoiding the need to bump major version.
I do not believe that this is a deal-breaker, but it's a gotcha that people should be aware of (package X would fail to build as soon as someone tried to insert the dependency, so it's not the sort of thing that can "slip through" qualifying a tag, but it can put a package owner in a situation where they cannot add an otherwise "natural" dependency because of how their package interacts with another.)
This would be detected during build of the Package X and reported as an error because X would have a dependency on Y to be able to reference C, so it shouldn't be possible to publish a version of Y that introduces such a dependency because the package won't build.
In general, I am in favour of lifting this restriction however I would like us to document this new behaviour somewhere. Specifically how this interacts with dependency resolution when on package is the root package.
When you are editing a root package (A) that depends on a package (B) which depends back on the root (A). Then there is no semantic version present for the root (A) so SwiftPM has to assume that the local version is working with the dependency (B). However, this could in fact result in build time failures locally. I think that's totally fine but I just would like us to call out how all of this works in the Swift PM documentation.