This is a draft proposal to serve as a starting point for discussing dependency mirroring and forking. We invite the community to help flesh out concrete proposal(s) for the swift-evolution process!
Package Manager Dependency Mirroring and Forking
This is a draft proposal for adding support for dependency mirroring and forking in SwiftPM.
Mirror: A mirror is an alternate source location for a dependency which exactly mirrors the contents of the original source.
Fork: A fork is a package dependency which may have different content than the original dependency.
Vendoring: Vendoring refers to checking in source code of the entire dependency graph so you always have your dependencies.
Dependency mirroring is highly useful for several reasons:
- Ensures that a dependency can be always fetched in case the original source is unavailable or even deleted.
- Access to the original source location is slow or forbidden on the current network.
- You want to validate or screen the upstream updates before making it available to use internally in a company.
Most use-cases of using a fork boils down to one major case: using a modified version of a dependency in your package before those modifications are available in the original package. Example: You made a bug fix in Swift-NIO but the fix is not merged in the upstream repository yet and your application requires this fix in order to work correctly.
Vendoring has some overlapping motivations with mirroring. It ensures that the dependencies are always available even if the original sources disappear. It allows building without network access, and it enables "archiving" a project so it can be built in future without worrying about the original dependency sources being available years from now.
A dependency mirror is supposed to exactly replicate the contents of the original dependency and given its use-cases, a user should be able to use the mirrors they have access to with any arbitary package. We can introduce a new "package mirror" file to store per-dependency mirroring information that SwiftPM can use as an additional input. The mirrors will work for both direct and transitive dependencies of a package. A separate file allows using a mirror config without modifying the package.
It would be nice to have some kind of validations to make sure that the content on the mirror is actually same as the original content. However, we don't need to tackle that problem initially and we can add the validations in future as an enhancement to the mirroring feature.
Consider a package graph with two packages:
app depends on
libCore. Resolving the
app package currently looks something like this:
$ swift package resolve Fetching https://github.com/example/libCore.git Resolving https://github.com/example/libCore.git @ 1.0.0
With support for package mirror files, we would be able to record a mirror for
libCore, perhaps with a
$ swift package mirror add \ --package libCore \ --url https://mygithub.com/myOrg/libCore.git \ --mirror-file Package.mirror
Then, you could use the mirror file to resolve the dependencies:
$ swift package --mirror-file Package.mirror resolve Fetching https://mygithub.com/myOrg/libCore.git Resolving https://mygithub.com/myOrg/libCore.git @ 1.0.0
We could also make the mirror file implicit if a file named
Package.mirror is present in the root directory of the package.
This leaves an open question of what do we store in the
Package.resolved file if a mirror file is used.
Things get a bit tricky with fork support. Forks may modify the contents of the original dependency and if you depend on the contents of the fork to build, the dependees of the package will also need to use that fork. However, propagating forks in downstream packages can be dangerous and it can create a new type of dependency hell where there could be two forks of a dependency in a package graph. Moreover, packages should always get the dependencies they asked or expected, and not some modified version from a different source location because one of the dependencies forked something.
For the reasons mentioned above, we should probably support declaring forks on only the root package. This does mean that if a library package publishes a version that use a fork, it would either fail to build when used as a dependency or its dependees will need to declare the forks as well. Note that forking a transitive dependency should be allowed.
Since forks are inherent part of how a package is configured and built, it makes sense to declare the fork information in the
Package.swift manifest file, similar to branch or git hash based dependencies.
Package.resolved is too fragile to hold this information as one might want to blow it away to get to a pristine state.
A strawman proposal for declaring forked dependencies:
let package = Package( name: "app", dependencies: [ // App depend on the libCore dependency. .package(url: "https://github.com/example/libCore.git", from: "1.0.0"), // Override the dependency with our fork until an important patch is merged upstream. .fork(package: "libCore", url: "https://github.com/myName/libCore.git", .branch("CVE-5715")), ], ... )
We're not planning to address the vendoring feature in this proposal but we're open to reconsideration based on feedback from the community. Vendoring has some overlap with problems mentioned in this proposal. It can be considered as a way to "archive" a project with all of its dependencies so it can be easily built without worrying about availability of the dependencies. It also solves the problem of fetching the dependencies when there is no or limited network access.
It is currently possible to technically vendor packages or create a "local" fork using the local dependencies feature, however, this would only work for root packages as SwiftPM will emit an error if the local dependency feature is used in a dependency referenced by a version.
We think mirroring and forking are the right choices to solve some of the problems that vendoring aims to address. This leaves the archiving use-case which can be considered as a separate feature in the future.