Package Manager Local Dependencies

Hi,

I would love to get some feedback on a draft proposal for adding support of local dependencies.

https://github.com/aciidb0mb3r/swift-evolution/blob/local-dependencies/proposals/NNNN-package-manager-local-dependencies.md

Package Manager Local Dependencies

Introduction

This proposal adds a new API in PackageDescription to support declaring
dependency on a package using its path on disk instead of the git URL.

Motivation

There are two primary motivations for this proposal:

  1. Reduce friction in bringing up a set of packages.

    Currently, there is a lot of friction in setting up a new set of interconnected
    packages. The packages must be in a git repository and the package author needs
    to run the swift package edit command for each dependency to develop them in tandem.

  2. Some package authors prefer to keep their packages in a single repository.
    This could be because of several reasons, for e.g.:

    • The packages are in very early design phase and are not ready to be published
      in their own repository yet.
    • The author has a set of loosely related packages which they may or may not
      want to publish as a separate repository in future.

Proposed solution

We propose to add the following API in the PackageDescription module:

extension Package.Dependency {
    public static func package(path: String) -> Package.Dependency
}
  • This API will only be available if the tools version of the manifest is
    greater than or equal to the Swift version this proposal is implemented in.

  • The value of path must be a valid absolute or relative path string.

  • The package at path must be a valid Swift package.

  • The local package will be used as-is and the package manager will not try to
    perform any git operation on the package.

  • The local package must be declared in the root package or in other local
    dependencies. This means, it is not possible to depend on a regular versioned
    dependency that declares a local package dependency.

  • A local package dependency will override any regular dependency in the package
    graph that has the same package name.

  • A local dependency will not be recorded in the Package.resolved file and
    if it overrides a dependency in the package graph, any existing entry will be
    removed. This is similar to the edit mode behaviour.

  • The package manager will not allow using the edit feature on local dependencies.

Impact on existing packages

None.

Alternatives considered

We considered piggybacking this on the multi-package repository feature, which
would also allow authors to publish subpackages from a repository. However, we
think local dependencies is a much simpler feature that stands on its own.

3 Likes

Allowing the paths to be absolute or to escape outside the root package’s folder (with ../) is obviously flexible, but it sounds like a recipe for versioning chaos.

I wonder whether you should restrict the paths to relative paths contained within the root package’s directory.

That would still achieve a much simpler workflow when bringing up a set of related packages, but would at least encourage them all to be committed mono-repo style.

(some real-world use cases might be helpful here)

Can you expand on what you mean by versioning chaos?

I doubt this feature will cause much issues in practice. You can’t depend on a package that uses this API because of this rule:

The local package must be declared in the root package or in other local
dependencies. This means, it is not possible to depend on a regular versioned
dependency that declares a local package dependency.

1 Like

I think the proposed API is confusing, as the PackageDescription4 API already allows to set local paths with something like .package(url: "../PythonKit", from: .branch("master")). For the case of non-repository packages something like this would be clearer:

public static func package(url: String, asRepository: Bool) -> Package.Dependency

You’re right, we should make the API more clear but I don’t think a boolean is required. It doesn’t matter if the package at the path is a repository or not. The package manager will use it as-is. How about “localPath” instead of “path”?

extension Package.Dependency {
    public static func package(localPath: String) -> Package.Dependency
}
1 Like

I suppose this is related to you Workspace proposal so I think we could go with something like this:

public static func package(localPath: String) -> Package.Dependency
public static func package(url: String, from version: Version) -> Package.Dependency
public static func package(url: String, from version: Version, localPath: Bool) -> Package.Dependency

With last version, if there is package in the local path Swift PM would use it, if not it would get it from the Git URL version. Also, I think the Workspace could be integrated in Package.swift.

The kind of chaos I’m envisaging is that people start developing a multi-package project with no source control - which is fair enough perhaps as a way of getting up and running quickly - and then at some point when they realise they need it, there’s a danger that they’ll put the packages into individual repos, but will continue to use relative paths instead of switching over to version tags. They will end up having to manually track which branch each of the repos needs to be on to work with their top-level package.

That’s not something that anyone with a bit of experience would do, so perhaps I’m worrying unnecessarily. If they’re on the ball they can commit the whole cluster of packages to the same repo until they’re ready to split them up (if they ever want to).

I just saw it as a way that some people might get themselves into an unnecessary mess if relative paths were presented as a quick way to get started with a multi-package project. If we encourage them to put all the sub-packages inside the folder containing the root package, it’s perhaps easier to then just commit the whole thing mono-repo style.

I guess I’m arguing for a bit less flexibility, in order to avoid people taking a wrong turn. I’m probably overthinking it though :)

2 Likes

Yeah, I don’t think we need to worry that much. There are all sorts of ways to misuse (even existing) features and create unnecessary issues. I believe good documentation can help people understand the difference and pick the right thing according to their use-case. This is a very common feature in other package managers and giving developers some flexibility is ok IMO.

Working with ‘interconnected’ packages is hands-down my biggest pain at the moment.

This proposal looks like a solution, but just to clarify: using the proposed API a symbolic link will be created in the Packages directory? So explicit package-edit/unedit commands will no longer be necessary?

In a perfect world, I would like to be able to see all the dependent local modules in XCode’s project viewer while each module’s code is stored in it’s own directory.

Also, is there any chance this proposal will be accepted in Swift 5 time-frame?

Yep, exactly!

I expect that we will enhance the Xcode project generator to work this way for the local dependencies.

Unfortunately, I don’t know :)

This proposal in now in review.

1 Like
Terms of Service

Privacy Policy

Cookie Policy