Unable to integrate a remote Package that has local Packages

AaronSofaer · 2021-10-28T22:55:32.115Z

I've got a Swift Package repo. It has a bunch of Packages in it; its manifest defines them via a local, relative path. (I've tried both "NameOfPackage" and "./NameOfPackage".)

When I try to import this repo locally/from the file system, it works perfectly. When I try to import it as a Github-based Package, it fails, throwing

invalidManifestFormat("'\NameOfPackage' is not a valid path for path-based dependencies; use relative or absolute path instead.",

Am I missing something here? The parser in ManifestJSONParser is a little hard to work through, alas, and I don't understand how a relative filesystem path that isn't using ~/ gets turned into an absolute path.

Peter-Schorn · 2021-10-29T01:32:02.441Z

AaronSofaer:

invalidManifestFormat("'\NameOfPackage' is not a valid path for path-based dependencies; use relative or absolute path instead.",

This suggests that you should be using the full file path to the package.

Post your Package.swift file.

AaronSofaer · 2021-10-29T01:37:55.094Z

If only the absolute path is supported, the error message should be updated and also that makes it useless to me, since it has to work both on developers' machines and also on CI.

Peter-Schorn · 2021-10-29T01:38:58.622Z

Post your Package.swift file.

AaronSofaer · 2021-10-29T16:21:12.129Z

This is the Package.swift file for the lib package, stripped of identifiable names (and with one dependency instead of a dozen). As I said I've tried it both like this and without the ./ at the beginning of the path.

// swift-tools-version:5.5
// The swift-tools-version declares the minimum version of Swift required to build this package.

import PackageDescription

let package = Package(
    name: "lib-ios",
    platforms: [
        .iOS(.v13)
    ],
    products: [
        // Products define the executables and libraries a package produces, and make them visible to other packages.
        .library(
            name: "lib-ios",
            targets: ["lib-ios"]),
    ],
    dependencies: [
        .package(path: "./Account"),
    ],
    targets: [
        // Targets are the basic building blocks of a package. A target can define a module or a test suite.
        // Targets can depend on other targets in this package, and on products in packages this package depends on.
        .target(
            name: "lib-ios",
            dependencies: ["Account"]),
        .testTarget(
            name: "lib-iosTests",
            dependencies: ["lib-ios"]),
    ]
)

AaronSofaer · 2021-11-02T18:38:50.596Z

I have thrown together a repository which, at least on my machine, reproduces the behavior.

GitHub

GitHub - AaronSofaer/spm-remote-local-demo: An SPM repo to demonstrate a...

An SPM repo to demonstrate a particular bug. Contribute to AaronSofaer/spm-remote-local-demo development by creating an account on GitHub.

If this is not a bug, I will happily open a PR against the documentation updating it to reflect that it is intended that trying to integrate a remote Package that itself integrates local-to-itself Packages is not supported, and will give a spurious/incorrect error message.

tomerd · 2021-11-02T19:00:22.402Z

cc @NeoNacho

NeoNacho · 2021-11-02T19:29:32.202Z

This works as designed, from SE-0201:

The local package must be declared in the root package or in other local dependencies. This means, it is not possible to depend on a regular versioned dependency that declares a local package dependency.

I believe we used to have a more specific diagnostic for this, it might be a regression that we're not showing that anymore.

AaronSofaer · 2021-11-02T20:30:12.506Z

While I did read that during my passes through the documentation, I didn't realize that "a regular versioned dependency" meant "any remote package". The error message being spurious is extremely frustrating, and a more specific diagnosis would have been great, for "not losing multiple days of time" values of great between those two things.

I'm disappointed that this is working as designed, but I guess if it's not a bug, it's not a bug. Is this simply a limit of the work that's been done to the parser, or is there a reason why it's intended that this workflow not be supported?

NeoNacho · 2021-11-02T21:47:48.261Z

I believe the intent here was to avoid a versioned dependency depending on a local package that's outside the same repository, since the path of a local package can point anywhere on the filesystem.

I could see us allowing local dependencies that are part of the same repository in the future.

AaronSofaer · 2021-11-02T22:19:22.872Z

When you say "in the future", are we talking "please feel free to make a feature request", "not in the foreseeable future", or "if you want to make this happen, we would welcome an implementation"?

(My problem would also be solved by being able to point to a package that's within a repo rather than at repo root, but from my searching that use-case is considered not supported even in the hypothetical future because the team believes Package Registries fulfills that need.)

NeoNacho · 2021-11-03T16:58:23.248Z

What I meant is that while we're not actively pursuing it, I wouldn't be opposed to changing it. So probably closest to "if you want to make this happen, we would welcome an implementation".

tomerd · 2021-11-20T06:04:46.017Z

@AaronSofaer if i understand the use case correctly, i would be interested in better understanding the value of depending on a local package that is in a sub-directory of a top level package compared to defining targets (modules) for those locations

k-o-d-e-n · 2022-07-18T15:03:56.861Z

My case slightly different, but related to this problem. I want to use local dependency in development environment:

let env = Context.environment["DEV_ENV"]
let isDev = env == "Dev"
if isDev {
    dependencies = [
        .package(name: <name>, path: "../<name>"),
    ]
} else {
    dependencies = [
        .package(url: "https://github.com/<user>/<name>.git", branch: "main")
    ]
}

So, other packages that depends on this package get the error above.

doozMen · 2022-11-18T09:23:54.040Z

If I reed the intend of the proposal yes then this is the clearly not supposed to work. I created an issues as I find the docs to be contradicting to what I read and experience. Remote versioned packages do not work with local packages, docs seam to state otherwise · Issue #5915 · apple/swift-package-manager · GitHub

markturnip · 2023-05-01T02:51:47.389Z

Also found myself here. Mostly just wanted to cleanup my Package.swift and use another local definition to wrap some binary targets.

More frustrating is the fact locally the package builds & remote is what fails resolution.

asendra · 2023-07-19T07:13:57.910Z

Literally the same issue, though process, and frustrating result

Sarunas · 2023-10-04T07:24:07.108Z

Also running into this issue, repository structure got too complicated and during active development phase working in > 10 repositories is very difficult and time consuming to perform code changes across the repositories. It would be nice to have commit atomicity via remote local packages (which would allow a simple to split out the repos once the repositories are mature).