Can we prefer local package and fallback to url package

Prefer local package and fallback to url package

Introduction

Add a new method to PackageDescription.Package.Dependency to create a package.

Motivation

Currently we use some logic like the following to toggle between local package and remote packages.

if ProcessInfo.processInfo.environment["USE_LOCAL"] == nil {
    package.dependencies += [
        .package(url: "git@github.com:apple/swift-docc.git", .branch("main")),
    ]
} else {
  	// For CI or local use
    package.dependencies += [
        .package(path: "../swift-docc"),
    ]
} 

But if we depend a lot of repo and we only have some of them locally, we have to either clone them all locally or use the remote version.

if ProcessInfo.processInfo.environment["USE_LOCAL"] == nil {
    package.dependencies += [
        .package(url: "git@github.com:apple/swift-docc1.git", .branch("main")),
	      .package(url: "git@github.com:apple/swift-docc2.git", .branch("main")),
        .package(url: "git@github.com:apple/swift-docc3.git", .branch("main")),
        .package(url: "git@github.com:apple/swift-docc4.git", .branch("main")),
        .package(url: "git@github.com:apple/swift-docc5.git", .branch("main")),
    ]
} else {
  	// We only have swift-docc2 and swift-docc5 locally
    package.dependencies += [
        .package(path: "../swift-docc1"),
      	.package(path: "../swift-docc2"),
        .package(path: "../swift-docc3"),
        .package(path: "../swift-docc4"),
        .package(path: "../swift-docc5"),
    ]
} 

I proprosal to add a new method to PackageDescription.Package.Dependency so that we can declare a local first package and if the local package does not exist we can use to remote one.

Proposed solution

extension Package.Dependency {
    static func package(preferLocal: Bool = true, localPath: String, url: String, _ requirement: Package.Dependency.Requirement) -> Package.Dependency {
        if preferLocal, FileManager.default.fileExists(atPath: localPath) {
            return .package(path: localPath)
        } else {
            return .package(url: url, requirement)
        }
    }
}

Currently due to the sandbox limitation, FileManager.default.fileExists(atPath:) will always return false. And we need to add --disable-sandbox to workaround.

My proposal is to add a method, so that we can get a bool result for whether a location has a valid package

or not. Something like this :point_down:

func successResolvePackage(at path: String) -> Bool

Alternatives considered

Use --disable-sandbox to workaround.

swift-docc/Package.swift at 0d61dc5550c9bdb04fd1579144679e7d3287f640 · apple/swift-docc · GitHub

2 Likes

Inspired by this PR, found another workaround. Enable library evolution for SwiftDocC library via a conditional by franklinsch · Pull Request #123 · apple/swift-docc · GitHub

We can use #filePath and deletingLastPathComponent API to workaround with the sandbox limitation.

Note: ".." will not work, use deletingLastPathComponent instead

extension Package.Dependency {
    static func package(
        preferLocal: Bool = true,
        name: String,
        location: String? = nil,
        url: String,
        _ requirement: Package.Dependency.Requirement) -> Package.Dependency {
        let manifestLocation = URL(fileURLWithPath: #filePath)
        let location = location ?? name
        let packageLocation = manifestLocation
            .deletingLastPathComponent()
            .deletingLastPathComponent()
            .appendingPathComponent(location)
        if preferLocal, FileManager.default.fileExists(atPath: packageLocation.path) {
            return .package(path: packageLocation.path)
        } else {
            return .package(url: url, requirement)
        }
    }
}

Using the above method, we could check the existence of some folder in the parent directory of the Package without adding --disable-sandbox flag. Is this a SPM sandbox bug or a known use case? cc @tomerd