Hey there folks First time contributing to the evolution of Swift, so let's see how this goes
I hope this has enough detail, and follows the respected pattern from previous pitches. Please share as many thoughts and feelings as possible -- I'll also accept tips on how to write better pitches in the future!
Introduction
Many developers operate under a "mono repo" design pattern where code for multiple packages or projects exists under one repository.
Swift Package Manager currently assumes that each repository only contains one package, this may not always be true - this proposal aims to add support for repositories which contain multiple packages.
Context
When I refer to a "package definition" I'm talking about the Package.swift
file.
In this pitch I'll reference Firebase a couple times, especially in examples. This is purely as this is a relatively widely known, public, package demonstrating where this proposal might be useful.
While I am basing this proposal on my own thoughts and findings, I want to highlight @ddunbar's concept from 4 years ago(!) which is along the same lines and some components have been brought across.
Motivation
Larger projects (such as Firebase), teams or organisations might wish to have multiple packages coexist within a single repository.
Adoption of Swift Package Manager within these projects requires some sort of ability to not only define multiple projects within a repository but also not have them tied to the root directory of the repository.
Workarounds
Currently developers wishing to have multiple packages within a single repository have to adopt a single package with many products.
The exact solution may vary (examples below) but they all suffer from extra noise, complexity and require a higher level of understanding.
Massive Package Definition
Firebase's definition currently has 16 products, 40 targets, 14 test targets, and a lot of configuration. It's daunting, hard to read, lengthy, and ultimately attempts to wrap multiple SDKs all under one umbrella.
The repository has all of the different products in separate directories, frequently with their own test suite.
This repository would be a key candidate for some sort of separation allowing for each product to be defined in a separate file. This would improve readability and management.
Automatically Generated
Personally, I have created a CLI tool which automatically generates a package definition for the root of the project based on a list of packages within the repository.
This allows each product to have its own definition, as is desirable - and allows for easy referencing of other packages within the same repository. But for external clients, the automatically generated Package.swift
enables them to use the subpackages.
This is problematic though as it introduces more complexity through an extra dependency, more work to keep it up-to-date with new updates of Swift Package Manager and still produces noise within the repository.
Proposal
This proposal would introduce the idea of "subpackages", this is a package within a repository which is addressed by its path relative to the root.
Every subpackage will be required to have a unique name defined in the package definition. This will be name which is used by users of the package within their dependency definitions.
When performing dependency resolution on a package graph, we will unify all of the requirements on any individual repository across all requested subpackages, to ensure only a single revision of the multi package repository is required.
Detailed Design
Folder Structure
A repository may choose to have a Package.swift
at the root level (referred to simply as a "package") and/or one or more Package.swift
files within directories (referred to as a "subpackage").
Package.swift // A "package"
Foo/Package.swift // A "subpackage"
Bar/Package.swift // A "subpackage"
Dependency Declaration
(Examples may be simplified for demonstration purposes)
let package = Package(
name: "Foo",
dependencies: [
.package(subpackage: "Bar"), // (1)
.package(url: "https://github.com/example/one", subpackage: "Baz", .branch("master")), // (2)
.package(url: "https://github.com/example/two", subpackages: [ "Baz", "path/to/Qux" ], .branch("master")), // (3)
]
)
- A developer may choose to add a dependency on a local subpackage. The path would be relative to the package definitions directory.
- The most common route will be adding a remote dependency with a subpackage path. This will be a path relative to the repository root.
- Very similary to number two but allows the user to provide an array of subpackages. This may be preferred as it enables the reference to and loading of multiple subpackages with one version ultimately simplifying updates (one place to update).
We will need to add support for scanning a package definition for multiple dependencies within the same repository. It's plausible for the same repository to be listed multiple times (with different package/subpackages) but we would need to ensure we only download it once -- and ensure every definition uses the same version.
Target Inclusion
Once a dependency has been defined, it needs to be used be used by a target. This will be done slightly different:
let package = Package(
// ...
targets: [
.target(
name: "Foo",
dependencies: [
"Bar", // (1)
.product(name: "Bar", subpackage: "path/to/Bar"), // (2)
]
]
)
- Where it's clear and not ambiguous, a target may be able to define simply by the name of the subpackage (as referenced to in the dependencies array) by it's package name.
- Where it's not clear or ambiguous, the user may need to be more explicitly reference the name of the package and the path to it.
Impact on Existing Packages
This change would not immediately break any packages and is entirely additive.
Developers can, at their own pace, choose to move their package definitions to wherever makes sense for their repository - at that point in time it would be deemed a breaking change and as such users would need to update their dependency to reference the new path.
Alternatives Considered
Import Statement
This alternative would introduce a new API to the 'products' array within the existing package definition. It would allow the Package.swift
file to "import" another Swift Package at a given relative local path.
This would address the motivation for this pitch by allowing the package definition in the root of the project to simply import packages from around the repository and to not define itself any specific or extra targets - reducing the noise.
This approach has a key benefit in that it does not alter the installation method of any Swift Package by any users, instead relying solely on a change by the package author. This further reduces the impact on the change as it would no longer be a breaking change.
Below is an example of the Package.swift
which would sit in the root of the repository.
// swift-tools-version:5.3
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription
let package = Package(
name: "Firebase",
products: [
.import(path: "Crashlytics"), // imports Crashlytics/Package.swift
.import(path: "path/to/Analytics"), // imports path/to/Analytics/Package.swift
]
)
Importantly though there is no reason why a package definition not at the root level should not also be able to import another package.
This is to say that imports can be chained. An error would be thrown if you attempt to create a circular reference where one package definitions reference another, and that package references the original.
I believe while this is a viable option and has its own list of benefits, it does also introduce some of its own complexity as mentioned above. I also don't think it fundamentally addresses the 1:1 relationship between package and repository -- purely focussing on reducing the noise and complexity of having a single, huge, package definition. It also requires that all packages still have a Package.swift
at the root of the repository - this may be desirable for some but I believe still contributes to noise which could be avoided.
There is however, no reason why this couldn't be implemented as well as the original proposal giving developers and users some options. Though this would probably be best to follow up in a future proposal.
Conclusion
I have not at this time been able to successfully implement the proposal within the Swift Package Manager project but I do not have any reason to believe it wouldn't be possible. I'm definitely willing to give it a go but would likely need some direction and support with it
I hope that I've at least provided justification for why I feel a change of this nature is required and hope we can work together to move this pitch to an official proposal -- and get it implemented!
Thanks for reading, and I look forward to discussing!