in theory, SPM is a normal swift package and you can build it (with the SPM that comes with the toolchain) as a normal swift package.
but the swift-package-manager repository has no recent semver tags, it uses the “toolchain” tagging scheme (swift-5.9.1-RELEASE) instead.
how do you depend on SPM as a library?
We depend on the swift-5.9-RELEASE revision:
.package(url: "https://github.com/apple/swift-package-manager.git", revision: "swift-5.9-RELEASE"),
doesn’t a revision requirement prevent anyone from depending on the client package itself as an SPM library?
As of right now, libSwiftPM doesn't maintain a stable API that could follow semantic versioning. Packages that you build with your own libSwiftPM will pull in the PackageDescription module from your current Swift installation, which may be incompatible with the version of libSwiftPM you'd use. This incompatibility will surface as differing serialization format (itself a private API) used to communicate information from package manifests and plugins, which will lead to obscure and hard to diagnose errors.
As someone who previously maintained a CLI tool built on top of libSwiftPM and now maintaining SwiftPM itself, I would advise against adding it as a dependency. It's not meant for consumption outside of the set of tools that are versioned together and distributed with the Swift toolchain.
If you need a CLI interface that could operate on packages, use SwiftPM command plugins instead, those do provide a stable API.
i’m less interested in operating on packages and more interested in using the package manifest metadata model that SwiftPM uses. i’m currently parsing JSON metadata dumps and doing all the graph traversal and dependency resolution logic myself, but naturally this has proven to be unreliable and highly fragile.
i’m also interested in integrating with SPM’s package identity system.
Then libSwiftPM won't help you, you'll have the same problem, but shifted to a different component. JSON is produced by the PackageDescription module distributed with the Swift toolchain and linked dynamically by package manifests. libSwiftPM would be linked statically into your tool, so without synchronizing the exact versions of libSwiftPM in your tool and PackageDescription in Swift installation it operates on it will remain fragile. This JSON format is a private API that can change without warning.
The fact that JSON itself will be used in the future can't be guaranteed, different and more efficient IPC implementations do exist.
That's also meant to be a private API and can change without warning. It's only exposed as a library product for tools like SourceKit-LSP that are distributed together with the Swift toolchain and are guaranteed to be of the same version and used together as parts of the same installation.
am i understanding correctly that to ingest package metadata from arbitrary swift packages, you would need to build and link multiple versions of libSwiftPM, chosen according to the swift-tools-version declared at the top of the manifest?
No, it's not about swift-tools-version, it's about the version of PackageDescription module installed with the toolchain that libSwiftPM would discover. Say you have Swift 5.8 installed, but your tool uses libSwiftPM from the 5.9 release, this isn't guaranteed to work. Your Swift 5.8 will come with a version of PackageDescription incompatible with libSwiftPM from the 5.9 release linked into your tool.
This may be even the case for patch versions, hypothetically 5.10.1 may require a fix in libSwiftPM and PackageDescription that make it incompatible with PackageDescription from 5.10.0, libSwiftPM for all intents and purposes is a private API that can arbitrarily change.
You don't see this problem in the wild manifested in the Swift toolchain, since SourceKit-LSP and SwiftPM itself always have the same version of libSwiftPM linked into them as the PackageDescription module that comes with them in the same Swift installation.
Package plugins already expose some information about targets declared in package manifests, is that not enough for your use case? If not, I'd be interested to see how we could solve this with plugins in the future. We do provide a stable API for SwiftPM plugins after all.
so what i gather is most of SwiftPM’s API is private and unstable. what parts (if any) are stable and can be depended on?
to bring this back to the problem i am trying to solve, i have a number of symbol graphs in the swiftinit DB that contain Package.resolved pins. for example, for vapor:
| Package | Requirement | Resolved Version |
|---|---|---|
| swift | 5.9.1 | |
| swift-atomics | 1.1.0..<2.0.0 | 1.2.0 |
| swift-collections | 1.0.5 | |
| swift-nio | 2.62.0..<3.0.0 | 2.62.0 |
| swift-nio-transport-services | 1.20.0 | |
| swift-nio-ssl | 2.8.0..<3.0.0 | 2.25.0 |
| swift-nio-http2 | 1.28.0..<2.0.0 | 1.29.0 |
| multipart-kit | 4.2.1..<5.0.0 | 4.5.4 |
| swift-crypto | 1.0.0..<4.0.0 | 3.1.0 |
| swift-nio-extras | 1.19.0..<2.0.0 | 1.20.0 |
| websocket-kit | 2.13.0..<3.0.0 | 2.14.0 |
| swift-log | 1.0.0..<2.0.0 | 1.5.3 |
| routing-kit | 4.5.0..<5.0.0 | 4.8.2 |
| console-kit | 4.10.0..<5.0.0 | 4.11.0 |
| async-http-client | 1.19.0..<2.0.0 | 1.19.0 |
| swift-metrics | 2.0.0..<3.0.0 | 2.4.1 |
| swift-numerics | 1.0.2 | |
| swift-algorithms | 1.0.0..<2.0.0 | 1.2.0 |
| async-kit | 1.15.0..<2.0.0 | 1.19.0 |
the problem is that the name async-kit is not specific or stable enough, many people can publish packages named async-kit, and the owner of vapor/async-kit can always rename the repository to something else.
on the other hand, repository URL is too specific, it can’t accommodate different URL formats that point to the same repository. and it still suffers from the ephemerality problem: the canonical repository URL can change or even usurp the URL for a “different” package. so tooling that tries to look up a ‘canonical name’ for a package based on its URL is going to run into problems.
PackageDescription and CompilerPluginSupport are two modules with stable API meant to be used in package manifests. PackagePlugin module provides a stable API for plugins. Everything else in the SwiftPM package doesn't follow semver and should not be depended on by anything that's distributed outside of the Swift toolchain (hence the exception for SourceKit-LSP, which itself doesn't have semver tags).