My goal is to fetch Package.swift of a repo and analyze it's dependencies, platforms, plugins, ... Ideally without having to store as a file in the file system and just load it as a string.
What would you say is the best way to do this these days? There is swift-package-manager/libSwiftPM.md at main · apple/swift-package-manager · GitHub, but any sort of documentation seems to be pretty nonexistent unless I'm missing something. Also Programmatically access information from Package.swift - #2 by SDGGiesbrecht seems to be outdated.
Looking at the content of Package.swift will give you the direct dependencies, but not the full dependency graph.
I implemented something similar to what you describe for SourceDocs. My solution was to use swift package to resolve and dump the dependency graph in JSON format. Then you can easily load and process it.
My understanding is this process requires the whole package source
2 Likes
That is still essentially the best way. The API has shifted a little over time, but the example file has kept up.
The primary SwiftPM team has not tagged semantic version releases since Swift 5.1, but I have continued tagging releases in my fork.
Yes, you do require a full checkout of the top‐level package (but not the repository itself; a --depth 1 clone or a zipped release is sufficient).
Regardless of whether you use SwiftPM as a library or from the command line, any action that queries the dependency graph will net you the standard .build folder and the dependencies will traverse the standard cache. If you plan on loading a large number of packages, you will want to think about rm -rf .build and swift package purge-cache.
1 Like
Thanks, I think that I'd be fine skipping the package resolution phase and literally just iterate through the dependencies section in the package file.
Regarding reading parsing a string instead of file instead of file, it looks like I might be able to mock he FileSystem injected into Workspace.
For that, you only need to load it as a Manifest type (through a ManifestLoader).
Yes, that should work, at least in theory. It was not designed to be used that way though, so if the file system does not match the manifest, it may fire all sorts of validation errors. I am not sure whether you can make a convincing enough mock without already knowing the contents of the manifest. There is no central list of the checks or their edge cases, so you will have to work it out by trial and error and be prepared for stricter checks to appear with each release.
Good luck!
1 Like
Got it, thanks! Might be easier to take a detour saving the content as a file and loading it again from the filesystem. I'll investigate.
No, what I meant was the general directory structure. For example, while checking that the manifest was loaded properly, I think it verifies that there are actually source files at the expected paths of each target. So your mocked file system might need to answer accordingly. I think it also checks things like whether C is mixed with Swift, or whether other file types are handled as resources. Your mock does not have to be truthful about everything, but it will have to be convincingly self‐consistent.
But those checks happen at sorts of levels, wherever it was most convenient. So you might get lucky and some of them will give you a free pass as long as you do not try to load it at a higher level of detail (the primary types at each level are Manifest, Package, PackageGraph, and Workspace).
So far I've been trying to load a package from a file. I'm executing the code from a vapor app on a Mac, which seems to be sandboxed. I'm copied the Package.swift file to my desktop and the app ask permission to access my desktop, but then fails anyway with following error
failure(PackageLoading.ManifestParseError.invalidManifestFormat("posix_spawn error: Permission denied (13), `[\"/\", \"-L\", \"/lib/swift/pm/4_2\", \"-lPackageDescription\", \"-Xlinker\", \"-rpath\", \"-Xlinker\", \"/lib/swift/pm/4_2\", \"-target\", \"x86_64-apple-macosx10.15\", \"-swift-version\", \"5\", \"-I\", \"/lib/swift/pm/4_2\", \"-sdk\", \"/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk\", \"-package-description-version\", \"5.5.0\", \"/Users/petr/Desktop/Package.swift\", \"-o\", \"/var/folders/wz/7gjdk17j3s57lknd9vn2w7q80000gn/T/TemporaryDirectory.TuvEIe/desktop-manifest\"]`", diagnosticFile: nil))
Here's the code I'm running
ManifestLoader.loadRootManifest(at: AbsolutePath("/Users/petr/Desktop"), swiftCompiler: AbsolutePath("/"), swiftCompilerFlags: [], identityResolver: DefaultIdentityResolver(), on: .global()) { result in
print("\(result)")
}
I wonder if the swiftCompiler is needs to point to actual swift compiler path or something.
Yes, that is the path of the executable (swiftc) that will be invoked to compile the manifest.
The sandbox will be a severe problem and possibly make it entirely impossible. To use SwiftPM as a library, you generally still need a matching toolchain to be present, so that it can execute the compiler, link the PackageDescription module, and possibly other things.
SwiftMP normally finds everything else relative to swiftc according to the layout of the standard toolchain, so the path supplied to that method must actually point the compiler executable in a toolchain, not just to a symlink or an isolated copy of the executable itself.
BTW, this is one of the use cases I had in mind when creating the VirtualFileSystem type in SwiftPM (swift-package-manager/VirtualFileSystem.swift at main · apple/swift-package-manager · GitHub). I don't have a shareable example at hand, but in principle it allows to load a package from a serialized form and through the use of a custom package container provider (swift-package-manager/Workspace.swift at main · apple/swift-package-manager · GitHub), it is even possible to introduce such "virtual" packages into a "real" package dependency graph.
4 Likes