SwiftPM not building build tool plugin's dependency

Using Swift
,
1

I am trying to create a build tool plugin for a package (SwiftGtk). It is a source generation plugin, so I have mostly just copied the SwiftGen example plugin.

I got the plugin working correctly at one point (for SwiftGLib, a dependency of SwiftGtk which I am working on first), but after deleting the .build directory to verify that it works for clean builds, I got an error. The error seems to be caused by SwiftPM not building the plugin's dependencies before running commands generated by the plugin. The plugin declares the tool as a dependency in the package manifest. Here's the error (I added newlines and indentation to make it readable):

error: failed: PrebuildCommand(
  configuration: SPMBuildCore.BuildToolPluginInvocationResult.CommandConfiguration(
    displayName: Optional("Running gir2swift"),
    executable: <AbsolutePath:"/Users/user/Desktop/Projects/Swift/SwiftGtk/SwiftGLib/.build/x86_64-apple-macosx/debug/gir2swift">,
    arguments: [
      "-o", "/Users/user/Desktop/Projects/Swift/SwiftGtk/SwiftGLib/.build/plugins/outputs/swiftglib/GLib/Gir2SwiftPlugin/Gir2SwiftOutputDir",
      "--manifest", "/Users/user/Desktop/Projects/Swift/SwiftGtk/SwiftGLib/gir2swift-manifest.yml"
    ],
    environment: [:],
    workingDirectory: nil
  ),
  outputFilesDirectory: <AbsolutePath:"/Users/user/Desktop/Projects/Swift/SwiftGtk/SwiftGLib/.build/plugins/outputs/swiftglib/GLib/Gir2SwiftPlugin/Gir2SwiftOutputDir">
)

Here are the relevant parts of the Package.swift:

let package = Package(
    name: "GLib",
    products: [ .library(name: "GLib", targets: ["GLib"]) ],
    dependencies: [
        // ...
    ],
    targets: [
        .systemLibrary(
            name: "CGLib",
            pkgConfig: "gio-unix-2.0",
            providers: [
                // ...
            ]
        ),
        .target(
            name: "GLib", 
            dependencies: ["CGLib"],
            plugins: ["Gir2SwiftPlugin"]
        ),
        .executableTarget(
            name: "gir2swift",
            dependencies: ["libgir2swift"]
        ),
        .target(name: "libgir2swift"),
        .plugin(name: "Gir2SwiftPlugin", capability: .buildTool(), dependencies: [.target(name: "gir2swift")]),
    ]
)

And here's the plugin:

@main struct Gir2SwiftPlugin: BuildToolPlugin {
    func createBuildCommands(context: PluginContext, target: Target) throws -> [Command] {
        let outputDir = context.pluginWorkDirectory.appending("Gir2SwiftOutputDir")
        try FileManager.default.createDirectory(atPath: outputDir.string, withIntermediateDirectories: true)
        
        return [.prebuildCommand(
            displayName: "Running gir2swift",
            executable: try context.tool(named: "gir2swift").path,
            arguments: [
                "-o", outputDir.string,
                "--manifest", context.package.directory.appending("gir2swift-manifest.yml"),
            ],
            outputFilesDirectory: outputDir
        )]
    }
}

Perhaps there is some quirk about dependencies in plugins that I don't know about? Can build tool plugins only rely on binary targets or something? From what I have seen that doesn't seem to be the case, because SwiftPM was actively building the gir2swift tool as part of the plugin's build originally, and I don't know what I changed for it to stop working.

Also: I have had so many weird issues with plugins trying to get this working, originally I had the plugin in a separate package but it wouldn't run, and then I moved the plugin into the package (but not the gir2swift tool) and it claimed that the gir2swift product didn't exist. Once I get the plugin working with everything in one package I may come back and post about some of the other issues I was facing. Even if I am just missing something, the error reporting for build tool plugins seems pretty bad. Many of the issues that I am facing have very minimal error messages, and when a plugin attempts to write outside of the sandbox, it seems to just silently fail (I will have to do further testing to confirm that, but it does seem to be the case from what I have observed).

If it's helpful I have uploaded a zip of the package to we transfer, but that link will only last a week so if anyone has a better place to host the file let me know (it's a pity files can't be attached to posts).

3 Likes
2

I hit this yesterday; turns out that when you return .prebuildCommand it tries to run it maybe at package install time, rather than at the start of the build for that target, and that's too soon to have built the dependency. If you return .buildCommand instead, it'll build the tool dependency first. Seems like a bug, and might not be a useful workaround in your case, of course.

4 Likes
3

Thanks! I’ll try that out tomorrow. I agree that it definitely seems like a bug, and if it’s not a bug, it’s terribly documented and reported.

2 Likes
4

Yep, that worked. Thank you so much!

5

Did you find a way to have a preBuildCommand that you can actually run?

6

The issue was that prebuildCommands require that the tools they run are all binary targets.

I eventually found documentation of that in the initial proposal for extensible build tools (SE-0303):

/// The package plugin may specify the executable targets or binary targets
/// that provide the build tools that will be used by the generated commands
/// during the build. In the initial implementation, prebuild commands can
/// only depend on binary targets. Regular build commands can depend on exe-
/// cutables as well as binary targets. This is due to limitations in how
/// SwiftPM's build system constructs its build plan. It is a goal to remove
/// this restriction in a future release.

I had a feeling that I had read about that somewhere in the past, but I couldn't find it anywhere and the error messages didn't mention the limitation at all, so I assumed that I had made it up lol.

In the end I solved the issue by using a buildCommand and defining the input and output files of my tool. I think the name prebuildCommand is a bit misleading because I thought my tool definitely counted as a 'prebuild' tool, but it seems to work fine as a regular buildCommand. I don't really understand what the purpose of a prebuildCommand is anymore.

5 Likes
7

I need my code to be ran before the build of swift package manager begins as it should download some dependencies before the SwiftPM starts the build. I read the documentation and found out that it would resolve the package.swift but not actually building until the commands are finished. This is what I would like to do but not sure if the current version supports it.

let package = Package(
    name: "Foo",
    products: [
        .library(name: "Foo", targets: ["Foo"])
    ],
    dependencies: [],
    targets: [
         .target(name: "Foo", dependencies: "Kit"),
        .binaryTarget(name: "Kit", path: "Kit.xcframework"),
.plugin(name: "KitDownloaderPlugin", capability: .buildTool(), dependencies: ["download tool with binary"])
    ]
)

The kit mentioned here is behind a user name and password protected url. So the script should simply download it before it is build. I have a curl command that does just that.

 curl -f -o "${framework_zip}" \
        --user "${USERNAME}${PASSWORD:+:${PASSWORD}}" \
        "${framework_url}"

So relatively simple if the preBuildCommand could simply execute a bash script. But it seams like a real overkill to create a binary build just for this?

8

Hmm I read your instructions again and I now agree with you and can ge it to work with the buildCommand as that will run according to the docs whenever some of the output files are missing. So effectively it runs before the build ...

Confusing docs indeed and this prebuildCommand seams redundant. Thanks a lot for the quick response.

9

Glad I could help!

10

To answer my own question. No it is not possible to write something like this as the plugins run in an environment without network access swift-evolution/0303-swiftpm-extensible-build-tools.md at main · apple/swift-evolution · GitHub.