SPM build tool plugins - bringing resources to the target?

Using Swift
,
1

Ok, ran into one more issue when trying build tool plugins - any hints on how to work around this issue cleanly is much appreciated.

Basically I have a simple prebuild tool that should generate source code using sourcery. As part of that, a .stencil template is defined in the package defining the plugin.

Now, when executing the plugin, I want to consume that resource - but all the directory accessor in the build tool plugin context references the current build target - there doesn't seem to be any way to get at resources/files from the checked out version that was used to build my build tool?!

I can see it is checked out in .build:

hassila@max ~/G/package-plugin-manager (feature/sc-376/autogenerate-all-boilerplate-code-for-apps) [1]> find . -name "*.stencil" -ls
50926654        8 -r--r--r--    1 hassila          staff                 471 Jun 29 22:50 ./.build/checkouts/package-plugin/Sources/PluginResources/PluginFactory.stencil
50902553        8 -rw-r--r--    1 hassila          staff                 196 Jun 29 17:38 ./Templates/PluginManagerLoader.stencil
hassila@max ~/G/package-plugin-manager (feature/sc-376/autogenerate-all-boilerplate-code-for-apps)>

So there seems to be no way to defined resources for a plugin that will be copied with it - perhaps that's simply a missing feature?

Or is there some way to access resources from the package that defines the build tool?

For tools that are small in nature and just operates on the same package where they are defined, this is not a problem of course, but that is less useful.

I don't want to add the stencil to the end-consumer package, the best approach I've come up with so far is to embed the stencil in the plugin source code as a string literal and to write it out to disk in the temporary work directory for the plugin so I can consume it, but that is not very nice indeed.

Am I missing something, or are there simply a gap here?

2

TL;DR

Can a build tool plugin access any resources from the repository/package where the plugin was defined when used by a client in another repository/package, or are we stuck with the client repo/target + the scratch writeable directory?

3

I am having trouble following. Is any of these what you are asking?

  • Can a plugin definition include resources?

    Unspecified, but the final answer will probably be no. They are designed to be minimal. They cannot even have dependencies.

  • Can an executable still include resources if it is invoked by a plugin?

    I am pretty sure this should be possible. If not I would consider it a bug.

  • Can an executable access the resources of another target?

    No. Resources are an implementation detail and the generated accessor is deliberately internal so that they are not leaked into the API without the developers deliberately forwarding them.

  • Does a plugin context contain information about the current target’s resources?

    I do not know off the top of my head. It is certainly a reasonable request if it is not available yet. (I can look up the current state if this actually is your question.)

4

Thanks @SDGGiesbrecht - I'll try to clarify.

Package A
  Plugin (target + product)
  Sourcery.stencil resource (needed/used by Plugin to perform its work)

and

Package B
  Target (that depends on plugin)

So while testing the plugin during development in Package A, the plugin has access to the Sourcery.stencil template by accessing it through e.g.

...
        let templateFile = context.package.directory.appending("Templates").appending("Sourcery.stencil")
...
        let commandArgs:[String] = [
            "--templates", templateFile.string,
            "--sources", inputDirectory.string,
            "--output", outputDirectory.string,
            "--disableCache"]

           let command: Command = .prebuildCommand(
            displayName: "Running Sourcery",
            executable: toolPath,
            arguments: commandArgs,
            outputFilesDirectory: outputDirectory)

            return [command]
...

Now, when starting using the plugin in Package B, that path evaluates to B:s package directory (context.package is now in B), which does not have the Sourcery.stencil - it does not even have a Template directory.

So I want the plugin to have access to a resource (Sourcery.stencil) from Package A (which defines the plugin) while executing the plugin in the context of Package B, as it's basically part of the definition of the plugins functionality in this case.

So I feel I am missing (most likely):

A resources: clause for the plugin definition.

This is the closest ;-)

They can have dependencies declared though - e.g. like a executable target.

That was my latest try to work around the problem, by defining a dummy target which the plugin depended on that just had a 'copy resource' phase - didn't get that working though, as it still is unclear how I can access those resources - at which path would they reside at plugin execution time)

5

While a resource in the general sense, this is not what SwiftPM calls a resource in the technical sense.

context exists to provide information about the execution context of the plugin (not the context of its own build). Hence the resulting path inside B is working as intended. If the plugin wants to know something about B in order to make a decision, context is where it should look.

On the other hand a resource in SwiftPM’s technical sense is something associated with a target that is to be bundled alongside it for runtime access. You register them with the target’s resources parameter and look them up at runtime with Bundle.module. In the manifest, plugin(...) does not even have a resources parameter, therefore resources are clearly not supported for plugin definitions.

What you likely want to do is create an executable that produces the template (executables do support resources). Your plugin would then depend on both the executable and Sourcery, first generating the template, then running Sourcery pointing at it.

Think of plugins as being composed of two pieces, the declaration of intent (.plugin) and one or more executables (.executableTarget) that actually do the work.

The declaration will be evaluated while SwiftPM is figuring out what it will need to do, before it actually does anything. Like the manifest, nothing is guaranteed to have been resolved yet, the source is not guaranteed to be available, and arbitrary imports do not work. The declaration should be as simple as possible.

On the other hand, the workhorse executable can be as elaborate as you want, and supports all features of SwiftPM. Since it will only be invoked if the build actually needs it, and participates in the incremental build cache, it is much less of an issue if it bloats into something cumbersome.

They can only depend on executable targets. And the executable is not actually available when the plugin is evaluated, but rather declared as needed when it comes time to execute the reported commands. Any other kind of dependency is not supported and will trigger an error (or fail silently in early versions).

1 Like
6

Thanks @SDGGiesbrecht - I clearly need to adjust my mental model such that the plugins only are drivers of actual work...

It would have been convenient though in this case to be able to have a resource included with the plugin as it's just a basic configuration file used by the tool that is driven by the plugin (now I'll need to create an executable target just to write out this config file which will be used by another tool.).

Thanks for clearing things up.

7

Sourcery templates can be also initialized from a String, e.g., a constant in your code. Could that be a workaround for you?

8

Thanks, I ended up with a minimal helper function writing out the template into the plug-in working directory for now - will repackage it as a proper executable and resource as suggested in the future if needing more complicated templates.


extension ApplicationLoader {
    func generateStencil(path: Path) {
        // This is a ugly workaround as SPM build tool plugins
        // Can't bring resources to the target package, so we'll write one out
        // instead. 'proper' would be to add dependency on a binary target
        // which in turn adds the stencil as a resource, which is accessed
        // using Foundations Bundle.module.xxx

        let stencil = """
        {% for type in types.all|annotated:"Application" %}

        import Application

        @_cdecl("pluginFactory")
        public func pluginFactory() -> UnsafeMutableRawPointer {
        Unmanaged.passRetained(ApplicationFactory({{ type.name }}.self)).toOpaque()
        }

        {% endfor %}
        """
        // We'll crash if failing to open for writing, we'll live with that
        let fd = fopen(path.string, "w")

        if fputs(stencil, fd) == EOF {
            print("Write to \(path) failed. errno = \(errno)")
            exit(EXIT_FAILURE)
        }

        fclose(fd)
    }
}