[Pitch] Package Manager Command Plugins

Evolution Pitches
1

Hello,

SE-0303 introduced the first kind of SwiftPM plugins, focusing on the ability to extend the build system with custom build tool invocations (in particular for the purpose of generating source code). Those plugins were always intended to be just the first kind of plugin supported by SwiftPM.

I'd like to pitch a draft proposal for adding another kind of more general-purpose "command plugin" to SwiftPM. These kinds of plugins would be directly invocable by users and would be intended for such things as source code formatting, documentation generation, test report generation, etc. A command plugin would not necessarily have anything to do with the build system.

One important aspect of these custom command plugins is that they can ask the plugin host (either SwiftPM or an IDE that supports packages) to generate specialized information on-demand or to initiate builds or test runs. This is the part of the draft proposal that could use the most scrutiny. There is an element of tension here between making the API rich enough to be as useful as possible, while also making it generic enough to be implementable not only in SwiftPM but also in IDEs that support Swift Packages.

I'd love to hear what everyone thinks and especially what the ideas are for striking the right balance with regard to the "plugin host services" API.

Thanks!

https://github.com/abertelrud/swift-evolution/blob/swiftpm-command-plugins/proposals/NNNN-swiftpm-command-plugins.md

14 Likes
2

This seems great. One reason for this is that it is so tantalisingly close to enabling developers to ditch entire those entire tech stacks used for automating tasks like uploading release builds to App Store Connect, distribute as hoc builds, attach code coverage reports to pull requests etc.

So I guess my impatience is motivating me to ask: is the security around network access really something that bloats the scope of the proposal?

3

To be sure, I know SwiftPM doesn’t allow us to build app targets yet, but the command could be invoked as a separate step, passing the path to the archived release build, until it becomes possible we can achieve the goal of not having to deal with multiple tech stacks and have automated, deterministic and secure processes.

1 Like
4

Thanks for your comments, and sorry it took me a while to reply. I'm certainly no networking expert, but it seems to me that, especially when it comes to authentication and how to safely pass credentials etc, a lot of questions would come up that might be a lot to tackle all at once.

There are a lot of other pieces to this proposal that could really use feedback and discussion (especially the proposed API for calling back to the host to run builds and tests), so I was hoping to come to be able to propose and implement something that would help a certain set of use cases now and then be extensible to network access later.

My hope was that there might be enough here to help automate some parts of the workflow, while still leaving the actual network access to caller scripts. And then focus a later proposal on network access specifically.

1 Like
5

Based on some offline feedback, I have added the PluginCommandIntent enum to the proposed API, so that command plugins that have one a common purpose can declare its intent in a semantically meaningful way.

Initially suggested intents include documentation generation and source code formatting.

The idea is that by declaring what category of functionality a plugin belongs to, the SwiftPM CLI and any IDEs that support packages can provide better ways of grouping the command plugins that are available.

There is a custom intent with a custom verb and description, as in the earlier edition of the pitch.

1 Like
6

I would love to hear other feedback about the direction of this pitch.

The part of the proposal that could particularly use feedback is in the API of services provided by SwiftPM or IDEs supporting packages (the PackageManager struct that the plugin can call to get more information or initiate actions). It seems tricky to find the right balance between providing enough functionality to plugins while still making it possible to implement this API in any IDE that supports plugins, so ideas or feedback in that area would be particularly helpful from anyone who is interested or has thoughts on that area. Thanks!

7

I have a security-related question inspired by CVE-2021-30892:

Is a plugin command run using the system's default shell? If it is, then will the shell start with --noprofile --norc or something similar, in order to prevent arbitrary code in user's shell configuration from running?

8

No, the plugin is invoked directly, with the compiled executable as the first argument to Foundation's Process. There is no shell involved. The same is true for the commands that build tool plugins return.

2 Likes
9

This proposal is great and I'm very excited about it!

  • The shape and number of exposed information to the plugins sounds about right.

[from the proposal] One key tension in this proposal is between providing rich functionality for plugins to use while still presenting that functionality in a way that's general enough to be implemented in both the SwiftPM CLI and in any IDE that supports packages. To that end, this proposal provides a minimal initial API, with the intention of adding more functionality in future proposals.

  • Realistically as long as an IDE can get a list of "command + description" they'll all be happy. This will work well with either CLion or Visual Studio Code I believe; they all generally simply expose "the commands" that they have discovered in some way as a list like this:

Screen Shot 2021-11-15 at 16.39.01
Screen Shot 2021-11-15 at 16.39.01790×102 16.4 KB

And for VS Code we're trying to get together and work on improving existing plugin or making a new one in the SSWG, so we'll figure it out. CLion has a good history supporting build tool commands, and I'm not worried about them as long as there's some way to "dump" what commands are available.

Question: So, how are external tools going to find commands they can invoke? Is there some swift plugin list --type=command or similar?

  • Great that a plugin can invoke build or test using the provided PackageManager, this is great. The name of that type is fine too, I think we had some worries about it at some point, but it seems fine to me :+1:
  • I would really really love re-visit is allowing commands to be "top level".

I.e.: swift doc rather than swift package doc, most notably because I'd love to offer "more fancy" ways to test that do things like "only this file changed, only re-run tests affected by it" which is something sbt can do and is called

I really would love to have "deployment" and documentation tasks available as top-level...

One can imagine that swift docc (to be honest I'd argue for swift doc but let's bikeshed that once that's on the table...) it much more what people will expect from other ecosystems:

  • mvn javadoc -- java's maven does not know about javadoc at all, it's a plugin; in our world we want to document the entire package of course
  • sbt test, sbt jmh, sbt testQuick

I think most notably, we should look at go that while does not have plugins, it does have the "CLI tool" share the name with the language (as does our swift command), and there are plenty "not really package task" involved here as well:

  • go doc
  • go vet (linting, in our world this would be swift lint which again is an external tool today).
  • go generate (generate sources) etc.

I think it really matters for discoverability and making swift feel simple if such tasks are possible to invoke just as swift. I was recently dreaming of swift deploy staging where staging is an environment name you want to deploy to (and have configured the plugin to do so). All those become rather tedious if we had to say "swift package deploy staging" IMHO...

We could say that if ambiguous (or the top level name is used up by something provided by Swift already) you have to invoke the "long form" of swift package <command> but in general I'd love commands to just be available on top level.

The PluginCommandIntent sounds good to me, but I wonder -- shouldn't this be a collection of static properties rather than an enum?

I can imagine browsing plugins by "intent" and looking for a good source code formatter etc, but why should specific names be blocked on major swift versions, as I understand we could not add new enum cases within minor swift releases?

Minor: Should we qualify as MyPlugin:doc rather than MyPlugin.doc? I guess the dot seems weird to me personally... Swift doesn't really have namespacing so we don't have much to lean on here...

The permissions affect the ways in which the command plugin can access external resources such as the file system or network. By default, command plugins have only read-only access to the file system (except for temporary-files locations) and cannot access the network.

We should clarify that this is only currently supported on apple platforms via sandboxing, as I understand we don't have this implemented on Linux yet...?

Feedback on the implementation of:

Many command plugins will invoke other tools to do the actual work. A plugin can use Foundation’s Process API to invoke executables, using the PluginContext.tool(named:) API to obtain the full path of the command line tool in the local file system (even if it originally came from a binary target or is provided by the Swift toolchain, etc).

When writing plugins I found it hard to understand what name I'm expected to pass there.

It would be nice if the error when crashing with an unknown name provided the available tools and where they came from.

It seems we want to call this:

var packageManager: PackageManagerServiceProvider { get }

just PackageManager instead? it seems so in the listing below where we discuss struct PackageManager.

I love the ways to interact with it -- great that it's an async function to build or test. This would allow me to implement a "quick test" (only run tests which failed the last time) thing that other package managers offer, very excited for this.

The TestResult seems to be missing information that I'd like to see in the proposal?

public struct TestResult {
   /// Path of the code coverage JSON file, if code coverage was requested.
   public var codeCoveragePath: Path?
   
   /// This should also contain information about the tests that were run
   /// and whether each succeeded/failed.

}

So if TestResult is "all the tests" how will be the individual tests reported? let tests: [????TestResult]? where each has a failed / succeeded / skipped? Would those also be able to have the time it took them to execute?

I love the way running tests is exposed, it is nice to be able to pass the filterest list there.

Overall design comment:

So this is a pretty simple plugin infrastructure; we can't have plugins produce outputs that other plugins take as inputs and have it be type-safe etc. That's fine I suppose, though every time I read these I'm left longing for a very powerful system like that (and that I'm used to from a prior life). I understand though this would be very hard to implement in compatible ways with existing build systems, so I'll drop the ball on that -- this should be good enough for all the needs we have today :+1:

I'm positive on the pitch! I'm excited to be able to write a few types of plugins I'm missing in Swift today: building docs, deployments, "test quick", and "special weird tests (that create multiple clustered nodes)" etc.

Thank you for the work on this, it's really promising!

5 Likes
10

Thanks a lot @ktoso for the extensive feedback!

Thanks, great to hear. One of the particular areas where I think it's difficult to find this balance is in the PackageManager type where SwiftPM or an IDE can vend services for plugins to "call back" into. It's easy here to make unintentional assumptions about the concepts and features vended by the environment in which the plugin is running.

One possibility is that there is some way of expressing whether a particular functionality is available, e.g. a plugin that requires access to symbol graphs can only work where those are available.

That's a good point — the intent was that tools that integrate with packages would use libSwiftPM and invoke tools that way, and there they could fairly easily find the plugins. They will also appear in package describe. But I think it's a great idea to also add some kind of list functionality, even for use from the command line.

1 Like
11

I can see the point, but I am concerned about possible ambiguity here. The direct subcommands of swift that are available today are a mixture of ones that conceptually operate on packages and ones that don't, and I think that's confusing. For example, swift build and swift demangle both look like peers but operate very differently (in particular one requires a package and one does not).

I would actually prefer to see us go the opposite way (and I think this was the intent at some point) to make any command that operates on packages be spelled as swift package <command>. With the introduction of a now open-ended set of verbs, it seems to me that having the Swift driver treat any unknown verb as a possible Swift Package plugin command would get confusing.

I'm pretty sure that @available will work for minor releases of SwiftPM (note these are SwiftPM version, not Swift language versions). For example in CLanguageStandard, there is:

    /// ISO C 2017.
    @available(_PackageDescription, introduced: 5.4)
    case c17

etc. So I don't think that static functions will give us anything here that enums don't. The only thing is that we couldn't have two cases with the same name but different sets of parameters, but I think that might be confusing to have anyway.

That makes sense, although we could also use a separate parameter that specifies which target (or package?) the plugin comes from.

Yes, this is a great point. That's the current case for manifests too. I think only Darwin is sandboxed at the moment.

Many command plugins will invoke other tools to do the actual work. A plugin can use Foundation’s Process API to invoke executables, using the PluginContext.tool(named:) API to obtain the full path of the command line tool in the local file system (even if it originally came from a binary target or is provided by the Swift toolchain, etc).

Thanks, that's a great point. Perhaps we should also have an iterable collection of the available tools.

Yes, I think that was just an oversight. Thanks!

Yes, you're right. This struct needs to be reworked a bit. The intent is to pass back a struct for each of the tests, grouped into the different test suites that were run. I'll update the pitch with a reworking of this part.

Thanks a lot for the detailed comments as well as this assessment. One challenge here has been to find a way to fit plugins into the existing Swift Package Manager, and to make it work in the different environments where packages are supported (with their own build systems etc).

It would be fantastic to reconsider how SwiftPM works at a more fundamental level, and I think that would open up a lot of possibilities. Having type safe information flow from one plugin to another might be possible in the current SwiftPM — I'd have to think more about that — but one more basic thing that would make a big difference would be to be able to have layered dependency graphs, e.g. to have plugins and build tools be able to have separate dependencies from the actual library or executable code that they operate on.

The goal with the current effort is to add some flexibility within the current model. But I would love to see a rethinking of SwiftPM that was built on plugins from the ground up rather than have them get added on as this and the previous plugin proposals are doing.

You mentioned that you'd worked with a plugin system that does this really well — which one was that, and do you think its approach could be made to work in SwiftPM without a fundamental restructuring?

Thanks again for all your thoughts and comments here!

2 Likes
12

CLion currently retrieves SwiftPM project structure by running swift package dump-package. It would indeed be really helpful for CLion, and probably for other IDE integrations as well, if SwiftPM had a similar command for dumping JSON with command names & descriptions.

1 Like
13

+1 on this, it looks really great!

Also, on a broader level, I really love these new plugin APIs that have been added recently. The idea of providing a special module which gives you access to package manager services is a really elegant solution. My experience of using other build systems (make, cmake, etc) is full of awkward DSLs to access the build system's understanding of the project so I can build some software the way I need. Having a richly-typed Swift API which lets me build parts of the package graph and access SwiftPM's understanding of the package is so much better.

One thing I would like is custom swift settings on BuildParameters. I think it would be useful to add a fuzzing plugin, which currently requires adding the following settings to your fuzzer executable targets:

.target(
  name: ...,
  dependencies: ...,
  swiftSettings: [.unsafeFlags(["-parse-as-library", "-sanitize=fuzzer,address"])]
),

And building with a script similar to the following to set some compiler flags:

if [ "$(uname)" == "Darwin" ]; then
  # Apple's SDK toolchains do not include fuzzer support, but swift.org toolchains do.
  xcrun -toolchain swift swift build -Xswiftc -sanitize=fuzzer,address -Xswiftc -g
else
  swift build -Xswiftc -sanitize=fuzzer,address -Xswiftc -g
fi

It would be nice if this could be done with a command plugin, so I could just add the plugin dependency, and start fuzzing with swift fuzz or swift package fuzz.

2 Likes
14

I had come up with the idea of introducing a new swiftpm shortcut which packs the core functionality of SwiftPM, with subcommands like build, run, clean, reset, update as well as custom ones. This would make new users comfortable because such command interface is generally shorter and more popular, but the details should be carefully handled.

15

Thanks a lot for chiming in @egor.zhdan! We'll make sure the plugins are discoverable in some way like this :+1:

16

Thanks for this suggestion, I can really see the use of this. Would it suffice to have the same flags for all targets, as in your example with the -Xswiftc flag? It would of course be more flexible to allow custom flags for different targets, but specifying that could get quite complicated. Maybe this proposal should start out by letting you specify the same things you can on the swift build command line (which applies to all targets) and then that can be further extended in a later proposal.

17

Thanks @egor.zhdan and @ktoso. Plugins would indeed be included in the swift package describe output (which has a JSON form in addition to text), but that only lists plugins that are defined in the package being described.

Since the set of command plugins that is available for a package includes those defined in the package as well as in any dependencies, it might also make sense to have a separate command (as suggested here) that's more focused on the set of plugins that are available for use and where they come from. That could be derived by describing each of the packages but then every IDE would need to reimplement that logic. So this might be worth addressing in the proposal as well.

1 Like
18

That's an interesting idea and would help keep some clarity, though it seems beyond the scope of this proposal unless there is a strong push to make plugins be invoked directly from the swift driver. I don't think that decision necessarily has to be made as part of this proposal — we could start out by having a more verbose way to invoke the plugins and then broaden that in a future proposal (while keeping the more verbose way working of course) if there is a strong desire to do that. That seems easier than going the opposite direction.

19

I have updated the pitch based on a lot of the feedback here in the pitch. The main changes are to:

  • add information about the outcomes of the unit test runs to the TestResult structure
  • add the ability to pass extra flags in the BuildParameters structure
  • add a section with a suggested command syntax for listing the command plugins available to a package
  • fix various inconsistencies and mistakes in the examples

The top-of-branch of the pitch proposal has the changes, and the differences can be seen at Comparing d4ad816be262b9fa7381d35a1946b82356f65738...swiftpm-command-plugins · abertelrud/swift-evolution · GitHub

2 Likes
20

Is there a guide on how to setup a local environment to try the branch?

My use case is a new plugin provided by swift-argument-parser to used to generate a man page. To to this the plugin would need to run after the primary build and invoke the built binary with --experimental-dump-help.

1 Like