Package manifest editing commands beyond SE-0301

Evolution Pitches
1

Hi everyone!

SE-0301 "Package editing commands" was accepted a while ago, but the implementation never managed to land. I went ahead and implemented the three commands it specifies:

  • swift package add-product
  • swift package add-target
  • swift package add-dependency

Now, it didn't take very long for me to want to add another command to edit package manifests, and adding new specific editing commands is really easy. I have a PR up for swift package add-target-dependency, which adds a new dependency to a named target. The command-line interface I have is this:

OVERVIEW: Add a new target dependency to the manifest

USAGE: swift package add-target-dependency <dependency-name> <target-name> [--package <package>]

ARGUMENTS:
  <dependency-name>       The name of the new dependency
  <target-name>           The name of the target to update

OPTIONS:
  --package <package>     The package in which the dependency resides
  --version               Show the version.
  -h, -help, --help       Show help information.

It's getting a little bit crowded in the swift package command space. Here's the full list from main:

SUBCOMMANDS:
  add-dependency          Add a package dependency to the manifest
  add-product             Add a new product to the manifest
  add-target              Add a new target to the manifest
  add-target-dependency   Add a new target dependency to the manifest
  clean                   Delete build artifacts
  purge-cache             Purge the global repository cache.
  reset                   Reset the complete cache/build directory
  update                  Update package dependencies
  describe                Describe the current package
  init                    Initialize a new package
  experimental-install    Offers the ability to install executable products of
                          the current package.
  experimental-uninstall  Offers the ability to uninstall executable products
                          previously installed by `swift package
                          experimental-install`.
  diagnose-api-breaking-changes
                          Diagnose API-breaking changes to Swift modules in a
                          package
  dump-symbol-graph       Dump Symbol Graph
  dump-package            Print parsed Package.swift as JSON
  edit                    Put a package in editable mode
  unedit                  Remove a package from editable mode
  config                  Manipulate configuration of the package
  resolve                 Resolve package dependencies
  show-dependencies       Print the resolved dependency graph
  tools-version           Manipulate tools version of the current package
  compute-checksum        Compute the checksum for a binary artifact.
  archive-source          Create a source archive for the package
  completion-command      Completion command (for shell completions)
  plugin                  Invoke a command plugin or perform other actions on
                          command plugins

It seems likely that we'll want to add more of these in the future. Should we continue to add them as subcommands of swift package, or should we revisit the decision noted in the Alternatives Considered in SE-0301?

I could imagine two groupings that would make sense here:

  • swift package manifest <command>, which all of the manifest-editing commands go under.
  • swift package product <command> and swift package target <command> for product- and target-specific commands, some of which are manifest editing.

Thoughts?

Doug

21 Likes
2

There are two ways command-line arguments are used: interactively, and batch (either directly from a shell script, or indirectly from another language). I don’t think either is served well by additional hierarchy—for the batch case it doesn’t really matter, so it’s more about what reads well and how it fits into an overall command line. For the interactive case, though, I think deep hierarchies make it harder to discover commands, either through smart autocomplete or through --help, because it’s easier to look in the wrong place, or to not know which subcommand to look into. And then even when you know it, it still takes longer to type. So while I sympathize with the busyness of the full list, I think it probably still makes sense to not go deeper than the effectively-two levels of hierarchy we already have (swift > package).

3 Likes
3

I agree that additional levels of hierarchy is worse for discoverability, but considering the amount of editing commands potentially available it seems maybe warranted:

swift package target add <name> [--type <executable, target, module>]
swift package target remove <name>
swift package target rename <name> <new-name>
swift package target remove-dependency ...
swift package target add-dependency ...

All of which would be valid commands for products too.

That said this logic extends to add|remove-dependency which could be expressed instead as:

swift package target dependency add <name>
swift package target dependency remove <name>

I dont have a really strong opinion because all options seem suboptimal.

Maybe we should extend ArgumentParser to be able to group subcommands like we currently do for titled groups of options?

6 Likes
4

I like the idea of keeping these as top level subcommands of swift package, but updating the help-text so related subcommands are visually grouped better.

I'm skeptical of the idea of adding an additional level of subcommands here because it's not clear to me we do want to add too many additional manifest editor subcommands. IMO the utility of a CLI for "verbs" like remove and rename is a little questionable, since editing the source in those cases is trivial. Where things get a little more interesting is when edit operations are composed. Something like a hypothetical swift package upgrade-dependencies which would bump the version specifiers of a dependency in a package manifest might be a useful feature, but it also doesn't fit into a neat hierarchy, so I think we'd want that at the top level too.

4 Likes
5

Totally a reasonable position. Small note, I would hope remove and rename would edit all uses of the target in the package.

E.g. rename Foo -> Baz

 _ = [
-  .target(name: "Foo"),
+  .target(name: "Baz"),
   .target(
      name: "Bar",
-     dependencies: ["Foo"])
+     dependencies: ["Baz"])
 ]

This would be more useful as an IDE integration IMO

2 Likes
6

Agreed on a deep level of hierarchy being harmful, but adding one more would simplify the organisation of the command and discoverability is impacted by both nesting and a very long API surface if you will. I think this can be solved by help commands verbosely display everything in all hierarchies and an easy to access web documentation too.

Organising the commands under a new hierarchy would give us a chance to grow the list of commands and actually enhance discoverability as it would organise the commands logically and would reduce the impact of growing each. Not sure how many layers would be too many but if I were to guess one more layer or two would be ok as in the benefits would outweigh the negatives.

I would vote for this change to go through.

7

I like this idea because I don't know that we can rely on introducing hierarchy to keep the top-level swift package tidy. Some kind of ad hoc grouping mechanism lets us categorize things over time without changing existing commands.

I suppose this is also an ArgumentParser feature request, to be able to group subcommands.

Doug

1 Like
8

The main issue I have no idea how we would expose this as API in argument parser

9

Spit-balling here, but would it be desirable to provide spm ... as an "alias" for swift package ...? Users would still use swift build and swift test for the common tasks, but swift package commands would significantly shorten (and probably be more discoverable in online resources to boot).

12 Likes
10

Seems reasonable, and sorta mirrors rust.

swiftc  <-> rustc
swiftly <-> rustup
spm     <-> cargo
swiftpm <-> cargo
3 Likes
11

wasn’t there previously an effort to get rid of the SPM spelling and standardize on SwiftPM as the official name of the tool?

3 Likes
12

I do think this is the right approach, because it allows for:

swift package target help

Or

swift package product help

Etc.

That way one can get a partial contextual help instead of a monster help message. It greatly improves usability I think.

It is also inherently more scalable in my opinion.

3 Likes
13

Agreed, hierarchy is needed and can help logically organise things and avoid weird verbs to otherwise add separation between commands which could be confused.

1 Like
14

Yeah, that has been reiterated but then “swiftpm”.

It’s quite often I have the prefix

swift package —disable-sandbox benchmark xxxxx

As we write to disk/network.

It would be great to also allow for disabling the sandbox somehow without interaction all the time. But that’s separate (but related to the overarching discussion on usability).

swiftpm benchmark xxx

Would be nicer indeed… it’s really related also to the swift language evolution when we try to reduce unnecessary ceremony and boilerplate.

2 Likes
15

I think revamping swift package would be great for several reasons, e.g. the current setup makes everything subcommands which bring a lot of baggage like requiring a package manifest in the working directory which doesn't necessarily make sense for things like compute-checksum. IIRC there are also some subcommands which try to hack around that in weird ways.

Maybe swift build could also finally become swiftpm build?

4 Likes
16

CommandConfiguration currently has a listing of subcommands. For swift package, it's a pretty long list:

        subcommands: [
            AddDependency.self,
            AddProduct.self,
            AddTarget.self,
            AddTargetDependency.self,
            Clean.self,
            PurgeCache.self,
            Reset.self,
            Update.self,
            Describe.self,
            Init.self,
            Format.self,

            Install.self,
            Uninstall.self,
            
            APIDiff.self,
            DeprecatedAPIDiff.self,
            DumpSymbolGraph.self,
            DumpPIF.self,
            DumpPackage.self,

            Edit.self,
            Unedit.self,

            Config.self,
            Resolve.self,
            Fetch.self,

            ShowDependencies.self,
            ToolsVersionCommand.self,
            ComputeChecksum.self,
            ArchiveSource.self,
            CompletionCommand.self,
            PluginCommand.self,

            DefaultCommand.self,
        ]

We could introduce some group structure with another argument, e.g.,

subgroups: [
  CommandGroup(
    name: "Package manifest editing",
    subcommands: [
            AddDependency.self,
            AddProduct.self,
            AddTarget.self,
            AddTargetDependency.self,
            Init.self,
    ]
  ),
  CommandGraph(
    name: "Package editing",
    subcommands: [
            Edit.self,
            Unedit.self,
    ]
  ),
]

Under this suggestion, swift package target will have two entries, and swift package product will have one. There will be no consistent place for "package manifest editing operations" the way the grouping described above provides.

I think the hierarchy sounds attractive because it feels right to put things into hierarchies, but I don't think it's actually going to actually make anything easier for people.

Doug

1 Like
17

I’ve used the GitHub CLI quite a lot and I’ve found it really intuitive. I consider it one of the best designed modern CLIs and it does have a pretty extensive hierarchy. However, the base command is “gh”: just two letters! I’m thus in favor of grouping commands and growing the hierarchy, but only if we alias “swift package” as either “spm” (similar to npm) or “swiftpm” (keeping to Swift’s verbose style).

4 Likes
18

I ended up implementing this with a result builder. It's pretty straightforward: here's the pull request.

[EDIT: I put up a discussion thread in the area for the argument parser library]

Doug

7 Likes
19

I’ll add another voice in favor of aliasing “swift package” with “spm” or “swiftpm”. It’s likely there are some down sides to this but I would be interested to hear both the arguments for and against.

Regardless, excited to see these package editing commands being revisited. I look forward to seeing how this simplifies package installation. Thanks @Douglas_Gregor!

2 Likes
20

I tested out these commands in the latest release/6.0 snapshot and they are great! I was able to run these commands to seamlessly add swift-argument-parser to a test project:

swift package add-dependency https://github.com/apple/swift-argument-parser --from 1.3.0
swift package add-target-dependency ArgumentParser MyTarget --package swift-argument-parser

I think these commands will go a long way to help simplify the process of adding new project dependencies, even by being able to replace installation instructions by providing example commands like the ones above.

Looking at the two commands, the first could be directly copied and executed by a package end user from a given project's documentation. The second one, which is required for actually starting to leverage the package in your code, would have to include a placeholder for the target name like so:

swift package add-target-dependency ArgumentParser <target> --package swift-argument-parser

Some (possibly even most) developers will quickly identify <target> as something that needs to be replaced in order to execute the command successfully, but those with less experience may not and even a helpful tip included with installation instructions may be overlooked and lead to confusion.

This made me want to have the ability to smash the two commands into one to get closer to the holy grail of a single, copy, paste and run command that would get someone up and running with a new package dependency in their project (Ă  la npm install <package-name>). This would be a convenience to devs of all experience, not just new comers to Swift. Something like this:

swift package add-dependency https://github.com/apple/swift-argument-parser --from 1.3.0 --product ArgumentParser --package swift-argument-parser

Of course, this still misses the mark because there is no specified target to add the new dependency to (among other reasons). There are probably a number of ways that this could be solved (with special cases for single target packages, or an interactive mechanism for developers to select the relevant targets), but this could easily build to have a scope beyond what should be included in the toolchain.

A thin, community provided wrapper around swift package could add functionality to achieve this, but it would be nice to have a first party solution that has these conveniences. All that being said, I'm interested to hear other's thoughts on this subject and what interest there is here.