[Pitch] Package Editor Commands

Evolution Pitches
,
10

Firstly, a question: what does this do about conditional-compilation blocks (#if os(linux), etc), imperative code (package.dependencies += [...]), and comments? Are they all preserved? Can we detect them and fail the operation if it will destroy information?

I think this is a problem any editing tool will face. FWIW, my preference would be to detect those situations and fail, with a flag to force the change.

Secondly, I'd like to propose an alternative - I agree that this is really important and useful functionality, but I think we could implement it better. You already mentioned some of the difficulties with this approach:

I'd like to add a couple more:

  • Limited functionality. It is always likely to remain very limited due to the aforementioned maintenance burden, and the limitations of the command-line interface. It doesn't scale to give every operation on every type it's own, individual CLI entry-point.

  • Difficult to remember. You mention as one of the motivations:

    Instead of remembering package layout conventions, they will now have to remember commands with very long names, each with unique sets of flags, and each flag itself also with a relatively long name (e.g. --up-to-next-minor-from). Your 4-line example has a lot of repetition (some of that is inherent to CLI interfaces, but it's still ugly and typo-prone).

    Simply put, I don't see the CLI ever being anybody's preferred way of sitting down and making a change to a Package.swift file; IMO the focus of this feature should be automation.

So here's the thing: what if we didn't do all that? Why are we even creating a parallel interface? For scripting? We already have a great Package interface, written in Swift, and Swift supports scripting. So why can't I write a script to mutate a Package instance?

Well, actually, we can! It turns out that loads of projects have secret little scripts hidden inside their Package.swift and are doing this exact thing already! SwiftPM itself does it, swift-driver has something similar, so does sourcekit-lsp, etc. Other projects may require custom build scripts because environment-specific options and paths cannot be written using one Package.swift file, and they lack the tools to automatically edit them.

On the one hand, these little scripts at the end of Package.swift files are for configuration, not for the specific use-cases presented in this pitch (integrating a package you found on the web). On the other hand, they are an example of machine-editing Package.swift files, using the Package API, which is extremely widespread today.

So the alternative that I'd like to pitch is to use Swift's existing script functionality to allow for focussed scripts which edit a Package object.

swift package update-script <script name>

We'd create a new module, EditablePackage (?), with a single global declaration. All package update scripts would implicitly import EditablePackage if they don't do it explicitly.

// module EditablePackage
import PackageDescription

var package: Package

We'd run the existing Package.swift file, resulting in us constructing a Package object named package, which we'd assign to EditablePackage's global. Then we'd run the script file in a special context where EditablePackage is implicitly imported. Once the script is finished, we'd validate and serialise the updated object found in EditablePackage.package.

Pros:

  • We get the full flexibility of the Package API, with no additional maintenance burden translating them in to CLI switches. This would be immensely more powerful, enabling use-cases from integrating packages to configuration scripts (such as those used by SwiftPM, sourcekit-lsp, etc). The possibilities are endless.

  • It's just Swift. There is one thing to remember, and you already know how it works. It's easy to read and verify that the script does what you expect.

Cons:

  • The command accepts a file, rather than a string. That isn't very convenient, since you often want to run some commands without writing a file first.

    However, there are ways to use stdin as your input file. The convention is to use a sole - as the file name. The swift compiler already supports that (clang also supports it, as do other tools like VSCode - it's a super useful feature). For example, if you run:

    xcrun swift -

    You can just start typing code, with newlines and quotes and everything, and finish off with EOF/CMD+D and the compiler will run it.

    For copy/pasting commands, you can use a "heredoc". For example, try copying and pasting the following in to a terminal:

    xcrun swift - <<EOF
var a = "hello"
a.append(", world")
print(a)
EOF

    The best thing? Everybody here can read what this code does.

  • Some situations can become a little more verbose. Consider as a CLI:

    swift package add-dependency https://github.com/apple/swift-argument-parser
swift package add-target MyLibrary
swift package add-target MyExecutable --no-test-target --dependencies MyLibrary,ArgumentParser
swift package add-product MyLibrary --targets MyLibrary

    Or as a script:

    swift package update-script - <<EOF
package.dependencies += .package(url: "https://github.com/apple/swift-argument-parser")
package.targets += [
  .target(name: "MyLibrary")
  .executableTarget(name: "MyExecutable", .dependencies: ["MyLibrary", "ArgumentParser"])
]
package.products += .library(name: "MyLibrary", targets: ["MyLibrary"])
EOF

    Which is disappointing. The Swift version has bits of ceremony (like .package(url:...)) which aren't required in the CLI. But the CLI does one operation at a time, so it has a bunch of ceremony of its own (swift package add-) that the script doesn't have.

    And let's not forget that we can add APIs to EditablePackage to make these scripts smaller. For example, we could add a function to add a dependency and add it as a dependency to some list of targets, in one move. It's a lot easier to do these things if we're writing APIs for Swift scripts using the Package type. Maybe that's also a way to expose things like --no-test-target.

3 Likes