Swift Run scripts

I don't think @Aciid meant to not include the feature per se, but rather that it might just not be specified inside the Package.swift file. I can imagine that there could be a Commands.swift file (just some strawman name) or similar where those commands could be placed.

2 Likes

BTW, a similar topic has come up before:

Even though this is more about running scripts after certain events, a solution could combine all those features.

1 Like

Yes my comment wasn't very clear. I was arguing against that kind of pattern. For one it seems odd that scripts/commands would be its own file but not anything else related to SPM manifests. If you did have a Commands.swift, why not allow a Dependencies.swift? Another reason I would argue against separate files is that Package.swift no longer describes all about a particular package. Whereas I believe that a proper manifest file should encompass everything about a specific package, not only how it's built and its products.

On the flip side I could kind of see why you would want seperate files for these kinds of things. Whereas npm just uses a plan .json file for its manifest, we have a .swift one that runs in an interpreter. I'll try not to rekindle the json/yaml/etc vs Swift manifest debate, but using a .swift files does mean we have to be more careful around adding too many things to the package constructor, less we make it even harder for people to remember the correct order of the init.

I wonder if we could compromise and extend Package.swift to allow defining a Commands type that SPM can look for?

2 Likes

When thinking about a simple example like a currency library, I can imagine there being a .gyb file which would produce a .swift file with all the currencies of the world as structs. Now processing this gyb file would be a classic example of a script that should be included in such a library. As such it would make sense to me to include it in the Package.swift file in a separate section.

In an application there could be a script to update the version numbers for all targets or similar (super-made-up dummy example). Come to think of it, there could actually be user-specific scripts outside the current Swift project but I digress…

I am not overly familiar how npm scripts are used, but it seems to me that there are two kind of scripts.
In a library, the scripts would be more or less part of the package, while in an application the scripts are somewhat specific shortcuts to commonly used commands pertaining to this specific app. Is this something that is expressed in npm or even worth distinguishing?

While that's a good point, I think the convenience of having it inside the Package file trumps it. I'm very happy to use the scripts section in node packages.

1 Like

I don't think this is really something worth distinguishing. "scripts" are best thought of as "anything you would need to do in a terminal, but automated". So you could have libraries that define scripts to publish a new version of the lib that runs the tests, does any extra work to setup before publishing, and then publish.

Personally, I would much rather like a Swift equivalent to Ruby's Rake (or plain Makefiles). I'm somewhat not convinced of the value of coupling a task runner to the package manager.

1 Like

The interesting thing here is that the Package.swift is both lib reference (Xcode target/Podspec/Gemspec) and project reference (Xcodeproj/Gemfile/Podfile), which means there might always be a bit of a tension between "is this useful when having the package.swift for building as a lib" vs "is this useful when it's the root of a project and needs to define project metadata"

This is really only useful for the latter case

In NPM, no this doesn't exist. How some of those types of projects handle it is by allowing a lot of the configuration to live inside the project definition file - I built something like this with PackageConfig - then you just need to run the main command for whatever tool via the nom run command.

This is quite an interesting idea, the two domains could be completely separated:

let package = Package(
    name: "danger-swift",
    products: [
        .library(name: "Danger", type: .dynamic, targets: ["Danger"]),
        .executable(name: "danger-swift", targets: ["Runner"])
    ],
    dependencies: [
        .package(url: "https://github.com/JohnSundell/Marathon.git", from: "3.1.0"),
        // Dev dependencies
        .package(url: "https://github.com/eneko/SourceDocs.git", from: "0.5.1"),
        .package(url: "https://github.com/realm/SwiftLint.git", from: "0.28.1"),
        .package(url: "https://github.com/nicklockwood/SwiftFormat.git", from: "0.35.7"),
    ],
 })

let project = Project(
    scripts: [
      .command(name:"docs", "sourcedocs generate --spm-module Danger --output-folder Documentation/reference"),a
      .command(name: "format", "swiftformat"),
      .command(name: "lint", "swiftlint"),
      .command(name: "ci", ["lint", "format"]),
      .command(name: "deploy", "node --require ts-node/register scripts/deploy.ts")
   ], 
    config: [
      .setting(name: "swiftlint", ["ignore":["MyModule"]]) 
   ]
)

and maybe that's more extensible and lean into the separation completely?

1 Like

Oh, I missed this example before. I think as soon as we start composing these commands, this feature starts looking more like a task runner. A task runner can become a big project of its own but maybe that is the right direction for such custom commands? We already have a way of writing powerful build systems using llbuild and I think it'll be great to write a Swift DSL on top of it for running custom commands. I am not sure if such a task runner should be part of SwiftPM or not though.

1 Like

Having a task runner bundled with swift would be really nice.
It could be a side project to SPM, even though the latter would need to be aware of it so that we can bundle our tasks along our dependencies.
In fact SPM may even use it internally to define its own task (resolve, edit, dump, ...).

Didn't test it but Beak approach to define tasks/commands seems interesting. Plus side IMO:

  • It's not text based
  • We can execute shell commands or swift pure code :)
  • A lot of possibilites (arguments, optional arguments, ...)
1 Like

I would love this functionality in SPM. I'm playing around in the node ecosystem right now and am finding a task runner to be a big boon to productivity. As far as I'm concerned it's a big +1 from me.

1 Like

I think it is dubious to have SPM to act as a task manager. Unless it will grow to systems like Gradle, where you can basically build the whole pipeline using their plugin APIs. It is not clear to me whether this is a development vector for SPM.

1 Like

In general I'm not sure how I feel about this, but with specific regard to what Rust calls build scripts I would be very much in favor. It is very helpful for metaprogramming (Make, CMake, and Autotools also easily support this).

To not have build scripts in SPM seems like a huge oversight to me.

I think we have to distinguish two features here:

  • running custom code as part of the build process. We have a draft proposal for this here: Package Manager Extensible Build Tools
  • specifying tasks which can be run manually outside of the build process, this is what @orta is proposing here.

These might seem similar, but I think we'd intentionally want to separate them in SwiftPM. We want to be much more strict about what happens during the build process due to the reasons outlined in the extensible build tools draft proposal, but tasks which are run manually do not need to meet the same high bar.

I would agree. I didn't know about the unnumbered, WIP, Package Manager Extensible Build Tools until now.

Coming from Node JS you could define run-script commands in package.json:

"scripts": {
    "start": "node server.js",
    "seed-db": "node ./scripts/seedDatabase.js 
}

And then simply run them from the command-line:

npm run-script seed-db

This pattern made testing and deployment easier as in this example I can call npm run-script seed-db to seed the database and then call the npm start to start up the server.

In swift there is a single start point in main.swift, so a work around where you could define the start point for each script would need to be sorted out.

All in all, I think this is a major feature that would benefit SwiftPM in one form or another.

Quick check here: How is this a win over writing Utilities/docs.sh, Utilities/format.sh, Utilities/lint.sh, Utilities/ci.sh, and Utilities/deploy.sh?

Shell scripts (or Swift scripts!) in a folder are way more flexible in terms of chaining commands together and implementing custom glue logic, they already have ways to handle problems like escaping and command-line arguments, and the package manager already supports them today (by not caring that the Utilities folder exists). What does having a scripts: section in the package description offer us?

I just gave a talk at swiftcloudwork.shop about building command line tools! There's an existing tool out there called Beak, which makes building task-runners in Swift super simple by leveraging SourceKit. It might be a good fit for what you're trying to do. GitHub - yonaskolb/Beak: A command line interface for your Swift scripts

My talk included lots of links to other open-source tools, too. If you're interested, here are the slides: The Command Line Toolbox - Michael Bates

2 Likes

I think the biggest thing it gives us is a unified development experience for Swift. One of the main strengths (and one could argue, weaknesses) of node.js is its package ecosystem and the defacto standard of npm and the unified experience it gives you. Most node projects I've worked on use npm as the primary tool used to interface with the project and its tasks.

It's this standardization around the "package manager" as the default tool for working with a Swift project that I think would be the value. Now like I mentioned upthread, I think there is room to explore what a unified development experience for Swift would look like. (Keep expanding the scope of the manifest, add a new kind of tool with a corresponding Tasks.swift, etc)

4 Likes

I wanted to add that the beauty of the Swift Package run scripts would be Xcode integration. Instead of schemes, we can have a drop down menu of run scripts.

1 Like