Pitch [stdlib]: Command Line Argument Parsing

Hi Everyone,

I’d like to get general consensus on whether or not the community believes a command line argument parser should be available in the standard library. If we gain consensus that we should move forward I will start working on the implementation and API details for the Swift Evolution proposal.

Command Line Argument Parsing

  • Proposal: SE-NNNN

  • Authors: Rahul Malik

  • Review Manager: TBD

  • Status: Awaiting implementation

Introduction

Swift is being embraced for building command line applications and scripting. The user experience of these types of applications expose configuration through the use of flags and subcommands. Currently to extract this configuration, a developer must either write their own solution that consumes CommandLine.arguments or rely on a third-party dependency which must be integrated into the project.

We have many first-party and third-party implementations in the Swift community of argument parsing to support these configurations and this pitch is advocating for a default implementation in the Swift standard library since this functionality has become essential to build these types of applications and is applicable for any application invoked from a terminal.

There is a precedent for argument parsing logic in the standard library of other languages:

  • Go’s flag package implements command-line flag parsing.
  • Python exposes the argparse library which provides much of the necessary functionality around argument parsing, displaying errors and deriving help documentation.
  • OCaml’s Arg module provides a general mechanism for extracting options and arguments from the command line.

Previous discussions on the subject:

Motivation

There are a lot of things that can make the experience of building command line applications better in Swift and I believe an argument parser in the standard library is a good start. A builtin argument parser will make scripting and command line applications easier to write, improve portability and user experience, and lastly make these use-cases first-class citizens for Swift development.

Luckily the Swift community has produced many different solutions to this problem which I believe we can use as inspiration to design a great solution. Below is a (very incomplete) list of popular argument parsing libraries:

Our design would provide an API to satisfy these requirements:

  • Declaring / Parsing / Validating flags
  • Declaring / Parsing / Validating subcommands
  • Generate “help” / “usage” documentation

Nice to have:

  • Shell completion utilities
  • External configuration for declaring command line interface (YAML, JSON, etc)

Proposed solution

TBD

Detailed design

TBD

Source compatibility

This proposal does not impact source compatibility.

Effect on ABI stability

This proposal does not impact ABI stability.

Effect on API resilience

N/A

Alternatives considered

TBD

20 Likes

Thank you so much for pitching this! I think having command-line parsing in stdlib will be a great win for the Swift community.

6 Likes

Agreed.

I am pretty happy with SwiftPM's so far as a start, I like the type safety and the testability, but a big pass on increasing usability to get to a standard solution would be awesome.

5 Likes

Thank you for pitching!

I have said this in a number of previous pitches, so I apologise if this is sounding repetitive, but I think the desire to put everything into the standard library is understandable but flawed. When pitching something into the standard library, I think it is critical to explain what advantage is gained by having the standard library carry the burden of this functionality.

The mere fact that many applications need to do command line parsing does not by itself make the case that the standard library should carry a command line parser. After all, many applications do not need to do command line parsing.

So let me ask this question: what advantage is there in the standard library shipping a command line parser instead of leaving this up to third party packages?

18 Likes

Seconded for SwiftPM’s approach. I’ve been using it in pretty much every project that needs to parse command-line args.

2 Likes

I don't think this is something that should live in the standard library.

There is more than one way to design command-line arguments and I don't think it's desirable for us to try to come to consensus around it.

This is a good example of where we should let the package ecosystem do its thing (and it seems to be doing OK so far).

7 Likes

As there's already a CommandLine type, the answer is already yes, the question is how much functionality should go into it. The minimum pragmatic thing to do is add some basic functionality to CommandLine. This could be just enough to remove the same friction that every person who has written a command line tool with Swift has faced.

I don't think this should be framed as a binary choice of standard library or ecosystem. It's reasonable to provide core functionality in the standard library and let the community provide alternatives for more specialized use. From the list above, I think the standard library could provide:

  • Declaring / Parsing / Validating flags
  • Generate “help” / “usage” documentation

And the others could be from the community.

5 Likes

I love the idea. For what it's forth, I've just finished work on an argument parser that could evolve very nicely to use the currently pitched Property Delegate proposal.

The library lets you define commands as types and arguments/options as properties:

import Yaap

class RandomCommand: Command {
    let documentation = "Generates a random integer in a certain interval"
    let maximum = Argument<Int>(documentation: "Maximum value")
    let minimum = Option<Int>(defaultValue: 0, documentation: "Minimum value")

    func run() throws {
        print(Int.random(in: minimum.value..<maximum.value))
    }
}

let tool = Tool(name: "rand", version: "1.0", command: RandomCommand())
tool.run()

If Property Delegates are accepted, arguments and options would become delegates, thus reducing the noise:

import Yaap

class RandomCommand: Command {
    let documentation = "Generates a random integer in a certain interval"

    @Argument(documentation: "Maximum value")
    let maximum: Int

    @Option(documentation: "Minimum value")
    let minimum: Int = 0

    func run() throws {
        print(Int.random(in: minimum.value..<maximum.value))
    }
}

let tool = Tool(name: "rand", version: "1.0", command: RandomCommand())
tool.run()
9 Likes

Awesome, this looks neat!

I echo the sentiments of many that having first-class command line argument parsing is a great idea, but it may not be necessary to include it in the standard library. Inclusion in the standard library means that these APIs are available by default everywhere swift is used. That includes iOS, tvOS, and watchOS applications. None of which would likely ever use command line parsing.

I think this should be in a separate CommandLine library or something along those lines so that it must be explicitly imported. I think this would set a precedence for other things which would be beneficial to include along with the standard library, but that not everyone will need.

6 Likes

I would love to have some built in argument parsing. My biggest questions would be (1) what approach and (2) is this the right time?

I think the timing question is important because these things inevitably become a survey of what's available. One of them becomes the official solution. Has this space settled down enough for us to pick a solution that we want to live with for the long term? (c.f. Guy Steele's ideas in "Growing a Language")

I think it's probably to early to pick a good solution. We're beginning to see Swift be used for command-line apps—especially with the help of SwiftPM. If we waited a year, we'd probably have a lot more collective experience with argument parser to pick a good winner.

There's also the possibility that new language features might drastically improve a potential API (cf @hartbit's post above). Is this the right time in the development of the language to settle on something?

Failing that, I'd like to see an emphasis on a solution that can easily be extended. i.e. if someone continues to experiment in this space, they should be able to leverage what's in the standard library to do so.

1 Like

This resonates with how I've been thinking about this space as well.

There are many "batteries" for software built using Swift to draw upon. There are various places to put those batteries, which includes the Standard Library but also a more federated approach to providing recommended APIs like the process done for the Swift Server Work Group. While shipping an API in the Standard Library makes it truly "batteries included" in Swift, it also has some downsides:

  • It ties updates of those APIs to Swift releases, which only have significant updates about twice a year.

  • It inflates the size of the Standard Library. Sometimes that cost is OK, but it really comes down to the benefits reaped by putting the API in the Standard Library. That's a combination of the density of users of the API, whether or not it is really "fundamental" API, etc.

I would like to find a way for the Swift project to support building up a corpus of APIs like this and making them feel very in reach to users. There are a variety of command line argument parsing libraries in existence already, and so recommending one over the other is something to be carefully considered so not to snuff out innovation.

13 Likes

I don't think we need to wait for perfection.

The downside of standardizing in the Standard Library is it possibly establishes a permanence that is over-inflated and artificial.

Having a good set of command line APIs today would benefit people today. If better affordances turned up in the language later, then new APIs or existing APIs can evolve to take advantage of those. The solutions people are already using can continue to work, and people can take advantage of better (newfangled) solutions when they feel it has benefit to them.

1 Like

I guess this would be yet again an opportunity to talk about a Boost like process/space where we as a community can develop packages that similar to the SSWG get an official blessing at some point after having matured.

2 Likes

I think experimentation and use is key. Having a model that allows people to build software on top of APIs that will potentially evolve I think makes this more fluid and lead to a better result.

1 Like

I don't think so either. But I think any proposal should at least consider what language improvements we expect to see in the next 1-2 years that might affect the ideal design. Then the question becomes whether the design can adopt with the introduction of those improvements or should we wait for the improvements first.

I proposed starting a boost-like library in a separate thread that would include things which are worthy of blessed apple support but are not sure-fire inclusions in the standard library. I definitely feel like command line parsing would be a useful inclusion in this and perhaps a good start.

This is probably the biggest reason to not include this in the standard library. If we build a new library or use an existing one, it will definitely undergo many changes and upgrades as it is used by the entire swift community. Currently all the available command line libraries have maybe a few thousand users at most. That is not representative of the entire community which would end up using an official CommandLine framework.


I think that this should be a completely new design which should build from the experiences and designs of what's already been done. Certain trade-offs and design decisions were made in each of the available libraries based on a single developer's viewpoint (at least in the beginning before they gained adoption/contributors).

Using the community and proposal/discussion approach we can come up with a set of API's that are flexible, extensible, and useful for the community as a whole. Building a batteries-included, but customization allowed framework for working with the command line.

2 Likes

I would love to see more robust and easier access to argument parsing libraries, how to use them, etc - but I have to agree that we shouldn't include these in the standard library.

1 Like

@Aciid, wasn't there discussion a while back about pulling some of SwiftPM's utilities out into a separate "libraries for tool development" repository? There's this repository (https://github.com/apple/swift-tools-support-core) but it's been empty for a year.

I'd love to see that happen instead, and would be willing to assist in such an effort, because it would benefit swift-format among other tools I develop. IMO, we should just take SwiftPM's excellent command line argument binders and move them into a package like that.

2 Likes

We're still interested in doing that so those utilities can at least be shared between the swift.org projects (llbuild, SwiftPM, SwiftSyntax, SourceKit-LSP, etc) but just haven't had the time to make it happen. The major complication there is getting everything built on Swift CI because of the dependency cycle. One pragmatic option is using CMake for bootstrapping but that is also not a trivial task.

1 Like
Terms of Service

Privacy Policy

Cookie Policy