[Pitch] Improving package creation with custom templates: SwiftPM Template Initialization

Introduction

Currently, SwiftPM supports a handful of hardcoded templates that act as a starting point for user projects. These built-in templates, accessible via swift package init, allow users to initialize basic packages such as libraries, tools, executables, and macros. More complex package initialization is achieved through third-party command-line tools or custom scripts, many of which operate outside the SwiftPM ecosystem. This disconnect increases the complexity and makes it harder for users to discover, share, or customize initialization workflows.

To address this gap, this proposal introduces a system that allows Swift packages to declare reusable templates with custom logic for generating a user’s package code. These templates can be distributed like any other Swift package and executed in a standardized, first-class manner via a new SwiftPM command. Template authors would also have access to tools to improve development experience and maintainability over time.

Motivation

While swift package init remains valuable for initializing simple packages, many Swift developers face more complex requirements when starting a new project, whether that may be initializing an HTTP server, or creating a package that implements an OpenAPI specification.

As a solution, this proposal introduces an extended version of swift package init. This extended command allows developers to easily get started with a new package based on a specific use case. Furthermore, swift package init supports invoking custom, user-defined templates that are:

  • Fully integrated with SwiftPM
  • Customizable through arguments and logic
  • Shareable just like standard Swift packages
  • Flexible enough for a wide range of use-cases

This enhancement will empower developers to create and reuse templates for specific use-cases, all while ensuring consistency and good practices.

Proposal

Package Generation from template

Swift Package Manager, along with the improved swift package init command, enables users to easily generate a package based on predefined templates. The init command currently allows users to select a template bundled within SwiftPM via the --type option. However, this functionality will now be extended to support selecting a template from an external package located either on local disk, in a git repository, or in a package registry, by specifying the template’s location using the --path, --url, or --package-id options along with any necessary versioning requirements. SwiftPM will then generate the package based on the selected template by performing the following steps:

  1. Creates a temporary working directory
  2. Initializes the appropriate base package
  3. Prompts the user for any required configurations
  4. Generates the final package structure in the temporary directory
  5. Copies the fully-initialized package into the user’s specified destination directory

Below is an example of generating a package from a template located in a registry.

% swift package init --type PartsService --package-id author.template-example
...
Build of product 'PartsService' complete! (7.15s)

Add a starting database  migration routine: [y/N] y

Add a README.md file with an introduction and tour of the code: [y/N] y

Choose from the following:

• Name: include-database
  About: Add full database support to your package.
• Name: exclude-database
  About: Create the package without database integration
include-database

Pick a database system for part storage and retrieval. [sqlite3, postgresql] (default: sqlite3):
sqlite3

Building for debugging...
[1/1] Write swift-version.txt
Build of product 'PartsService' complete! (0.42s)

After the project is generated, the user is left with a scaffolded package and is ready to start hacking.

.
├── Package.swift
├── Scripts
│   └── create-db.sh
├── README.md
├── Sources
│   ├── Models
│   │   └── Part.swift
│   │  
│   └── App
│       └── main.swift
│ 
└── Tests
    └── PartsServiceTests
        └── PartsServiceTests.swift

Below is the new output of swift package init --help:

% swift package init --help
OVERVIEW: Initialize a new package.

USAGE: swift package init [<options>] [<args> ...]

ARGUMENTS:
  <args>                  Template arguments to auto-fill prompts and skip input.

OPTIONS:
  --type <type>           Specifies the package type or template.
        Valid values include:

        library           - A package with a library.
        executable        - A package with an executable.
        tool              - A package with an executable that uses
                            Swift Argument Parser. Use this template if you
                            plan to have a rich set of command-line arguments.
        build-tool-plugin - A package that vends a build tool plugin.
        command-plugin    - A package that vends a command plugin.
        macro             - A package that vends a macro.
        empty             - An empty package with a Package.swift manifest.
        <custom>          - When used with --path, --url, or --package-id,
                            this resolves to a template from the specified 
                            package or location.
  --name <name>           Provide custom package name.
  --path <path>           Path to the package containing a template.
  --url <url>             The git URL of the package containing a template.
  --package-id <package-id>
                          The package identifier of the package containing a template.
  --exact <exact>         The exact package version to depend on.
  --revision <revision>   The specific package revision to depend on.
  --branch <branch>       The branch of the package to depend on.
  --from <from>           The package version to depend on (up to the next major version).
  --up-to-next-minor-from <up-to-next-minor-from>
                          The package version to depend on (up to the next minor version).
  --to <to>               Specify upper bound on the package version range (exclusive).
  --build-package         Run 'swift build' after package generation to validate the template.
  --version               Show the version.
  -h, -help, --help       Show help information.

To encourage community reuse and sharing, templates are intended to be distributed using the same mechanism as Swift packages themselves. This ensures that developers can rely on familiar distribution workflows:

  • Git repositories (public or private)
  • Swift package Registry entries
  • Local paths for internal or enterprise use

SwiftPM will resolve these sources similarly to how it handles regular dependencies, caching them for offline use and version resolution.

This change opens the door to an ecosystem of templates maintained by framework authors as well as enterprise-internal scaffolds tailored to a company’s needs.

Security & Trust

Given that templates may execute arbitrary Swift code, a design decision was made to execute templates via SwiftPM command-line plugins, a secure and permission-aware mechanism already established in the Swift ecosystem. By leveraging this infrastructure, template authors can include Swift-based logic while SwiftPM maintains clear security boundaries.

SwiftPM plugins run in a sandboxed process separate from the SwiftPM itself, enforcing the principle of least privilege. As such, they operate with limited permissions and reduce the risk of malicious code affecting critical system components. Furthermore, each plugin is paired with a singular template, allowing developers to have clear visibility into the sources of their respective templates, reducing the risk of hidden malicious logic. Finally, they have a scoped execution environment i.e. they cannot arbitrarily modify files outside of allowed directories, reducing the risk of corruption or backdoor insertion.

IDE Integrations

To further enhance the usability and accessibility of custom templates, the implementation and API surface of templates should enable consumption and integration with IDEs, supporting package initialization with templates through graphical and automated workflows. These integrations would provide intuitive interfaces for discovering, configuring, and initializing packages from templates, making the feature more approachable for a broader audience.

Authoring a template

Maintaining templates over time, especially those with big decision trees, can be challenging for authors. Ensuring that a template behaves as expected across different configurations is crucial to its reliability and long-term usability. As such, the template-authoring experience centers around two facets:

  1. Flexibility
  2. Testability

Template authors should have the flexibility to design their templates however they prefer, whether that preference may be using templating engines, string interpolation or another approach. It should be up to the template author to choose the style that brings them comfort and familiarity.

At the same time this flexibility requires a reliable way for templates to communicate with SwiftPM and vice-versa. In order to allow this communication, template authors must fulfill the following requirements.

Information required by the template must be defined through swift-argument-parser

At its very essence, a template is an executable that is run via command-line plugin. As such, its arguments are declared via swift-argument-parser syntax:

 @Flag(help: "Add a README.md file with an introduction and tour of the code")
 var readme: Bool = false

 @Option(help: "Pick a database system for part storage and retrieval.")
 var database: Database = .sqlite3
 
 @Flag(help: "Add a starting database migration routine.")
 var migration: Bool = false

The reasoning by this decision is to leverage Swift’s powerful JSON parsing capabilities alongside --experimental-dump-help, which generates a JSON output of available arguments, in order to prompt the consumer for their choices.

However, a template is not simply a list of options, flags, and arguments. It is a branching decision tree, where specific choices influence which options are available next. To model these branching paths, authors can organize them into subcommands, which help users navigate complex template structures more naturally.

@main
struct ServerGenerator: ParsableCommand {
    public static let configuration = CommandConfiguration(
        commandName: "server-generator",
        abstract: "This template gets you started with starting to experiment with servers in swift.",
        subcommands: [
            CRUD.self,
            Bare.self
        ],
    )

    @Flag(help: "Add a README.md file with an introduction and tour of the code")
    var readme: Bool = false
 
    mutating func run() throws {
         ...   
    }
}

struct Bare: ParsableCommand {
    static let configuration = CommandConfiguration(
        commandName: "bare",
        abstract: "Generate a bare server"
    )

    @OptionGroup
    var serverOptions: SharedOptionsServers

    func run() throws {
        try? FileManager.default.removeItem(atPath: packageDir / "Package.swift")
        try packageSwift(serverType: .bare).write(toFile: packageDir / "Package.swift")
    }
}

public struct CRUD: ParsableCommand {
    public static let configuration = CommandConfiguration(
        commandName: "crud",
        abstract: "Generate CRUD server",
    )

    @Option(help: "Set the logging level.")
    var logLevel: LogLevel = .debug

    @OptionGroup
    var serverOptions: SharedOptionsServers
    
    public func run() throws {
        try? FileManager.default.removeItem(atPath: packageDir / "Package.swift")
        try packageSwift(serverType: .bare).write(toFile: packageDir / "Package.swift")
    }
}

The example code provided demonstrates how a template with multiple decision branches might be structured to generate files, but the actual approach to templating, including how source code is generated, organized, or customized, is entirely up to the template author. Swift Package Manager simply provides the mechanism to invoke templates; the template author defines the logic, content, and structure of the generated code according to their needs.

Below is what the consumer might see when initializing a project based on the template above:

% swift package init --type ServerTemplate --path <path/to/template>
...
Choose from the following:

• Name: crud
  About: Generate CRUD server

• Name: bare
  About: Generate a bare server
  
Type the name of the option:
crud

Set the logging level. [trace, debug, info, notice, warning, error, critical] (default: debug):
notice

Building for debugging...
[1/1] Write swift-version.txt
Build of product 'ServerTemplate' complete! (0.42s)

Template Target and product

This pitch introduces a new kind of target and product in SwiftPM: .template. This new syntax abstracts the declaration of templates within a package and allows authors to declare their presence within the Package.swift file.

Here is an example of a Swift package that defines a simple template:

let package = Package(
    name: "TemplateExample",
    products: .template(name: "Template1")
    dependencies: [
        .package(url: "https://github.com/apple/swift-argument-parser", from "1.3.0")
        .package(url: "https://github.com/swift-server/async-http-client.git", from: "1.9.0")
    ],
    targets: .template(
            name: "Template1",
            dependencies: [
                .product(name: "ArgumentParser", package: "swift-argument-parser"),
                .product(name: "AsyncHTTPClient", package: "async-http-client")
            ],
            initialPackageType: .executable,
            description: "A simple template that requires network access",
            permissions: [
                .allowNetworkConnections(scope: .none, reason: "Need network access to help generate a template"
            ]
        )
   )

The .template() API allows for full control over a template’s dependencies, permissions, and descriptive metadata. Dependencies declared in the dependencies array are made available only to the generated executable target, not the plugin. This separation ensures that plugin code used to instantiate the template remains isolated, improving encapsulation and security.

The initialPackageType parameter corresponds to the current set of types of packages available for initialization. Each template type corresponds to a predefined package configuration that is used by the template executable as a base for scaffolding a new Swift package.

public enum initialType {
    case library
    case executable
    case tool
    case buildToolPlugin
    case commandPlugin
    case `macro`
    case empty
}

Finally, the permissions parameter is primarily focused on specifying security requirements and access scopes needed by the package. Whenever a permission is required, the template will prompt the user just as plugins do. This process helps ensure that users understand why certain permissions are required when executing an author’s template, ensuring transparency and establishing trust between users and authors.

Author’s Package structure

To maintain organization and consistency within an author’s package, unless specified differently by the author, all template-related code must be placed under the /Templates directory. Below is an example of an author’s package directory.

.
├── Package.swift
│
├── Templates
│   └── Template1
│       └── Template1.swift
├── Plugins
│   └── Template1Plugin
│       └── Template1Plugin.swift
│
└── Tests
    └── FooTests
        └── FooTests.swift

As every template is an executable and a command-line plugin, templates must have an entry point along with arguments, options and flags to represent required configurations needed from the consumer. Afterwards, the author is free to write their template however they want.

Below is an example of a template based on Swift’s string interpolation that generates the basic structure for a simple Swift project, including a main.swift file, and optionally a README.md file.

import ArgumentParser
import Foundation
import SystemPackage

extension FilePath {
    static func / (left: FilePath, right: String) -> FilePath {
        left.appending(right)
    }
}

extension String {
    func write(toFile: FilePath) throws {
        try self.write(toFile: toFile.string, atomically: true, encoding: .utf8)
    }
}

//basic structure of a template that uses string interpolation
@main
struct HelloTemplateTool: ParsableCommand {
    
    //swift argument parser needed to expose arguments to template generator
    @Option(help: "The name of your app")
    var name: String

    @Flag(help: "Include a README?")
    var includeReadme: Bool = false

    //entrypoint of the template executable, that generates just a main.swift and a readme.md
    func run() throws {
        let fs = FileManager.default

        let rootDir = FilePath(fs.currentDirectoryPath)

        let mainFile = rootDir / "Sources" / name / "main.swift"

        try fs.createDirectory(atPath: mainFile.removingLastComponent().string, withIntermediateDirectories: true)

        try """
            // This is the entry point to your command-line app
            print("Hello, \(name)!")

            """.write(toFile: mainFile)

        if includeReadme {
            try """
                # \(name)
                This is a new Swift app!
                """.write(toFile: rootDir / "README.md")
        }

        print("Project generated at \(rootDir)")
    }
}

Testing

Templates, like any other shippable software, should be testable. The flexibility of creating a template allows authors to write various types of tests, including unit tests, and integration tests. Authors should strive to deliver high-quality templates while minimizing unnecessary risk, complexity, and rework. Therefore, authors are strongly encouraged to write focused tests that ensure reliability and maintainability.

Below is an example of a simple unit test, verifying whether a file generation function correctly interpolates configuration values into an output file and reflects logging-related settings in the generated code.:

import Testing
import Foundation
@testable import ServerTemplate

struct CrudServerFilesTests {
    @Test
    func testGenTelemetryFileContainsLoggingConfig() {
        
        let generated = CrudServerFiles.genTelemetryFile(
            logLevel: .info,
            logFormat: .json,
            logBufferSize: 2048
        )

        #expect(generated.contains("let logBufferSize: Int = 2048"))
        #expect(generated.contains("Logger.Level.info"))
        #expect(generated.contains("LogFormat.json"))
    }
}

To support end-to-end testing of templates, Swift Package Manager introduces a new subcommand: swift test template. This subcommand allows authors to verify if a given template can successfully generate a package and that a generated package builds correctly. By specifying both a template and an output directory, SwiftPM performs the following actions:

  1. Reads the package, locates the template, and extracts all available options, flags, arguments, and subcommands,
  2. Prompts for all required inputs (options, flags, and arguments).
  3. Generates each possible path in the decision tree, from the root command to the leaf subcommands, using the user’s given inputs.
  4. Validates that each variant is created successfully and builds without error.
  5. Logs any errors to a file located within the generated package directory.
% swift test template --template-name ServerTemplate --output-path <output/directory/path>
...
Build of product 'ServerTemplate' complete! (3.37s)

Set logging buffer size (in bytes). (default: 1024):
2048

Server Port (default: 8080):
80

Set the logging format. [json, keyValue] (default: json):
keyValue

Set the logging level. [trace, debug, info, notice, warning, error, critical] (default: debug):
critical

Add a README.md file with an introduction to the server + configuration?: [y/N] y

Generating server-generator-crud

Generating server-generator-bare

Argument Branch           Gen Success Gen Time(s) Build Success Build Time(s) Log File
server-generator-crud-mtls    true      11.17         true         92.09         -
server-generator-crud-no-mtls true      11.53         true         91.65         -

Each variant of a template is written to a subdirectory within the specified output path, with directory names assembled as <command>-<subcommand>-... to reflect the structure of the decision tree path. This layout not only aids in testing but also enables author review of the generated output of each variant, further increasing confidence in the correctness and reliability of the template. This end-to-end test support ensures an author’s template performs correctly across all decision branches, increasing confidence in its reliability.

Impact on Existing packages

This is an additive feature. Existing packages will not be affected by this change. It adds the ability to define and invoke templates as part of a Swift Package.

Alternatives

None.

Future

We have several plans for Swift templates, including:

  • Enabling the ability to also enhance a package using a selected template
  • Extending Swift registries to support template metadata, making templates more searchable and discoverable
  • Providing an API to simplify modifications to Package.swift, allowing users to add dependencies, targets, target-dependencies and products without rewriting the file from scratch.

However, we want to also gauge community feedback before fully fleshing out the future of this new feature.

17 Likes

Conceptually, I love where this is going:

  • :white_check_mark: template based structure
  • :white_check_mark: testable
  • :white_check_mark: projects and packages can provide their own templates to customize more

I'd love to understand a bit more detail about what the templating enables, at least at an initial cut of this. Is it a single static setup that you provide arguments into, or, as the demo you included, does it have interactive choice points that some CLI oriented user interface provides, asking for input to configure and adopt what you want?

The line:

  • Customizable through arguments and logic

This makes me think I could provide my own logic to provide arbitrary question and answer flows, possibly validate the inputs, and so on.

In the whole process to establishing a template. If that's the case - where and how would I add this logic, and what base-line options would exist with an initial functional prototype of this? Is there a protocol I'd conform to, or an example of what this might look like, if you're not that far down this path? Would this be a new Swift Package plugin type perhaps, or extend one of the existing plugin types (a variation on the command plugin seems possible)?

In particular, I'd love to better understand what authoring control I could have when establishing a new template, getting feedback through questions, and choosing further options, or potentially exiting out if the combination requested was invalid.

Will the existing --type options remain and be amended by additional templates available using --path, --url, or --package-id?

Other questions:

  • Are you anticipating that we'll have another repository somewhere in the Swift project for extended templates that are available by default? Is that being considered in future directions? I'm asking because I'd like to make sure if we enable something like this, we have the pieces in place, or at last a directional idea, of how to make additional templates easily discoverable for newcomers to the Swift project, or people getting invested within specific Swift use case scenarios (server side code, embedded, windows, wasm, etc).

(Assuming that this runs forward into reality, I'd highly recommend working with the kind folks at Swift Package Index to support highlighting that a package includes a template there)

And one potential concern:

  • For the reliance on the Swift Argument Parser, I'll note that you're setting a dependency on a package that isn't part of the Swiftlang core libraries (although I'd love it to be!), but more importantly, last I heard the output format of --experimental-dump-help was intentionally not being locked down and guaranteed, which may leave this pitch's reliance on it a bit awkward.
2 Likes

This is a very exciting feature!

I don't entirely like the idea of introducing new target/product types, did you explore adding some kind of generic payload to the existing plugin type to model the new attributes?

How does the build model work for the templates? Are they built as regular targets or plugins? I see that the example has dependencies, that suggests the former, but a lot of prose suggests the later.

The Swift Argument Parser is super popular with Swift and has this powerful feature with the ability to emit a complete JSON representation of the command-line structure. It's also showcased in the existing out-of-the-box templates for a few years. It feels like a really good fit to help drive the decision tree protocol for the template engine.

The experimental dump help has been experimental for a few years now, and changes appear to have been mostly additive, compatible ones so far. In the near term, this affords a level of stability. In the longer term, this might be an argument for releasing a stable version of the feature (ie. --dump-help) to establish a first version of it, or to look towards other emerging command-line tool schemas, such as OpenCLI.

1 Like

The examples in the proposal are showcasing new functions in the PackageDescription, but the underlying model is still producing command plugins and executables, both targets and products for them. There's very minimal changes to the underlying types, only enough to capture the new template metadata and no impact to the build graph, so it can work independently of the build system.

Is it a single static setup that you provide arguments into, or, as the demo you included, does it have interactive choice points that some CLI oriented user interface provides, asking for input to configure and adopt what you want?

Firstly, the template system does support dynamic interactive flows, not just static scaffolding. Templates are implemented using SAP, thus allowing for branching logic using subcommands (such as the server template example), where certain options can influence subsequent prompts. Furthermore, you can validate user inputs inside the run() methods.

So yes, templates can support arbitrary logic and control flow.

In the whole process to establishing a template. If that's the case - where and how would I add this logic, and what base-line options would exist with an initial functional prototype of this? Is there a protocol I'd conform to, or an example of what this might look like, if you're not that far down this path? Would this be a new Swift Package plugin type perhaps, or extend one of the existing plugin types (a variation on the command plugin seems possible)?

There is no protocol to conform to beyond the existing capabilities of the Swift argument parser. Templates are simply SwIftPM command-line plugins with executable entry points. The plugin will feed the executable the user’s cli inputs, while the executable validates inputs (if needed), run custom file generation, and can conditionally generate files or directories. So, author control is near-total, you just need to use Argument Parser.

There is no new plugin type, but the .template product and target declarations mark the executable as a template and exposes it via swift package init.

As of now, I want to try as hard as possible to avoid making any changes to the package data structure.

However, with this new system, there comes the possibility of templates making templates too. So, in theory, we can have templates that can be used to help authors get started writing their templates.

In particular, I'd love to better understand what authoring control I could have when establishing a new template, getting feedback through questions, and choosing further options, or potentially exiting out if the combination requested was invalid.

Yes. Since your template logic is just regular swift code, you can:

  • validate combinations of inputs in run()
  • Print errors and throw errors
  • customize flow with conditionals and guard clauses

Templates have full control over execution logic, including quitting early if inputs are incompatible.

Will the existing --type options remain and be amended by additional templates available using --path, --url, or --package-id?

Yes, the proposal retains the existing —type values and extends it to support —path, —url, and —package-id. The —type flag is just now a multipurpose flag for initializing a package based on existing built-in + custom templates.

1 Like

This is an exciting feature! I appreciate having more flexibility around SwiftPM's templates.

One question/comment I had is around the proposed swift test template command:

It sounds like this subcommand will work by attempting to instantiate and build every permutation of option the template supports—is that correct? I'm a little worried that might not scale well, if for example there are many potential options or a large number of choices for a single option. I'm also unsure whether only attempting to build the generated template variations is sufficient to have confidence that they are correct.

I do like the idea of allowing SwiftPM to automatically gather all the potential template options; that helps a template author know which options they might need to consider writing tests to validate. (In that sense, it could serve a role somewhat similar to code coverage tools.) But it seems like you might not need to actually instantiate every template permutation, and (more importantly, perhaps) you may want to have test code which inspects the instantiated templates and confirms their contents are correct.

I think it could be even more helpful if the subcommand could begin by gathering the options of a template (as currently proposed) but then generate test file templates containing tests which communicate back to SwiftPM and instantiate the template with varying options. Then, the template author could add their own validation logic into those generated tests to confirm each template file contains content they expect. To accomplish this might require SwiftPM to offer some API which could be called from within a test target to say "Please instantiate template […] with the options […] and return the generated path".

It sounds like this subcommand will work by attempting to instantiate and build every permutation of option the template supports—is that correct?

Your understanding is somewhat correct. The swift test template command will attempt to:

  1. Extract all the template variants at the subcommand level
  2. Enumerate all paths through the decision tree
  3. Instantiate each variant
  4. build each generated package
  5. log results.

So, we try to mitigate the combinatorial explosion by not creating variants of templates based on different types of options, flags, or arguments, but rather based on subcommands (arguments that will have an impact on the template’s decision tree). Any arguments that are additive or do not impact the decision tree will be prompted for. However, you are correct that it does not validate the content correctness, and that there would still be a threat of combinatorial explosion (even if we try our best to mitigate it).

However, to deal with the combinatorial explosion even more, I am planning on introducing a filter option to be able to allow users to focus on specific branches of the decision tree. For example, say we have the following decision tree:

server-generator
│
├── crud
│   ├── include-database
│   └── exclude-database
└── bare

where the swift test template subcommand tells us that the branches server-generator-crud-include-database and server-generator-crud-exclude-database failed, we would be able to filter out testing the server-generator-bare branch by simply writing something along the lines of swift test template –filter server-generator-crud to only test that specific part of the decision tree.

I think it could be even more helpful if the subcommand could begin by gathering the options of a template (as currently proposed) but then generate test file templates containing tests which communicate back to SwiftPM and instantiate the template with varying options. Then, the template author could add their own validation logic into those generated tests to confirm each template file contains content they expect. To accomplish this might require SwiftPM to offer some API which could be called from within a test target to say "Please instantiate template […] with the options […] and return the generated path".

I actually am trying to do something similar where, users can precise a –dry-run flag, where instead of running the swift test template command, they can get as output the command line invocation of the template. However, I would like to discuss with you further this idea as I think this can really drive forward testing and greatly improve validating templates.

3 Likes

Overall I am very excited for this feature, especially, the options of multi-step templates. This is something that I wanted in the server space for a long time where it is common for project generators to ask questions like “What database?” or “What cache?”. I’m happy to see this problem being tackled in the initial version.

I share the same concern as @Joseph_Heck does around the reliance on SwiftArgumentParser and in particular the experimental feature. Concretely, I have those concerns:

  1. SwiftArgumentParser is not part of the Swift project at this time. The Ecosystem Steering Group has discussed sponsoring the inclusion of it but there are more details to work out here.
  2. While the --experimental-dump-help has been around for a long time it doesn’t change the fact that it is in fact experimental. The authors of the package can change it any time.
  3. Even if the dump command became stable I still have concerns of essentially tying SwiftPM and SwiftArgumentParser together. SwiftPM has entirely different API stability guarantees than SwiftArgumentParser which is in theory free to create a new major completely changing its APIs. What is SwiftPM doing then?

While I think we can solve the first and second concern, I personally would like us to explore if we can only loosely couple the proposed feature to SwiftArgumentParser. In particular, I was thinking if something like this could work:

  1. We define a new TemplateSupport library in SwiftPM that defines types that express express command options, arguments and flags
  2. We concretely define an invocation of the template executable that must generate the SwiftPM expected JSON using the types defined in 1
  3. SwiftArgumentParser adds a new if canImport(TemplateSupport)part where it adds the command defined in 2 and maps its configuration to the SwiftPM types

This should allow users to implement generators without using SwiftArgumentParser but still allows everyone to use it if they want to. The coupling in the end is just the expected JSON structure and SwiftPM is providing a Swift type representation of it as convenience.

1 Like

Thank you for your insights @FranzBusch.

SwiftArgumentParser is not part of the Swift project at this time. The Ecosystem Steering Group has discussed sponsoring the inclusion of it but there are more details to work out here.

It’s great to see that SwiftArgumentParser is moving closer to joining SwiftPM. This means the two can work together more easily in the future.

While the --experimental-dump-help has been around for a long time it doesn’t change the fact that it is in fact experimental. The authors of the package can change it any time.

It is important to consider that not all changes made to the —experimental-dump-help command may result in breaking changes. Since it is a JSON output, widely used techniques in REST APIs can be employed, including version-less approaches to enhance cross-revision compatibility.

Given that this feature is integral to the Swift ecosystem, its long-standing availability and the presence of numerous dependencies, it will be prudent to evaluate the potential impact of making breaking changes, such as removals or type narrowing, to the JSON output generated by —experimental-dump-help. If such changes are necessary and compatibility must be compromised, it is important to engage in a thorough discussion regarding their implications, including the utilization by SwiftPM package templates.

Even if the dump command became stable I still have concerns of essentially tying SwiftPM and SwiftArgumentParser together. SwiftPM has entirely different API stability guarantees than SwiftArgumentParser which is in theory free to create a new major completely changing its APIs. What is SwiftPM doing then?

While this pitch attempts to illustrate the template author’s experience in a manner familiar to many Swift developers, it is crucial to emphasize that there is no strict dependency on SwiftArgumentParser. The template generator must be capable of responding to well-known flags and generating a JSON describing its complete argument structure. This JSON serves as a decision tree that can be transmitted, stored and utilized to encode a user’s decisions as an invocation to the corresponding CLT.

The objective is to adopt the most comprehensive standard possible to support this functionality. Although there is currently no standard that spans multiple languages and frameworks (e.g., OpenCLI may eventually achieve this), the Swift argument parser is widely recognized within this programming community. It frequently becomes the preferred choice for more than just SwiftPM and its templates across various versions. There are use cases, such as ad-hoc testing, automation, and (IDE) integration, that can benefit from standardization.

Regarding version compatibility, REST is exemplary because it supports version decoupling through the flexible JSON decoding described earlier. If a new version is warranted due to a fundamental breaking change, clients can choose to use a specific version, typically indicated through a header or URL prefix (e.g., /api/v2/…). However, unless a service guarantees that no one will continue using the old version, the old endpoint remains accessible to clients.

In the context of content negotiation with CLTs, an analog could be established by employing a command-line flag to convey the client’s expectations. For instance, the SwiftArgumentParser currently uses —experimental-dump-help for this purpose. Should a breaking change be implemented, —dump-help could be added to accommodate the new content. In the future, it might introduce —dump-help-v2 or potentially a standardized approach across programming languages and argument parsing frameworks. To maintain compatibility with existing clients, including those that have adopted experimental methods, the old flags should remain available.

We define a new TemplateSupport library in SwiftPM that defines types that express express command options, arguments and flags

Thank you for your suggestion. An approach similar to this was considered at one point. I have some reservations.

  1. This strategy positions SwiftPM in the domain of command-line descriptions, which deviates from its primary focus. There are specialized components that excel in this area.

  2. The approach is specific to templates and SwiftPM. Ideally, a template generator should be a CLT that can be invoked in the anticipated ways to enable alternative use cases and tooling. It should utilize the most comprehensive standards to achieve this.

  3. Similar compatibility mismatches as previously described persist. However, Swift types restrict flexibility compared to a wire format like JSON.

  4. I am uncertain whether an optional dependency from Swift Argument Parser on SwiftPM is optimal from an architectural standpoint.

As a co-maintainer of ArgumentParser, 4 is almost certainly a non starter for us.

The dump help format is versioned and has tests to ensure it doesn't break.

2 Likes

I really like this new direction. Probably not alone here: rolled my own template system to quickly setup new Swift projects. So this really serves a need.

And I :heart: that you create a testable solution.

Two questions:

  1. Do you have an example of a template available, so we can see how the templating syntax would work?
  2. what is your idea about additional steps that can be performed, like creating (remote) Git repo’s, bootstrapping CI/CD pipelines or initialising a database? Do you feel this could be in scope of this project?

I think this is a great start to expanding the capabilities of SwiftPM and making it easy to generate different types of projects.

For some prior art, have a look at the Vapor Toolbox which does exactly this - build SwiftPM projects from templates. Happy to talk through the experiences we’ve had of using that over the years, and different edge cases we’ve had to solve.

One thing that has sprung to mind when reading this was the test command - what platforms is it going to test on? Because if I’m building a server template locally on macOS I want to be able to test it on Linux to make sure it works. And how are you going to test it when relying on external services? Compilation is only half of the problem, and unless you execute the unit tests it’s easy to ship broken templates that compile but don't actually work (ask us how we know), which can lead to a frustrating user experience.

4 Likes

I have been thinking about your reply and the general problem of tying the ArgumentParser and SwiftPM together more. I agree with @rauhul that option 4 is not feasible for two reasons:

  1. Swift PM doesn’t offer a semantically versioned API
  2. Swift PM has a dependency on ArgumentParser already for the various CLI tools it offers

I think your second reservation here is essentially what I’m driving at. A template generator in the end is just a binary invoked in a specific way from SwiftPM. So let’s specify this contract. In particular, let’s specify:

  1. That the CLT needs to have a --dump-help command that outputs JSON
  2. The JSON format expected from 1
  3. The way the arguments, options and flags defined in the JSON are passed to the CLT

This provides us with a specification for template generators, it allows us to point users towards ArgumentParser since it matches the above, but importantly it also allows other ways to implement such template generators without ArgumentParser. While the last part is possible with the current state of the pitch, it isn’t spelled out anywhere what the expectations of such a template generator are.

4 Likes

Hi maartene!

Do you have an example of a template available, so we can see how the templating syntax would work?

There are code snippets within the pitch for how the syntax of a template looks. If you are looking for a package that contains the entire picture, you will have to wait, as I want to publish a couple starter templates alongside the proposal for this project.

what is your idea about additional steps that can be performed, like creating (remote) Git repo’s, bootstrapping CI/CD pipelines or initialising a database? Do you feel this could be in scope of this project?

Personally, I believe that SwiftPM should not be in charge of handling this. Luckily, since authors are free to write templates however they want, nothing is stopping them from coding workflows for creating git repos, or bootstrapping CI/CD pipelines. In fact, in the proposal, there is an example of a user getting prompted about choosing to initialize a database (which was taken from a real example of a template that has the ability to set up a database):

% swift package init --type PartsService --package-id author.template-example
...
Build of product 'PartsService' complete! (7.15s)

Add a starting database  migration routine: [y/N] y

Add a README.md file with an introduction and tour of the code: [y/N] y

Choose from the following:

• Name: include-database
  About: Add full database support to your package.
• Name: exclude-database
  About: Create the package without database integration
include-database

Pick a database system for part storage and retrieval. [sqlite3, postgresql] (default: sqlite3):
sqlite3

Building for debugging...
[1/1] Write swift-version.txt
Build of product 'PartsService' complete! (0.42s)

1 Like

I would really love to have a talk about not just checking if templates are capable of generating projects that build but also projects that are not broken i.e. projects without missing or incorrect files, failing runtime behaviours, etc. Testability was one of the key things I wanted to focus on when introducing swift templates, as I feel that often times, template authors either have a hard time testing, or users receive templates that just do not work. I would love to set up some time to chat further regarding ways we can try and tackle this issue.

1 Like

Feel free to DM me and we can work out a time to have a call/chat