Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to me as the review manager by email or DM. When contacting the review manager directly, please put "SE-0500" in the subject line.
Trying it out
If you'd like to try this proposal out, you can download a toolchain supporting it for macOS.
What goes into a review?
The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:
What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
More information about the Swift evolution process is available at:
+1. I gave the proposal a quick read and it looks great. This seems much more powerful than any configuration script or tools like the Vapor Toolbox. One thing I think would be very important is to offer APIs to be able to insert all the template variables through a GUI, as we do in the Vapor VS Code extension, so that we can add this functionality to the Swift VS Code extension as well.
Massive +1, this does exactly what I would ask from this tool. The Hummingbird Project has been discussing a similar templating tool that seems to fit like a glove with this proposal. I'm very enthusiastic!
A big +1 from me as well. I was tracking the pitch and looking forward to this review. It does add a fair bit of complexity to the package manager interfaces, but the flexibility is well worth it.
During the review, I was concerned about potential reliance on the internal format of the help-dump JSON output from swift-argument-parser, but that's mitigated by including the format specifics in this proposal. I would, however, suggest that this detail (as well as most of the new pieces around the types of targets and modes of interaction) need to be formally included in documentation for Swift Package Manager. I'd like to suggest that if this isn't already an expected part of review, it should be explicitly considered for this feature before this feature is marked as "implemented" in the review dashboard.
My own suggestion would be to leverage the example in the proposal and convert that into a "how to create a package template" article, to be matched with reference content for the expected argument parser content and the related updates to PackageDescription. Given the complexity allows for the flexibility this proposal provides, an example project may even be warranted.
Templates must be declared as products by their authors; otherwise, a manifest compilation error will occur. This requirement exists because initializing a package from a template involves attaching the author's package as a dependency to the consumer's base package.
Q1: I'll need help understanding the full implications of this. For Swift OpenAPI Generator, for example, we might want to host a template that doesn't however depend on the Generator package, and instead depends only on the Swift OpenAPI Runtime package. Would this support it, or does the package hosting the template always have to become a dependency of the generated package?
Q2: “Version Requirements Resolution” describes that if no constraints are provided, a registry dependency resolves to latest version, whereas a git dependency will fail without an explicit version. Why? Could we make both use the same fallback?
Q3: “If multiple templates are present, and no name is provided, an error is thrown.” - can the error please contain the list of discovered template names, to make it easy to pick and rerun? Otherwise hunting down the exact template name can get tedious.
We have written documentation for this, and planning on re-reviewing it once the review phase has been completed and no major changes are needed to be made.
Q1: Does the template-hosting package always become a dependency of the generated package?
Yes, the package that provides the template will become a dependency of the base package, but only for the duration of the template execution. It will be up to the author to decide whether they will want to keep the dependency in the final generated package.
This dependency is only introduced in the staging process, not necessarily in the final user package. Also, we delete any build artifacts before presenting the final generated package to the consumer
So, yes, you can host a template that generates a package that only depends on the Swift OpenAPI Runtime package.
Q2: Package registries can reliably provide a latest version since they maintain a structured index of published semantic versions. Git repositories however do not have a standardized notion of latest version. A git source might use tags, branches, or multiple release streams. Tags might not even follow semantic version. Thus, SwiftPM cannot infer the intentions of the user when resolving to a “latest git version”. Thus, until we can define such a thing, SwiftPM will require an explicit version requirement instead of choosing an arbitrary fallback.
SwiftPM already knows how to work with git tags that represent semantic versions, and how to ignore all the other tags. SemVer 2.0 defines precedence rules between semantic versions.
Why couldn't SwiftPM, in this use case, run the same algorithm to collect all the semantic versions from tags, sort them, and pick the greatest one? Unless I'm missing something, I think all the building blocks for this are already implemented in SwiftPM?
I have added the changes to the implementation. Now, if version requirements are omitted, SwiftPM will try and find the latest SemVer compliant tag. If not, then it will throw an error.
Some projects surely did, but for example I'd love to adopt it in Swift OpenAPI Generator once available, but I never would have wanted to maintain bespoke template machinery myself.
Is the problem being addressed significant enough to warrant a change to Swift?
Yes, I think there's clear motivation to make package creation more extensible so that we can have templates which make it easy to get started with new libraries, use cases, etc. and can be developed outside the core package manager.
Does this proposal fit well with the feel and direction of Swift?
Yes, I think the proposed design here is in line with SwiftPM's other plugin types and does a good job of providing a flexible interface to template authors.
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?
Read through the proposal and an example template from the SwiftPM PR, discussed in the build and packaging workgroup meeting.
One item that did come up in the workgroup discussion was the initialPackageType: Target.TemplateType argument to template targets which can be used to provide a scaffold for a user-authored template to build on mirroring the --type flag of swift package init. A template authored using this feature today will need to remain compatible with future versions of SwiftPM, which may limit the types of changes we can safely make to these "base" templates going forward. It might be good to add a short section to the proposal about what kind of guarantees are provided to template authors, if any. For example, we might say that the filesystem layout setup by initialPackageType: .library may vary based on the tools-version of the user-authored template, and therefore might not match swift package init --type library, depending on which version of SwiftPM is being used.
One question that comes to mind is the implications of sandboxing the network. I can see why, but it’d potentially be very interesting to allow for pulling in an LLM that is remotely accessed from the command line tool.
we have discussed exactly that for a tool to help our customers to get a smarter template starting point. It would be pretty near to get that in this context - one would need to input some user prompting or course.
The idea would be then to have an LLM which is primed with our documentation and a host of sample code…
I will add a section detailing how the filesystem layout setup provided by initialPackageTypewill vary based on the tools-version of the user-authored template. However, one layout that might never change is .empty, which provides simply a Package.swift file. Otherwise, I agree that for other types of starting layouts, the layout will change and might not offer as many guarantees as the .empty package
One question that comes to mind is the implications of sandboxing the network. I can see why, but it’d potentially be very interesting to allow for pulling in an LLM that is remotely accessed from the command line tool.
Sounds interesting! The implication around sandboxing was to offer certain guarantees to consumers that templates cannot make unexpected or malicious outbound requests.
If a template needs network access, then you can explicitly declare this in the template’s manifest, within a .template target:
templatePermissions: [ .allowNetworkConnections(scope: .none, reason: "Need to remotely access a LLM"),]
When this is present, the end-user that invokes this specific template will be prompted to grant or deny network access before the template runs.
