SE-0152: Package Manager Tools Version


(Anders Bertelrud) #1

Hello Swift community,

The review of SE-0152 "Package Manager Tools Version" begins now and runs through February 13, 2017. The proposal is available here:

  https://github.com/apple/swift-evolution/blob/master/proposals/0152-package-manager-tools-version.md

Reviews are an important part of the Swift evolution process. All reviews should be sent to the swift-build-dev and swift-evolution mailing lists at:

  https://lists.swift.org/mailman/listinfo/swift-build-dev
  https://lists.swift.org/mailman/listinfo/swift-evolution

or, if you would like to keep your feedback private, directly to the review manager. When replying, please try to keep the proposal link at the top of the message:

Proposal link:

https://github.com/apple/swift-evolution/blob/master/proposals/0152-package-manager-tools-version.md

Reply text

Other replies

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer:

  * What is your evaluation of the proposal?
  * Is the problem being addressed significant enough to warrant a change to Swift?
  * Does this proposal fit well with the feel and direction of Swift?
  * If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
  * How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available at

  https://github.com/apple/swift-evolution/blob/master/process.md

Thank you,

- Anders

Review Manager


(Martin Waitz) #2

Hi,

The review of SE-0152 "Package Manager Tools Version" begins now and runs through February 13, 2017. The proposal is available here:

  https://github.com/apple/swift-evolution/blob/master/proposals/0152-package-manager-tools-version.md

From the proposal:

Not changing this API would still leave the problem of figuring out which Swift language compatibility version to interpret the manifest in. It's possible that Package.swift manifests won't be significantly affected by Swift language changes in Swift 4, and could mostly work in either language compatibility mode without changes. However, we don't know whether that will be the case, and it would be a significant risk to assume that it will be.

I really assume that the simple `Package.swift` files will be able to be compiled in swift 3 and swift 4 modes.
Do we really need this proposal now?
If we really encounter problems, we can always add it later.
So what exactly is the risk?

When we want to introduce it already now, I’d like to revisit the way the tool version is specified.

Can we _please_ not use a comment?
Comments are comments and should have no influence on how to parse the file.

By the way, in XML/HTML, the DTD/Doctype is not specified in a comment, but in a processor instruction (`<!DOCTYPE html>` vs. `<!— comment —>`).
So the XML example would more closely resemble something like `#swift(3.0)` in Swift.

Maybe the package manager could just grep for the `swiftLanguageVersion` specification?
This has to be specified by the package anyway.
We could simply specify that the swift compiler always uses the same mode for `Package.swift` and the rest of the sources,
and that the package manager obtains this version from the manifest without using the full Swift compiler.
I.e. this line would be fixed to some special syntax (`^\s*swiftLanguageVersion\s*:\s*(\d+)\s*,\s*$` or something).

— Martin


(Will Field-Thompson) #3

* What is your evaluation of the proposal?

I fully support the goals of the proposal and have a question about the
implementation.

* Is the problem being addressed significant enough to warrant a change to

Swift?

Yep. It provides a good way to evolve the API, and putting it in place for
Swift 3.1 gives us the freedom to evolve the API as (or if) needed for
Swift 4.

* Does this proposal fit well with the feel and direction of Swift?

I agree that the placement of the tools version in a comment is
unfortunate, but I think that it's the best way to include it in
Package.swift.

I'm interested in other people's thoughts on a .swift-tools-version file.
I'm not sure I prefer it to the comment in Package.swift, but the reasons
included in the proposal don't convince me the .swift-tools-version shouldn't
be used either.

   - *Eliminates the possibility of including either Package.swift or
   .swift-tools-version*: Seems like exactly the kind of mistake you make
   once and then don't make again. I guess with a leading dot it's more likely
   to be forgotten than a non-dot-file, but not a major issue IMO.
   - *Users may like to see the version in the Package.swift*: Probably
   true, though I'm not sure how much of an issue it is.

Are these significant problems for other people? They don't seem that way
to me, but I may not represent the majority here as my work is in iOS dev
and the only time I get to use swiftpm is for side projects.

The .swift-tools-version has the advantage that it is extendable in a
prettier (i.e. easier to read and easier to diff for VCS purposes) way than
the semicolon-separated syntax from the proposal. How likely is it that
we'll ever have to extend it? I think that, for me, is the determining
factor for which format I prefer.

* If you have used other languages or libraries with a similar feature,

how do you feel that this proposal compares to those?

The main package manager I use is CocoaPods and as far as I am aware
CocoaPods doesn't allow you to specify the tools version in the Podfile.
This did cause some (albeit minimal) pain on transition to CocoaPods 1.0. I
think Podfile.lock records the version of CocoaPods, but I don't think that
actually helps in this regard.

* How much effort did you put into your review? A glance, a quick

reading, or an in-depth study?

I spent some time reading through the proposal, but I lost track of the
discussion pre-proposal.

- Will


(Rick Ballard) #4

The review of SE-0152 "Package Manager Tools Version" begins now and runs through February 13, 2017. The proposal is available here:

  https://github.com/apple/swift-evolution/blob/master/proposals/0152-package-manager-tools-version.md

Hi Martin,

Thanks again for your feedback.

Regarding your first point, about whether this is necessary:

We could indeed choose to make the Swift 3 tools always interpret the manifest as Swift 3 and the Swift 4 tools always interpret it as Swift 4, and hope that no language changes affect code in manifests, but that's a pretty big risk -- either that manifests won't build, or that subtle incompatibilities between the modes could have unexpected side effects. Simple manifests might not be affected by language changes (though we really don't know that yet), but more complex manifests might be, which would still be a a problem.

If this was the main point of this proposal, I might agree that it isn't worth it for this single need, despite the risk. But there are two other things that we need this proposal for; the most important being allowing packages to avoid breaking dependency resolution from older versions of the Swift tools when they adopt new PackageDescription API (or any other Swift features, which might not be reflected in the Swift language compatibility version, e.g. if it's a feature added in a minor release). Since we want a Swift tools version defined for that need, we can get proper control "for free" over the language version used to parse the manifest.

Unfortunately, we can't just wait and see how much of a problem this is in practice and add it later if we need it; this mechanism needs to be understood by both a new breaking version of Swift (Swift 4) and the version prior (Swift 3.1) in order to have any effect, since the whole point is to keep the prior version from trying to use incompatible packages. Our window to get changes into 3.1 will close before too long, so it's now or never for this language transition.

Regarding your second point, about this being a comment:

That's a good point about the SGML DOCTYPE not actually being a comment; I can revise the proposal to fix that inaccuracy. I don't think that materially affects the tradeoffs between making this a comment vs. other approaches, though.

We discuss some alternatives to using a comment, and why we favor the comment, here:
  https://github.com/apple/swift-evolution/blob/master/proposals/0152-package-manager-tools-version.md#store-the-swift-tools-version-in-a-separate-file
and here:
  https://github.com/apple/swift-evolution/blob/master/proposals/0152-package-manager-tools-version.md#specify-the-swift-tools-version-with-a-more-swifty-syntax

There was a more detailed discussion of the tradeoffs in a version of the document we floated prior to formal review, before we had decided on the comment:
  https://github.com/rballard/swift-evolution/blob/fd28010aaef077b9783d738eacd912915b55c67e/proposals/NNNN-package-manager-tools-version.md#how-the-swift-tools-version-is-specified

You suggest an alternative of grepping the Package.Swift for the string "swiftLanguageVersions". That approach seems like pretty unexpected, non-formal behavior for the tools. What happens if this string appears multiple times, which we expect will be the case once we have a proper build settings model and you can set swiftLanguageVersions on a per-module basis within the package? Would we parse (and be able to machine-edit) all syntaxes for setting the swiftLanguageVersions (initial initializer, mutating the property after Package() is initialized, expression to derive the RHS of the setter)? What would it mean if swiftLanguageVersions has multiple values, which is allowed? I don't see concrete benefits to this idea, and it adds new complexities and possibilities for error.

Note that the swiftLanguageVersions does _not_ actually have to be specified by the package otherwise. In the common case of a package whose language version is the same as its Swift tools version, no swiftLanguageVersions property need be specified -- and I expect that `swift package init` won't give you one by default.

We don't want to stipulate that the package manager needs to use the same mode for `Package.swift` as the rest of your sources, because we want you to be able to adopt new package manager API without having to simultaneously convert all your package's sources from Swift 3 to Swift 4. It's easier for our users if they can tackle language conversion when they're ready to, and not make it a prerequisite of using new SwiftPM features.

Ultimately, I think no one is thrilled with parsing the Swift tools version out of a comment, but the feedback we've received so far (off-list) has strongly favored that approach, and when we evaluated the tradeoffs vs. the other approaches we considered, it seemed the best.

Thanks,

  - Rick

···

On Feb 8, 2017, at 12:17 AM, Martin Waitz via swift-build-dev <swift-build-dev@swift.org> wrote:

From the proposal:

Not changing this API would still leave the problem of figuring out which Swift language compatibility version to interpret the manifest in. It's possible that Package.swift manifests won't be significantly affected by Swift language changes in Swift 4, and could mostly work in either language compatibility mode without changes. However, we don't know whether that will be the case, and it would be a significant risk to assume that it will be.

I really assume that the simple `Package.swift` files will be able to be compiled in swift 3 and swift 4 modes.
Do we really need this proposal now?
If we really encounter problems, we can always add it later.
So what exactly is the risk?

When we want to introduce it already now, I’d like to revisit the way the tool version is specified.

Can we _please_ not use a comment?
Comments are comments and should have no influence on how to parse the file.

By the way, in XML/HTML, the DTD/Doctype is not specified in a comment, but in a processor instruction (`<!DOCTYPE html>` vs. `<!— comment —>`).
So the XML example would more closely resemble something like `#swift(3.0)` in Swift.

Maybe the package manager could just grep for the `swiftLanguageVersion` specification?
This has to be specified by the package anyway.
We could simply specify that the swift compiler always uses the same mode for `Package.swift` and the rest of the sources,
and that the package manager obtains this version from the manifest without using the full Swift compiler.
I.e. this line would be fixed to some special syntax (`^\s*swiftLanguageVersion\s*:\s*(\d+)\s*,\s*$` or something).

— Martin


(Boris Buegling) #5

The main package manager I use is CocoaPods and as far as I am aware CocoaPods doesn't allow you to specify the tools version in the Podfile. This did cause some (albeit minimal) pain on transition to CocoaPods 1.0. I think Podfile.lock records the version of CocoaPods, but I don't think that actually helps in this regard.

This is correct. The Podfile.lock records the last used version, but does not enforce it in any way. The officially recommended approach is using Bundler’s Gemfile/Gemfile.lock to ensure consistency of used versions of the CP gem.

Cheers,
Boris

···

On 8. Feb 2017, at 10:49, Will Field-Thompson via swift-evolution <swift-evolution@swift.org> wrote:

  * How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

I spent some time reading through the proposal, but I lost track of the discussion pre-proposal.

- Will
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Rick Ballard) #6

The review of SE-0152 "Package Manager Tools Version" begins now and runs through February 13, 2017. The proposal is available here:

  https://github.com/apple/swift-evolution/blob/master/proposals/0152-package-manager-tools-version.md

Hi Will,

Thanks for taking the time to review this.

Is there a particular reason you're envisioning that we might need to extend the metadata for the Swift tools version?

We've left ourself an escape hatch in case we ever do need to, where anything after a ';' character in the tools version comment will be ignored for now, but I'm not expecting that we'll ever need to use that escape hatch. The Swift tools version itself needs to be stored in a special way since it dictates how we'll interpret the manifest, but once we've determined that, all other data should be able to be stored as Swift code in the manifest as per usual.

If you can think of some good reasons why we'd actually need this extensibility, please do suggest them. Otherwise I'd hope that we wouldn't need to choose a less convenient mechanism here for the sake of more convenient extensibility that will likely never be used.

Thanks,

  - Rick

···

On Feb 8, 2017, at 10:49 AM, Will Field-Thompson via swift-evolution <swift-evolution@swift.org> wrote:

  * What is your evaluation of the proposal?

I fully support the goals of the proposal and have a question about the implementation.

  * Is the problem being addressed significant enough to warrant a change to Swift?

Yep. It provides a good way to evolve the API, and putting it in place for Swift 3.1 gives us the freedom to evolve the API as (or if) needed for Swift 4.

  * Does this proposal fit well with the feel and direction of Swift?

I agree that the placement of the tools version in a comment is unfortunate, but I think that it's the best way to include it in Package.swift.

I'm interested in other people's thoughts on a .swift-tools-version file. I'm not sure I prefer it to the comment in Package.swift, but the reasons included in the proposal don't convince me the .swift-tools-version shouldn't be used either.
Eliminates the possibility of including either Package.swift or .swift-tools-version: Seems like exactly the kind of mistake you make once and then don't make again. I guess with a leading dot it's more likely to be forgotten than a non-dot-file, but not a major issue IMO.
Users may like to see the version in the Package.swift: Probably true, though I'm not sure how much of an issue it is.
Are these significant problems for other people? They don't seem that way to me, but I may not represent the majority here as my work is in iOS dev and the only time I get to use swiftpm is for side projects.

The .swift-tools-version has the advantage that it is extendable in a prettier (i.e. easier to read and easier to diff for VCS purposes) way than the semicolon-separated syntax from the proposal. How likely is it that we'll ever have to extend it? I think that, for me, is the determining factor for which format I prefer.

  * If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

The main package manager I use is CocoaPods and as far as I am aware CocoaPods doesn't allow you to specify the tools version in the Podfile. This did cause some (albeit minimal) pain on transition to CocoaPods 1.0. I think Podfile.lock records the version of CocoaPods, but I don't think that actually helps in this regard.

  * How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

I spent some time reading through the proposal, but I lost track of the discussion pre-proposal.

- Will


(Martin Waitz) #7

Hello Rick,

thanks again for your time and to explain everything in so much detail! :slight_smile:

···

Am 08.02.2017 um 18:34 schrieb Rick Ballard <rballard@apple.com>:

Ultimately, I think no one is thrilled with parsing the Swift tools version out of a comment, but the feedback we've received so far (off-list) has strongly favored that approach, and when we evaluated the tradeoffs vs. the other approaches we considered, it seemed the best.

Yeah, I can follow your argumentation now.
And I don’t really have any better idea.
Thanks for the explanations.

— Martin


(Will Field-Thompson) #8

Is there a particular reason you're envisioning that we might need to extend the metadata for the Swift tools version?

I can't think of any reason myself, I just noticed the escape hatch and was wondering if there were any planned uses for it!

We've left ourself an escape hatch in case we ever do need to, where anything after a ';' character in the tools version comment will be ignored for now, but I'm not expecting that we'll ever need to use that escape hatch. The Swift tools version itself needs to be stored in a special way since it dictates how we'll interpret the manifest, but once we've determined that, all other data should be able to be stored as Swift code in the manifest as per usual.

Given that, I think the comment in Package.swift is almost definitely the right way to go.

If you can think of some good reasons why we'd actually need this extensibility, please do suggest them. Otherwise I'd hope that we wouldn't need to choose a less convenient mechanism here for the sake of more convenient extensibility that will likely never be used.

I completely agree. Thank you for explaining!

- Will

···

Thanks,

  - Rick

On Feb 8, 2017, at 10:49 AM, Will Field-Thompson via swift-evolution <swift-evolution@swift.org> wrote:

  * What is your evaluation of the proposal?

I fully support the goals of the proposal and have a question about the implementation.

  * Is the problem being addressed significant enough to warrant a change to Swift?

Yep. It provides a good way to evolve the API, and putting it in place for Swift 3.1 gives us the freedom to evolve the API as (or if) needed for Swift 4.

  * Does this proposal fit well with the feel and direction of Swift?

I agree that the placement of the tools version in a comment is unfortunate, but I think that it's the best way to include it in Package.swift.

I'm interested in other people's thoughts on a .swift-tools-version file. I'm not sure I prefer it to the comment in Package.swift, but the reasons included in the proposal don't convince me the .swift-tools-version shouldn't be used either.
Eliminates the possibility of including either Package.swift or .swift-tools-version: Seems like exactly the kind of mistake you make once and then don't make again. I guess with a leading dot it's more likely to be forgotten than a non-dot-file, but not a major issue IMO.
Users may like to see the version in the Package.swift: Probably true, though I'm not sure how much of an issue it is.
Are these significant problems for other people? They don't seem that way to me, but I may not represent the majority here as my work is in iOS dev and the only time I get to use swiftpm is for side projects.

The .swift-tools-version has the advantage that it is extendable in a prettier (i.e. easier to read and easier to diff for VCS purposes) way than the semicolon-separated syntax from the proposal. How likely is it that we'll ever have to extend it? I think that, for me, is the determining factor for which format I prefer.

  * If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

The main package manager I use is CocoaPods and as far as I am aware CocoaPods doesn't allow you to specify the tools version in the Podfile. This did cause some (albeit minimal) pain on transition to CocoaPods 1.0. I think Podfile.lock records the version of CocoaPods, but I don't think that actually helps in this regard.

  * How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

I spent some time reading through the proposal, but I lost track of the discussion pre-proposal.

- Will