Package manager support for versioning (both language and tools)


(Rick Ballard) #1

As part of Swift source compatibility, the Swift compiler added a new flag for controlling what Swift language version should be used to compile with. Currently there is no good way to specify this for Swift packages. We've got a proposal ready that provides a mechanism for controlling this. I'd appreciate any feedback the community would like addressed before this goes through evolution review.

The Swift language version support proposal can be viewed and commented on at https://github.com/rballard/swift-evolution/pull/2/files

Additionally, we are proposing to introduce a "Swift minimum tools version" mechanism for Swift packages. This mechanism will allow us to evolve the package manager – adding new API, and doing a one-time revision of our existing API for Swift 4 – without breaking the package ecosystem. It will also manage the Swift 3–>4 language transition for Package.swift files themselves. This is a pretty lengthy proposal, but I'd appreciate any important feedback before this goes to evolution review. There is also one decision left to make before this proposal is final; I lay out several possibilities for how we could store the "Swift tools version". I'd appreciate feedback on which of the possibilities we should pick before I bring this to review.

The Swift tools version support proposal can be viewed and commented on at https://github.com/rballard/swift-evolution/pull/1/files

Absent any serious feedback which requires us to make major revisions, I'm expecting to bring both of these to Swift evolution review early next week.

Thank you,

  - Rick


(Matthew Johnson) #2

It’s clear that a lot of work has gone into identifying and evaluating several different approaches. Thank you for doing the hard work here!

I think I might have one additional alternative to consider. I like the idea of using Package.swift to specify the tools version, but I think the disadvantages of using a DSL in a comment is less than ideal. Is there a way we can design this to rely on normal Swift syntax? For example, what if we required Package.swift to include a line whose first non-whitespace character is the following (possibly allowing trailing comments):

let toolsVersion = ToolsVersion._3_1

This would be supported by a ToolsVersion enum to which we add a new case with each new version of the tools. If no such declaration is found it will be it would be considered to be 3.0.0 just as with the comment approach.

Is there any reason why using a comment DSL is preferable to something like this?

···

On Feb 3, 2017, at 3:03 PM, Rick Ballard via swift-build-dev <swift-build-dev@swift.org> wrote:

As part of Swift source compatibility, the Swift compiler added a new flag for controlling what Swift language version should be used to compile with. Currently there is no good way to specify this for Swift packages. We've got a proposal ready that provides a mechanism for controlling this. I'd appreciate any feedback the community would like addressed before this goes through evolution review.

The Swift language version support proposal can be viewed and commented on at https://github.com/rballard/swift-evolution/pull/2/files

Additionally, we are proposing to introduce a "Swift minimum tools version" mechanism for Swift packages. This mechanism will allow us to evolve the package manager – adding new API, and doing a one-time revision of our existing API for Swift 4 – without breaking the package ecosystem. It will also manage the Swift 3–>4 language transition for Package.swift files themselves. This is a pretty lengthy proposal, but I'd appreciate any important feedback before this goes to evolution review. There is also one decision left to make before this proposal is final; I lay out several possibilities for how we could store the "Swift tools version". I'd appreciate feedback on which of the possibilities we should pick before I bring this to review.

The Swift tools version support proposal can be viewed and commented on at https://github.com/rballard/swift-evolution/pull/1/files

Absent any serious feedback which requires us to make major revisions, I'm expecting to bring both of these to Swift evolution review early next week.

Thank you,

  - Rick

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev


(Rick Ballard) #3

Hi Matthew,

The reason I was suggesting a simple DSL in a comment instead of using Swift for that declaration is because we need to know what this value is before we start the Swift interpreter, so that we know what Swift language compatibility mode to run the interpreter in. I suppose that it'd be possible to work around that; for example, we could grab the first line of the file and run it through the Swift interpreter in the latest language compatibility mode, assuming that a simple let statement and enum will always compile the same way in all future versions of Swift. We'd figure out the Swift tools version from that and then re-run the interpreter over the entire file in the right language compatibility mode.

That's some fairly magic behavior to understand, though; if it's a place that you can write Swift code, it's non-obvious that we're only reading a single line of it. It would also be difficult to enforce the restrictions that we'd like on the first line; if we're using the Swift interpreter, how do we make sure that you haven't written a more complicated expression that might not compile in all future versions of Swift, and/or referenced any of the version-specific PackageDescription API beyond that enum? And from an aesthetic standpoint, I imagine that many people would be annoyed to have a single line of Swift code at the top of the file right before any comment header that they'd otherwise lead their file with (e.g. containing a copyright header, author info, etc).

Overall, I don't see any concrete benefit to making that first line be Swift, and it adds new complications. This was worth considering, however, so thanks for suggesting it!

  - Rick

···

On Feb 4, 2017, at 8:28 AM, Matthew Johnson <matthew@anandabits.com> wrote:

It’s clear that a lot of work has gone into identifying and evaluating several different approaches. Thank you for doing the hard work here!

I think I might have one additional alternative to consider. I like the idea of using Package.swift to specify the tools version, but I think the disadvantages of using a DSL in a comment is less than ideal. Is there a way we can design this to rely on normal Swift syntax? For example, what if we required Package.swift to include a line whose first non-whitespace character is the following (possibly allowing trailing comments):

let toolsVersion = ToolsVersion._3_1

This would be supported by a ToolsVersion enum to which we add a new case with each new version of the tools. If no such declaration is found it will be it would be considered to be 3.0.0 just as with the comment approach.

Is there any reason why using a comment DSL is preferable to something like this?

On Feb 3, 2017, at 3:03 PM, Rick Ballard via swift-build-dev <swift-build-dev@swift.org> wrote:

As part of Swift source compatibility, the Swift compiler added a new flag for controlling what Swift language version should be used to compile with. Currently there is no good way to specify this for Swift packages. We've got a proposal ready that provides a mechanism for controlling this. I'd appreciate any feedback the community would like addressed before this goes through evolution review.

The Swift language version support proposal can be viewed and commented on at https://github.com/rballard/swift-evolution/pull/2/files

Additionally, we are proposing to introduce a "Swift minimum tools version" mechanism for Swift packages. This mechanism will allow us to evolve the package manager – adding new API, and doing a one-time revision of our existing API for Swift 4 – without breaking the package ecosystem. It will also manage the Swift 3–>4 language transition for Package.swift files themselves. This is a pretty lengthy proposal, but I'd appreciate any important feedback before this goes to evolution review. There is also one decision left to make before this proposal is final; I lay out several possibilities for how we could store the "Swift tools version". I'd appreciate feedback on which of the possibilities we should pick before I bring this to review.

The Swift tools version support proposal can be viewed and commented on at https://github.com/rballard/swift-evolution/pull/1/files

Absent any serious feedback which requires us to make major revisions, I'm expecting to bring both of these to Swift evolution review early next week.

Thank you,

  - Rick

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev


(Matthew Johnson) #4

Hi Matthew,

The reason I was suggesting a simple DSL in a comment instead of using Swift for that declaration is because we need to know what this value is before we start the Swift interpreter, so that we know what Swift language compatibility mode to run the interpreter in. I suppose that it'd be possible to work around that; for example, we could grab the first line of the file and run it through the Swift interpreter in the latest language compatibility mode, assuming that a simple let statement and enum will always compile the same way in all future versions of Swift. We'd figure out the Swift tools version from that and then re-run the interpreter over the entire file in the right language compatibility mode.

That's some fairly magic behavior to understand, though; if it's a place that you can write Swift code, it's non-obvious that we're only reading a single line of it. It would also be difficult to enforce the restrictions that we'd like on the first line; if we're using the Swift interpreter, how do we make sure that you haven't written a more complicated expression that might not compile in all future versions of Swift, and/or referenced any of the version-specific PackageDescription API beyond that enum? And from an aesthetic standpoint, I imagine that many people would be annoyed to have a single line of Swift code at the top of the file right before any comment header that they'd otherwise lead their file with (e.g. containing a copyright header, author info, etc).

Overall, I don't see any concrete benefit to making that first line be Swift, and it adds new complications. This was worth considering, however, so thanks for suggesting it!

Hi Rick,

Thanks for the reply. I wasn’t suggesting that you actually use the Swift interpreter to parse this line. My idea was that if the package format uses a top-level constant named `toolsVersion` that *must* be initialized with a direct reference to the enum case it would be easy enough to parse *without* using the Swift interpreter. You would just look for a line that begins with `let toolsVersion = ToolsVersion._n_n_n`. It wouldn’t necessarily need to be the first line. This statement has a very simple grammar that could be parsed directly just as easily as a DSL we invent and put in comments.

I agree that the primary downside of this is that it would be specified using normal Swift syntax but would not allow the full expressivity that Swift allows (this constant must be initialized via directly specifying a case name rather than with any value of the correct type). However, it has the advantage of using normal Swift syntax and all the benefits derived from that (for example, cmd-click in Xcode to see the valid values, reading like a normal part of the package specification, etc). If the user attempted to use an expression that is valid Swift but does not a direct reference to one of the case names the package manager would handle that in the same way it would handle an invalid tools version in the comment DSL.

It’s not immediately obvious to me which approach makes the best overall tradeoff. It seems like a topic worthy of discussion by the community, especially since you have already listed several options and asked for community input on the subject. Does anyone else have any thoughts on this?

···

On Feb 6, 2017, at 2:06 PM, Rick Ballard <rballard@apple.com> wrote:

  - Rick

On Feb 4, 2017, at 8:28 AM, Matthew Johnson <matthew@anandabits.com> wrote:

It’s clear that a lot of work has gone into identifying and evaluating several different approaches. Thank you for doing the hard work here!

I think I might have one additional alternative to consider. I like the idea of using Package.swift to specify the tools version, but I think the disadvantages of using a DSL in a comment is less than ideal. Is there a way we can design this to rely on normal Swift syntax? For example, what if we required Package.swift to include a line whose first non-whitespace character is the following (possibly allowing trailing comments):

let toolsVersion = ToolsVersion._3_1

This would be supported by a ToolsVersion enum to which we add a new case with each new version of the tools. If no such declaration is found it will be it would be considered to be 3.0.0 just as with the comment approach.

Is there any reason why using a comment DSL is preferable to something like this?

On Feb 3, 2017, at 3:03 PM, Rick Ballard via swift-build-dev <swift-build-dev@swift.org> wrote:

As part of Swift source compatibility, the Swift compiler added a new flag for controlling what Swift language version should be used to compile with. Currently there is no good way to specify this for Swift packages. We've got a proposal ready that provides a mechanism for controlling this. I'd appreciate any feedback the community would like addressed before this goes through evolution review.

The Swift language version support proposal can be viewed and commented on at https://github.com/rballard/swift-evolution/pull/2/files

Additionally, we are proposing to introduce a "Swift minimum tools version" mechanism for Swift packages. This mechanism will allow us to evolve the package manager – adding new API, and doing a one-time revision of our existing API for Swift 4 – without breaking the package ecosystem. It will also manage the Swift 3–>4 language transition for Package.swift files themselves. This is a pretty lengthy proposal, but I'd appreciate any important feedback before this goes to evolution review. There is also one decision left to make before this proposal is final; I lay out several possibilities for how we could store the "Swift tools version". I'd appreciate feedback on which of the possibilities we should pick before I bring this to review.

The Swift tools version support proposal can be viewed and commented on at https://github.com/rballard/swift-evolution/pull/1/files

Absent any serious feedback which requires us to make major revisions, I'm expecting to bring both of these to Swift evolution review early next week.

Thank you,

  - Rick

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev


(Xiaodi Wu) #5

I think I agree with Rick. I see no benefit to making it look like this
line is Swift when it's really not parsed as such. Like a doctype in HTML
or a shebang line in Python, a special comment is lightweight, familiar,
and does not suggest any capability it doesn't have.

···

On Mon, Feb 6, 2017 at 17:45 Matthew Johnson via swift-evolution < swift-evolution@swift.org> wrote:

> On Feb 6, 2017, at 2:06 PM, Rick Ballard <rballard@apple.com> wrote:
>
> Hi Matthew,
>
> The reason I was suggesting a simple DSL in a comment instead of using
Swift for that declaration is because we need to know what this value is
before we start the Swift interpreter, so that we know what Swift language
compatibility mode to run the interpreter in. I suppose that it'd be
possible to work around that; for example, we could grab the first line of
the file and run it through the Swift interpreter in the latest language
compatibility mode, assuming that a simple let statement and enum will
always compile the same way in all future versions of Swift. We'd figure
out the Swift tools version from that and then re-run the interpreter over
the entire file in the right language compatibility mode.
>
> That's some fairly magic behavior to understand, though; if it's a place
that you can write Swift code, it's non-obvious that we're only reading a
single line of it. It would also be difficult to enforce the restrictions
that we'd like on the first line; if we're using the Swift interpreter, how
do we make sure that you haven't written a more complicated expression that
might not compile in all future versions of Swift, and/or referenced any of
the version-specific PackageDescription API beyond that enum? And from an
aesthetic standpoint, I imagine that many people would be annoyed to have a
single line of Swift code at the top of the file right before any comment
header that they'd otherwise lead their file with (e.g. containing a
copyright header, author info, etc).
>
> Overall, I don't see any concrete benefit to making that first line be
Swift, and it adds new complications. This was worth considering, however,
so thanks for suggesting it!

Hi Rick,

Thanks for the reply. I wasn’t suggesting that you actually use the Swift
interpreter to parse this line. My idea was that if the package format
uses a top-level constant named `toolsVersion` that *must* be initialized
with a direct reference to the enum case it would be easy enough to parse
*without* using the Swift interpreter. You would just look for a line that
begins with `let toolsVersion = ToolsVersion._n_n_n`. It wouldn’t
necessarily need to be the first line. This statement has a very simple
grammar that could be parsed directly just as easily as a DSL we invent and
put in comments.

I agree that the primary downside of this is that it would be specified
using normal Swift syntax but would not allow the full expressivity that
Swift allows (this constant must be initialized via directly specifying a
case name rather than with any value of the correct type). However, it has
the advantage of using normal Swift syntax and all the benefits derived
from that (for example, cmd-click in Xcode to see the valid values, reading
like a normal part of the package specification, etc). If the user
attempted to use an expression that is valid Swift but does not a direct
reference to one of the case names the package manager would handle that in
the same way it would handle an invalid tools version in the comment DSL.

It’s not immediately obvious to me which approach makes the best overall
tradeoff. It seems like a topic worthy of discussion by the community,
especially since you have already listed several options and asked for
community input on the subject. Does anyone else have any thoughts on this?

>
> - Rick
>
>> On Feb 4, 2017, at 8:28 AM, Matthew Johnson <matthew@anandabits.com> > wrote:
>>
>> It’s clear that a lot of work has gone into identifying and evaluating
several different approaches. Thank you for doing the hard work here!
>>
>> I think I might have one additional alternative to consider. I like
the idea of using Package.swift to specify the tools version, but I think
the disadvantages of using a DSL in a comment is less than ideal. Is there
a way we can design this to rely on normal Swift syntax? For example, what
if we required Package.swift to include a line whose first non-whitespace
character is the following (possibly allowing trailing comments):
>>
>> let toolsVersion = ToolsVersion._3_1
>>
>> This would be supported by a ToolsVersion enum to which we add a new
case with each new version of the tools. If no such declaration is found
it will be it would be considered to be 3.0.0 just as with the comment
approach.
>>
>> Is there any reason why using a comment DSL is preferable to something
like this?
>>
>>
>>
>>> On Feb 3, 2017, at 3:03 PM, Rick Ballard via swift-build-dev < > swift-build-dev@swift.org> wrote:
>>>
>>> As part of Swift source compatibility, the Swift compiler added a new
flag for controlling what Swift language version should be used to compile
with. Currently there is no good way to specify this for Swift packages.
We've got a proposal ready that provides a mechanism for controlling this.
I'd appreciate any feedback the community would like addressed before this
goes through evolution review.
>>>
>>> The Swift language version support proposal can be viewed and
commented on at https://github.com/rballard/swift-evolution/pull/2/files
>>>
>>> Additionally, we are proposing to introduce a "Swift minimum tools
version" mechanism for Swift packages. This mechanism will allow us to
evolve the package manager – adding new API, and doing a one-time revision
of our existing API for Swift 4 – without breaking the package ecosystem.
It will also manage the Swift 3–>4 language transition for Package.swift
files themselves. This is a pretty lengthy proposal, but I'd appreciate any
important feedback before this goes to evolution review. There is also one
decision left to make before this proposal is final; I lay out several
possibilities for how we could store the "Swift tools version". I'd
appreciate feedback on which of the possibilities we should pick before I
bring this to review.
>>>
>>> The Swift tools version support proposal can be viewed and commented
on at https://github.com/rballard/swift-evolution/pull/1/files
>>>
>>> Absent any serious feedback which requires us to make major revisions,
I'm expecting to bring both of these to Swift evolution review early next
week.
>>>
>>> Thank you,
>>>
>>> - Rick
>>>
>>> _______________________________________________
>>> swift-build-dev mailing list
>>> swift-build-dev@swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-build-dev
>>
>

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Daniel Dunbar) #6

I think I agree with Rick. I see no benefit to making it look like this line is Swift when it's really not parsed as such. Like a doctype in HTML or a shebang line in Python, a special comment is lightweight, familiar, and does not suggest any capability it doesn't have.

+1

I agree with Matthew it is unfortunate to have to have a (very tiny) embedded DSL, but this seems to be the best alternative.

- Daniel

···

On Feb 6, 2017, at 4:18 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

On Mon, Feb 6, 2017 at 17:45 Matthew Johnson via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

> On Feb 6, 2017, at 2:06 PM, Rick Ballard <rballard@apple.com <mailto:rballard@apple.com>> wrote:
>
> Hi Matthew,
>
> The reason I was suggesting a simple DSL in a comment instead of using Swift for that declaration is because we need to know what this value is before we start the Swift interpreter, so that we know what Swift language compatibility mode to run the interpreter in. I suppose that it'd be possible to work around that; for example, we could grab the first line of the file and run it through the Swift interpreter in the latest language compatibility mode, assuming that a simple let statement and enum will always compile the same way in all future versions of Swift. We'd figure out the Swift tools version from that and then re-run the interpreter over the entire file in the right language compatibility mode.
>
> That's some fairly magic behavior to understand, though; if it's a place that you can write Swift code, it's non-obvious that we're only reading a single line of it. It would also be difficult to enforce the restrictions that we'd like on the first line; if we're using the Swift interpreter, how do we make sure that you haven't written a more complicated expression that might not compile in all future versions of Swift, and/or referenced any of the version-specific PackageDescription API beyond that enum? And from an aesthetic standpoint, I imagine that many people would be annoyed to have a single line of Swift code at the top of the file right before any comment header that they'd otherwise lead their file with (e.g. containing a copyright header, author info, etc).
>
> Overall, I don't see any concrete benefit to making that first line be Swift, and it adds new complications. This was worth considering, however, so thanks for suggesting it!

Hi Rick,

Thanks for the reply. I wasn’t suggesting that you actually use the Swift interpreter to parse this line. My idea was that if the package format uses a top-level constant named `toolsVersion` that *must* be initialized with a direct reference to the enum case it would be easy enough to parse *without* using the Swift interpreter. You would just look for a line that begins with `let toolsVersion = ToolsVersion._n_n_n`. It wouldn’t necessarily need to be the first line. This statement has a very simple grammar that could be parsed directly just as easily as a DSL we invent and put in comments.

I agree that the primary downside of this is that it would be specified using normal Swift syntax but would not allow the full expressivity that Swift allows (this constant must be initialized via directly specifying a case name rather than with any value of the correct type). However, it has the advantage of using normal Swift syntax and all the benefits derived from that (for example, cmd-click in Xcode to see the valid values, reading like a normal part of the package specification, etc). If the user attempted to use an expression that is valid Swift but does not a direct reference to one of the case names the package manager would handle that in the same way it would handle an invalid tools version in the comment DSL.

It’s not immediately obvious to me which approach makes the best overall tradeoff. It seems like a topic worthy of discussion by the community, especially since you have already listed several options and asked for community input on the subject. Does anyone else have any thoughts on this?

>
> - Rick
>
>> On Feb 4, 2017, at 8:28 AM, Matthew Johnson <matthew@anandabits.com <mailto:matthew@anandabits.com>> wrote:
>>
>> It’s clear that a lot of work has gone into identifying and evaluating several different approaches. Thank you for doing the hard work here!
>>
>> I think I might have one additional alternative to consider. I like the idea of using Package.swift to specify the tools version, but I think the disadvantages of using a DSL in a comment is less than ideal. Is there a way we can design this to rely on normal Swift syntax? For example, what if we required Package.swift to include a line whose first non-whitespace character is the following (possibly allowing trailing comments):
>>
>> let toolsVersion = ToolsVersion._3_1
>>
>> This would be supported by a ToolsVersion enum to which we add a new case with each new version of the tools. If no such declaration is found it will be it would be considered to be 3.0.0 just as with the comment approach.
>>
>> Is there any reason why using a comment DSL is preferable to something like this?
>>
>>
>>
>>> On Feb 3, 2017, at 3:03 PM, Rick Ballard via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:
>>>
>>> As part of Swift source compatibility, the Swift compiler added a new flag for controlling what Swift language version should be used to compile with. Currently there is no good way to specify this for Swift packages. We've got a proposal ready that provides a mechanism for controlling this. I'd appreciate any feedback the community would like addressed before this goes through evolution review.
>>>
>>> The Swift language version support proposal can be viewed and commented on at https://github.com/rballard/swift-evolution/pull/2/files
>>>
>>> Additionally, we are proposing to introduce a "Swift minimum tools version" mechanism for Swift packages. This mechanism will allow us to evolve the package manager – adding new API, and doing a one-time revision of our existing API for Swift 4 – without breaking the package ecosystem. It will also manage the Swift 3–>4 language transition for Package.swift files themselves. This is a pretty lengthy proposal, but I'd appreciate any important feedback before this goes to evolution review. There is also one decision left to make before this proposal is final; I lay out several possibilities for how we could store the "Swift tools version". I'd appreciate feedback on which of the possibilities we should pick before I bring this to review.
>>>
>>> The Swift tools version support proposal can be viewed and commented on at https://github.com/rballard/swift-evolution/pull/1/files
>>>
>>> Absent any serious feedback which requires us to make major revisions, I'm expecting to bring both of these to Swift evolution review early next week.
>>>
>>> Thank you,
>>>
>>> - Rick
>>>
>>> _______________________________________________
>>> swift-build-dev mailing list
>>> swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
>>> https://lists.swift.org/mailman/listinfo/swift-build-dev
>>
>

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution
_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution