Proposal: SwiftPM Support for Language Versions


(Anders Bertelrud) #1

Hello,

This is a proposal to enhance the SwiftPM package selection logic so that packages can be differentiated based on Swift version in addition to package version. It does not introduce any source-breaking changes, but defines a way for package authors to annotate repositories and package manifests so that SwiftPM can exclude those that are usable only for particular versions of Swift.

You can find the proposal here:

https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md

and the current text is also included below.

Time is short, but feedback on this proposal is very welcome.

Thanks,

Anders

Package Manager Support for Language Versions

Proposal: SE-NNNN <https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md>
Author: Anders Bertelrud <https://github.com/abertelrud>
Status: Awaiting Review
Review manager: TBD
<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#introduction>Introduction

As new, source-incompatible versions of Swift come into use, there is a growing need for packages to be authored in a way that makes them usable from multiple versions of Swift. While package authors want to adopt new Swift versions as soon as possible, they also need to support their existing clients.

Source incompatibilities can arise not only from changes to the language syntax, but also from changes to the Swift Standard Library and the Package Description API of the Swift Package Manager itself.

Support for multiple Swift versions could in theory be implemented using #if directives in the package source code, but that approach can become unwieldy when the required code differences are significant.

The Swift Package Manager should therefore provide facilities that make it as easy as possible for package authors to support clients using different versions of Swift. The proposal described here intends to solve an immediate need for Swift Package Manager 3; the need for version-specific packages will hopefully diminish as the language and libraries stabilize. We can revisit the need for this support in a future version of Swift.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#motivation>Motivation

It is important to allow Swift users to migrate to new Swift versions as easily as possible. At the same time, packages need to stay compatible with existing clients who are not yet ready to migrate.

A new version of Swift means a new version of the language, the Standard Library API, and SwiftPM's own Package Description API. In some cases it's possible to use #if directives to let a single source base build using different versions of Swift. When the code differences are significant, however, it's impractical to use conditional code, and some other way to differentiate is needed. This is particularly true for new versions of the Swift Package Manager, as the manifest format evolves.

Making this practical requires some improvements to the Package Manager, but to see why, it is useful to look at why current workarounds would be impractical:

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impractical-workaround-1>Impractical workaround #1:

A strategy that wouldn't require any Swift Package Manager changes would be to require package authors to tie the semantic versions(1 <http://semver.org/>) of their packages to specific versions of Swift itself. For example, a package author could decide that version 1 of their package would work only with Swift 2.3, and package version 2 would work only with Swift 3.

Using different package versions for the different Swift versions would take advantage of the Package Manager's existing version matching logic to make sure that the right package is chosen, and the right package for the Swift version would automatically be chosen.

However, this doesn't seem like a particularly acceptable restriction, since it ties the release cycles of packages to those of Swift itself. This coupling between new Swift versions and new versions of all packages in use by a client would introduce significant revlock, which may seriously impact adoption. In particular, a client that wanted or needed to migrate to a new version of a package couldn't do so until they also migrated to a new version of Swift itself. In addition, they would at the same time have to migrate to newer versions of any other package on which they depend (assuming that such newer versions even exist).

This is particularly unfortunate in cases in which only the package manifest needs to be different, since that would cause a package that could otherwise support multiple Swift versions to have to bifurcate.

What is needed is a way to allow differentiation of packages by not just their own semantic version, but also the version of Swift being used.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impractical-workaround-2>Impractical workaround #2:

Another possible strategy would be to choose package version numbers that also incorporate the version of Swift they require, thereby "flattening" the two version numbers involved (package version and Swift version) into one.

One could come up with various schemes for this, such as assigning odd-valued package version numbers to prerelease versions of the Swift language, and even-valued package version numbers to release versions of the Swift language.

But this would pollute the version space, and it also sends the wrong message about semantic versioning (which is a strategy that we want package authors to use). Encouraging the abuse of the major version number for other things than the package API is not in the best interest of the maintainability of the package ecosystem.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#solution-goals>Solution Goals

The solution needs to:

ensure that the established package ecosystem graph continues to work as-is, even as packages supporting new Swift versions are published

support parallel co-existence of actively maintained package graphs for:

the latest stable (release) version of Swift

the most recent pre-release version of Swift

any older versions of Swift that still need to be supported

(note that "Swift version" here includes not just the syntax of the language, but also the Standard Library API and the Package Description API)

The Swift Package Manager should make it as easy as possible for a package to support multiple Swift versions using a single package repository, when that is possible with respect to the magnitude of the requires source differences).

An additional, more abstract goal, is to encourage the whole Swift ecosystem to move forward as quickly as possible. This means optimizing for a workflow that involves latest-GM, current-prerelease, and current-development version, but ideally not a long tail of old GM versions.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#proposed-solution>Proposed Solution

There are two parts to the proposed solution:

provide package authors with a way to differentiate package repository version tags by Swift version, to support multiple editions of a single semantic package version

provide package authors with a way to provide multiple package manifests for a single package version tag, differentiated by Swift version

In both cases, version-based name suffixes are used to allow Package Manager to resolve dependencies based on package version as well as Swift version.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#version-differentiated-package-tags>Version-differentiated package tags

When selecting the version of a package dependency to use for a particular client, the Swift Package Manager uses a repository tag naming convention based on the specified version restrictions of the package.

For example, a client that specifies this dependency in its Package.swift:

.Package(url: "https://github.com/apple/example-package-deckofplayingcards.git",
         majorVersion: 2)
causes the package manager to look for tags in the form of a semantic version (see semver.org <http://semver.org/> for more information).

In this example, only tags having a major version number of 2 are considered, and the highest version among them is the one that is selected.

Other restrictions involving minor versions and version ranges can be specified in the client's Package.swift manifest.

Regardless of how the desired version restrictions are specified, the highest semantic version that matches the restrictions is the one that is selected when resolving dependencies.

This proposal would allow an optional Swift version to appended to the package version, separated from it by a @swift-string.

This can be used to provide two separate tags for the same package version, differentiated only by Swift version. For example, version 1.0 of MyPackage could have a 1.0@swift-2.3 tag and a 1.0@swift-3 tag.

The expected use case for this is when the differences required to support two or more versions of Swift are large enough that it would be impractical to implement them in the same checkout of the repository.

The new logic would first look for tag names having such a Swift version suffix that matches the version of Swift the client wants to use, and if found, omits from consideration any tags that do not have that suffix. The existing logic would be applied to the remaining set of tag candidates.

The format of the swift version is not itself a semantic version, but instead follows the Swift marketing versions used. For matching, the number of digits specified affects the precision of the matching; for example, @swift-3 would match any version Swift 3.x.x version, while @swift-3.0 would match only Swift 3.0.x but not Swift 3.1 etc.

The most specific version suffix matching the client's Swift version is used. For example, if both a @swift-3 and a @swift-3.1 tag are found, Swift 3.1 would use the latter and Swift 3.2 would use the former.

If no tag names have the Swift version suffix, the matching would work as it currently does, using only the package version restrictions.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#version-differentiated-package-manifests>Version-differentiated package manifests

Creating Swift-versioned tags for a particular package version has maintenance consequences. When possible, it's more maintainable for a package to support multiple Swift versions in a single tag of a repository. In the ideal case, no source changes are required at all in order to support two different Swift versions (this is the case, for example, between Swift 2.2 and Swift 2.3).

When the required changes are minimal and a single package manifest can be used, #if directives in the source code can be used to support any other differences between the Swift versions. This already works today.

For the Package.swift manifest itself, though, it can be somewhat unwieldy to express differences using only #ifdirectives. This is the case whether the differences are due to language syntax changes (recall that Package.swiftmanifests are actually Swift source files) or due to changes in the Package Description API.

To support this, this proposal would allow a Swift version to be appended to the base Package base name, separated from it by the string @swift-. For example, a package manifest specific to Swift 2.3 would have the file namePackage@swift-2.3.swift, while one that worked for any Swift 3.x version would be Package@swift-3.swift.

As with versioned tags, in the absence of a version-specific Package.swift file, the Package Manager would use the regular Package.swift file.

Often, a package author would use either version-differentiated package tags or version-differentiated package manifests, but they could also be used together when that makes sense.

It is hoped that as the Swift language stabilizes and packages eventually drop support for older versions of Swift, many packages will be able to discard the version-specific variants and keep only Package.swift.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impact-on-existing-code>Impact on existing code

There is not expected to be any impact on existing code, since this proposal adds on top of existing functionality.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#alternatives>Alternatives

Do nothing and let package authors invent their own ways. For the reasons spelled out in the Motivation section, this very undesirable. However, given how close we are to Swift 3's completion date, there is a possibility that there will not be time to implement this proposal for Swift 3.

The consequence would be that future changes to the Package Description API would cause existing packages to break, which could significantly obstruct package adoption of new versions of Swift.

Add declarations to the Package.swift manifest to specify the required Swift version or version range. This has a number of problems, including the fact that all of the various manifests would need to be checked out before deciding which to exclude from consideration. Another problems is that such minimum-version requirement declarations would have to be able to be parsed by older versions of the Package Manager, and this is not a guarantee we can make (the manifest might not be parseable using an older version of Swift).


(Honza Dvorsky) #2

Hi Anders,

thanks for the proposal, I agree this is something to be supported
explicitly in SwiftPM. I really like the Goals section and mostly support
the Solutions you proposed. Just a few comments below:

*> The new logic would first look for tag names having such a Swift version
suffix that matches the version of Swift the client wants to use, and if
found, omits from consideration any tags that do not have that suffix. The
existing logic would be applied to the remaining set of tag candidates.*

Do I understand correctly that if we have the tag list: [2.1, 2.2, 2.3] and
we're compiling swift-3, all three tags will be considered; when we add
2.1@swift-3, we now have [2.1, 2.1@swift-3, 2.2, 2.3] and only [2.1@swift-3]
will be considered? If so, I'm not sure I like that non-continuity that
suddenly takes away 2.2 and 2.3 from considered tags by adding 2.1@swift-3.
The alternative would be that when 2.1@swift-3 is added, the considered
list would be [2.1@swift-3, 2.2, 2.3] (only the non-specific 2.1 was not
considered now). I could imagine this emerging when a security fix is
needed in 2.1 and the fix needs Swift version-specific code to be applied.
I think I'd like that behavior better, but please do let me know if you've
already considered this.

*> 2. support parallel co-existence of actively maintained package graphs
for:*

···

-

   *the most recent pre-release version of Swift*

Would that work for snapshots in the form of
swift-DEVELOPMENT-SNAPSHOT-2016-07-25-a as well? For pre-release Swift, in
my projects I release new minor versions for each new snapshot, until we
get to the final release. Being able to have a tag of
1.2.3@swift-DEVELOPMENT-SNAPSHOT-2016-07-25-a is not pretty, but it would
definitely help many projects that adopt the latest snapshot at all times,
while not breaking older projects paused on older snapshots. If it is
intended to work with snapshot names, not just full Swift versions, than
I'm happy with it.

Thanks!
Honza

On Wed, Jul 27, 2016 at 9:54 AM Anders Bertelrud via swift-build-dev < swift-build-dev@swift.org> wrote:

Hello,

This is a proposal to enhance the SwiftPM package selection logic so that
packages can be differentiated based on Swift version in addition to
package version. It does not introduce any source-breaking changes, but
defines a way for package authors to annotate repositories and package
manifests so that SwiftPM can exclude those that are usable only for
particular versions of Swift.

You can find the proposal here:

https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md

and the current text is also included below.

Time is short, but feedback on this proposal is very welcome.

Thanks,

Anders

Package Manager Support for Language Versions

   - Proposal: SE-NNNN
   <https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md>
   - Author: Anders Bertelrud <https://github.com/abertelrud>
   - Status: Awaiting Review
   - Review manager: TBD

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#introduction>
Introduction

As new, source-incompatible versions of Swift come into use, there is a
growing need for packages to be authored in a way that makes them usable
from multiple versions of Swift. While package authors want to adopt new
Swift versions as soon as possible, they also need to support their
existing clients.

Source incompatibilities can arise not only from changes to the language
syntax, but also from changes to the Swift Standard Library and the Package
Description API of the Swift Package Manager itself.

Support for multiple Swift versions could in theory be implemented using
#if directives in the package source code, but that approach can become
unwieldy when the required code differences are significant.

The Swift Package Manager should therefore provide facilities that make it
as easy as possible for package authors to support clients using different
versions of Swift. The proposal described here intends to solve an
immediate need for Swift Package Manager 3; the need for version-specific
packages will hopefully diminish as the language and libraries stabilize.
We can revisit the need for this support in a future version of Swift.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#motivation>
Motivation

It is important to allow Swift users to migrate to new Swift versions as
easily as possible. At the same time, packages need to stay compatible with
existing clients who are not yet ready to migrate.

A new version of Swift means a new version of the language, the Standard
Library API, and SwiftPM's own Package Description API. In some cases it's
possible to use #if directives to let a single source base build using
different versions of Swift. When the code differences are significant,
however, it's impractical to use conditional code, and some other way to
differentiate is needed. This is particularly true for new versions of the
Swift Package Manager, as the manifest format evolves.

Making this practical requires some improvements to the Package Manager,
but to see why, it is useful to look at why current workarounds would be
impractical:

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impractical-workaround-1>Impractical
workaround #1:

A strategy that wouldn't require any Swift Package Manager changes would
be to require package authors to tie the semantic versions(1
<http://semver.org/>) of their packages to specific versions of Swift
itself. For example, a package author could decide that version 1 of their
package would work only with Swift 2.3, and package version 2 would work
only with Swift 3.

Using different package versions for the different Swift versions would
take advantage of the Package Manager's existing version matching logic to
make sure that the right package is chosen, and the right package for the
Swift version would automatically be chosen.

However, this doesn't seem like a particularly acceptable restriction,
since it ties the release cycles of packages to those of Swift itself. This
coupling between new Swift versions and new versions of all packages in use
by a client would introduce significant revlock, which may seriously impact
adoption. In particular, a client that wanted or needed to migrate to a new
version of a package couldn't do so until they also migrated to a new
version of Swift itself. In addition, they would at the same time have to
migrate to newer versions of any other package on which they depend
(assuming that such newer versions even exist).

This is particularly unfortunate in cases in which only the package
manifest needs to be different, since that would cause a package that could
otherwise support multiple Swift versions to have to bifurcate.

What is needed is a way to allow differentiation of packages by not just
their own semantic version, but also the version of Swift being used.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impractical-workaround-2>Impractical
workaround #2:

Another possible strategy would be to choose package version numbers that
also incorporate the version of Swift they require, thereby "flattening"
the two version numbers involved (package version and Swift version) into
one.

One could come up with various schemes for this, such as assigning
odd-valued package version numbers to prerelease versions of the Swift
language, and even-valued package version numbers to release versions of
the Swift language.

But this would pollute the version space, and it also sends the wrong
message about semantic versioning (which is a strategy that we want package
authors to use). Encouraging the abuse of the major version number for
other things than the package API is not in the best interest of the
maintainability of the package ecosystem.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#solution-goals>Solution
Goals

The solution needs to:

   1.

   ensure that the established package ecosystem graph continues to work
   as-is, even as packages supporting new Swift versions are published
   2.

   support parallel co-existence of actively maintained package graphs
   for:
   -

      the latest stable (release) version of Swift
      -

      the most recent pre-release version of Swift
      -

      any older versions of Swift that still need to be supported

(note that "Swift version" here includes not just the syntax of the
language, but also the Standard Library API and the Package Description API)

The Swift Package Manager should make it as easy as possible for a package
to support multiple Swift versions using a single package repository, when
that is possible with respect to the magnitude of the requires source
differences).

An additional, more abstract goal, is to encourage the whole Swift
ecosystem to move forward as quickly as possible. This means optimizing for
a workflow that involves latest-GM, current-prerelease, and
current-development version, but ideally not a long tail of old GM versions.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#proposed-solution>Proposed
Solution

There are two parts to the proposed solution:

   1.

   provide package authors with a way to differentiate package repository
   version tags by Swift version, to support multiple editions of a single
   semantic package version
   2.

   provide package authors with a way to provide multiple package
   manifests for a single package version tag, differentiated by Swift version

In both cases, version-based name suffixes are used to allow Package
Manager to resolve dependencies based on package version as well as Swift
version.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#version-differentiated-package-tags>Version-differentiated
package tags

When selecting the version of a package dependency to use for a particular
client, the Swift Package Manager uses a repository tag naming convention
based on the specified version restrictions of the package.

For example, a client that specifies this dependency in its Package.swift:

.Package(url: "https://github.com/apple/example-package-deckofplayingcards.git",
         majorVersion: 2)

causes the package manager to look for tags in the form of a semantic
version (see semver.org for more information).

In this example, only tags having a major version number of 2 are
considered, and the highest version among them is the one that is selected.

Other restrictions involving minor versions and version ranges can be
specified in the client's Package.swift manifest.

Regardless of how the desired version restrictions are specified, the
highest semantic version that matches the restrictions is the one that is
selected when resolving dependencies.

This proposal would allow an optional Swift version to appended to the
package version, separated from it by a @swift-string.

This can be used to provide two separate tags for the same package
version, differentiated only by Swift version. For example, version 1.0 of
MyPackage could have a 1.0@swift-2.3 tag and a 1.0@swift-3 tag.

The expected use case for this is when the differences required to support
two or more versions of Swift are large enough that it would be impractical
to implement them in the same checkout of the repository.

The new logic would first look for tag names having such a Swift version
suffix that matches the version of Swift the client wants to use, and if
found, omits from consideration any tags that do not have that suffix. The
existing logic would be applied to the remaining set of tag candidates.

The format of the swift version is not itself a semantic version, but
instead follows the Swift marketing versions used. For matching, the number
of digits specified affects the precision of the matching; for example,
@swift-3 would match any version Swift 3.x.x version, while @swift-3.0 would
match only Swift 3.0.x but not Swift 3.1 etc.

The most specific version suffix matching the client's Swift version is
used. For example, if both a @swift-3 and a @swift-3.1 tag are found,
Swift 3.1 would use the latter and Swift 3.2 would use the former.

If no tag names have the Swift version suffix, the matching would work as
it currently does, using only the package version restrictions.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#version-differentiated-package-manifests>Version-differentiated
package manifests

Creating Swift-versioned tags for a particular package version has
maintenance consequences. When possible, it's more maintainable for a
package to support multiple Swift versions in a single tag of a repository.
In the ideal case, no source changes are required at all in order to
support two different Swift versions (this is the case, for example,
between Swift 2.2 and Swift 2.3).

When the required changes are minimal and a single package manifest can be
used, #if directives in the source code can be used to support any other
differences between the Swift versions. This already works today.

For the Package.swift manifest itself, though, it can be somewhat
unwieldy to express differences using only #ifdirectives. This is the
case whether the differences are due to language syntax changes (recall
that Package.swiftmanifests are actually Swift source files) or due to
changes in the Package Description API.

To support this, this proposal would allow a Swift version to be appended
to the base Package base name, separated from it by the string @swift-.
For example, a package manifest specific to Swift 2.3 would have the file
namePackage@swift-2.3.swift, while one that worked for any Swift 3.x
version would be Package@swift-3.swift.

As with versioned tags, in the absence of a version-specific Package.swift file,
the Package Manager would use the regular Package.swift file.

Often, a package author would use either version-differentiated package
tags or version-differentiated package manifests, but they could also be
used together when that makes sense.

It is hoped that as the Swift language stabilizes and packages eventually
drop support for older versions of Swift, many packages will be able to
discard the version-specific variants and keep only Package.swift.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impact-on-existing-code>Impact
on existing code

There is not expected to be any impact on existing code, since this
proposal adds on top of existing functionality.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#alternatives>
Alternatives

   1.

   Do nothing and let package authors invent their own ways. For the
   reasons spelled out in the *Motivation* section, this very
   undesirable. However, given how close we are to Swift 3's completion date,
   there is a possibility that there will not be time to implement this
   proposal for Swift 3.

   The consequence would be that future changes to the Package
   Description API would cause existing packages to break, which could
   significantly obstruct package adoption of new versions of Swift.
   2.

   Add declarations to the Package.swift manifest to specify the required
   Swift version or version range. This has a number of problems, including
   the fact that all of the various manifests would need to be checked out
   before deciding which to exclude from consideration. Another problems is
   that such minimum-version requirement declarations would have to be able to
   be parsed by older versions of the Package Manager, and this is not a
   guarantee we can make (the manifest might not be parseable using an older
   version of Swift).

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev


(Rob Allen) #3

Hi,

Does this tie-in with the post from Ted?

The subject line "[swift-dev] End of source-breaking changes for Swift 3" where he says:
"The Swift team at Apple has reflected on this and decided what it "means" for Swift 3 to be source compatible with Swift 4 and later releases going forward. Our goal is to allow app developers to combine a mix of Swift modules (e.g., SwiftPM packages), where each module is known to compile with a specific version of the language (module A works with Swift 3, module B works with Swift 3.1, etc.), then combine those modules into a single binary. The key feature is that a module can be migrated from Swift 3 to 3.1 to 4 (and beyond) independently of its dependencies."

[...]

"To make this more concrete, suppose an application is written to use Swift 4, but uses packages via SwiftPM that are written using Swift 3. A single compiler would build both the app and the packages — thus ensuring that all the compiled sources are binary compatible."

Regards,

Rob...

···

On 27 Jul 2016, at 09:54, Anders Bertelrud via swift-build-dev <swift-build-dev@swift.org> wrote:

Hello,

This is a proposal to enhance the SwiftPM package selection logic so that packages can be differentiated based on Swift version in addition to package version. It does not introduce any source-breaking changes, but defines a way for package authors to annotate repositories and package manifests so that SwiftPM can exclude those that are usable only for particular versions of Swift.

You can find the proposal here:

https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md

and the current text is also included below.

Time is short, but feedback on this proposal is very welcome.

Thanks,

Anders

Package Manager Support for Language Versions

Proposal: SE-NNNN <https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md>
Author: Anders Bertelrud <https://github.com/abertelrud>
Status: Awaiting Review
Review manager: TBD
<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#introduction>Introduction

As new, source-incompatible versions of Swift come into use, there is a growing need for packages to be authored in a way that makes them usable from multiple versions of Swift. While package authors want to adopt new Swift versions as soon as possible, they also need to support their existing clients.

Source incompatibilities can arise not only from changes to the language syntax, but also from changes to the Swift Standard Library and the Package Description API of the Swift Package Manager itself.

Support for multiple Swift versions could in theory be implemented using #if directives in the package source code, but that approach can become unwieldy when the required code differences are significant.

The Swift Package Manager should therefore provide facilities that make it as easy as possible for package authors to support clients using different versions of Swift. The proposal described here intends to solve an immediate need for Swift Package Manager 3; the need for version-specific packages will hopefully diminish as the language and libraries stabilize. We can revisit the need for this support in a future version of Swift.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#motivation>Motivation

It is important to allow Swift users to migrate to new Swift versions as easily as possible. At the same time, packages need to stay compatible with existing clients who are not yet ready to migrate.

A new version of Swift means a new version of the language, the Standard Library API, and SwiftPM's own Package Description API. In some cases it's possible to use #if directives to let a single source base build using different versions of Swift. When the code differences are significant, however, it's impractical to use conditional code, and some other way to differentiate is needed. This is particularly true for new versions of the Swift Package Manager, as the manifest format evolves.

Making this practical requires some improvements to the Package Manager, but to see why, it is useful to look at why current workarounds would be impractical:

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impractical-workaround-1>Impractical workaround #1:

A strategy that wouldn't require any Swift Package Manager changes would be to require package authors to tie the semantic versions(1 <http://semver.org/>) of their packages to specific versions of Swift itself. For example, a package author could decide that version 1 of their package would work only with Swift 2.3, and package version 2 would work only with Swift 3.

Using different package versions for the different Swift versions would take advantage of the Package Manager's existing version matching logic to make sure that the right package is chosen, and the right package for the Swift version would automatically be chosen.

However, this doesn't seem like a particularly acceptable restriction, since it ties the release cycles of packages to those of Swift itself. This coupling between new Swift versions and new versions of all packages in use by a client would introduce significant revlock, which may seriously impact adoption. In particular, a client that wanted or needed to migrate to a new version of a package couldn't do so until they also migrated to a new version of Swift itself. In addition, they would at the same time have to migrate to newer versions of any other package on which they depend (assuming that such newer versions even exist).

This is particularly unfortunate in cases in which only the package manifest needs to be different, since that would cause a package that could otherwise support multiple Swift versions to have to bifurcate.

What is needed is a way to allow differentiation of packages by not just their own semantic version, but also the version of Swift being used.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impractical-workaround-2>Impractical workaround #2:

Another possible strategy would be to choose package version numbers that also incorporate the version of Swift they require, thereby "flattening" the two version numbers involved (package version and Swift version) into one.

One could come up with various schemes for this, such as assigning odd-valued package version numbers to prerelease versions of the Swift language, and even-valued package version numbers to release versions of the Swift language.

But this would pollute the version space, and it also sends the wrong message about semantic versioning (which is a strategy that we want package authors to use). Encouraging the abuse of the major version number for other things than the package API is not in the best interest of the maintainability of the package ecosystem.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#solution-goals>Solution Goals

The solution needs to:

ensure that the established package ecosystem graph continues to work as-is, even as packages supporting new Swift versions are published

support parallel co-existence of actively maintained package graphs for:

the latest stable (release) version of Swift

the most recent pre-release version of Swift

any older versions of Swift that still need to be supported

(note that "Swift version" here includes not just the syntax of the language, but also the Standard Library API and the Package Description API)

The Swift Package Manager should make it as easy as possible for a package to support multiple Swift versions using a single package repository, when that is possible with respect to the magnitude of the requires source differences).

An additional, more abstract goal, is to encourage the whole Swift ecosystem to move forward as quickly as possible. This means optimizing for a workflow that involves latest-GM, current-prerelease, and current-development version, but ideally not a long tail of old GM versions.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#proposed-solution>Proposed Solution

There are two parts to the proposed solution:

provide package authors with a way to differentiate package repository version tags by Swift version, to support multiple editions of a single semantic package version

provide package authors with a way to provide multiple package manifests for a single package version tag, differentiated by Swift version

In both cases, version-based name suffixes are used to allow Package Manager to resolve dependencies based on package version as well as Swift version.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#version-differentiated-package-tags>Version-differentiated package tags

When selecting the version of a package dependency to use for a particular client, the Swift Package Manager uses a repository tag naming convention based on the specified version restrictions of the package.

For example, a client that specifies this dependency in its Package.swift:

.Package(url: "https://github.com/apple/example-package-deckofplayingcards.git",
         majorVersion: 2)
causes the package manager to look for tags in the form of a semantic version (see semver.org <http://semver.org/> for more information).

In this example, only tags having a major version number of 2 are considered, and the highest version among them is the one that is selected.

Other restrictions involving minor versions and version ranges can be specified in the client's Package.swift manifest.

Regardless of how the desired version restrictions are specified, the highest semantic version that matches the restrictions is the one that is selected when resolving dependencies.

This proposal would allow an optional Swift version to appended to the package version, separated from it by a @swift-string.

This can be used to provide two separate tags for the same package version, differentiated only by Swift version. For example, version 1.0 of MyPackage could have a 1.0@swift-2.3 tag and a 1.0@swift-3 tag.

The expected use case for this is when the differences required to support two or more versions of Swift are large enough that it would be impractical to implement them in the same checkout of the repository.

The new logic would first look for tag names having such a Swift version suffix that matches the version of Swift the client wants to use, and if found, omits from consideration any tags that do not have that suffix. The existing logic would be applied to the remaining set of tag candidates.

The format of the swift version is not itself a semantic version, but instead follows the Swift marketing versions used. For matching, the number of digits specified affects the precision of the matching; for example, @swift-3 would match any version Swift 3.x.x version, while @swift-3.0 would match only Swift 3.0.x but not Swift 3.1 etc.

The most specific version suffix matching the client's Swift version is used. For example, if both a @swift-3 and a @swift-3.1 tag are found, Swift 3.1 would use the latter and Swift 3.2 would use the former.

If no tag names have the Swift version suffix, the matching would work as it currently does, using only the package version restrictions.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#version-differentiated-package-manifests>Version-differentiated package manifests

Creating Swift-versioned tags for a particular package version has maintenance consequences. When possible, it's more maintainable for a package to support multiple Swift versions in a single tag of a repository. In the ideal case, no source changes are required at all in order to support two different Swift versions (this is the case, for example, between Swift 2.2 and Swift 2.3).

When the required changes are minimal and a single package manifest can be used, #if directives in the source code can be used to support any other differences between the Swift versions. This already works today.

For the Package.swift manifest itself, though, it can be somewhat unwieldy to express differences using only #ifdirectives. This is the case whether the differences are due to language syntax changes (recall that Package.swiftmanifests are actually Swift source files) or due to changes in the Package Description API.

To support this, this proposal would allow a Swift version to be appended to the base Package base name, separated from it by the string @swift-. For example, a package manifest specific to Swift 2.3 would have the file namePackage@swift-2.3.swift <mailto:Package@swift-2.3.swift>, while one that worked for any Swift 3.x version would be Package@swift-3.swift <mailto:Package@swift-3.swift>.

As with versioned tags, in the absence of a version-specific Package.swift file, the Package Manager would use the regular Package.swift file.

Often, a package author would use either version-differentiated package tags or version-differentiated package manifests, but they could also be used together when that makes sense.

It is hoped that as the Swift language stabilizes and packages eventually drop support for older versions of Swift, many packages will be able to discard the version-specific variants and keep only Package.swift.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impact-on-existing-code>Impact on existing code

There is not expected to be any impact on existing code, since this proposal adds on top of existing functionality.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#alternatives>Alternatives

Do nothing and let package authors invent their own ways. For the reasons spelled out in the Motivation section, this very undesirable. However, given how close we are to Swift 3's completion date, there is a possibility that there will not be time to implement this proposal for Swift 3.

The consequence would be that future changes to the Package Description API would cause existing packages to break, which could significantly obstruct package adoption of new versions of Swift.

Add declarations to the Package.swift manifest to specify the required Swift version or version range. This has a number of problems, including the fact that all of the various manifests would need to be checked out before deciding which to exclude from consideration. Another problems is that such minimum-version requirement declarations would have to be able to be parsed by older versions of the Package Manager, and this is not a guarantee we can make (the manifest might not be parseable using an older version of Swift).

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev

--
Development thoughts at http://akrabat.com
Daily Jotter for Mac OS X at http://dailyjotter.com


(Anders Bertelrud) #4

Hello Honza,

thanks for the proposal, I agree this is something to be supported explicitly in SwiftPM. I really like the Goals section and mostly support the Solutions

Great! Thanks for taking the time to comment. Responses inline.

you proposed. Just a few comments below:

> The new logic would first look for tag names having such a Swift version suffix that matches the version of Swift the client wants to use, and if found, omits from consideration any tags that do not have that suffix. The existing logic would be applied to the remaining set of tag candidates.

Do I understand correctly that if we have the tag list: [2.1, 2.2, 2.3] and we're compiling swift-3, all three tags will be considered; when we add 2.1@swift-3, we now have [2.1, 2.1@swift-3, 2.2, 2.3] and only [2.1@swift-3] will be considered?

In your example, would 2.1 be the only one that has to have a SwiftPM 3 specific edition? I wonder how realistic that is... would 2.2 and 2.3 have been modified to support both SwiftPM 3 and SwiftPM 4, or would 2.2 and 2.3 have dropped support for SwiftPM 3?

If so, I'm not sure I like that non-continuity that suddenly takes away 2.2 and 2.3 from considered tags by adding 2.1@swift-3.

I agree that it does feel a bit odd, but at least it's entirely under the package author's control. Once they start needing to differentiate based on Swift version, they might need to add more differentiation to existing tags, that's true.

The alternative would be that when 2.1@swift-3 is added, the considered list would be [2.1@swift-3, 2.2, 2.3] (only the non-specific 2.1 was not considered now). I could imagine this emerging when a security fix is needed in 2.1 and the fix needs Swift version-specific code to be applied.

For small fixes, we expect that a multi version project would be a much better solution, so in that case the tag shouldn't be versioned at all. Versioning the tag is really quite a big hammer, as the proposal notes.

I think I'd like that behavior better, but please do let me know if you've already considered this.

We did indeed consider this, and the problem is that in this case, SwiftPM 3 won't know that it cannot use 2.2 or 2.3 (assuming that that's the case). So there would be no way to say that the highest version that SwiftPM 3 can match against is 2.1, but SwiftPM 4 (for example) could use 2.2 or 2.3.

> 2. support parallel co-existence of actively maintained package graphs for:
the most recent pre-release version of Swift

Would that work for snapshots in the form of swift-DEVELOPMENT-SNAPSHOT-2016-07-25-a as well? For pre-release Swift, in my projects I release new minor versions for each new snapshot, until we get to the final release. Being able to have a tag of 1.2.3@swift-DEVELOPMENT-SNAPSHOT-2016-07-25-a is not pretty, but it would definitely help many projects that adopt the latest snapshot at all times, while not breaking older projects paused on older snapshots. If it is intended to work with snapshot names, not just full Swift versions, than I'm happy with it.

We discussed this, but that is not currently a part of the proposal. Part of the problem is that the naming of snapshots isn't a formal format. The proposal as written is definitely geared toward major versions, with the expectation that people will update to new snapshots fairly quickly (much quicker than going from a stable GM release to a prerelease).

Anders

···

On 2016-07-27, at 13.06, Honza Dvorsky <jan.dvorsky@me.com> wrote:

Thanks!
Honza

On Wed, Jul 27, 2016 at 9:54 AM Anders Bertelrud via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:
Hello,

This is a proposal to enhance the SwiftPM package selection logic so that packages can be differentiated based on Swift version in addition to package version. It does not introduce any source-breaking changes, but defines a way for package authors to annotate repositories and package manifests so that SwiftPM can exclude those that are usable only for particular versions of Swift.

You can find the proposal here:

https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md

and the current text is also included below.

Time is short, but feedback on this proposal is very welcome.

Thanks,

Anders

Package Manager Support for Language Versions

Proposal: SE-NNNN <https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md>
Author: Anders Bertelrud <https://github.com/abertelrud>
Status: Awaiting Review
Review manager: TBD
<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#introduction>Introduction

As new, source-incompatible versions of Swift come into use, there is a growing need for packages to be authored in a way that makes them usable from multiple versions of Swift. While package authors want to adopt new Swift versions as soon as possible, they also need to support their existing clients.

Source incompatibilities can arise not only from changes to the language syntax, but also from changes to the Swift Standard Library and the Package Description API of the Swift Package Manager itself.

Support for multiple Swift versions could in theory be implemented using #if directives in the package source code, but that approach can become unwieldy when the required code differences are significant.

The Swift Package Manager should therefore provide facilities that make it as easy as possible for package authors to support clients using different versions of Swift. The proposal described here intends to solve an immediate need for Swift Package Manager 3; the need for version-specific packages will hopefully diminish as the language and libraries stabilize. We can revisit the need for this support in a future version of Swift.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#motivation>Motivation

It is important to allow Swift users to migrate to new Swift versions as easily as possible. At the same time, packages need to stay compatible with existing clients who are not yet ready to migrate.

A new version of Swift means a new version of the language, the Standard Library API, and SwiftPM's own Package Description API. In some cases it's possible to use #if directives to let a single source base build using different versions of Swift. When the code differences are significant, however, it's impractical to use conditional code, and some other way to differentiate is needed. This is particularly true for new versions of the Swift Package Manager, as the manifest format evolves.

Making this practical requires some improvements to the Package Manager, but to see why, it is useful to look at why current workarounds would be impractical:

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impractical-workaround-1>Impractical workaround #1:

A strategy that wouldn't require any Swift Package Manager changes would be to require package authors to tie the semantic versions(1 <http://semver.org/>) of their packages to specific versions of Swift itself. For example, a package author could decide that version 1 of their package would work only with Swift 2.3, and package version 2 would work only with Swift 3.

Using different package versions for the different Swift versions would take advantage of the Package Manager's existing version matching logic to make sure that the right package is chosen, and the right package for the Swift version would automatically be chosen.

However, this doesn't seem like a particularly acceptable restriction, since it ties the release cycles of packages to those of Swift itself. This coupling between new Swift versions and new versions of all packages in use by a client would introduce significant revlock, which may seriously impact adoption. In particular, a client that wanted or needed to migrate to a new version of a package couldn't do so until they also migrated to a new version of Swift itself. In addition, they would at the same time have to migrate to newer versions of any other package on which they depend (assuming that such newer versions even exist).

This is particularly unfortunate in cases in which only the package manifest needs to be different, since that would cause a package that could otherwise support multiple Swift versions to have to bifurcate.

What is needed is a way to allow differentiation of packages by not just their own semantic version, but also the version of Swift being used.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impractical-workaround-2>Impractical workaround #2:

Another possible strategy would be to choose package version numbers that also incorporate the version of Swift they require, thereby "flattening" the two version numbers involved (package version and Swift version) into one.

One could come up with various schemes for this, such as assigning odd-valued package version numbers to prerelease versions of the Swift language, and even-valued package version numbers to release versions of the Swift language.

But this would pollute the version space, and it also sends the wrong message about semantic versioning (which is a strategy that we want package authors to use). Encouraging the abuse of the major version number for other things than the package API is not in the best interest of the maintainability of the package ecosystem.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#solution-goals>Solution Goals

The solution needs to:

ensure that the established package ecosystem graph continues to work as-is, even as packages supporting new Swift versions are published

support parallel co-existence of actively maintained package graphs for:

the latest stable (release) version of Swift

the most recent pre-release version of Swift

any older versions of Swift that still need to be supported

(note that "Swift version" here includes not just the syntax of the language, but also the Standard Library API and the Package Description API)

The Swift Package Manager should make it as easy as possible for a package to support multiple Swift versions using a single package repository, when that is possible with respect to the magnitude of the requires source differences).

An additional, more abstract goal, is to encourage the whole Swift ecosystem to move forward as quickly as possible. This means optimizing for a workflow that involves latest-GM, current-prerelease, and current-development version, but ideally not a long tail of old GM versions.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#proposed-solution>Proposed Solution

There are two parts to the proposed solution:

provide package authors with a way to differentiate package repository version tags by Swift version, to support multiple editions of a single semantic package version

provide package authors with a way to provide multiple package manifests for a single package version tag, differentiated by Swift version

In both cases, version-based name suffixes are used to allow Package Manager to resolve dependencies based on package version as well as Swift version.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#version-differentiated-package-tags>Version-differentiated package tags

When selecting the version of a package dependency to use for a particular client, the Swift Package Manager uses a repository tag naming convention based on the specified version restrictions of the package.

For example, a client that specifies this dependency in its Package.swift:

.Package(url: "https://github.com/apple/example-package-deckofplayingcards.git",
         majorVersion: 2)
causes the package manager to look for tags in the form of a semantic version (see semver.org <http://semver.org/> for more information).

In this example, only tags having a major version number of 2 are considered, and the highest version among them is the one that is selected.

Other restrictions involving minor versions and version ranges can be specified in the client's Package.swift manifest.

Regardless of how the desired version restrictions are specified, the highest semantic version that matches the restrictions is the one that is selected when resolving dependencies.

This proposal would allow an optional Swift version to appended to the package version, separated from it by a @swift-string.

This can be used to provide two separate tags for the same package version, differentiated only by Swift version. For example, version 1.0 of MyPackage could have a 1.0@swift-2.3 tag and a 1.0@swift-3 tag.

The expected use case for this is when the differences required to support two or more versions of Swift are large enough that it would be impractical to implement them in the same checkout of the repository.

The new logic would first look for tag names having such a Swift version suffix that matches the version of Swift the client wants to use, and if found, omits from consideration any tags that do not have that suffix. The existing logic would be applied to the remaining set of tag candidates.

The format of the swift version is not itself a semantic version, but instead follows the Swift marketing versions used. For matching, the number of digits specified affects the precision of the matching; for example, @swift-3 would match any version Swift 3.x.x version, while @swift-3.0 would match only Swift 3.0.x but not Swift 3.1 etc.

The most specific version suffix matching the client's Swift version is used. For example, if both a @swift-3 and a @swift-3.1 tag are found, Swift 3.1 would use the latter and Swift 3.2 would use the former.

If no tag names have the Swift version suffix, the matching would work as it currently does, using only the package version restrictions.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#version-differentiated-package-manifests>Version-differentiated package manifests

Creating Swift-versioned tags for a particular package version has maintenance consequences. When possible, it's more maintainable for a package to support multiple Swift versions in a single tag of a repository. In the ideal case, no source changes are required at all in order to support two different Swift versions (this is the case, for example, between Swift 2.2 and Swift 2.3).

When the required changes are minimal and a single package manifest can be used, #if directives in the source code can be used to support any other differences between the Swift versions. This already works today.

For the Package.swift manifest itself, though, it can be somewhat unwieldy to express differences using only #ifdirectives. This is the case whether the differences are due to language syntax changes (recall that Package.swiftmanifests are actually Swift source files) or due to changes in the Package Description API.

To support this, this proposal would allow a Swift version to be appended to the base Package base name, separated from it by the string @swift-. For example, a package manifest specific to Swift 2.3 would have the file namePackage@swift-2.3.swift <mailto:Package@swift-2.3.swift>, while one that worked for any Swift 3.x version would be Package@swift-3.swift <mailto:Package@swift-3.swift>.

As with versioned tags, in the absence of a version-specific Package.swift file, the Package Manager would use the regular Package.swift file.

Often, a package author would use either version-differentiated package tags or version-differentiated package manifests, but they could also be used together when that makes sense.

It is hoped that as the Swift language stabilizes and packages eventually drop support for older versions of Swift, many packages will be able to discard the version-specific variants and keep only Package.swift.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impact-on-existing-code>Impact on existing code

There is not expected to be any impact on existing code, since this proposal adds on top of existing functionality.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#alternatives>Alternatives

Do nothing and let package authors invent their own ways. For the reasons spelled out in the Motivation section, this very undesirable. However, given how close we are to Swift 3's completion date, there is a possibility that there will not be time to implement this proposal for Swift 3.

The consequence would be that future changes to the Package Description API would cause existing packages to break, which could significantly obstruct package adoption of new versions of Swift.

Add declarations to the Package.swift manifest to specify the required Swift version or version range. This has a number of problems, including the fact that all of the various manifests would need to be checked out before deciding which to exclude from consideration. Another problems is that such minimum-version requirement declarations would have to be able to be parsed by older versions of the Package Manager, and this is not a guarantee we can make (the manifest might not be parseable using an older version of Swift).

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev


(Daniel Dunbar) #5

Can you clarify your question. What do you mean "does this tie-in"?

Yes, they definitely interact. However, this proposal is largely targeted at making sure SwiftPM 3.0 _can_ remain viable for some period of time, and it will not have the features mentioned here. So what we need is a mechanism by which package authors can make the graph continue to work with SwiftPM 3.0 while still being able to move forward. Does that help clarify?

- Daniel

···

On Jul 27, 2016, at 11:11 PM, Rob Allen via swift-build-dev <swift-build-dev@swift.org> wrote:

Hi,

Does this tie-in with the post from Ted?

The subject line "[swift-dev] End of source-breaking changes for Swift 3" where he says:
"The Swift team at Apple has reflected on this and decided what it "means" for Swift 3 to be source compatible with Swift 4 and later releases going forward. Our goal is to allow app developers to combine a mix of Swift modules (e.g., SwiftPM packages), where each module is known to compile with a specific version of the language (module A works with Swift 3, module B works with Swift 3.1, etc.), then combine those modules into a single binary. The key feature is that a module can be migrated from Swift 3 to 3.1 to 4 (and beyond) independently of its dependencies."

[...]

"To make this more concrete, suppose an application is written to use Swift 4, but uses packages via SwiftPM that are written using Swift 3. A single compiler would build both the app and the packages — thus ensuring that all the compiled sources are binary compatible."

Regards,

Rob...

On 27 Jul 2016, at 09:54, Anders Bertelrud via swift-build-dev <swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>> wrote:

Hello,

This is a proposal to enhance the SwiftPM package selection logic so that packages can be differentiated based on Swift version in addition to package version. It does not introduce any source-breaking changes, but defines a way for package authors to annotate repositories and package manifests so that SwiftPM can exclude those that are usable only for particular versions of Swift.

You can find the proposal here:

https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md

and the current text is also included below.

Time is short, but feedback on this proposal is very welcome.

Thanks,

Anders

Package Manager Support for Language Versions

Proposal: SE-NNNN <https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md>
Author: Anders Bertelrud <https://github.com/abertelrud>
Status: Awaiting Review
Review manager: TBD
<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#introduction>Introduction

As new, source-incompatible versions of Swift come into use, there is a growing need for packages to be authored in a way that makes them usable from multiple versions of Swift. While package authors want to adopt new Swift versions as soon as possible, they also need to support their existing clients.

Source incompatibilities can arise not only from changes to the language syntax, but also from changes to the Swift Standard Library and the Package Description API of the Swift Package Manager itself.

Support for multiple Swift versions could in theory be implemented using #if directives in the package source code, but that approach can become unwieldy when the required code differences are significant.

The Swift Package Manager should therefore provide facilities that make it as easy as possible for package authors to support clients using different versions of Swift. The proposal described here intends to solve an immediate need for Swift Package Manager 3; the need for version-specific packages will hopefully diminish as the language and libraries stabilize. We can revisit the need for this support in a future version of Swift.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#motivation>Motivation

It is important to allow Swift users to migrate to new Swift versions as easily as possible. At the same time, packages need to stay compatible with existing clients who are not yet ready to migrate.

A new version of Swift means a new version of the language, the Standard Library API, and SwiftPM's own Package Description API. In some cases it's possible to use #if directives to let a single source base build using different versions of Swift. When the code differences are significant, however, it's impractical to use conditional code, and some other way to differentiate is needed. This is particularly true for new versions of the Swift Package Manager, as the manifest format evolves.

Making this practical requires some improvements to the Package Manager, but to see why, it is useful to look at why current workarounds would be impractical:

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impractical-workaround-1>Impractical workaround #1:

A strategy that wouldn't require any Swift Package Manager changes would be to require package authors to tie the semantic versions(1 <http://semver.org/>) of their packages to specific versions of Swift itself. For example, a package author could decide that version 1 of their package would work only with Swift 2.3, and package version 2 would work only with Swift 3.

Using different package versions for the different Swift versions would take advantage of the Package Manager's existing version matching logic to make sure that the right package is chosen, and the right package for the Swift version would automatically be chosen.

However, this doesn't seem like a particularly acceptable restriction, since it ties the release cycles of packages to those of Swift itself. This coupling between new Swift versions and new versions of all packages in use by a client would introduce significant revlock, which may seriously impact adoption. In particular, a client that wanted or needed to migrate to a new version of a package couldn't do so until they also migrated to a new version of Swift itself. In addition, they would at the same time have to migrate to newer versions of any other package on which they depend (assuming that such newer versions even exist).

This is particularly unfortunate in cases in which only the package manifest needs to be different, since that would cause a package that could otherwise support multiple Swift versions to have to bifurcate.

What is needed is a way to allow differentiation of packages by not just their own semantic version, but also the version of Swift being used.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impractical-workaround-2>Impractical workaround #2:

Another possible strategy would be to choose package version numbers that also incorporate the version of Swift they require, thereby "flattening" the two version numbers involved (package version and Swift version) into one.

One could come up with various schemes for this, such as assigning odd-valued package version numbers to prerelease versions of the Swift language, and even-valued package version numbers to release versions of the Swift language.

But this would pollute the version space, and it also sends the wrong message about semantic versioning (which is a strategy that we want package authors to use). Encouraging the abuse of the major version number for other things than the package API is not in the best interest of the maintainability of the package ecosystem.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#solution-goals>Solution Goals

The solution needs to:

ensure that the established package ecosystem graph continues to work as-is, even as packages supporting new Swift versions are published

support parallel co-existence of actively maintained package graphs for:

the latest stable (release) version of Swift

the most recent pre-release version of Swift

any older versions of Swift that still need to be supported

(note that "Swift version" here includes not just the syntax of the language, but also the Standard Library API and the Package Description API)

The Swift Package Manager should make it as easy as possible for a package to support multiple Swift versions using a single package repository, when that is possible with respect to the magnitude of the requires source differences).

An additional, more abstract goal, is to encourage the whole Swift ecosystem to move forward as quickly as possible. This means optimizing for a workflow that involves latest-GM, current-prerelease, and current-development version, but ideally not a long tail of old GM versions.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#proposed-solution>Proposed Solution

There are two parts to the proposed solution:

provide package authors with a way to differentiate package repository version tags by Swift version, to support multiple editions of a single semantic package version

provide package authors with a way to provide multiple package manifests for a single package version tag, differentiated by Swift version

In both cases, version-based name suffixes are used to allow Package Manager to resolve dependencies based on package version as well as Swift version.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#version-differentiated-package-tags>Version-differentiated package tags

When selecting the version of a package dependency to use for a particular client, the Swift Package Manager uses a repository tag naming convention based on the specified version restrictions of the package.

For example, a client that specifies this dependency in its Package.swift:

.Package(url: "https://github.com/apple/example-package-deckofplayingcards.git",
         majorVersion: 2)
causes the package manager to look for tags in the form of a semantic version (see semver.org <http://semver.org/> for more information).

In this example, only tags having a major version number of 2 are considered, and the highest version among them is the one that is selected.

Other restrictions involving minor versions and version ranges can be specified in the client's Package.swift manifest.

Regardless of how the desired version restrictions are specified, the highest semantic version that matches the restrictions is the one that is selected when resolving dependencies.

This proposal would allow an optional Swift version to appended to the package version, separated from it by a @swift-string.

This can be used to provide two separate tags for the same package version, differentiated only by Swift version. For example, version 1.0 of MyPackage could have a 1.0@swift-2.3 tag and a 1.0@swift-3 tag.

The expected use case for this is when the differences required to support two or more versions of Swift are large enough that it would be impractical to implement them in the same checkout of the repository.

The new logic would first look for tag names having such a Swift version suffix that matches the version of Swift the client wants to use, and if found, omits from consideration any tags that do not have that suffix. The existing logic would be applied to the remaining set of tag candidates.

The format of the swift version is not itself a semantic version, but instead follows the Swift marketing versions used. For matching, the number of digits specified affects the precision of the matching; for example, @swift-3 would match any version Swift 3.x.x version, while @swift-3.0 would match only Swift 3.0.x but not Swift 3.1 etc.

The most specific version suffix matching the client's Swift version is used. For example, if both a @swift-3 and a @swift-3.1 tag are found, Swift 3.1 would use the latter and Swift 3.2 would use the former.

If no tag names have the Swift version suffix, the matching would work as it currently does, using only the package version restrictions.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#version-differentiated-package-manifests>Version-differentiated package manifests

Creating Swift-versioned tags for a particular package version has maintenance consequences. When possible, it's more maintainable for a package to support multiple Swift versions in a single tag of a repository. In the ideal case, no source changes are required at all in order to support two different Swift versions (this is the case, for example, between Swift 2.2 and Swift 2.3).

When the required changes are minimal and a single package manifest can be used, #if directives in the source code can be used to support any other differences between the Swift versions. This already works today.

For the Package.swift manifest itself, though, it can be somewhat unwieldy to express differences using only #ifdirectives. This is the case whether the differences are due to language syntax changes (recall that Package.swiftmanifests are actually Swift source files) or due to changes in the Package Description API.

To support this, this proposal would allow a Swift version to be appended to the base Package base name, separated from it by the string @swift-. For example, a package manifest specific to Swift 2.3 would have the file namePackage@swift-2.3.swift <mailto:Package@swift-2.3.swift>, while one that worked for any Swift 3.x version would be Package@swift-3.swift <mailto:Package@swift-3.swift>.

As with versioned tags, in the absence of a version-specific Package.swift file, the Package Manager would use the regular Package.swift file.

Often, a package author would use either version-differentiated package tags or version-differentiated package manifests, but they could also be used together when that makes sense.

It is hoped that as the Swift language stabilizes and packages eventually drop support for older versions of Swift, many packages will be able to discard the version-specific variants and keep only Package.swift.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#impact-on-existing-code>Impact on existing code

There is not expected to be any impact on existing code, since this proposal adds on top of existing functionality.

<https://github.com/abertelrud/swift-evolution/blob/swiftpm-language-version-support/proposals/NNNN-package-manager-support-for-language-versions.md#alternatives>Alternatives

Do nothing and let package authors invent their own ways. For the reasons spelled out in the Motivation section, this very undesirable. However, given how close we are to Swift 3's completion date, there is a possibility that there will not be time to implement this proposal for Swift 3.

The consequence would be that future changes to the Package Description API would cause existing packages to break, which could significantly obstruct package adoption of new versions of Swift.

Add declarations to the Package.swift manifest to specify the required Swift version or version range. This has a number of problems, including the fact that all of the various manifests would need to be checked out before deciding which to exclude from consideration. Another problems is that such minimum-version requirement declarations would have to be able to be parsed by older versions of the Package Manager, and this is not a guarantee we can make (the manifest might not be parseable using an older version of Swift).

_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org <mailto:swift-build-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-build-dev

--
Development thoughts at http://akrabat.com <http://akrabat.com/>
Daily Jotter for Mac OS X at http://dailyjotter.com <http://dailyjotter.com/>
_______________________________________________
swift-build-dev mailing list
swift-build-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-build-dev


(Rob Allen) #6

It does, thanks.

Rob...

···

On 28 Jul 2016, at 18:37, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

Can you clarify your question. What do you mean "does this tie-in"?

Yes, they definitely interact. However, this proposal is largely targeted at making sure SwiftPM 3.0 _can_ remain viable for some period of time, and it will not have the features mentioned here. So what we need is a mechanism by which package authors can make the graph continue to work with SwiftPM 3.0 while still being able to move forward. Does that help clarify?