RFC: Deprecating version specific manifests

Development Package Manager
1

Hi folks,

I just opened a PR to add a deprecation warning for the presence of version-specific manifests: Deprecate version-specific manifests by neonichu · Pull Request #3120 · apple/swift-package-manager · GitHub, the motivation here being to simplify our story regarding support for different Swift tools versions. For reference, this feature was introduced in SE-0135.

Much like RFC: Deprecating generate-xcodeproj, I'd like to use this thread to gather any feedback from people who are currently using the feature to see how we can address those needs going forward.

2

If adding the feature had to go through evolution, wouldn't removing it need to as well? If so, it seems like this should be a pitch.

Given that there's no alternative to this functionality, removing it without some equivalent seems like a bad idea. Alamofire's (and AFNetworking's) use of this feature is the obvious: allow adoption of new package features while still supporting older toolsets. This also enables support for new SPM capabilities, like test only dependencies, where older package versions would break the feature generally.

2 Likes
3

Thanks for the feedback @Jon_Shier

We are considering deprecating this feature to reduce complexity for both users and maintainers of SwiftPM. The goal of this thread is to help determine if the feature still provides material value or can be deprecated. If the conclusion is its no longer providing material value than a more formal process can take place to actually deprecate it.

IIUC version-specific-manifests (SE-0135) were added before SwiftPM tools-version (SE-0152) was introduced, and they both provide similar and overlapping functionality. In other words, the core of the question here is if tools-version (in conjunction with semantic versioning) is a strong enough alternative to version-specific-manifests, or if there are compelling use cases in which version-specific-manifests are the only solution.

When a new version of Swift/SwiftPM is released, we believe it is typical for package authors to want and take advantage of the new features in both the Language and SwiftPM, and as such the package's codebase as a whole becomes aligned with the Swift/SwiftPM release. Obviously, package authors need to support older versions of Swift/SwiftPM for existing clients, but for that they could/should use semantic versioning and keep the Package.swift for those versions aligned with the Swift/SwiftPM version that package version is designed to support. To emphasize, the core assumption here is that the whole codebase becomes aligned with a Swift/SwiftPM release, and its unlikely to only make changes in Package.swift without also making changes to other parts of the source corresponding to those SwiftPM features. Understanding if this assumption is correct would be helpful to determine if version-specific-manifests are still materially useful.

To give one concrete example where tools-version + semantic versioning is used in practice:
SwiftNIO 1.x is aligned to tools-version 4: https://github.com/apple/swift-nio/blob/nio-1.14/Package.swift#L1
SwiftNIO 2.x is aligned to tools-version 5: https://github.com/apple/swift-nio/blob/nio-2.16/Package.swift#L1

4

How does tools-version replace version specific manifests when you can only have one tools-version in a manifest? It's not just the version specificity that is beneficial, it's the ability to provide multiple versions at the same time.

This isn't true for any library which provides support for Swift versions more than 6 months at a time, at least if the library wants to ship any early support for some package feature. It also isn't true for tools-versions when there are new features (e.g. a package may wish to support test-only dependencies and thus need a 5.2 (or 5.3?) tools-version, but supports back to Swift 5).

In general, I'm not sure it's tenable to make the tools-version another source of major version changes, but I guess that's a larger discussion. What kind of Swift version requirement change corresponds to which semantic versioning rule?

1 Like
5

Echoing what Jon is saying.

Swift Protobuf currently supports back to Swift 4.2 (GitHub - apple/swift-protobuf: Plugin and runtime library for using protobuf with Swift), it has a policy of always supporting back atleast one major version (swift-protobuf/INTERNALS.md at main · apple/swift-protobuf · GitHub).

In the past to do this, it has required having version specific manifests. Right now we happen to be able to get with one Package.json claiming 4.2 support, but with proposals like the executableTarget one, I can see the need for version specific ones again in the future.

The only other option I'm aware of would be to use versioned branches, but that's a much larger burden on SDK authors as it means you have to maintain multiple copies of the code instead of just an extra file or two.

The recent addition of resources support to SwiftPM is another example where the version support is useful/needed. On some Google projects, we were going to have to use Package.swift with the versioned feature support so we can add the tests (they use resources), but still support older Swift versions by having a second Package.swift without the tests for those users on the older toolchains.

1 Like
6

I'm surprised that the feature is deprecated without providing any alternatives, especially as it was introduced through the evolution process. I also would expect the removal of the feature to go through evolution review as well.

I currently maintain multiple packages that require version-specific manifests to support Swift 5.0 and Swift 5.1 (as those versions don't allow platform requirements in target dependencies) and even have a couple of packages that go as far back as Swift 4.2.

Deprecating and removing this without providing an adequate replacement for this would make it impossible to support old versions of Swift in such packages. We have enough breaking changes in every Swift version, so supporting old versions in packages is critical. In my opinion, removing support for this harms the ecosystem.

1 Like
7

I agree, this would be a harmful change. In OpenCombine we still support Swift 5.0, but we need to use the more recent package manifest features like conditional target build settings to support Wasm. So we have to have multiple package manifest files, since the .wasi static property of the Platform struct from the PackageDescription module has only been introduced in Swift 5.3.

I believe this is an important case.

It is very unfortunate to see the trend of dropping important features for the sake of “simplicity” without providing an alternative. This makes the development of nontrivial packages more and more unpleasant. It is already hard enough.

2 Likes
8

I don't think that it even requires a non-trivial package to stumble upon this problem. Things that I maintain that support Swift 4.2 are fairly straightforward. Target-based dependency resolution introduced in Swift 5.2 is very valuable even for simplest packages that want to get something better than XCTest for their testing needs. Getting that to work and supporting Swift 4.2 at the same time is going to be impossible if support for version-specific manifests is dropped.

9

Also introduced (or at least mentioned) in SE-0135 are Version-differentiated package tags (e.g. 1.0.0@swift-4.2). Are we considering deprecating this feature as well? Or is the intention for version-specific tags to be an alternative?

For what it's worth, I only recently learned about version-specific tags, having never seen them in the wild. Given the choice between the two, I prefer version-specific manifests, which is primarily motivated by version-specific tags being SemVer-incompatible.

One other point: This deprecation would impact SE-0292, which is currently under review. The endpoint for fetching package release manifests takes a swift-version query parameter. To avoid an immediate revision to the specification, should we take that part out before SE-0292 is accepted?

2 Likes
10

thank you @Max_Desiatov @broadway_lamb @mattt for the feedback, to be clear we are not deprecating version-specific-manifests yet, but rather exploring the implications of deprecating it with the community given there are other ways to achieve the same thing. In other word, our goal with this thread is to help determine if the feature still provides material value or can be deprecated. If the conclusion is its no longer providing material value than a more formal process (ie evolution) will take place.

To give more background, there are 3 different ways today to manage tools-version specific manifests:

  1. version-specific-manifests introduced in SE-0135
  2. version-specific-tags introduced in SE-0135
  3. Package.swift tools-version introduced in SE-0152 (in conjunction with semantic versioning)

Theoretically, using tools-version in conjunction with semantic versioning (as demonstrated in the SwiftNIO example above) should cover all use cases, but as few folks pointed out on this thread, that approach only makes sense when the codebase as a whole changes with the SwiftPM/Swift version, and there are cases (e.g. supporting for 5.2 just for target-based dependencies) where that may not be the case, so could be too heavy handed.

11

I'm hoping this functionality remains for a long time, as it can be really useful when major changes are introduced by new Swift versions.

An example where "Version-differentiated package manifests" is useful is to allow embedding resources (xibs, images, etc) on a package when building for Swift 5.3, while still supporting Swift 5.2 and below (without those resources).

Example Package.swift

let package = Package(
    name: "Foo",
    dependencies: [],
    targets: [
        .target(name: "FooLib", dependencies: []),
        .target(name: "FooResources", dependencies: [])
    ]
)

Example Package@swift-5.2.swift

let package = Package(
    name: "Foo",
    dependencies: [],
    targets: [
        .target(name: "FooLib", dependencies: [])
    ]
)

I have no experience with "version-specific-tags" so cannot say one way or another, however I understand the reasoning and benefits of simplifying Swift Package Manager internals.

2 Likes
12

version-specific manifests and version-specific tags address different needs, and keeping only tools-version would cut off functionality that it can't address. Others have covered the manifests angle; the version-specific tags are good for ending support for an older language mode without leaving the package in an unusable state for someone still using it. It's not a substitute for version-specific manifests because in general it means maintaining multiple trees and multiple sets of tags, and no one would choose to do that. I think all three features are needed.

13

Could you speak more to how you might use version-specific tags to support older versions of Swift that aren't otherwise possible or as convenient with version-specific manifests? Like others in this thread, I would miss version-specific manifests if they were removed, however I couldn't say the same for version-specific tags.

14

I have used a version-specific tag to establish the last version that works for a particular version of the tools. You could obviously do this by incrementing the major version number, but that might not be appropriate. I dropped support for Swift 4 at version 6.5 of a package, but then added a "6.4.1@swift-4". This allows a client still on Swift 4 to continue using the dependency with no change in its manifest.

1 Like