Removing PackageDescription API v3 (master branch)

Development Package Manager
1

I am planning to refactor the runtime targets from the package model to increase maintainability and extensibility of the different manifest versions that SwiftPM supports. It makes sense to remove the version 3 of PackageDescription API before starting work on the refactor. This is a master change only, the swift-4.2-branch will contain the older version of the PackageDescription API. Once the older API is removed, you will not be able to build a package that is using the older manifest API with trunk snapshots. Updating the manifest to Swift 4 should be simple for most packages; you can read this blog post to see the summary of the changes between v3 and v4.

Note that moving to a newer manifest API does not mean that you also need to update your source code. You can select any supported Swift language version in the manifest file.

Edit: Add rationale

The Swift compiler is removing Swift 3 language compatibility mode from Swift 5. SwiftPM's manifest API v3 relies on the compatibility mode for parsing the manifest file. While it is technically possible to parse the v3 manifests with a different language mode, it will not always work and can easily break the package graph. The manifest version is also used to determine the -swift-version for package's source, if swiftLanguageVersion is not declared in the manifest. This means, packages which are still Swift 3 manifest are likely to fail to build with Swift 5 tools. Manifest API v3 has a lot of magic behavior that was revised in SwiftPM with SE-0158. SwiftPM has evolved a lot in terms of API and behavior with that proposal. Migrating packages from v3 to v4 behavior should be trivial for most packages using the APIs provided in manifest v4 API and should not require changes in the disk layout.

3 Likes
2

I’ll bite. While support for v3 manifests has to be dropped some day this is going to break a whole lot of repos. Convenience of refactoring the SPM source shouldn’t be the primary consideration here.

1 Like
3

I'm inclined to agree with John, we should bend over backwards to maintain support and only drop if it is very difficult to retain support.

If we do want to drop it, then I think we need to back it up with statistics on current level of use, and ideally we would have a depreciation cycle where we warn you that it will become unsupported.

1 Like
4

Refactoring isn't the primary consideration. v3 manifests are parsed using Swift 3 language mode which is going away in Swift 5. Technically, we can support v3 API by parsing it with other language modes but that doesn't seem like a good idea to me, and it can easily break the packages anyway. I think it's good idea to tie it with the removal of Swift 3 language mode, at least this time. There are various legacy behavior that SwiftPM needs to currently conditionalize because of the v3 mode, for e.g. v3 API exposed a bad products API that we are still supporting even though it can produce weird results. Basing these decisions on statistics seems reasonable but they're not easy to collect right now because of GitHub API limits. In future, I hope we can collect accurate data from a package index.

5

I’d be fine with parsing in swift 4 mode.

I would expect this to keep working.

import PackageDescription

let package = Package(
    name: "Accumulate"
)
6

Yes, it'll work for the manifests which do not use a Swift 3 language behavior that was changed in Swift 4. However, that doesn't provide a guarantee that all manifests will still work, which is the point of having backwards compatible behavior. Updating the manifest should be relatively simple for most packages and we're talking about Swift 5 here which is currently estimated to be released next year. I see this as a good opportunity to remove the complexity that we've been carrying since the package manager first came out. We have evolved significantly in terms of API and behavior with the Swift 4 manifest API redesign proposal.

7

Isn’t infered packages and products a v3 api only or are you saying that this implicit behavior will still be supported in v4?

8

The inferred targets and products is v3 API feature but packages should move away from that legacy behavior. I was talking about the language modes, if we parse v3 manifests with Swift 4 mode, they may or may not work depending on how the manifest is written.

9

I am okay with swift 3.0 parsing going away.
Not excited about v3 package description going away. I spent a lot of time organizing the file system into the supported versions.

10

In most cases, you just need to declare your targets in the manifest. If your package is opensource, I can help with the list of changes you need to do.

1 Like
11

I see v3 manifests are no more looking at the number of breakages in the compatibility suite. A lot of code could have been salvaged by compiling a v3 manifest in Swift4 and many of these now broken repos where for C and module map imports only. The total amount saved by not having to maintain compatibility is going to be far exceeded by the time lost on the community cleaning up after this. Was there no executive oversight of this decision?

12

Can you please clarify this? Can you confirm there wont be breaking changes in swift 4.2 timeframe?

13

Yep, this change is Swift 5 and above only. PackageDescription API v3 will be available in Swift 4.2

14

Perfect. Thank you!

15

We just hit an interesting side effect of this.

Using a 5 snapshot, you can't build something depending on swift-protobuf because it falls back to the Package.swift which I guess it assumes is only Swift 3. (there's no comment marker, so I guess I was hoping it would assume something more current)

So until RFC: Tools version with version-specific manifests lands, it won't use the Package@Swift-4.2.swift as a fall back, and instead Swift 5 decides it can't build swift-protobuf at all.

I guess to support folks wanting to use a snapshot we have to make a new release with a Package@swift-5.swift or with a change to the Package.swift. But the existing releases just won't work?

16

That is correct, we need to implement the new behaviour to get support for the existing releases of swift-protobuf back.