Swift package diagnose-api-breaking-changes

What looks to be very useful feature is the new SPM diagnose-api-breaking-changes (previously experimental-api-diff ) that is queued up for 5.6 (based on the swift-api-digester). I just wanted to post about it and hear if anyone else have used it and to point out a somewhat hidden gem that I stumbled over recently so more people can play with it.

Here are some of the background:
https://bugs.swift.org/browse/SR-10540

I found out about it while digging in e.g. we need to automatically detect unintentional API changes Β· Issue #965 Β· apple/swift-nio Β· GitHub

Sample output could be:

jocke@linux ~/s/swift-plugin (main)> swift package diagnose-api-breaking-changes 0.1.0
[4/4] Build complete!

1 breaking change detected in Plugin:
  πŸ’” API breakage: func PluginFactory.x() has been added as a protocol requirement
jocke@linux ~/s/swift-plugin (main) [1]> nano Sources/Plugin/Plugin.swift
jocke@linux ~/s/swift-plugin (main)> swift package diagnose-api-breaking-changes 0.1.0
[4/4] Build complete!

1 breaking change detected in Plugin:
  πŸ’” API breakage: func PluginFactory.create() has been renamed to func create(_:)
jocke@linux ~/s/swift-plugin (main) [1]> swift --version
Swift version 5.6-dev (LLVM 3647999e44ce50a, Swift 22aa3c5dbe4e319)
Target: aarch64-unknown-linux-gnu
jocke@linux ~/s/swift-plugin (main)> 

It seems like a promising way of automating semver generation for packages - perhaps it would even make sense to have something like swift package tag-release that would use this to automate the semver tagging in the future?

14 Likes

This is a neat tool however I think it can create a false sense of security. Breaking changes can occur in the behavior of an API even with the exact same interface. For example, say a throwing function takes a String argument, a new version of the library may start throwing validation errors on Strings that were valid arguments in a previous version. Or perhaps a new version of the library expects methods to be called in a different order than before. So semantic versioning should always be done with care by the engineer making changes to the library, it would be dangerous to fully outsource that job to a tool which can only see breaking changes in the API surface. This might be nice to generate warnings of breaking changes in PRs for example but not to fully automate versioning decisions.

5 Likes

That's definitely a fair point, but I still think that the absolute majority of bumps will be automated which is neat.

Perhaps it'd be nice with a manual way to force a breakage that the tool would take into account, so that when an engineer knowingly breaks things that wouldn't reflect in the obvious API surface.

E.g. one can add a new value to en enumeration that is part of the public interface (assuming that would force a break) and check it in or similar to force diagnose-api-breaking-changes to flag it when one does such a more intricate change you suggest.

Just tried that β€” could work ok as a workflow I think;

Just defined:

public enum manualBreakingEnumerationTest {
  case manualBreak_2 // Manually increase this counter to force a semver API break 
}

This gives this output:

jocke@linux ~/s/swift-diagnose-api-breaking-changes-test (main)> swift package diagnose-api-breaking-changes 0.3.2
[4/4] Build complete!
2 breaking changes detected in swift_diagnose_api_breaking_changes_test:
  πŸ’” API breakage: enumelement manualBreakingEnumerationTest.manualBreak_2 has been added as a new enum case
  πŸ’” API breakage: enumelement manualBreakingEnumerationTest.manualBreak_1 has been removed
jocke@linux ~/s/swift-diagnose-api-breaking-changes-test (main) [1]> 

Of course, doesn't help with understanding whether its a minor or patch version, just for major semver breaks - so perhaps you're right its only enough for PR checks - that's not too bad either though as a start.

In the best of worlds, the diagnose-api-breaking-changes command could be extended in the future to not only detect breaks, but to also optionally display added API which would allow for automation of detection of minor/patch semver changes too. diagnose-api-changes perhaps?

Then I think a fully automated workflow with the above manual enumeration "hack" for the more sophisticated breaks that can't be automatically detected would actually be quite usable.

1 Like

I think PR checks are a great application for this. Detecting breaking changes in the API surface that are not accompanied with a major version bump would be a great safety check, even if it can’t detect all types of breaking changes. Detecting incremental changes accompanied with minor version bumps would also be helpful but perhaps more challenging. I think that automating a version bump may be a step too far but rejecting PRs that have obvious versioning issues is a great start.

1 Like
Terms of Service

Privacy Policy

Cookie Policy