Swift package diagnose-api-breaking-changes

What looks to be very useful feature is the new SPM diagnose-api-breaking-changes (previously experimental-api-diff ) that is queued up for 5.6 (based on the swift-api-digester). I just wanted to post about it and hear if anyone else have used it and to point out a somewhat hidden gem that I stumbled over recently so more people can play with it.

Here are some of the background:

I found out about it while digging in e.g. we need to automatically detect unintentional API changes · Issue #965 · apple/swift-nio · GitHub

Sample output could be:

jocke@linux ~/s/swift-plugin (main)> swift package diagnose-api-breaking-changes 0.1.0
[4/4] Build complete!

1 breaking change detected in Plugin:
  💔 API breakage: func PluginFactory.x() has been added as a protocol requirement
jocke@linux ~/s/swift-plugin (main) [1]> nano Sources/Plugin/Plugin.swift
jocke@linux ~/s/swift-plugin (main)> swift package diagnose-api-breaking-changes 0.1.0
[4/4] Build complete!

1 breaking change detected in Plugin:
  💔 API breakage: func PluginFactory.create() has been renamed to func create(_:)
jocke@linux ~/s/swift-plugin (main) [1]> swift --version
Swift version 5.6-dev (LLVM 3647999e44ce50a, Swift 22aa3c5dbe4e319)
Target: aarch64-unknown-linux-gnu
jocke@linux ~/s/swift-plugin (main)> 

It seems like a promising way of automating semver generation for packages - perhaps it would even make sense to have something like swift package tag-release that would use this to automate the semver tagging in the future?

15 Likes

This is a neat tool however I think it can create a false sense of security. Breaking changes can occur in the behavior of an API even with the exact same interface. For example, say a throwing function takes a String argument, a new version of the library may start throwing validation errors on Strings that were valid arguments in a previous version. Or perhaps a new version of the library expects methods to be called in a different order than before. So semantic versioning should always be done with care by the engineer making changes to the library, it would be dangerous to fully outsource that job to a tool which can only see breaking changes in the API surface. This might be nice to generate warnings of breaking changes in PRs for example but not to fully automate versioning decisions.

5 Likes

That's definitely a fair point, but I still think that the absolute majority of bumps will be automated which is neat.

Perhaps it'd be nice with a manual way to force a breakage that the tool would take into account, so that when an engineer knowingly breaks things that wouldn't reflect in the obvious API surface.

E.g. one can add a new value to en enumeration that is part of the public interface (assuming that would force a break) and check it in or similar to force diagnose-api-breaking-changes to flag it when one does such a more intricate change you suggest.

Just tried that — could work ok as a workflow I think;

Just defined:

public enum manualBreakingEnumerationTest {
  case manualBreak_2 // Manually increase this counter to force a semver API break 
}

This gives this output:

jocke@linux ~/s/swift-diagnose-api-breaking-changes-test (main)> swift package diagnose-api-breaking-changes 0.3.2
[4/4] Build complete!
2 breaking changes detected in swift_diagnose_api_breaking_changes_test:
  💔 API breakage: enumelement manualBreakingEnumerationTest.manualBreak_2 has been added as a new enum case
  💔 API breakage: enumelement manualBreakingEnumerationTest.manualBreak_1 has been removed
jocke@linux ~/s/swift-diagnose-api-breaking-changes-test (main) [1]> 

Of course, doesn't help with understanding whether its a minor or patch version, just for major semver breaks - so perhaps you're right its only enough for PR checks - that's not too bad either though as a start.

In the best of worlds, the diagnose-api-breaking-changes command could be extended in the future to not only detect breaks, but to also optionally display added API which would allow for automation of detection of minor/patch semver changes too. diagnose-api-changes perhaps?

Then I think a fully automated workflow with the above manual enumeration "hack" for the more sophisticated breaks that can't be automatically detected would actually be quite usable.

1 Like

I think PR checks are a great application for this. Detecting breaking changes in the API surface that are not accompanied with a major version bump would be a great safety check, even if it can’t detect all types of breaking changes. Detecting incremental changes accompanied with minor version bumps would also be helpful but perhaps more challenging. I think that automating a version bump may be a step too far but rejecting PRs that have obvious versioning issues is a great start.

1 Like

Hi all. What are the parameters to run swift package api-breaking-changes for iOS?

@hassila would you mind helping me running the tool for iOS simulator or iOS device?
more info on my question is here The way to run diagnose-api-breaking-changes for iOS

Hi, I have never used an iOS simulator nor written any app for iOS so can’t really help you with more than is written above - running that from command line can help checking for obvious breaks in api - can’t see why you would use a simulator / device. Perhaps someone can help you if you describe the problem you try to solve more concretely?

@hassila more details about the question are here The way to run diagnose-api-breaking-changes for iOS
In other words, I'm looking for command line params to start diagnose-api-breaking-changes for the app being built for iOS or iOS simulator. In the link I've started with some extra params, which should change the SDK from Mac to iOS, but those params seem not enough for success, since the resulting binary has Mac architecture.

Any help is appreciated.