Adding support for linting and formatting to SwiftPM's extensible build tools?

Two of the items that were specifically called out in SE-0303 were the use of SwiftPM's new extensible build tools to run code linting and formatting as part of a package's build/prebuild:

A future proposal should add specific support for code linters. In particular, there should be a way for build tools to convey fixits and other mechanical editing instructions to SwiftPM or to an IDE that uses libSwiftPM.

One approach would be to use the existing Clang diagnostics file format, possibly together with a library making it easy to generate such files, and to extend the PackagePlugin API to allow the plugin to configure commands to emit this information in a way that SwiftPM and IDEs can apply it. Such a capability could also be useful for build tools such as source translators, if they want to be able to apply fixits to their input files.

Code formatters (which typically modify the source code directly) should probably be supported using a new plugin capability that allows some specific action that a package author can take to run the formatter on their code, since it seems a bit subtle to allow source code to be modified as a side effect of a regular build action.

Source: 0303-swiftpm-extensible-build-tools.md

What does the process look like to get that moving? Is that something only Apple can drive, or could the community help out? These are both important tools to my team's workflow, and right now we're resorting to manual processes or relying on things being caught by CI (which has a really high latency).

1 Like

I think it is just a matter of adding a plugin product to the official swift-format package (or some custom equivalent), and then reporting back to SwiftPM if you find things that you think could interact better.

I suspect the only reason no one has done so yet is that the plugin feature is newer than the latest stable release of the toolchain. For now you would have to use a snapshot instead of a release.

I don't know if it's a limitation of the current implementation, or bugs in Xcode/SwiftPM, but for me neither of these use cases actually work:

So I'm not sure that it's as quick as adding plugins to packages — I think there are some genuine limitations at play here?

Then the next step would be to build SwiftPM from source (swift build) and then try it on one of your packages (.build/debug/swift-build --package-path /wherever/your/package/is for build tool plugins, .build/debug/swift-package --package-path /wherever/your/package/is whatever‐command for command plugins). If it fails, you can debug the problem and submit a PR. If it succeeds, then whatever issue you are hitting has already been fixed and all you need to do is wait. If you want, you can try it again in the release/5.6 branch and if necessary find and submit a cherry‐pick PR of the relevant fix into 5.6.

If it works in SwiftPM but not in Xcode, then it is entirely in Apple’s court and all you can do is make a filing with the Feedback Assistant.

Thanks for that advice. I'll follow through on it, but it doesn't really answer the original question: these were acknowledged limitations of the initial implementation of build tool plugins. I'd really like to know how I can help get the next phase of that implementation off the ground so that support for these tools can be added.

My understanding is that these tools (linting/formatting) not working is not a bug - SwiftPM hasn't been designed to do what they need yet.

Since SE‐0303 that you quoted earlier, there has also been SE‐0332. Does that not contain what you are talking about?

No, not really: command plugins are not run as part of the build of a target.

Command plugins appear to be useful for extraneous steps like packaging. You could certainly run formatting and linting outside of a build, but the linting in particular becomes a lot less useful if it's not happening (and being reported) during the build (or pre-build).

You are correct in saying that the current implementation is not catered towards linters or formatters, according to the proposal at least:

New capabilities may focus on specific areas of functionality, such as source code formatters or linters, new types of unit tests, or actions that can be invoked on-demand to perform particular tasks.

However, I believe that most of the functionality required to create a formatter/linter plugin is included in the proposal.

I did try creating a lint plugin as a build tool, but due to some bugs with the plugin implementation in Xcode 13.3 beta 1 I couldn't get it working (it may have been fixed since then). I think build tool output also isn't displayed in Xcode which is another roadblock (especially for linters).

I did however manage to get SwiftLint working as a basic command plugin for now as a compromise (swift-lint-plugin). Which is definitely still useful because it doesn't require contributors to have SwiftLint installed and allows it to easily be added as a pre-commit hook.

One of the examples demonstrated in the already‐implemented SE‐0332 proposal is a formatter/linter.

You can just as easily rearrange it to be a build tool instead.

If the existing (but yet‐unreleased) features are not enough, then please be specific about what in particular is still missing. For example, this tidbit is the sort of concrete information I am trying to fish from you:

Unfortunately, as I said before, Xcode is a Feedback Assistant matter. But here on the forums, we would like to know how well you think raw SwiftPM communicates output and diagnostics. If you have improvements in mind, then you can create a pitch for whatever API you need. Or if no new API is necessary, simply submit a pull request. Xcode inherits much of SwiftPM’s implementation, so it may also benefit from any fixes you submit.

Yeah, I know that this isn't for Xcode, I was saying that because most people probably use Xcode and it's something people would want to know about.

@tonyarnold was asking about support for linters and formatters at build time, that's why I referred to the earlier built tools proposal instead of the command plugins one.

I couldn't get either of them working as build tools (I couldn't even get a simple hello world build tool plugin working in the Swift version that shipped with Xcode 13.3 beta 1, I'll try updating to beta 3 because iirc the Swift version it has fixes a few plugin issues).

You're right, I should open another topic with the issues I'm having. If I get around to trying to create a build tool plugin again I'll do that.

1 Like

There are other bugs in Xcode 13.3 beta 3 that will impact you - I've had some helpful interactions with @abertelrud to try and work out what's going on, and he's kindly created the following SR's:

I can, but that's not the point of my post. The lacking implementation right now seems to only be that there's no way to report messages, warnings and errors back from a build/prebuild step to SwiftPM (or Xcode) from a called tool like SwiftLint. I've filed that as FB9937439, and this was proposed as a future direction in SE-0303: Specific Support for Code Linters and Formatters.

1 Like

Thanks for following up on the issues you found and getting SR's for them. Hopefully they get fixed soon :crossed_fingers:. I'll get the latest Swift snapshot and see whether I can find any more issues preventing a basic build tool from being built on my end.

It would definitely be very nice to have a mechanism for linters/formatters to emit fixits that IDEs can display and action. I agree that the current plugin API doesn't have the right functionality to create a linter/formatter plugin. Is the main issue (other than bugs) not being able to emit diagnostics and fixits? If so, do you want to create a specific pre-pitch for that? I'd be happy to if you don't want to/don't have time or something :+1:

Yes, that's the functionality I'd most like to see added. I really need to work on my communication skills, because all I wanted was advice on how to get the process started (with some understandable rationalisation of my understanding) - sounds like a pre-pitch is the right way?

I will probably have time over the weekend to have a look, but feel free to take the pre-pitch and run with if you want to. I'm not precious, I just want to see this feature added at some point!

No, it was very much on topic, and happened to be precisely what @tonyarnold needed in order to figure out what each other was trying to ask. Thank you for joining the discussion.

I only meant it as a disclaimer that if the issues turned out to be unique to Xcode, the “driving it forward” @tonyarold wanted to do would dead‐end at the Feedback Assistant. But even from your first comment, it sounded like it might be caused by deeper issues or gaps in SwiftPM itself, and that is what I wanted to zero in on.

I recommend building SwiftPM from source instead when you tinker. (See my earlier comment for instructions.) There have been a lot of relevant patches lately, including for some of the SRs that have been mentioned here. You might also consider looking at the open PRs and merging any relevant ones into your working branch, even if they are still on their way through review.

It is precisely because of the recent flurry of work around plugins that I kept trying to point you to main to determine if what you wanted to do might actually have been done already or not.

2 Likes

Ok thanks for the tip. I have a dual-core i5 MacBook air so it might take a while, but sounds like it might be worth.

You can make it :+1: Now that I think about it, it might even make sense to just start it as a regular pitch because it doesn't seem like a super contentious issue, and it's a very specific and achievable feature/goal.

1 Like

I think the diagnostics reporting is exactly what's missing as far as linting is concerned. The initial support for build plugins focused on generating source code (and resources, though there seem to be a bug with output file handling which Tony referenced). At the time of the plugin proposal there was good discussion about how to represent diagnostics but to get a start at this and keep things somewhat bounded, that had to be deferred. So the main thing preventing good support for linting should be reporting of errors and especially fixits. As a next step, a pitch with ideas about the best way for the plugin to convey that to the plugin host (SwiftPM or an IDE) would be great. The structured-diagnostics file that the Swift compiler generates already has that information, but the missing piece is conveying that to the host.

Separately: with SwiftLint in particular there seems to be an issue with sandboxing that causes an error even when trying to run it, but this seems to be a bug and not something that would require additional API. It looks like SwiftLint is doing XPC calls that are getting blocked.

3 Likes

I think it is just a matter of adding a plugin product to the official swift-format package (or some custom equivalent), and then reporting back to SwiftPM if you find things that you think could interact better.
I suspect the only reason no one has done so yet is that the plugin feature is newer than the latest stable release of the toolchain. For now you would have to use a snapshot instead of a release.

I attempted to do this today. But it seems build plugins cannot depend on source dependencies.
I couldn't figure out how to build a binary of swift-format to actually get the plugin running.

if someone wants to take a crack at it, my branch is here