I have left this on the back burner for way too long but I made a stab at trying to fill in the blank spaces of the proposal that was already in progress: https://gist.github.com/bppr/bdbb9bb4c73a50cf9f52e6c15595c1a2
edit: we may have to adjust some of the nomenclature to avoid conflicting with the SwiftPM proposal for workspaces. I wrote this before that, but... I yield courteously
Hi folks,
This is a great discussion; thank you everyone who's been thinking about (and prototyping!) these problems.
In seems like there are a few issues being wrapped up together here:
With whatever solution we provide to any of these problems, I think it's very important to make sure that SwiftPM maintains a full and precise dependency specification. This allows us to build features like smart caching and precise incremental builds, which will be very important for build performance (as well as incremental build correctness!).
As much as possible, I think it's desirable to keep the core specification of what to build in the Package.swift manifest, without needing additional "levels" to the build. We should have a strong build settings model (though that deserves its own proposal), and we can have conditional build settings there as needed. And machine-editable manifests will eventually make it much easier to maintain that "how to build" without needing package-specific magic to allow maintainers to specify it in their own preferred way.
Telling SwiftPM how to build things that it doesn't natively know about is, I think, the point of Daniel's original (skeletal) plug-in proposal. While having native knowledge of common tools is great, there are always going to be things SwiftPM doesn't know how to do, like running less-common tools and bits of build logic that we don't want specific native support for. We should add support for this sort of custom logic, but when we do, we should make sure that we're still able to understand what that logic is doing, to maintain that precise dependency specification. That means that we need to know what the inputs and products are from any custom step, and ideally would be able to carefully control its environment to only expose it to declared inputs (e.g. using sandboxing), though that last step is an optional enhancement.
For how to get tools on your system, I think it's desirable to vend them via packages that are just part of your normal dependency graph, and subject to the normal constraints around how dependencies get resolved. Ideally, most common custom logic can be distributed as its own package that any client can use, and the precise configuration needed to use it can then be specified using the normal modeling in Package.swift.
Per-package scripts that you don't want to distribute as their own package should probably be possible at some point, but I suspect that they're a lot more error-prone (in terms of properly declaring inputs/outputs and not depending on state in the package manager that we may not want to commit to, like the current build directory layout). Unless this is really blocking people from doing what they need to do, I'd suggest we defer freeform scripts if possible until we're able to better control what they can do and make sure that we won't break them as SwiftPM evolves.
Ankit and Daniel have been working on fleshing out a proposal in this space, and while I haven't read it in detail yet, I think it hits all of these criteria. I'd suggest that he start a new thread for that discussion, since this thread has veered pretty far afield from the original topic.