Getting involved late, but I think @compnerd is absolutely right that the best answer, the right answer, is to get SwiftPM and Swift together to become substantially more friendly to being able to locate and integrate with system dependencies. I say both SwiftPM and Swift because I think both tools offer challenges that make this kind of integration harder than it needs to be, and a holistic solution would involve changes to both.
I am also cognisant of the fact that this is a problem for people now, today, and that nobody has committed to making the wide ranging changes that Swift and SwiftPM would need to achieve sensible and deep integration with system dependencies. I don’t want us to get stuck letting the perfect be the enemy of the good enough.
Simply vendoring zlib falls into the category of “good enough”. I think that goes double if we’re willing to make that behaviour optional, gated by a trait, such that it can be enabled situationally. The downside of that optionality of course is that it adds another multiplicative factor into the state space of NIO-based programs, yet another configuration to test.
A related note is that I’ll need to consult some legal experts to confirm that there are no licensing difficulties around including zlib as a source dependency before any such PR can be merged.
Quickly, on the question of what it would take to achieve great system package support in the Swift ecosystem, I think we’d need the following:
-
A way to enable SwiftPM to check whether a package has already been installed via a system package manager. This is necessary in a world where we want to be able to distribute pre-compiled binaries for packages, for example if we want Linux distributions to be able to rely on Swift packages as part of their core OS components, but it’s convenient in all sorts of cases, including making it feasible to distribute pre-compiled copies of some Swift packages on platforms with stable ABIs (e.g. via Homebrew).
-
A clear supported path to being able to import non-modularised package headers.
SwiftPM has decent support via the
systemLibrarytarget type for importing C libraries from the platform, but it awkwardly basically makes you write the umbrella header yourself. This is a) essentially undocumented, and b) forces your library to evolve in lockstep with the library you’re using, making it quite painful to support rapidly changing libraries whose list of header files changes over time.I can think of lots of ways to make this easier, with changes at different levels. We could change clang modulemaps to let you specify an umbrella directory without using an absolute path, searching the regular header search path for that directory. Or we could enable a way for build plugins to generate modulemaps on the fly. Or we could have a small DSL to express certain patterns for modulemap building. Or we could remove the requirement on clang modulemaps altogether. Or SwiftPM could transparently generate modulemaps from package-config files.
Regardless of the path we take, it is clearly not tenable to expect upstreams to modularise their imports when we’re the only ecosystem that expects them to do so.
-
Improvements to SwiftPM result packaging such that SwiftPM can produce more complex distributable artifacts. This would enable SwiftPM to lean more heavily on dynamic libraries while making sure that a single distributable unit is available.
I think this is a good start, but I’m sure there are more.