Swift Package Installation

@migueldeicaza what would you like to do next? If you are interested in splitting out an installable artifact proposal, I would be very willing to actively work on that.

Alternately, if you want to focus on the installation side, I would be willing to start work on an installable artifact proposal on my own, and we can see if we can get the two to dovetail?

I'm quite in favour of this (for binaries). I usually endup making a makefile just for make installation of CLIs easy :) Also as mentioned other modern languages have all these real world powers in their own CLIs so I'm pretty keen on Swift getting there too. (another thing I would love to see is a tool to install swift version as part of the Swift project)

Speaking from my experience as a user of exherbo, Gentoo, and a variety of other distributions and my experience of supporting the package manager on exherbo and Gentoo - it wouldn't be second but in fact closer to fourth class.

  1. System managed packages are definitely far more preferable on Linux and are best supported
  2. Packages installed by the user are usually accounted for in /usr/local and will have handling designed specifically for that
  3. Binary package distributions that are recognised as being important (e.g. VMware Workstation) have package manager workarounds to install the binary package itself
  4. Packages controlled by the language package manager are generally the worst supported and even somtimes ignored, requiring multiple parallel installations one done more closely as a binary package.

This includes CPAN, Ruby Gems, Python pip, Haskell Cabal, and most of all TeX packages. All of these are extremely difficult to integrate, and as a result have some of the worst packages available.

I think that making packages more amenable to packaging and building in system package managers (be it paludis, portage, pacman, or something else) is going to yield a much more smooth experience for users, at least in a subset of environments.

Id be interested in more details on this. I suspect that you are describing the fact that there are multiple API incompatible releases, something which at least Gentoo and exerhbo have solved using the concept that they call slots.

5 Likes

I haven’t used either of those distros, but I’ll read up on slots as soon as I can.

IIRC, mostly it was that one project would have a dependency on whatever version of something for some internal thing that never got exposed, some other dependency depending on some other version for another reason that never got exposed, with the result being a phantom incompatibility. Or something like that... this was a while ago. Anyway, IIRC the result was that if you went through the “recommended” process for compiling everything from source, you’d start having compiler errors halfway through getting the entire stack up and running because “the system” didn’t have (or maybe just didn’t use) a way to handle having multiple versions of the same dependency being installed.

I want to echo what @compnerd said here. The moment you start installing code outside of the build directory you need to address the possibility that what you’re doing will clash with what someone else is doing. For example, imagine a (beautiful) world where SwiftNIO is in the Ubuntu 22.04 apt repositories.

If you also built and installed that from SwiftPM, where do they go? Do they clash? How do you address this problem? What about the fact that a default Ubuntu build would be resilient but a default SwiftPM build would not?

Python’s pip ended up having to understand where distributions put the Python packages they installed with their own package manager in order to manage conflicts in this way. We should be wary of going too far down this road lest we find ourselves in the same place.

6 Likes

I really like @ddunbar's idea of using something like Nix, or build on the same ideas. Installing things globally like Homebrew does often leads to issues because the contract between dependencies break and that results in a poor experience for the developer.

SPM could ship with Nix, if Apple is comfortable with that idea, and expose an API that serves a proxy to nix-env. If the Swift Package Manager exports information about the inputs and outputs of each buildable artifact, Nix should be able to do the rest. Besides the advantage of resulting in a deterministic environment, which is what the developers would expect of something like this, you can also leverage other Nix features like caching, which would make the installation of packages faster.

I'm very happy about seeing this being discussed :smile:

3 Likes

That sounds like something to be integrated into Nix, not into SPM.

I would be very interested to see more exploration along these lines.

As many people on this thread have commented, it’s a large undertaking to replace a system package manager.

However, finding ways where SwiftPM can integrate natively with one, and Nix is to my mind the most obvious one because of technical alignment, can deliver a lot of the benefits while also improving the OSS ecosystem collectively.

I’m familiar with Nix conceptually, and not very much practically, so I don’t personally have any concrete ideas on how to accomplish this,, but I love the direction.

Perhaps, figuring out the right layering is always important. It could be Nix already had mechanisms to be “extended” or integrated with in such a way? Or if not, it’s possible the SwiftPM OSS project could work with them to develop it?

I would hope any such hooks would be agnostic to whatever tool is calling SPM to build executables. Ideally, the same hooks could be implemented by other language package managers so that all platform package managers could support such interop. Ideally, whatever integration is designed should have as little impact on the language package managers as possible so they can focus on their own unique features.

1 Like

Right, exactly.

I feel like this feature is out-of-scope for Swift Package Manager. While such functionality is useful, it should be provided in a separate tool.
Such a tool already exists, actually: Mint.