SE-0482: Binary Static Library Dependencies

Hi folks,

The review of "SE-0482: Binary Static Library Dependencies" begins now and runs through 15 May 2025.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to the review manager. When emailing the review manager directly, please keep the proposal link at the top of the message.

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

  • What is your evaluation of the proposal?
  • Is the problem being addressed significant enough to warrant a change to Swift?
  • Does this proposal fit well with the feel and direction of Swift?
  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available at

swift-evolution/process.md at main · swiftlang/swift-evolution · GitHub

Kuba (@kubamracek) will be your review manager for this proposal.

Thank you in advance for your comments,

Alastair.

18 Likes

Thank you so much, @daniel-grumberg, @Max_Desiatov, and @FranzBusch!

This was one of the biggest missing features in SwiftPM for me and I'm beyond excited to see this come to fruition!

This is a very important addition to SwiftPM. Will this proposal also work with C++ libraries, as they also provide .a file, or will this be limited to C only?

2 Likes

I'm really excited to see the progress on the static library proposal in Swift — it's been very beneficial to me. It would be great to see support for dynamic libraries (.so) on Linux platforms in the future as well.

Like @technogen, I'm hugely much in favor of this proposal - it would have been an incredible boon when I was working on Automerge, in order to support the Swift wrappers on Linux for server use cases. I've felt it's been a large hole in the cross-platform story for Swift ever since XCFrameworks appeared, but only being relevant to Apple platforms.

I did have a specific question about the proposal though. There are three "paths" outlined in the manifest, two of which explicitly assert they're relative paths (path and headerPaths) but the third doesn't (moduleMapPath). From context, I'd also expect that may be a relative path, but it wasn't clear in the proposal. Is it (also) expected to be a relative path?

I would like to see some sort of verification hash included (a checksum of some kind to validate the binary is what's expected). I do recognize that's a large potential can of worms. Still, something to validate the checksums of the binaries are the expected versions as a baseline is important to me. Maybe that's something that is envisioned around this manifest, but I'd like to see it addressed and mentioned in some form, even if it is "Future Directions".

2 Likes

The auditing tool is intended to provide guarantees to users who decide to opt in. You would be able to make this work with a C++ library as well but we can't guarantee that it would be ABI compatible with your deployment target.

2 Likes

Yes absolutely let me clarify that.

As for the checksum, as with other binary targets, the client needs to provide the checksum it expects in its Package manifest. This checksum should be computed using swift package compute-checksum as described in SE-0272.

2 Likes

While it's my intention (or at least my dream :grinning_face:) to eliminate the cases where SwiftPM can't build these libraries, I'm satisfied with this proposal. Binary compatibility is a hard thing to get right especially as we grow the number and types of platforms Swift supports. The triple will take us a long way but we'll need something more explicit to deal with runtime ABI in the future. But this will certainly help us now.

3 Likes

What does this mean? There will always be a need for binary-only libraries, for people who don't want to share their source code.

1 Like

Could this proposal be extended to include pre-built static Swift libraries? Doing so would allow Swift libraries to be pre-compiled and sourced in lieu of requiring the client to download and build the source directly.

Pulling from the example given in SE-0305, if you wanted to pre-compile the protobuf code that is generated for a given set of schema and vend the precompiled library, this would be a huge win for the client compiling the code. There's not really a huge benefit of compiling these files every single time (e.g. when running in CI) where a pre-compiled version would be sufficient

1 Like

Agreed. My point was on the first part of this:

The Swift Package Manager’s binaryTarget type lets packages vend libraries that either cannot be built in Swift Package Manager for technical reasons, or for which the source code cannot be published for legal or other reasons.

I want to make sure there aren't "technical" reasons why you can't build these libraries from SwiftPM.

3 Likes

One technical reason that fundamentally can't be solved is build times. Software projects can scale up massively and retaining short build times is a big issue. Very often you can and prefer to build your packages from source, but you may have hundreds of them, each of which using macros, build tool plugins and/or compiling thousands of source files, and then run extensive unit tests. In this scenario, the only viable solution is to build the packages separately, doing all the tedious time-consuming work in advance only at the point of change. Whenever you need to build the whole project, you never recompile anything that hasn't changed, so your project build time stays roughly constant, regardless of how much it scales up.

3 Likes

This is intentionally out of scope of this proposal for two reasons. First consumed pre-built Swift libraries has different ABI concerns especially on non-ABI stable platforms. This is mentioned in the future directions section of this proposal. Secondly, if the reason for consuming pre-built Swift libraries is build times then the proper solution is improving the caching capabilities of the build system. Many modern build systems support remote and local caching to reduce the amount of times code gets compiled and speed up overall build time.

3 Likes

But this doesn't take into account running unit tests and generating coverage reports. For those, we don't merely want to cache the build artifacts, we want to know when the package changed and we want to have a separate build flow for it for independent diagnostics reasons. Moreover, for production builds, it's very often forbidden to use build caches for build consistency reasons.

EDIT:

Essentially, for this purpose, the pre-built library is itself the intended build artifact, just like executables can be.

1 Like

Now if one only had a CAS based caching system as a foundation for that…

5 Likes

Most if not all of those problems have been solved by modern build systems like Bazel. Those systems often allow fine-grained control when cached artifacts should be used.

Taking a step back though. Adding pre-built Swift static library is a reasonable feature that has multiple use-cases. This proposal doesn't include this though to focus on the ABI considerations with regular static libraries. Swift static libraries require additional considerations and should in my opinion be explored in a dedicated proposal.

2 Likes

I'm really looking forward to that!

1 Like

After working on the implementation in SwiftPM we have decided to change the way to provide the header paths and modulemap path in the artifact manifest.
The updated JSON schema looks like:

{
    "schemaVersion": "1.0",
    "artifacts": {
        "<identifier>": {
            "version": "<version number>",
            "type": "staticLibrary",
            "variants": [
                {
                    "path": "<relative-path-to-library-file>",
                    "supportedTriples": ["<triple1>", ... ],
                    "staticLibraryMetadata": {
                        "headerPaths": ["<relative-path-to-header-directory-1>, ...],
                        "moduleMapPath": "<path-to-module-map>"
                    }
                },
                ...
            ]
        },
        ...
    }
}

This makes it clearer which metadata is required by library artifacts as opposed to tool artifacts.
A sample manifest might look like:

{
    "schemaVersion": "1.0",
    "artifacts": {
        "example": {
            "type": "staticLibrary",
            "version": "1.0.0",
            "variants": [
                {
                    "path": "libExample.a",
                    "supportedTriples": ["arm64-apple-macosx"],
                    "staticLibraryMetadata": {
                          "headerPaths": ["include"],
                          "moduleMapPath": "include/example.modulemap"
                    }
                }
            ]
        }
    }
}

Additionally it is worth noting (and the proposal will be updated to reflect this), that it is required to explicitly provide a modulemap if the artifact is to be imported directly by a Swift target.

3 Likes

Additionally it is worth noting (and the proposal will be updated to reflect this), that it is required to explicitly provide a modulemap if the artifact is to be imported directly by a Swift target.

I'd be interested to learn more about the reasoning behind this restriction. While specifying the module map independently of the header search paths may be useful for libraries which are modularized retroactively after vendoring sources from elsewhere, for a library which has authored its own module maps in the original source I think only needing to specify search paths to the colocated headers and modulemaps would be a nice simplification. It's also possible that someone may want to distribute a static library which encapsulates multiple modules, in which case it would be useful to have the ability to specify > 1 modulemap.

I'm very eager in this landing. It would enable me to package Swift wrappers for various interesting C/C++ libraries without having to go through a large hassle of repackaging that library in a SwiftPM structure.

2 Likes