SE-0490: Environment Constrained Shared Libraries

Evolution Proposal Reviews
1

Hello, Swift community!

The review of SE-0490: Environment Constrained Shared Libraries begins now and runs through September 18th, 2025.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to me as the review manager by email or DM. When contacting the review manager directly, please put "SE-0490" in the subject line.

Trying it out

If you would like to try out this proposal, you can clone the implementation PR of SwiftPM and build the modified package manager, then try it out with the end-to-end example projects below:

Example Producer: swift-dynamic-library-example

Example Consumer: swift-dynamic-library-example-client

There is also some documentation available in the implementation PR.

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

  • What is your evaluation of the proposal?
  • Is the problem being addressed significant enough to warrant a change to Swift?
  • Does this proposal fit well with the feel and direction of Swift?
  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available at:

https://github.com/swiftlang/swift-evolution/blob/main/process.md

Thank you,

Alastair Houghton
Review Manager

10 Likes
2

We are currently focused on bringing Swift Build as our suggested build system in the next release and will be deprecating the current native build system. I am still learning what features that will enable for Packages and will be producing proposals and requests for comment as soon as I get closer to having some firm ideas.

In the meantime, we really shouldn’t be adding new features to the native build system such as this. I am hoping we can get features like this for free from Swift Build. Well, free-ish since we need to be able to cleanly add them to the Package model and maybe extend it in some exciting ways. And if it’s not there, we need to be adding them there.

5 Likes
3

We have this same problem with the swift-syntax prebuilts for macros, which are essentially binary artifacts. There we distribute a build of the library for each linux distribution we have toolchains for. To identify the distribution we check the contents of the /etc/os-release file in addition to the triple. It provides enough information to identify Ubuntu 24.04, for example. The code for that is in Workspace+Prebuilts.swift.

It is working for that purpose, I wonder if this is something that can be generalized for all binary artifact types.

3 Likes
4

The implementation for this would most likely be primarily on the SwiftPM side. We'd at least need some support for the new artifact type in Swift Build PIF generation. Probably no changes needed on the Swift Build side as far as I can think of at the moment (not sure if that's what you meant by "for free").

It is working for the golden path use cases on the standard deployments of the most popular architectures of a handful of the most popular Linux-based operating systems, yes. That’s a far cry from full generalization to all possible Linux ABIs, which is not something I’m sure we should even attempt. There’s some great discussion on the Linux ABI problem in the original binary dependencies pitch thread: [PITCH] Support for binary dependencies

Also, the swift-syntax prebuilts are fundamentally a performance optimization of the build process, where we always have the option to fall back to building from source if needed. In my view that’s a distinct use case from binary artifact dependencies for the build outputs.

3 Likes
5

We’d be happy to try to help out there in the future if necessary too, but if it is largely independent from the native build system work, I don’t see why we should block this for an indeterminate amount of time?

I think this is an important point - the discussion on full generalisation is a different beast.

More importantly, I would like to stress that in addition to the motivation you can read in the PR, it really is also unblocking third parties from deploying certain enterprise solutions - just very practically, we at Ordo One currently need to build our own toolchain and distributed to our customers to be able to ship properly - SE-0490 would make it possible to use a stock toolchain as it provides the required escape hatch for such deployments - so it is not just a performance optimization, but it is an enabler for shipping solution based on Swift with requirements of being able to ship closed-source libraries on Linux (and in our case with library evolution…).

It would be unfortunate to block such deployments indefinitely searching for the “holy grail” of full generalisation - and if we ever get that, I think anyone using SE-0490 would be super happy to start using such a new facility - but until such a day, this escape hatch would make a big practical difference.

3 Likes
6

Sorry, I was mainly reacting to the changes to the BuildPlan which is part of the native build system. We need to make sure this works with –build-system swiftbuild as well.

There is a handful of people working on the SwiftPM transition to Swift Build. They are working hard to get it done in the next release or two. My feeling is that it’s more imminent than indeterminate :).

3 Likes
7

The proposal doesn’t mention the Windows system at all. Is there any plan for that? I’d love to have a working solution for using prebuilt shared libraries as dependencies. Internally, we use Swift extensively on macOS/Linux/Windows, and building software from source hinders productivity and CI times considerably.

1 Like
8

I don't see anything in the proposal that indicates that this solution would not also work for Windows.

1 Like
9

What is your evaluation of the proposal?

I don’t support the proposal in its current form.

Is the problem being addressed significant enough to warrant a change to Swift?

Yes. While the use case may not be especially common, I think it’s a goal of Swift that the build tools should not limit the type of artifacts users can build, even where they provide opinionated defaults.

Does this proposal fit well with the feel and direction of Swift?

I don’t think so. In my opinion, there are a couple areas where it feels in conflict with Swift’s goals.

  1. It targets a very specific set of use cases, and introduces restrictions on future evolution which make it difficult to generalize going forward. I agree that narrowly applicable features can be important to unblock important use cases when a fully general solution is much more complex, but I’m not convinced this proposal is the right short/medium term feature and I think it unnecessarily limits future evolution.

  2. As currently designed, this feature is difficult to use correctly without fully understanding its limitations, and does not have a mechanism to enforce correct use. The proposal addresses this, arguing that this is an advanced tool not suitable for most users. Nevertheless, I believe it could be designed to introduce guardrails which prevent the most common patterns of misuse without limiting the proposal's potential use cases. In the spirit of progressive disclosure and providing tools with safe and predictable default behaviors, I think the proposal should include these rather than placing the entire burden of correctness on the end user.

My specific concerns are:

swiftmodule layout in artifacts

The current proposal requires the following layout for the new artifact bundle type, with a top-level swiftinterface file:


đź“‚ example.artifactbundle

    đź“‚ MyLibrary

        ⚙️ libMyLibrary.so

        đź“ť MyLibrary.swiftinterface

    đź“ť info.json

The ecosystem has generally standardized on the directory-based swiftmodule layout when distributing swiftinterfaces and/or binary swiftmodules as toolchain/SDK/external content, and I think it would be a mistake to diverge here. The existing directory based layout naturally extends to multi-arch libraries if that is needed in the future, and has existing conventions for distributing supplementary content like ABI baselines, swiftsourceinfo, etc. which may be of interest to higher level tooling. In certain cases, the compiler can also produce better module loading diagnostics for common build config issues like architecture mismatches when modules are laid out in this format.

Naming

The use of “environment” in the feature’s terminology conflicts with the existing use of the “environment" field of target triples, which I think is likely to cause confusion as users of the proposed feature are likely to need an understanding of both concepts.

Choosing an environment

The proposal states:

Users are responsible for computing the correct URL of the Artifact Bundle for the environment they are building for, possibly within the package manifest. Swift tooling will not, on its own, diagnose or prevent the installation of an incompatible ECSL.

And the demo package currently reads this information from the environment: https://github.com/tayloraswift/swift-dynamic-library-example-client/blob/master/Package.swift. I think providing a better mechanism to propagate the selected variant to the build system should be a prerequisite for this feature. Although today techniques like this are necessary to work around limitations of the tools, we shouldn’t rely on them when building new features from scratch or encourage increasing their use. Overreliance on the environment has a significant impact on predictable build behavior and incremental build perf. It also makes it more difficult to setup long-lived or GUI tools like the VSCode Swift extension. Introducing a new flag or config file to provide this information instead would ensure we don’t exacerbate existing problems.

Forbidding use of triples in the artifact bundle

The proposal states:

The artifact must have exactly one variant in the variants list, and the supportedTriples field is forbidden.

While checking for a matching target triple is not sufficient to diagnose all possible configuration mistakes using this feature, it would make it easier to diagnose and prevent a subset of issues where the user introduces a trivial platform/arch/vendor mismatch. This restriction also potentially blocks use of the feature in multi-architecture builds.

Compiler Compatibility

Since non-Darwin platforms aren’t ABI stable, their use of swiftinterfaces has been extremely minimal up to now, and the compiler does not currently reject swiftinterfaces generated by an incompatible compiler. As the first feature encouraging use of textual interfaces on these platforms, I think the scope of this proposal should cover changes to the compiler to reject incompatible interfaces, whether that be by using the swift-compiler-version/-interface-compiler-version embedded in the interface, or some new mechanism. Without such a mechanism, I think there’s a very high risk of accidental miscompiles going initially unnoticed by users and surfacing in production.

How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

I reviewed the proposal and sample packages demonstrating the feature

11 Likes
10

Thanks Owen! Well said.

My biggest concern with this proposal as well as, retroactively, the static library proposal, is they solve specific problems without looking at a layer up, and solve them differently from each other. How do we properly distribute binary artifacts that we can securely and safely plug into builds? I have to think we can solve this at a slightly higher layer of abstraction that solves them for all.

That said, I am guilty of this as well with the swift-syntax prebuilts which solves a very thin problem space but from which we’ve learned a lot about the complexities in this area, i.e. compiler version compatibility, SDK compatibility (which we haven’t solved yet), linux distribution detection and support, publishing challenges, etc. It would be great to see a solution that solves it for all binary artifacts from what we’ve learned in all these implementations.

3 Likes
11

I am very sympathetic to the problem we are trying to solve here but I agree with what @owenv and @dschaefer2 have raised here as concerns.

This is and the fact that the proposal recommends to build libraries with -enable-library-evolution is one of the largest problems to me. Currently Swift only has a stable ABI on Darwin platforms and the fact that this flag works on non-Darwin platforms is not guaranteed. We also don't need ABI stability for this use-case since the expectation is that both the binary and the dynamically linked library are built with the same compiler.

4 Likes
12

I am also sympathetic to checking if the swiftinterface is built with an incompatible compiler, that makes sense and tightens things up. That is good feedback!

This is not really right though, if I may quote @Slava_Pestov in the original announcement post for library evolution :

Platform support

The Swift compiler currently only guarantees binary compatibility among different compiler versions on Apple platforms. This means that on Linux and other platforms, an application and a library built with different versions of the Swift compiler will not necessarily link or behave correctly at runtime.

However, stable module interfaces and library evolution can be used on all platforms supported by Swift. So on non-Apple platforms, you can still use multiple versions of the same library without recompiling a client application, as long as all binaries were built with the same version of the Swift compiler.

As mentioned in ABI stability and more, as development of Swift on Linux, Windows, and other platforms matures, the Swift Core Team will evaluate stabilizing the ABI on those platforms as well. This will lift the restriction on mixing and matching artifacts built with different compiler versions.

What this pitch suggests and tries to accomplish, is exactly what the middle paragraph outlines - we want stable module interfaces and use library evolution so we can use multiple versions of the same library without recompiling a client application, as long as all binaries are built with the same version of the compiler.

We need the library evolution flag to be passed to get the appropriate indirections to be able to support our clients to not have to recompile their binaries that link against our evolution-enabled framework. Specifically, again quoting from the original post, we want to:

Library evolution support should only be used when a framework is going to be built and updated separately from its clients. In this scenario, a client built against an old version of the framework can be run with a new version of the framework without being recompiled.

This is exactly what we want to do, “since frameworks built without library evolution do not provide any binary compatibility guarantees”.

6 Likes
13

Just to put this on the record in a friendly way: this feels a little bit unfair, and maybe even a bit of a double standard.

We originally pitched this feature back on February 2 (after starting the discussion January 25). That predates the binary static library dependencies pitch (March 18) as well as Swift-Syntax prebuilt for Macros (May 29). So the fact that this pitch was already in the pipeline was known, yet those other two went through without requiring a “proper distribution of binary artifacts.”

Given that it then took almost eight months for our pitch to actually be reviewed, it feels odd that the responsibility is now on us to solve the broader binary artifact distribution problem.

Realistically, I don’t think we could push through such a general solution as a third party—just getting this narrowly targeted fix pitched took nearly a year, so a larger effort would be even harder.

Hypothetically, if our pitch had been accepted before those two, would they have been held up with the same argument?

We’re happy to work through concrete points—like the compiler version check, and possibly other parts of Owen’s concrete feedback (thanks!) once we’ve had time to digest them. But right now, being left without any way to use library evolution in controlled environments feels like letting the perfect be the enemy of the good. It also blocks us from shipping properly (happy to share more details offline if that helps).

4 Likes
14

Just FWIW, and while I agree that it has taken longer than strictly necessary to get this proposal to the point of review, the decision to move to the new build system was announced on swift.org on February 1st, so I think it's reasonable to suppose that the folks working on the build system would have raised similar objections to your proposal back then too.

As far as the other proposals you mention, I think @dschaefer2 did specifically mention that:

so it's clear that there is awareness of the tension between having a perfect solution and being able to actually ship things.

I'm not going to comment on the proposal itself since that isn't really my function as review manager, but I would encourage you to work with others in the thread to come to some understanding on the way forward.

Questions that IMO are worth asking:

  • What are the implications for ongoing build system work if we accept this proposal, either as-is or with changes? Are there changes that would make it more acceptable and might mitigate the objections that have been raised?

  • Could we accept this proposal — and the changes to the old build system — on the basis that the new build system will support something similar but not necessarily the same? Maybe with a way to flag up in the Package.swift that we require the old build system (assuming we can't work that out from the tools version comment)?

  • Or should we decline this proposal, but commit to collaborating on providing, in a timely manner, a clean way to distribute binary artefacts based on the new build system, or in a way that we think will fit in with the new build system?

(Obviously these are not the only options open to us, but they are three that I think are worth contemplating.)

1 Like
15

FWIW, I think distribution is a separate problem than how to link against them which is what both this proposal and the static binary proposal have been focused on. There is already enough history with two proven ways of distribution of binaries both remote (URL + checksum) and local (path). This API has been used for binary targets since the introduction.

I'm curious what you think we should do differently here? I personally see a lot of room for binary support in the package registry specification but even with having that I think we always need to support both URL and path based binary dependencies as well.

4 Likes
16

The ABI might change across compiler versions on non-Darwin platforms, but this flag, and textual interface files, should work if you use it within one compiler release.

4 Likes
17

My concerns are mainly about the implementation. I’ve learned a lot doing the prebuilts feature and I really think we could put together a proposal that solves this for everyone. I’m not sure I agree with this statement from the proposal.

This means it is pretty much impossible for a public Swift library to vend precompiled binaries that will Just Work for everyone, and we are not going to try to solve that problem in this proposal.

I do believe the swift-syntax prebuilts are a counter example to this. They were a lot of work and took a while to get right, but they do Just Work for everyone and where they don’t we fall back to build them from source. This was done by providing binaries for each combination of Swift compiler and for each platform the toolchain supports including the individual Linux distributions. I’d like to see mention of doing something similar here in the Alternative Considered section.

I also wonder about this:

Deploying ECSLs does not involve SwiftPM or Artifact Bundles at all. You would deploy an ECSL by copying the latest binaries to the appropriate @rpath location on each machine in your fleet. The @rpath location is part of the organization-specific environment definition, and is not modeled by SwiftPM.

Some organizations might choose to forgo the @rpath mechanism entirely and simply install the ECSLs in a system-wide location.

Another alternative could be to just use unsafe flags on the targets that consume these libraries to point at these locations. Or am I misunderstanding why that doesn’t work. Clearly having the manifest deal with these as a first class citizen has benefits. I just worry about the stress on the architecture.

One other issue with the implementation is that Swift Build is imminent and any features we add in the next release need to support it. Our hope is to make it the recommended build system so we can start work on getting SwiftPM to take advantage of its power. To do that though, it needs to do everything the Native build system which is why were cautious at this time.

18

The review period for this proposal has now ended. This proposal has been returned for revision.

19

I am working towards a vision document for Swift Packages that I will be working with the community to ensure we meet everyone’s needs in a hopefully consistent manner. I should have something ready to start sharing in a few weeks. Let’s make sure we cover this use case there.

5 Likes
20

i think this is valid, and it would not be a major change to layout the swiftinterface files under another directory level.

i disagree with this assessment, as environment i think, is a straightforward elaboration of the concept as expressed by target triples, which essentially models a much less-granular version of an “environment” today.

that said, i really don’t want to get bogged down in naming, and i’m happy to adopt whatever name for the feature you feel is best, so long as it’s something all stakeholders at Apple agree on.

i’m not happy with the reliance on environment variables for variant selection either, but this was a pragmatic choice that was made out of recognition that there were no actual alternative pathways currently available to us, and that designing a general purpose mechanism to replace environment variables in Package.swift is far out-of-scope for a proposal like this one.

it’s worth noting that using environment variables in Package.swift was a forward-looking choice intended specifically to avoid blocking future work along this axis. nothing about the proposed implementation itself actually assumes that variant selection is being driven by environment variables, it just so happens that environment variables are currently the most convenient way for end users to compute the necessary URL to download the correct artifact from. by taking only a URL string as input, the proposal actually avoids any direct dependency on environment variables, and therefore would not make it any more difficult to migrate the ecosystem away from environment variables in the distant future.

this is not really valuable for the envisioned use case. the platform information encoded by a target triple is likely to be constant across an organization-scale fleet, it is the Swift/C runtime version that is far more likely to experience rolling upgrades and version mismatches. while you can always imagine situations where this diagnostic might catch something, i believe the real-world effectiveness would be very limited and that here is, at a high level, the wrong layer to be performing this type of compatibility check.

i’m also not clear on why this would block use of the feature in multi-architecture setups.

although i am sympathetic to this concern, i am not sure if the prerequisite machinery exists within SwiftPM to parse and understand toolchain compatibility version from inside a swiftinterface/swiftmodule file, as my (limited) understanding is that only the Swift compiler is capable of decoding this information.

1 Like