Many of the largest libraries out there, such as Alamofire, do not have module stability. However, I don't know of a clear way to determine which ones do (e.g. in Alamofire, when I tried, it just crashes). Is there an easy way to determine module stability for a library? If not, is there a conclusive list somewhere, or can we come up with one, of all the things that prevent module stability?
Unless they say otherwise, you can pretty much assume the developers have not considered their library’s ABI or use of
-enable-library-evolution. The vast majority would rather you built the library from source alongside its client and without
-enable-library-evolution, because the result will be optimized better.
If they build the library themselves and give you the resulting binary instead of their source, then ABI stability is more likely, but still not guaranteed. Ask them.
Just because a library doesn’t use evolution mode doesn’t mean you have to build it every time, it just means you have to treat every patch release of either the library or Swift itself as breaking.
You can read more about library evolution here.
There are a number of problems with building it from source for us at the moment, they are complicated and won't change soon. And distributing a binary for each version of Xcode we want to support is cumbersome. But I guess my question is, how do library maintainers even know if they have module stability? What are all of the requirements they have to look at?
This link I posted has the answer to that. It contains both a general overview and a further link to a more technical exhaustive list:
I see that list now. What’s interesting is that it talks about parts of the API changing across versions of the library, but that sounds like something that would apply only to dynamic libraries? After all, there’s nothing to change from if everything is being linked from scratch
Bumping this. I think the list in its current form is not as simple as it could be for (1) helping consumers to check if each of their third-party dependencies has module stability and (2) helping open-source maintainers to understand if their projects are stable. That some of our libraries at Spotify were not module-stable surprised many of us platform devs, so clearly more documentation would help. Lastly, the alternative of building each library for each version of Xcode, and then selecting the right one based on the Xcode version, is prohibitively cumbersome to do at scale.
There's not really a concept of whether a library "has module stability". There's a stable module format (swiftinterface or "module interface files") and an unstable, tied-to-the-compiler module format (swiftmodule files or "serialized AST files", even though they have SIL in them too). Module interface files (currently?) depend on library evolution mode because they erase private information about types, which then has to be recovered at runtime. Modules without library evolution don't provide that information in a runtime-compatible way because it would be a waste of code size.
Now, it is a little unfortunate that turning on library evolution changes the way clients can use a library. We tried to keep this from being the case, but it turns out that Swift 4 already allowed clients to depend on implementation details in ways that would be source-breaking and performance-affecting to change unilaterally in Swift 5, so those changes were only made in library evolution mode. (Frozen enums are of course the most obvious case; there are also some subtle ones around the types of private stored properties.)
Is it reasonable to say "module interfaces should be available in all modes"? Absolutely. But it is also much more work, since now these files have to represent things like struct layout without actually exposing those private members. (Or do something else to get the same effect.)
Note that "module stability in all modes" depends on something else that I'm not sure has been publicly promised: ABI stability in all modes. That is, if Swift 5.4 has a faster way to call generic functions, is it allowed to use that for the public API of modules not compiled with library evolution? Or are we stuck with what we have for freshly-compiled libraries as well as those that are supposed to be ABI-compatible across versions?
(If we do decide to change the non-evolution ABI and also want to support non-evolution module interfaces, we can at least keep it from being a mysterious run-time problem by marking newly-generated interfaces as incompatible with the old compiler.)
Separately, I've heard this before but I don't really understand it:
For in-house projects, why are you using more than, say, two versions of Xcode—your production one, and the "next" one you're trying to adopt? And what's keeping people from building their own copies of these libraries as a one-off thing when they need to use a different one?
If by modes you mean with library evolution vs. without, I'm fine compiling everything with library evolution and paying a reasonable performance/app size cost. What I struggle with is knowing if the libraries can work with the module stability flag at all, e.g. even with the flag Alamofire will still break. Now, when I look through the docs to find what all the pitfalls are, they seem focused on changing versions of dynamically-linked frameworks. I'm not concerned with that, since everything is statically linked. I'm just concerned with stability across different versions of Xcode. I struggle to find a concise list of what to look for.
As for why it matters, it's not the count of Xcode versions we have to support. It's providing the ability to seamlessly pick which static library to use for the version of Xcode that's compiling. If we open the same .pbxproj with two versions of Xcode, how can we pick a different library to compile the second time? And even if we could, there are many other challenges (build scripts, remote caching, etc.). Then we'll have to make sure people aren't breaking each version of Xcode in CI. The list goes on. As for why we'd use static libraries and don't build from source, I don't know the history of it, but the important thing is that we have lots and lots of static libraries we'd have to migrate over. It can also be difficult to build certain projects, and eventually we will see closed-source SDK companies trying to use Swift.
I feel like I'm not going to have an answer that satisfies you. Turning on library evolution is a semantic change, even if that only surfaces in a few ways. Open source packages generally do not give any thought to it. Therefore, you should assume that open source libraries do not work with library evolution turned on.
It's reasonable to ask for "module interfaces should support modules without library evolution turned on" and I think it's similarly reasonable to ask for "there should be a way to make use of the indirection provided for module stability, but don't change the semantics of the built code and its clients, so that the existing implementation of module interfaces works without affecting semantics". But neither of those currently exist.
Fair point, I can't think of an easy way to do that without a Run Script build phase.