A Tale of Swift and LLVM (ODR Violation in Swift Runtime)

The Swift runtime is a small C++ library that forms some of the underpinnings
for the Swift standard library. The library itself is written in the style of
LLVM with careful implementation for avoiding costs for things like exceptions
and type information. LLVM has developed a set of extremely useful abstractions
which are implemented in a low level library called LLVMSupport.

These data structures are well suited for the operations required for the
runtime. As such, the runtime made use of the LLVMSupport library. Because
LLVMSupport makes no ABI stability guarantees, the runtime used it by treating
the library as header only, copying over parts of the implementation of the
library into the runtime and relying on the linker to dead strip the references
for other symbols which it did not use. This worked well enough for a while,
but as the library evolved more symbols had to be added with more complicated
logic (e.g. ABI breaking checks).

This worked but required that usage of the LLVMSupport headers and types were
done with caution to avoid pulling in more of the implementation and to avoid
increasing the API surface that was pulled in indirectly.

LLVM continues to evolve and Swift pins itself to a specific version of LLVM. A
recent change in LLVMSupport split one of the headers and this caused the
LLVMSupport version in the runtime to differ from the current version of LLVM.
This would serve as a catalyst for unearthing a latent issue in the runtime.

When any LLVM based library is integrated into a single executable/library which
also has a Swift component, there are now two different implementations of the
LLVMSupport symbols which are linked together. Because these are implemented
largely in the headers, they are emitted everywhere and rely on the linker to
unique the definitions and must ensure that a single definition is selected.
There is no guarantee of which symbol would be selected. The drift between the
LLVM repository and Swift now becomes a problem, resulting in possible ODR
violations when they are mismatched. Note that symbol visibility in ELF and
MachO are not sufficient to solve this issue as with static linking, all symbols
are (locally) visible, and so you have the symbol being exposed.

The previously mentioned change resulted in a violation of the ODR rules for
C++. In order to resolve this, there are two options. You could build the
Swift runtime against an unpinned (floating) version of LLVMSupport revision
which still requires the consumer to ensure that all components in the address
space are built at the same revision of the LLVMSupport library.
Alternatively, you could create a local copy of LLVMSupport which allows you
change the library and ensure that changes to LLVMSupport do not impact the
Swift runtime.

After discussions with @Mike_Ash and @Joe_Groff , we decided that it was best to
sever the dependency on LLVM for the Swift runtime and integrate a copy of
LLVMSupport into the runtime. This ensures that the Swift runtime can
continue to evolve without concern over LLVM's evolution and can make changes to
the data types to better serve its needs.

Thanks to @Mike_Ash and @Joe_Groff for their helpful conversations in tackling this issue. Also, thanks to @Michael_Gottesman for convincing me that sharing this short snippet of this particular issue and the approach to solving it to keep others abreast of what is happening in various parts of the project can be useful.

13 Likes

Out of curiosity, what's the plan for dealing with the divergence of the two codebases over time (if any)? Is it strictly an integrated fork that will evolve independently from now on, or will changes be integrated from LLVM over time?

1 Like

For types that affect the ABI layout of runtime types, we would want to intentionally keep the Swift version of the types independent of the LLVM ones. We should only update them as needed, with an eye toward maintaining backward compatibility for target platforms with ABI constraints.

On that point (and just general curiosity): what kinds of utilities does the runtime actually use from LLVMSupport? How likely is it that they will change in any meaningful way?

The utilities are primarily StringRef, ArrayRef, DenseMap, SmallVector, and death (bad_alloc_error, llvm_unreachable, etc). Thereโ€™s maybe one or two more types at most.

The ArrayRef and StringRef could be replaced with array_view and string_view respectively. The rest would are relatively stable in evolution.

The changes that occurred that prompted the work involved the memory allocation routines being outlined.

I just want to give @compnerd a huge thank you for doing this work. It's not a lot of fun but it makes things easier for all of us.

7 Likes