The Swift runtime is a small C++ library that forms some of the underpinnings
for the Swift standard library. The library itself is written in the style of
LLVM with careful implementation for avoiding costs for things like exceptions
and type information. LLVM has developed a set of extremely useful abstractions
which are implemented in a low level library called LLVMSupport
.
These data structures are well suited for the operations required for the
runtime. As such, the runtime made use of the LLVMSupport
library. Because
LLVMSupport
makes no ABI stability guarantees, the runtime used it by treating
the library as header only, copying over parts of the implementation of the
library into the runtime and relying on the linker to dead strip the references
for other symbols which it did not use. This worked well enough for a while,
but as the library evolved more symbols had to be added with more complicated
logic (e.g. ABI breaking checks).
This worked but required that usage of the LLVMSupport
headers and types were
done with caution to avoid pulling in more of the implementation and to avoid
increasing the API surface that was pulled in indirectly.
LLVM continues to evolve and Swift pins itself to a specific version of LLVM. A
recent change in LLVMSupport
split one of the headers and this caused the
LLVMSupport
version in the runtime to differ from the current version of LLVM.
This would serve as a catalyst for unearthing a latent issue in the runtime.
When any LLVM based library is integrated into a single executable/library which
also has a Swift component, there are now two different implementations of the
LLVMSupport
symbols which are linked together. Because these are implemented
largely in the headers, they are emitted everywhere and rely on the linker to
unique the definitions and must ensure that a single definition is selected.
There is no guarantee of which symbol would be selected. The drift between the
LLVM repository and Swift now becomes a problem, resulting in possible ODR
violations when they are mismatched. Note that symbol visibility in ELF and
MachO are not sufficient to solve this issue as with static linking, all symbols
are (locally) visible, and so you have the symbol being exposed.
The previously mentioned change resulted in a violation of the ODR rules for
C++. In order to resolve this, there are two options. You could build the
Swift runtime against an unpinned (floating) version of LLVMSupport
revision
which still requires the consumer to ensure that all components in the address
space are built at the same revision of the LLVMSupport
library.
Alternatively, you could create a local copy of LLVMSupport
which allows you
change the library and ensure that changes to LLVMSupport
do not impact the
Swift runtime.
After discussions with @Mike_Ash and @Joe_Groff , we decided that it was best to
sever the dependency on LLVM for the Swift runtime and integrate a copy of
LLVMSupport
into the runtime. This ensures that the Swift runtime can
continue to evolve without concern over LLVM's evolution and can make changes to
the data types to better serve its needs.
Thanks to @Mike_Ash and @Joe_Groff for their helpful conversations in tackling this issue. Also, thanks to @Michael_Gottesman for convincing me that sharing this short snippet of this particular issue and the approach to solving it to keep others abreast of what is happening in various parts of the project can be useful.