Linking LLVM into a Swift app

We have a Swift project which is is aggregating a bunch of high level libraries, and one of them transitively uses LLVM. This almost works fine, except that the Swift runtime library redefines an LLVM symbol here:

We’re building on Linux and statically linking everything together. I think that the symbol above ends up being “not a problem” on apple platforms, because this code gets linked into a dylib with -fvisibility-hidden to hide the symbol. Is there some other reasonable way for the Swift runtime to achieve whatever it is doing?

For example, Dmitri suggests that we could go with a header-only solution like this:

namespace llvm { namespace hashing { namespace detail {
template<typename Ignored = void>
struct FixedSeedOverrideHolder {
static size_t fixed_seed_override;
};

template<typename Ignored>
size_t FixedSeedOverrideHolder<Ignored>::fixed_seed_override = 0;

size_t GetFixedSeedOverride() {
return FixedSeedOverrideHolder<>::fixed_seed_override;
}
}}}

Other ideas?

-Chris

Does it work if you make the fixed_seed_override definition in the runtime a weak definition? That should let it get replaced by the definition from your other library.

Yes, it would probably work. How would you prefer that to be spelled? __weak or some macro?

It looks like LLVM already provides a macro LLVM_ATTRIBUTE_WEAK.

We generally don’t want to expose weak symbols because they affect launch time. (I know Apple build processes flag any libraries containing weak definitions.) So that’s not going to fly either.

Maybe we shouldn’t use LLVM’s hashing directly in the runtime.

The runtime’s definition is only used internally and has hidden visibility. It won’t be exported from the dylib as a weak definition.

Yeah, llvm’s hashing stuff should probably work differently, or swift runtime shouldn’t use it.

Thanks for the pointer! I tried this, by changing in llvm/include/llvm/ADT/Hashing.h from
extern size_t fixed_seed_override;
to
extern size_t LLVM_ATTRIBUTE_WEAK fixed_seed_override;

We are still getting linker error: /linux/libswiftCore.a(GlobalObjects.pic.o): multiple definition of 'llvm::hashing::detail::fixed_seed_override'
(recall the project links with another copy of LLVM)

From existing code it seems LLVM_ATTRIBUTE_WEAK is only applied to functions.

https://en.wikipedia.org/wiki/Weak_symbol suggested attribute((weak)), which I also tried but got the same linker error.

LLVM_ATTRIBUTE_WEAK is a macro that expands to __attribute__((weak)). If you change the declaration in the LLVM header then both the Swift runtime and LLVM definitions are going to end up being weak, and we only want the Swift runtime’s definition to be. Try declaring fixed_seed_override as weak only in the Swift runtime source file that defines it.

I tried changing
size_t llvm::hashing::detail::fixed_seed_override = 0;

to
size_t LLVM_ATTRIBUTE_WEAK llvm::hashing::detail::fixed_seed_override = 0;

in the Swfit copy of llvm/lib/Support/Hashing.cpp, and am still getting the same linker error.

Update:

I changed
size_t fixed_seed_override = 0;
to
size_t LLVM_ATTRIBUTE_WEAK fixed_seed_override = 0;

in swift/stdlib/public/stubs/GlobalObjects.cpp, and that appears to work.

Thanks Joe for the suggestion.


Update: I spoke too soon. This is still hitting the same linker error. Any other suggestions?

Hm, I’m not sure why you would still be getting linker errors in that case, but I agree in principle with Jordan that it’d be better if a statically-linked Swift runtime didn’t include symbols that might collide with LLVM. There are a few other things we could do to mitigate the problem. As Jordan suggested, we could copy the LLVM hashing libraries Swift uses into the runtime and tweak them not to collide with LLVM. It might also be a good idea to change how static libswiftCore.a is built so that it’s incrementally linked into a single .o file, which then only exports the public interface of the runtime and standard library. This would ensure in the long term that implementation details of the Swift runtime and standard library don’t interfere with any other library a statically-linked executable may use in tandem with them.

I tried the weak symbol approach again and it worked. Probably was using a bad config env before – sorry about the confusion.

There’s a PR to upstream this change: https://github.com/apple/swift/pull/14791

I’m sorry I didn’t reply to this sooner, but this isn’t good enough for the static version of the stdlib. We might be able to get away with this some day in the future when the stdlib only shipped in dylib form as part of the OS, but we can’t take it today.

This definition will still have hidden visibility even if you statically link it into a command-line executable on macOS. I don’t know what you’re concerned about.

FWIW, this solution has seemed to worked for us on Linux with libswiftCore.a statically linked into the same binary as other libraries that include LLVMSupport.

Thanks, Joe explained this to me more explicitly and assuaged my concerns. Sorry for the disruption.