Hopefully this is the correct category to post this.
Currently on Ubuntu Linux, the Swift toolchain uses the system provided libicu which presents 2 problems:
The version of ICU is quite old on 14.04 and 16.04, which is the source of some bugs in scl-foundation. Currently some tests are disabled to prevent CI failures.
The way that libicu is built requires the use of dlopen(), thus making it unusable in a statically linked executable.
I propose building libicu by default as part of the Linux build. This will allow using the latest version and also it can be built without the use of dlopen().
I want to sound a note of caution here: while this would work, it presents risks if you want to link (either directly or transitively) to the system libicu.
If this approach is taken, we need to ensure that the shipped ICU does not expose clashing symbols with the ICU provided by the system. This will likely require mangling them, which may be troublesome.
libICU can be built with symbol mangling and this is how it is currently done on Ubuntu16.04, eg:
$ nm ~/swift-DEVELOPMENT-SNAPSHOT-2018-08-26-a-ubuntu16.04/usr/lib/swift/linux/libswiftCore.so |grep 'U ubrk'
U ubrk_close_55
U ubrk_following_55
U ubrk_open_55
U ubrk_preceding_55
U ubrk_setText_55
U ubrk_setUText_55
and on Ubuntu18.04
$ nm swift-4.2-RELEASE-ubuntu18.04/usr/lib/swift/linux/libswiftCore.so |grep 'U ubrk'
U ubrk_close_60
U ubrk_following_60
U ubrk_open_60
U ubrk_preceding_60
U ubrk_setText_60
In fact the libXML used by swift-corelibs-foundation will still link to the system ICU unless it is recompiled.
I think we've talked about this before, but this is a direction I'd absolutely like to go with the support of the standard library team (so we're all on the same page on using one version of ICU).
This also might help move us to a position where we don't have to create a toolchain per version of Ubuntu. Not sure what other factors are forcing that, but this is a big one.
Huge +1. I've mentioned this in a couple PRs and JIRAs but the Unicode.Scalar.Properties APIs are difficult to trust on Linux without a known version of ICU being built into the standard library. On Apple platforms we can at least make specific properties @available based on OS releases that map to ICU versions, but we don't have that ability for Linux, so there's no way (without something extreme, like checking the ICU version at runtime) to distinguish between "this property is false" vs. "this property isn't supported" in client code.
Has anyone got any updates on this? It looks like having a standard way to compile ICU together with the toolchain would definitely help in unifying the build process for other Linux distributions, Android toolchain and WebAssembly toolchain. While developing WebAssembly toolchain I currently clone https://github.com/unicode-org/icu at one level above the workspace directory and then link icu/icu4c from that clone to ${WORKSPACE}/icu, but that's not very convenient. It would be great if utils/update-checkout script could clone or download the correct version of ICU and put in a place that the current build infrastructure can agree on across all platforms.
Yes, having official Linux Swift packages usable in any distro would be a huge step. Other languagesdistribute a single package for all Linux distros instead of one per Ubuntu release.
Right, there are multiple possibilities I see here:
bundling libicu allows Swift compiler to be used on Linux distros that don't have a specific version of libicu installed. But there are still a couple of other dependencies like Glibc that won't allow us to run the compiler on musl distros like Alpine, right?
statically linked self-contained ELF executables compiled from Swift packages. I assume these would link with the bundled libicu? And would also require some support from SwiftPM as well? These binaries would still link with Glibc, but statically, which would allow a user to just copy it to an Alpine instance and run there without a problem, I guess?
This was implemented for Swift 5. If you are building on Linux you can add the --libicu option to the build script and it will build libICU as part of the build.
The option has also been added into the buildbot_linux preset so it will build automatically if that preset it used - this is how CI builds and tests it.