building static binaries / reducing library dependencies?


(Brian Swetland) #1

I've noticed that the standard "swift build" binaries are dynamic (which
makes sense), and there are quite a few dependencies. For a trivial hello
world, the hello binary depends on:
libswiftCore.so
libstdc++.so.6
libm.so
libgcc_s.so.1
libc.so.6

and libswiftCore.so further depends on:
libpthread.so.0
libdl.so.2
libicuuc.so.55
libicui18n.so.55
libbsd.so.0
libstdc++.so.6
libm.so.6
libgcc_s.so.1
libc.so.6
ld-linux-x86-64.so.2

If I want to build something I can hope will run on other linux machines
with potentially arbitrary distributions, it'd be really nice to be able to
build fully static, or at least more self-contained binaries.

Both license-wise and operationally, linking glibc staticly is problematic,
but at least glibc has very strong backward compatibility with itself.
libstdc++ on the other hand is something I'd always want staticly linked to
not be at the whim of ABI breakages from release to release.

I'm curious about the libstdc++ dependency, since I was under the
impression the swift runtime was written in swift. What part of the
runtime or generated code causes this dependency to exist?

The libc dependency could be simplified by supporting musl-libc (small, MIT
licensed, pretty source-compatible with glibc).

Given that libbsd and libicu are under permissive licenses and libstdc++
(if it's needed) has a linkage exception), it'd be nice to have a way to
include those staticly (in either libswiftCore.so or final binaries),
resulting in the only remaining dependencies being on
libc/libc/libpthread/libdl which have extremely robust and well maintained
API/ABI backward compatibility.

This should allow for linux binaries which have significantly better
chances of working correctly on an arbitrary linux machine.


(Joe Groff) #2

It should be possible to configure Swift to statically link libswiftCore.so on Linux. We dynamically link on Apple platforms for future-proofing, but that's not a concern on Linux as far as I can see. The Swift core runtime is written in C++, as are some edges of the standard library, hence the libstdc++ dependency. If you're concerned about licensing issues statically linking libstdc++, then building against LLVM's UIUC-licensed libc++ might be a possibility.

-Joe

···

On Dec 3, 2015, at 1:54 PM, Brian Swetland <swetland@frotz.net> wrote:

I've noticed that the standard "swift build" binaries are dynamic (which makes sense), and there are quite a few dependencies. For a trivial hello world, the hello binary depends on:
libswiftCore.so
libstdc++.so.6
libm.so
libgcc_s.so.1
libc.so.6

and libswiftCore.so further depends on:
libpthread.so.0
libdl.so.2
libicuuc.so.55
libicui18n.so.55
libbsd.so.0
libstdc++.so.6
libm.so.6
libgcc_s.so.1
libc.so.6
ld-linux-x86-64.so.2

If I want to build something I can hope will run on other linux machines with potentially arbitrary distributions, it'd be really nice to be able to build fully static, or at least more self-contained binaries.

Both license-wise and operationally, linking glibc staticly is problematic, but at least glibc has very strong backward compatibility with itself. libstdc++ on the other hand is something I'd always want staticly linked to not be at the whim of ABI breakages from release to release.

I'm curious about the libstdc++ dependency, since I was under the impression the swift runtime was written in swift. What part of the runtime or generated code causes this dependency to exist?

The libc dependency could be simplified by supporting musl-libc (small, MIT licensed, pretty source-compatible with glibc).

Given that libbsd and libicu are under permissive licenses and libstdc++ (if it's needed) has a linkage exception), it'd be nice to have a way to include those staticly (in either libswiftCore.so or final binaries), resulting in the only remaining dependencies being on libc/libc/libpthread/libdl which have extremely robust and well maintained API/ABI backward compatibility.

This should allow for linux binaries which have significantly better chances of working correctly on an arbitrary linux machine.


(Brian Swetland) #3

libstdc++ has a pretty clear linkage exception that allows it to be bolted
into whatever, so the license is less of a concern than the fact that it
often is not forward/backward compatible across versions. Static linking
libc/glibc is problematic license-wise, but since it's very stable from a
compatibility viewpoint linking it dynamically is not a worry. Also
glibc's dns resolver and some other features do not work correctly when
it's linked staticly.

An option to link the non-libc/libm/libpthread/libdl dependencies of
libswiftCore.so into it on Linux would allow for far more robust binaries
(worst case you bundle the particular libswiftCore.so you built against
with your app and you should be pretty safe).

For the case of 100% static binaries, an option to build with musl-libc
would probably be the simplest, but this is a less critical improvement
than simply removing dynamic dependencies on more fragile libraries.

Brian

···

On Thu, Dec 3, 2015 at 1:57 PM, Joe Groff <jgroff@apple.com> wrote:

> On Dec 3, 2015, at 1:54 PM, Brian Swetland <swetland@frotz.net> wrote:
>
> If I want to build something I can hope will run on other linux machines
with potentially arbitrary distributions, it'd be really nice to be able to
build fully static, or at least more self-contained binaries.
>
> Both license-wise and operationally, linking glibc staticly is
problematic, but at least glibc has very strong backward compatibility with
itself. libstdc++ on the other hand is something I'd always want staticly
linked to not be at the whim of ABI breakages from release to release.
>
> I'm curious about the libstdc++ dependency, since I was under the
impression the swift runtime was written in swift. What part of the
runtime or generated code causes this dependency to exist?
>
> The libc dependency could be simplified by supporting musl-libc (small,
MIT licensed, pretty source-compatible with glibc).
>
> Given that libbsd and libicu are under permissive licenses and libstdc++
(if it's needed) has a linkage exception), it'd be nice to have a way to
include those staticly (in either libswiftCore.so or final binaries),
resulting in the only remaining dependencies being on
libc/libc/libpthread/libdl which have extremely robust and well maintained
API/ABI backward compatibility.
>
> This should allow for linux binaries which have significantly better
chances of working correctly on an arbitrary linux machine.

It should be possible to configure Swift to statically link
libswiftCore.so on Linux. We dynamically link on Apple platforms for
future-proofing, but that's not a concern on Linux as far as I can see. The
Swift core runtime is written in C++, as are some edges of the standard
library, hence the libstdc++ dependency. If you're concerned about
licensing issues statically linking libstdc++, then building against LLVM's
UIUC-licensed libc++ might be a possibility.


(Joe Groff) #4

Sounds good. Statically linking libstdc++ and libswiftCore while leaving glibc dynamically linked should be fine. Porting to musl libc might be interesting too, but I'm not sure how dependent the core libs are/will be on glibc stuff.

-Joe

···

On Dec 3, 2015, at 3:24 PM, Brian Swetland <swetland@frotz.net> wrote:

On Thu, Dec 3, 2015 at 1:57 PM, Joe Groff <jgroff@apple.com <mailto:jgroff@apple.com>> wrote:

> On Dec 3, 2015, at 1:54 PM, Brian Swetland <swetland@frotz.net <mailto:swetland@frotz.net>> wrote:
>
> If I want to build something I can hope will run on other linux machines with potentially arbitrary distributions, it'd be really nice to be able to build fully static, or at least more self-contained binaries.
>
> Both license-wise and operationally, linking glibc staticly is problematic, but at least glibc has very strong backward compatibility with itself. libstdc++ on the other hand is something I'd always want staticly linked to not be at the whim of ABI breakages from release to release.
>
> I'm curious about the libstdc++ dependency, since I was under the impression the swift runtime was written in swift. What part of the runtime or generated code causes this dependency to exist?
>
> The libc dependency could be simplified by supporting musl-libc (small, MIT licensed, pretty source-compatible with glibc).
>
> Given that libbsd and libicu are under permissive licenses and libstdc++ (if it's needed) has a linkage exception), it'd be nice to have a way to include those staticly (in either libswiftCore.so or final binaries), resulting in the only remaining dependencies being on libc/libc/libpthread/libdl which have extremely robust and well maintained API/ABI backward compatibility.
>
> This should allow for linux binaries which have significantly better chances of working correctly on an arbitrary linux machine.

It should be possible to configure Swift to statically link libswiftCore.so on Linux. We dynamically link on Apple platforms for future-proofing, but that's not a concern on Linux as far as I can see. The Swift core runtime is written in C++, as are some edges of the standard library, hence the libstdc++ dependency. If you're concerned about licensing issues statically linking libstdc++, then building against LLVM's UIUC-licensed libc++ might be a possibility.

libstdc++ has a pretty clear linkage exception that allows it to be bolted into whatever, so the license is less of a concern than the fact that it often is not forward/backward compatible across versions. Static linking libc/glibc is problematic license-wise, but since it's very stable from a compatibility viewpoint linking it dynamically is not a worry. Also glibc's dns resolver and some other features do not work correctly when it's linked staticly.

An option to link the non-libc/libm/libpthread/libdl dependencies of libswiftCore.so into it on Linux would allow for far more robust binaries (worst case you bundle the particular libswiftCore.so you built against with your app and you should be pretty safe).