Whenever I update the installed Linux snapshot, my binaries break, with errors of the form:
[programname]: symbol lookup error: [programname]: undefined symbol: _TFOs7Processau9argumentsGSaSS_
I assume this is the standard library symbol Process.arguments (which is used in my program).
I guess this is because the standard library is not statically linked/embedded in this executable. This happens by default on OSX, but I guess might not on Linux.
Is that a bug?
Is there a way to force statically link the standard library on Linux, so my binaries work without a recompile each new snapshot?
AFAIK, the ABI isn’t stable, and therefore whenever a binary is compiled all of its dependencies must also be re-compiled.
- Brandon
···
From: <swift-users-bounces@swift.org<mailto:swift-users-bounces@swift.org>> on behalf of Drew Crawford via swift-users <swift-users@swift.org<mailto:swift-users@swift.org>>
Reply-To: Drew Crawford <drew@sealedabstract.com<mailto:drew@sealedabstract.com>>
Date: Saturday, February 13, 2016 at 7:54 PM
To: "swift-users@swift.org<mailto:swift-users@swift.org>" <swift-users@swift.org<mailto:swift-users@swift.org>>
Subject: [swift-users] statically link the standard library on Linux
Whenever I update the installed Linux snapshot, my binaries break, with errors of the form:
[programname]: symbol lookup error: [programname]: undefined symbol: _TFOs7Processau9argumentsGSaSS_
I assume this is the standard library symbol Process.arguments (which is used in my program).
I guess this is because the standard library is not statically linked/embedded in this executable. This happens by default on OSX, but I guess might not on Linux.
* Is that a bug?
* Is there a way to force statically link the standard library on Linux, so my binaries work without a recompile each new snapshot?
We have a ticket open to implement static linking of swift libs.
If anyone on the Swift team knows how to do this and can comment here then I’m sure we can get it implemented pretty quickly.
···
On Feb 13, 2016, at 7:54 PM, Drew Crawford via swift-users <swift-users@swift.org> wrote:
Whenever I update the installed Linux snapshot, my binaries break, with errors of the form:
[programname]: symbol lookup error: [programname]: undefined symbol: _TFOs7Processau9argumentsGSaSS_
I assume this is the standard library symbol Process.arguments (which is used in my program).
I guess this is because the standard library is not statically linked/embedded in this executable. This happens by default on OSX, but I guess might not on Linux.
Is that a bug?
Is there a way to force statically link the standard library on Linux, so my binaries work without a recompile each new snapshot?
tll;dr the standard library is forcibly dynamically-linked on Linux, and we need a way to opt out of that from the swift frontend. I have not gone further with that since A) my C++-foo is not very excellent and B) core hasn't blessed any particular solution I proffered in that thread. If you can make progress there I would be grateful.
Then it is just a matter of getting the flags right; in practice, OSX wants -L path/to/swift_static -lc++ -framework Foundation -Xlinker -force_load_swift_libs, whereas Linux wants (I believe) -licuuc -licu18n -lbsd /path/to/swift_static/a.a /path/to/swift_static/b.a /path/to/swift_static/c.a ... There may be an equivalent to force_load_swift_libs that knows what libs to load, but I haven't found it yet.
However as long as the Swift frontend forcibly injects the dynamic link we are SOL, so some progress needs to be made in the frontend first.
···
On Feb 17, 2016, at 1:37 PM, Max Howell <max.howell@apple.com> wrote:
We have a ticket open to implement static linking of swift libs.
If anyone on the Swift team knows how to do this and can comment here then I’m sure we can get it implemented pretty quickly.
On Feb 13, 2016, at 7:54 PM, Drew Crawford via swift-users <swift-users@swift.org <mailto:swift-users@swift.org>> wrote:
Whenever I update the installed Linux snapshot, my binaries break, with errors of the form:
[programname]: symbol lookup error: [programname]: undefined symbol: _TFOs7Processau9argumentsGSaSS_
I assume this is the standard library symbol Process.arguments (which is used in my program).
I guess this is because the standard library is not statically linked/embedded in this executable. This happens by default on OSX, but I guess might not on Linux.
Is that a bug?
Is there a way to force statically link the standard library on Linux, so my binaries work without a recompile each new snapshot?
Static vs dynamic building of libraries may also have an interesting impact on the build system, particularly in how we set visibility for emitted symbols. For executables and dylibs, our current mapping of access control to visibility and linkage makes sense:
- public has default visibility,
- internal has hidden visibility (__attribute__((visibility("hidden"))) in Clang),
- private has private linkage (like 'static' in C).
However, when statically linking a library into another executable or dylib, the static library's public symbols in most cases probably ought to become hidden in the final target, unless the dylib or executable explicitly reexports the static library's module somehow.
-Joe
···
On Feb 17, 2016, at 11:37 AM, Max Howell via swift-users <swift-users@swift.org> wrote:
We have a ticket open to implement static linking of swift libs.
If anyone on the Swift team knows how to do this and can comment here then I’m sure we can get it implemented pretty quickly.
We also want to move to using swift_begin.o and swift_end.o files
instead of the linker script. With these files, it is *critical* that
they are added at the very beginning and at the very end of the
command line, otherwise one is guaranteed to get a subtle silent
miscompile.
I think we need to add a user-friendly driver-level flag for static
linking, that allows to decide on per-library basis.
Dmitri
···
On Wed, Feb 17, 2016 at 12:40 PM, Drew Crawford via swift-users <swift-users@swift.org> wrote:
tll;dr the standard library is forcibly dynamically-linked on Linux, and we
need a way to opt out of that from the swift frontend. I have not gone
further with that since A) my C++-foo is not very excellent and B) core
hasn't blessed any particular solution I proffered in that thread. If you
can make progress there I would be grateful.
Then it is just a matter of getting the flags right; in practice, OSX wants
-L path/to/swift_static -lc++ -framework Foundation -Xlinker
-force_load_swift_libs, whereas Linux wants (I believe) -licuuc -licu18n
-lbsd /path/to/swift_static/a.a /path/to/swift_static/b.a
/path/to/swift_static/c.a ... There may be an equivalent to
force_load_swift_libs that knows what libs to load, but I haven't found it
yet.
We have a ticket open to implement static linking of swift libs.
If anyone on the Swift team knows how to do this and can comment here then I’m sure we can get it implemented pretty quickly.
Static vs dynamic building of libraries may also have an interesting impact on the build system, particularly in how we set visibility for emitted symbols. For executables and dylibs, our current mapping of access control to visibility and linkage makes sense:
- public has default visibility,
- internal has hidden visibility (__attribute__((visibility("hidden"))) in Clang),
- private has private linkage (like 'static' in C).
However, when statically linking a library into another executable or dylib, the static library's public symbols in most cases probably ought to become hidden in the final target, unless the dylib or executable explicitly reexports the static library's module somehow.
I’ve worried about such things, but my conclusions were there weren’t any practically compelling reasons to worry about it too much.
My reasoning being: when writing a Swift file the compiler won’t give you access to those symbols unless you `import` that module, in which case you want them.
Maybe not huge problems, but it's a secrecy violation and unwanted ABI liability, and also limits LTO-level optimization opportunities since exported functions have to preserve their existence and calling convention.
-Joe
···
On Feb 19, 2016, at 12:37 PM, Max Howell <max.howell@apple.com> wrote:
We have a ticket open to implement static linking of swift libs.
If anyone on the Swift team knows how to do this and can comment here then I’m sure we can get it implemented pretty quickly.
Static vs dynamic building of libraries may also have an interesting impact on the build system, particularly in how we set visibility for emitted symbols. For executables and dylibs, our current mapping of access control to visibility and linkage makes sense:
- public has default visibility,
- internal has hidden visibility (__attribute__((visibility("hidden"))) in Clang),
- private has private linkage (like 'static' in C).
However, when statically linking a library into another executable or dylib, the static library's public symbols in most cases probably ought to become hidden in the final target, unless the dylib or executable explicitly reexports the static library's module somehow.
I’ve worried about such things, but my conclusions were there weren’t any practically compelling reasons to worry about it too much.
My reasoning being: when writing a Swift file the compiler won’t give you access to those symbols unless you `import` that module, in which case you want them.
Static vs dynamic building of libraries may also have an interesting impact on the build system, particularly in how we set visibility for emitted symbols. For executables and dylibs, our current mapping of access control to visibility and linkage makes sense:
- public has default visibility,
- internal has hidden visibility (__attribute__((visibility("hidden"))) in Clang),
- private has private linkage (like 'static' in C).
However, when statically linking a library into another executable or dylib, the static library's public symbols in most cases probably ought to become hidden in the final target, unless the dylib or executable explicitly reexports the static library's module somehow.
I’ve worried about such things, but my conclusions were there weren’t any practically compelling reasons to worry about it too much.
My reasoning being: when writing a Swift file the compiler won’t give you access to those symbols unless you `import` that module, in which case you want them.
Unless there is something I’m missing?
Maybe not huge problems, but it's a secrecy violation and unwanted ABI liability, and also limits LTO-level optimization opportunities since exported functions have to preserve their existence and calling convention.
Makes sense, can we solve it? Our current direction is certainly one of: stuff all the (library) modules into a single linkage unit, be that a single dylib, a static library or a framework.
If the frontend knows it's building for a static library, it could conceivably set the correct visibility contingent on that flag. There may also be ld flags to suppress exporting symbols from particular .o and .a files post-hoc.
-Joe
···
On Feb 19, 2016, at 1:06 PM, Max Howell <max.howell@apple.com> wrote:
Static vs dynamic building of libraries may also have an interesting impact on the build system, particularly in how we set visibility for emitted symbols. For executables and dylibs, our current mapping of access control to visibility and linkage makes sense:
- public has default visibility,
- internal has hidden visibility (__attribute__((visibility("hidden"))) in Clang),
- private has private linkage (like 'static' in C).
However, when statically linking a library into another executable or dylib, the static library's public symbols in most cases probably ought to become hidden in the final target, unless the dylib or executable explicitly reexports the static library's module somehow.
I’ve worried about such things, but my conclusions were there weren’t any practically compelling reasons to worry about it too much.
My reasoning being: when writing a Swift file the compiler won’t give you access to those symbols unless you `import` that module, in which case you want them.
Unless there is something I’m missing?
Maybe not huge problems, but it's a secrecy violation and unwanted ABI liability, and also limits LTO-level optimization opportunities since exported functions have to preserve their existence and calling convention.
Makes sense, can we solve it? Our current direction is certainly one of: stuff all the (library) modules into a single linkage unit, be that a single dylib, a static library or a framework.
The --exclude-libs linker option does exactly that.
···
On Fri, Feb 19, 2016 at 4:08 PM, Joe Groff via swift-users < swift-users@swift.org> wrote:
> On Feb 19, 2016, at 1:06 PM, Max Howell <max.howell@apple.com> wrote:
>
>>>> Static vs dynamic building of libraries may also have an interesting
impact on the build system, particularly in how we set visibility for
emitted symbols. For executables and dylibs, our current mapping of access
control to visibility and linkage makes sense:
>>>>
>>>> - public has default visibility,
>>>> - internal has hidden visibility
(__attribute__((visibility("hidden"))) in Clang),
>>>> - private has private linkage (like 'static' in C).
>>>>
>>>> However, when statically linking a library into another executable or
dylib, the static library's public symbols in most cases probably ought to
become hidden in the final target, unless the dylib or executable
explicitly reexports the static library's module somehow.
>>>
>>> I’ve worried about such things, but my conclusions were there weren’t
any practically compelling reasons to worry about it too much.
>>>
>>> My reasoning being: when writing a Swift file the compiler won’t give
you access to those symbols unless you `import` that module, in which case
you want them.
>>>
>>> Unless there is something I’m missing?
>>
>> Maybe not huge problems, but it's a secrecy violation and unwanted ABI
liability, and also limits LTO-level optimization opportunities since
exported functions have to preserve their existence and calling convention.
>
> Makes sense, can we solve it? Our current direction is certainly one of:
stuff all the (library) modules into a single linkage unit, be that a
single dylib, a static library or a framework.
If the frontend knows it's building for a static library, it could
conceivably set the correct visibility contingent on that flag. There may
also be ld flags to suppress exporting symbols from particular .o and .a
files post-hoc.