Build Error: Relocation R_X86_64_PC32


(Ryan Lovelett) #1

I have a daily build of Swift that I make on a Arch Linux box. On the
morning of February 23rd the build starting failing with the following
error:

/usr/bin/ld: stdlib/public/core/linux/x86_64/Swift.o: relocation
R_X86_64_PC32 against protected symbol `_TMPSa' can not be used when
making a shared object
/usr/bin/ld: final link failed: Bad value
clang-3.7: error: linker command failed with exit code 1 (use -v to see
invocation)

I build with these arguments:

utils/build-script --assertions --no-swift-stdlib-assertions --llbuild
--swiftpm --xctest --build-subdir=buildbot_linux --lldb --release --test
--validation-test --foundation -- --swift-enable-ast-verifier=0
--install-swift --install-lldb --install-llbuild --install-swiftpm
--install-xctest --install-prefix=/usr
'--swift-install-components=compiler;clang-builtin-headers;stdlib;sdk-overlay;license'
--build-swift-static-stdlib=1 --build-swift-stdlib-unittest-extra=1
--skip-test-lldb=1 --test-installable-package=1
--install-destdir=/build/swiftc/pkg/swiftc/
--installable-package=/build/swiftc/swift-3.0.20160208a.r613.g0a87b5a.tar.xz
--install-foundation --reconfigure --skip-test-cmark=1
--skip-test-lldb=1 --skip-test-swift=1 --skip-test-llbuild=1
--skip-test-swiftpm=1 --skip-test-xctest=1 --skip-test-foundation=1
--skip-test-osx=1 --skip-test-ios=1 --skip-test-ios-simulator=1
--skip-test-tvos=1 --skip-test-tvos-simulator=1 --skip-test-watchos=1
--skip-test-watchos-simulator=1 --skip-test-validation=1
--skip-test-optimized=1 --reconfigure=1 --verbose-build=1

Would anyone have an idea of what could have gone wrong (before I spend
time running `git bisect` and building on all the different repos)?


(Dmitri Gribenko) #2

Hi Ryan,

If you are running binutils 2.26, this is a known issue. A workaround
is to use the gold linker (build-script-impl has an option for this).

Dmitri

···

On Mon, Feb 29, 2016 at 7:51 PM, Ryan Lovelett via swift-dev <swift-dev@swift.org> wrote:

I have a daily build of Swift that I make on a Arch Linux box. On the
morning of February 23rd the build starting failing with the following
error:

/usr/bin/ld: stdlib/public/core/linux/x86_64/Swift.o: relocation
R_X86_64_PC32 against protected symbol `_TMPSa' can not be used when
making a shared object
/usr/bin/ld: final link failed: Bad value
clang-3.7: error: linker command failed with exit code 1 (use -v to see
invocation)

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Ryan Lovelett) #3

Perfect.

That corresponds with what I am seeing. On the morning of the 22nd (when
it built fine) it used binutils-2.25.1-3 and then on the morning of the
23rd (when it failed) it used binutils-2.26-1.

I'll look into the build flag for the gold linker

Out of curiosity is there a issue in the bug tracker for this that I
could follow; or should I file one?

Thanks again for the quick response.

···

On Mon, Feb 29, 2016, at 10:55 PM, Dmitri Gribenko wrote:

On Mon, Feb 29, 2016 at 7:51 PM, Ryan Lovelett via swift-dev > <swift-dev@swift.org> wrote:
> I have a daily build of Swift that I make on a Arch Linux box. On the
> morning of February 23rd the build starting failing with the following
> error:
>
> /usr/bin/ld: stdlib/public/core/linux/x86_64/Swift.o: relocation
> R_X86_64_PC32 against protected symbol `_TMPSa' can not be used when
> making a shared object
> /usr/bin/ld: final link failed: Bad value
> clang-3.7: error: linker command failed with exit code 1 (use -v to see
> invocation)

Hi Ryan,

If you are running binutils 2.26, this is a known issue. A workaround
is to use the gold linker (build-script-impl has an option for this).

Dmitri

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Dmitri Gribenko) #4

IIRC there is no issue in the bug tracker, please feel free to file one!

Also, if you are familiar with binutils internals, it might be helpful
to bisect the linker.

Meador Inge has also mentioned this bug:
https://llvm.org/bugs/show_bug.cgi?id=26580

Right now, it does not seem clear to me that we have an understanding
what the issue is, which component is broken (LLVM, Swift's IRGen, or
binutils), and if the breakage was accidental or intentional (e.g., to
fix something else).

Previous thread on this topic:

https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20160215/001176.html
https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20160222/001190.html

Dmitri

···

On Mon, Feb 29, 2016 at 8:04 PM, Ryan Lovelett <swift-dev@ryan.lovelett.me> wrote:

Perfect.

That corresponds with what I am seeing. On the morning of the 22nd (when
it built fine) it used binutils-2.25.1-3 and then on the morning of the
23rd (when it failed) it used binutils-2.26-1.

I'll look into the build flag for the gold linker

Out of curiosity is there a issue in the bug tracker for this that I
could follow; or should I file one?

Thanks again for the quick response.

On Mon, Feb 29, 2016, at 10:55 PM, Dmitri Gribenko wrote:

On Mon, Feb 29, 2016 at 7:51 PM, Ryan Lovelett via swift-dev >> <swift-dev@swift.org> wrote:
> I have a daily build of Swift that I make on a Arch Linux box. On the
> morning of February 23rd the build starting failing with the following
> error:
>
> /usr/bin/ld: stdlib/public/core/linux/x86_64/Swift.o: relocation
> R_X86_64_PC32 against protected symbol `_TMPSa' can not be used when
> making a shared object
> /usr/bin/ld: final link failed: Bad value
> clang-3.7: error: linker command failed with exit code 1 (use -v to see
> invocation)

Hi Ryan,

If you are running binutils 2.26, this is a known issue. A workaround
is to use the gold linker (build-script-impl has an option for this).

Dmitri

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Michael Buckley) #5

I have been trying to compile on a fresh Ubuntu 15.10 image since the 27th,
and have been unable to succeed, even when using the gold linker. Any
chance this is the same issue?

I installed the OS, and ran the following command.

sudo apt-get install git cmake ninja-build clang python uuid-dev
libicu-dev icu-devtools libbsd-dev libedit-dev libxml2-dev
libsqlite3-dev swig libpython-dev libncurses5-dev pkg-config

Then I ran

git clone https://github.com/apple/swift.git
cd swift
./utils/update-checkout --clone

Finally, I ran

cd ..
./swift/utils/build-script -c -R --llbuild --swiftpm --xctest --foundation
-- --reconfigure --use-gold-linker

Relevant output below.

Manually-specified variables were not used by the project:

    LLVM_TOOL_SWIFT_BUILD

-- Build files have been written to:
/home/buckley/swift/build/Ninja-ReleaseAssert/llvm-linux-x86_64
++ cmake_config_opt llvm
++ product=llvm
++ [[ Ninja == \X\c\o\d\e ]]
+ /usr/bin/cmake --build
/home/buckley/swift/build/Ninja-ReleaseAssert/llvm-linux-x86_64 -- -j4 all
[1484/2025] Building CXX object
tools/clang/lib/ASTMatchers/Dynamic/CMakeFiles/clangDynamicASTMatchers.dir/Registry.cpp.o
FAILED: /usr/bin/clang++ -DCLANG_ENABLE_ARCMT
-DCLANG_ENABLE_OBJC_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER
-DGTEST_HAS_RTTI=0 -D_DEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS
-D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -fno-stack-protector -fPIC
-fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings
-Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long
-Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
-std=c++11 -fcolor-diagnostics -ffunction-sections -fdata-sections
-fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3
-Itools/clang/lib/ASTMatchers/Dynamic
-I/home/buckley/swift/llvm/tools/clang/lib/ASTMatchers/Dynamic
-I/home/buckley/swift/llvm/tools/clang/include -Itools/clang/include
-Iinclude -I/home/buckley/swift/llvm/include -UNDEBUG -fno-exceptions
-fno-rtti -MMD -MT
tools/clang/lib/ASTMatchers/Dynamic/CMakeFiles/clangDynamicASTMatchers.dir/Registry.cpp.o
-MF
tools/clang/lib/ASTMatchers/Dynamic/CMakeFiles/clangDynamicASTMatchers.dir/Registry.cpp.o.d
-o
tools/clang/lib/ASTMatchers/Dynamic/CMakeFiles/clangDynamicASTMatchers.dir/Registry.cpp.o
-c /home/buckley/swift/llvm/tools/clang/lib/ASTMatchers/Dynamic/Registry.cpp
clang: error: unable to execute command: Killed
clang: error: clang frontend command failed due to signal (use -v to see
invocation)
Ubuntu clang version 3.6.2-1 (tags/RELEASE_362/final) (based on LLVM 3.6.2)
Target: x86_64-pc-linux-gnu
Thread model: posix
clang: note: diagnostic msg: PLEASE submit a bug report to
http://bugs.debian.org/ and include the crash backtrace, preprocessed
source, and associated run script.
clang: note: diagnostic msg:

···

********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang: note: diagnostic msg: /tmp/Registry-1ff941.cpp
clang: note: diagnostic msg: /tmp/Registry-1ff941.sh
clang: note: diagnostic msg:

********************
[1484/2025] Building CXX object
tools/clang/lib/ASTMatchers/CMakeFiles/clangASTMatchers.dir/ASTMatchFinder.cpp.o
ninja: build stopped: subcommand failed.
./swift/utils/build-script: command terminated with a non-zero exit status
1, aborting

On Mon, Feb 29, 2016 at 8:16 PM, Dmitri Gribenko via swift-dev < swift-dev@swift.org> wrote:

IIRC there is no issue in the bug tracker, please feel free to file one!

Also, if you are familiar with binutils internals, it might be helpful
to bisect the linker.

Meador Inge has also mentioned this bug:
https://llvm.org/bugs/show_bug.cgi?id=26580

Right now, it does not seem clear to me that we have an understanding
what the issue is, which component is broken (LLVM, Swift's IRGen, or
binutils), and if the breakage was accidental or intentional (e.g., to
fix something else).

Previous thread on this topic:

https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20160215/001176.html

https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20160222/001190.html

Dmitri

On Mon, Feb 29, 2016 at 8:04 PM, Ryan Lovelett > <swift-dev@ryan.lovelett.me> wrote:
> Perfect.
>
> That corresponds with what I am seeing. On the morning of the 22nd (when
> it built fine) it used binutils-2.25.1-3 and then on the morning of the
> 23rd (when it failed) it used binutils-2.26-1.
>
> I'll look into the build flag for the gold linker
>
> Out of curiosity is there a issue in the bug tracker for this that I
> could follow; or should I file one?
>
> Thanks again for the quick response.
>
> On Mon, Feb 29, 2016, at 10:55 PM, Dmitri Gribenko wrote:
>> On Mon, Feb 29, 2016 at 7:51 PM, Ryan Lovelett via swift-dev > >> <swift-dev@swift.org> wrote:
>> > I have a daily build of Swift that I make on a Arch Linux box. On the
>> > morning of February 23rd the build starting failing with the following
>> > error:
>> >
>> > /usr/bin/ld: stdlib/public/core/linux/x86_64/Swift.o: relocation
>> > R_X86_64_PC32 against protected symbol `_TMPSa' can not be used when
>> > making a shared object
>> > /usr/bin/ld: final link failed: Bad value
>> > clang-3.7: error: linker command failed with exit code 1 (use -v to
see
>> > invocation)
>>
>> Hi Ryan,
>>
>> If you are running binutils 2.26, this is a known issue. A workaround
>> is to use the gold linker (build-script-impl has an option for this).
>>
>> Dmitri
>>
>> --
>> main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
>> (j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev


(Ryan Lovelett) #6

IIRC there is no issue in the bug tracker, please feel free to file one!

Also, if you are familiar with binutils internals, it might be helpful
to bisect the linker.

I'm finally getting around to bisecting binutils. I'm making some
progress but I'm getting hung up on one thing. The build script keeps
finding the "regular" linker at `/usr/bin/ld` instead of mine
`/tmp/binutils/2.25/usr/local/bin/ld`.

I tried to update the PATH environment variable such that mine is before
`/usr/bin` but that doesn't seem to work. Is there an argument to send
that can force mine?

···

On Tue, Mar 1, 2016, at 12:16 AM, Dmitri Gribenko wrote:

Meador Inge has also mentioned this bug:
https://llvm.org/bugs/show_bug.cgi?id=26580

Right now, it does not seem clear to me that we have an understanding
what the issue is, which component is broken (LLVM, Swift's IRGen, or
binutils), and if the breakage was accidental or intentional (e.g., to
fix something else).

Previous thread on this topic:

https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20160215/001176.html
https://lists.swift.org/pipermail/swift-dev/Week-of-Mon-20160222/001190.html

Dmitri

On Mon, Feb 29, 2016 at 8:04 PM, Ryan Lovelett > <swift-dev@ryan.lovelett.me> wrote:
> Perfect.
>
> That corresponds with what I am seeing. On the morning of the 22nd (when
> it built fine) it used binutils-2.25.1-3 and then on the morning of the
> 23rd (when it failed) it used binutils-2.26-1.
>
> I'll look into the build flag for the gold linker
>
> Out of curiosity is there a issue in the bug tracker for this that I
> could follow; or should I file one?
>
> Thanks again for the quick response.
>
> On Mon, Feb 29, 2016, at 10:55 PM, Dmitri Gribenko wrote:
>> On Mon, Feb 29, 2016 at 7:51 PM, Ryan Lovelett via swift-dev > >> <swift-dev@swift.org> wrote:
>> > I have a daily build of Swift that I make on a Arch Linux box. On the
>> > morning of February 23rd the build starting failing with the following
>> > error:
>> >
>> > /usr/bin/ld: stdlib/public/core/linux/x86_64/Swift.o: relocation
>> > R_X86_64_PC32 against protected symbol `_TMPSa' can not be used when
>> > making a shared object
>> > /usr/bin/ld: final link failed: Bad value
>> > clang-3.7: error: linker command failed with exit code 1 (use -v to see
>> > invocation)
>>
>> Hi Ryan,
>>
>> If you are running binutils 2.26, this is a known issue. A workaround
>> is to use the gold linker (build-script-impl has an option for this).
>>
>> Dmitri
>>
>> --
>> main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
>> (j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Dmitri Gribenko) #7

This is an unrelated issue. Your host clang crashes when compiling
the new clang. I don't think there's much you can do except updating
your clang.

Dmitri

···

On Wed, Mar 2, 2016 at 10:13 AM, Michael Buckley via swift-dev <swift-dev@swift.org> wrote:

I have been trying to compile on a fresh Ubuntu 15.10 image since the 27th,
and have been unable to succeed, even when using the gold linker. Any chance
this is the same issue?

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Dmitri Gribenko) #8

IIRC there is no issue in the bug tracker, please feel free to file one!

Also, if you are familiar with binutils internals, it might be helpful
to bisect the linker.

I'm finally getting around to bisecting binutils. I'm making some
progress but I'm getting hung up on one thing. The build script keeps
finding the "regular" linker at `/usr/bin/ld` instead of mine
`/tmp/binutils/2.25/usr/local/bin/ld`.

I tried to update the PATH environment variable such that mine is before
`/usr/bin` but that doesn't seem to work. Is there an argument to send
that can force mine?

I'm afraid '/usr/bin/ld' is hardcoded in clang. A few days ago Clang
added a command-line option to override that:

commit 635bc7fefc12cc1333ba6ec77e563b7c8af01265
Author: Peter Zotov <whitequark@whitequark.org>

    Accept absolute paths in the -fuse-ld option.

    This patch extends the -fuse-ld option to accept a full path to an
executable
    and use it verbatim to invoke the linker. There are generally two reasons
    to desire this.

    The first reason relates to the sad truth is that Clang is retargetable,
    Binutils are not.

    While any Clang from a binary distribution is sufficient to compile code
    for a wide range of architectures and prefixed BFD linkers (e.g.
    installed as /usr/bin/arm-none-linux-gnueabi-ld) as well as cross-compiled
    libc's (for non-bare-metal targets) are widely available, including on all
    Debian derivatives, it is impossible to use them together because
    the -fuse-ld= option allows to specify neither a linker prefix nor
    a full path to one.

    The second reason is linker development, both when porting existing linkers
    to new architectures and when working on a new linker such as LLD.

    Differential Revision: http://reviews.llvm.org/D17952

    git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@262996
91177308-0d34-0410-b5e6-96231b3b80d8

Dmitri

···

On Mon, Mar 14, 2016 at 3:38 PM, Ryan Lovelett <swift-dev@ryan.lovelett.me> wrote:

On Tue, Mar 1, 2016, at 12:16 AM, Dmitri Gribenko wrote:

Date: Wed Mar 9 05:18:16 2016 +0000

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Greg Parker) #9

Dumb answer: temporarily move /usr/bin/ld aside and install your linker there?

···

On Mar 14, 2016, at 3:38 PM, Ryan Lovelett via swift-dev <swift-dev@swift.org> wrote:

On Tue, Mar 1, 2016, at 12:16 AM, Dmitri Gribenko wrote:

IIRC there is no issue in the bug tracker, please feel free to file one!

Also, if you are familiar with binutils internals, it might be helpful
to bisect the linker.

I'm finally getting around to bisecting binutils. I'm making some
progress but I'm getting hung up on one thing. The build script keeps
finding the "regular" linker at `/usr/bin/ld` instead of mine
`/tmp/binutils/2.25/usr/local/bin/ld`.

I tried to update the PATH environment variable such that mine is before
`/usr/bin` but that doesn't seem to work. Is there an argument to send
that can force mine?

--
Greg Parker gparker@apple.com Runtime Wrangler


(Ryan Lovelett) #10

>> IIRC there is no issue in the bug tracker, please feel free to file one!
>>
>> Also, if you are familiar with binutils internals, it might be helpful
>> to bisect the linker.
>
> I'm finally getting around to bisecting binutils. I'm making some
> progress but I'm getting hung up on one thing. The build script keeps
> finding the "regular" linker at `/usr/bin/ld` instead of mine
> `/tmp/binutils/2.25/usr/local/bin/ld`.
>
> I tried to update the PATH environment variable such that mine is before
> `/usr/bin` but that doesn't seem to work. Is there an argument to send
> that can force mine?

I'm afraid '/usr/bin/ld' is hardcoded in clang. A few days ago Clang
added a command-line option to override that:

Interesting learned something new.

···

On Mon, Mar 14, 2016, at 06:42 PM, Dmitri Gribenko wrote:

On Mon, Mar 14, 2016 at 3:38 PM, Ryan Lovelett > <swift-dev@ryan.lovelett.me> wrote:
> On Tue, Mar 1, 2016, at 12:16 AM, Dmitri Gribenko wrote:

commit 635bc7fefc12cc1333ba6ec77e563b7c8af01265
Author: Peter Zotov <whitequark@whitequark.org>
Date: Wed Mar 9 05:18:16 2016 +0000

    Accept absolute paths in the -fuse-ld option.

    This patch extends the -fuse-ld option to accept a full path to an
executable
    and use it verbatim to invoke the linker. There are generally two
    reasons
    to desire this.

    The first reason relates to the sad truth is that Clang is
    retargetable,
    Binutils are not.

    While any Clang from a binary distribution is sufficient to compile
    code
    for a wide range of architectures and prefixed BFD linkers (e.g.
    installed as /usr/bin/arm-none-linux-gnueabi-ld) as well as
    cross-compiled
    libc's (for non-bare-metal targets) are widely available, including
    on all
    Debian derivatives, it is impossible to use them together because
    the -fuse-ld= option allows to specify neither a linker prefix nor
    a full path to one.

    The second reason is linker development, both when porting existing
    linkers
    to new architectures and when working on a new linker such as LLD.

    Differential Revision: http://reviews.llvm.org/D17952

    git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@262996
91177308-0d34-0410-b5e6-96231b3b80d8

Dmitri

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Ryan Lovelett) #11

>
>>
>> IIRC there is no issue in the bug tracker, please feel free to file one!
>>
>> Also, if you are familiar with binutils internals, it might be helpful
>> to bisect the linker.
>
> I'm finally getting around to bisecting binutils. I'm making some
> progress but I'm getting hung up on one thing. The build script keeps
> finding the "regular" linker at `/usr/bin/ld` instead of mine
> `/tmp/binutils/2.25/usr/local/bin/ld`.
>
> I tried to update the PATH environment variable such that mine is before
> `/usr/bin` but that doesn't seem to work. Is there an argument to send
> that can force mine?

Dumb answer: temporarily move /usr/bin/ld aside and install your linker
there?

Hopefully not dumb since that's what I've been doing! :laughing:

I build other things on the machine and I was hoping not to pollute
those builds with my tinkering. If there was an easy flag to specify it
that would have been ideal.

Instead I'll just stop those builds while I tinker.

Thanks for the suggestions.

···

On Mon, Mar 14, 2016, at 06:44 PM, Greg Parker wrote:

> On Mar 14, 2016, at 3:38 PM, Ryan Lovelett via swift-dev <swift-dev@swift.org> wrote:
>> On Tue, Mar 1, 2016, at 12:16 AM, Dmitri Gribenko wrote:

--
Greg Parker gparker@apple.com Runtime Wrangler


(Ryan Lovelett) #12

>> IIRC there is no issue in the bug tracker, please feel free to file one!
>>
>> Also, if you are familiar with binutils internals, it might be helpful
>> to bisect the linker.
>
> I'm finally getting around to bisecting binutils. I'm making some
> progress but I'm getting hung up on one thing. The build script keeps
> finding the "regular" linker at `/usr/bin/ld` instead of mine
> `/tmp/binutils/2.25/usr/local/bin/ld`.
>
> I tried to update the PATH environment variable such that mine is before
> `/usr/bin` but that doesn't seem to work. Is there an argument to send
> that can force mine?

I'm afraid '/usr/bin/ld' is hardcoded in clang. A few days ago Clang
added a command-line option to override that:

commit 635bc7fefc12cc1333ba6ec77e563b7c8af01265
Author: Peter Zotov <whitequark@whitequark.org>
Date: Wed Mar 9 05:18:16 2016 +0000

    Accept absolute paths in the -fuse-ld option.

    This patch extends the -fuse-ld option to accept a full path to an
executable
    and use it verbatim to invoke the linker. There are generally two
    reasons
    to desire this.

    The first reason relates to the sad truth is that Clang is
    retargetable,
    Binutils are not.

    While any Clang from a binary distribution is sufficient to compile
    code
    for a wide range of architectures and prefixed BFD linkers (e.g.
    installed as /usr/bin/arm-none-linux-gnueabi-ld) as well as
    cross-compiled
    libc's (for non-bare-metal targets) are widely available, including
    on all
    Debian derivatives, it is impossible to use them together because
    the -fuse-ld= option allows to specify neither a linker prefix nor
    a full path to one.

    The second reason is linker development, both when porting existing
    linkers
    to new architectures and when working on a new linker such as LLD.

    Differential Revision: http://reviews.llvm.org/D17952

    git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@262996
91177308-0d34-0410-b5e6-96231b3b80d8

Dmitri

Phew. Well I finally got everything ready to where I could bisect
binutils. I won't bore with the details of what it took to actually be
able to bisect binutils (unless someone actually wants to know).

I bisected from 71090e7a9dde8d28ff5c4b44d6d83e270d1088e4 to
2bd25930221dea4bf33c13a89c111514491440e2. 2bd259 was good and 71090 was
bad.

ca3fe95e469b9daec153caa2c90665f5daaec2b5 is the first bad commit
commit ca3fe95e469b9daec153caa2c90665f5daaec2b5
Author: H.J. Lu <hjl.tools@gmail.com>

    Add extern_protected_data and set it for x86
    
    With copy relocation, address of protected data defined in the
    shared
    library may be external. This patch adds extern_protected_data and
    changes _bfd_elf_symbol_refs_local_p to return false for protected
    data
    if extern_protected_data is true.
    
    bfd/
    
      PR ld/pr15228
      PR ld/pr17709
      * elf-bfd.h (elf_backend_data): Add extern_protected_data.
      * elf32-i386.c (elf_backend_extern_protected_data): New.
      Defined to 1.
      * elf64-x86-64.c (elf_backend_extern_protected_data): Likewise.
      * elflink.c (_bfd_elf_adjust_dynamic_copy): Don't error on
      copy relocs against protected symbols if extern_protected_data
      is true.
      (_bfd_elf_symbol_refs_local_p): Don't return true on protected
      non-function symbols if extern_protected_data is true.
      * elfxx-target.h (elf_backend_extern_protected_data): New.
      Default to 0.
      (elfNN_bed): Initialize extern_protected_data with
      elf_backend_extern_protected_data.
    
    ld/testsuite/
    
      PR ld/pr15228
      PR ld/pr17709
      * ld-i386/i386.exp (i386tests): Add a test for PR ld/17709.
      * ld-i386/pr17709-nacl.rd: New file.
      * ld-i386/pr17709.rd: Likewise.
      * ld-i386/pr17709a.s: Likewise.
      * ld-i386/pr17709b.s: Likewise.
      * ld-i386/protected3.d: Updated.
      * ld-i386/protected3.s: Likewise.
      * ld-x86-64/pr17709-nacl.rd: New file.
      * ld-x86-64/pr17709.rd: Likewise.
      * ld-x86-64/pr17709a.s: Likewise.
      * ld-x86-64/pr17709b.s: Likewise.
      * ld-x86-64/protected3.d: Updated.
      * ld-x86-64/protected3.s: Likewise.
      * ld-x86-64/x86-64.exp (x86_64tests): Add a test for PR
      ld/17709.

:040000 040000 7fc861c288f9bed44c6444d1b04e2f6e688aeeaf
fca3f6ce979e7c00ed44c04f506880015235806d M bfd
:040000 040000 a5e358abb99b2b4089765f16904f9ebc496c0f12
7ccba1e77448a0155e56e8155073b40804b00daa M ld

I'd like to write this up as an issue, if one has not been created in
the 8-days it took me to get this working, is that a good idea or a bad
one?

···

On Mon, Mar 14, 2016, at 06:42 PM, Dmitri Gribenko wrote:

On Mon, Mar 14, 2016 at 3:38 PM, Ryan Lovelett > <swift-dev@ryan.lovelett.me> wrote:
> On Tue, Mar 1, 2016, at 12:16 AM, Dmitri Gribenko wrote:

Date: Thu Mar 5 06:34:39 2015 -0800

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Michael Buckley) #13

Thanks. Apt-get reports that my clang is up to date (Ubuntu clang version
3.6.2-1), but I'm happy to wait for a newer version. I was just attempting
to compile it to make sure a change I made worked on Linux, but it's not
urgent.

I'm running my Linux VM in VMWare. I'll give it a shot in VirtualBox to see
if it might be a virtualization issue with VMWare.

···

On Wed, Mar 2, 2016 at 10:28 AM, Dmitri Gribenko <gribozavr@gmail.com> wrote:

On Wed, Mar 2, 2016 at 10:13 AM, Michael Buckley via swift-dev > <swift-dev@swift.org> wrote:
> I have been trying to compile on a fresh Ubuntu 15.10 image since the
27th,
> and have been unable to succeed, even when using the gold linker. Any
chance
> this is the same issue?

This is an unrelated issue. Your host clang crashes when compiling
the new clang. I don't think there's much you can do except updating
your clang.

Dmitri

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Dmitri Gribenko) #14

Thank you for the hard work! It is definitely a good idea to create a
bug, I am not aware of one existing on bugs.swift.org.

Dmitri

···

On Tue, Mar 22, 2016 at 9:55 AM, Ryan Lovelett <swift-dev@ryan.lovelett.me> wrote:

I'd like to write this up as an issue, if one has not been created in
the 8-days it took me to get this working, is that a good idea or a bad
one?

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(Joe Groff) #15

Does this mean we need to do something on our side to suppress "extern_protected_data"?

-Joe

···

On Mar 22, 2016, at 9:55 AM, Ryan Lovelett <swift-dev@ryan.lovelett.me> wrote:

On Mon, Mar 14, 2016, at 06:42 PM, Dmitri Gribenko wrote:

On Mon, Mar 14, 2016 at 3:38 PM, Ryan Lovelett >> <swift-dev@ryan.lovelett.me> wrote:

On Tue, Mar 1, 2016, at 12:16 AM, Dmitri Gribenko wrote:

IIRC there is no issue in the bug tracker, please feel free to file one!

Also, if you are familiar with binutils internals, it might be helpful
to bisect the linker.

I'm finally getting around to bisecting binutils. I'm making some
progress but I'm getting hung up on one thing. The build script keeps
finding the "regular" linker at `/usr/bin/ld` instead of mine
`/tmp/binutils/2.25/usr/local/bin/ld`.

I tried to update the PATH environment variable such that mine is before
`/usr/bin` but that doesn't seem to work. Is there an argument to send
that can force mine?

I'm afraid '/usr/bin/ld' is hardcoded in clang. A few days ago Clang
added a command-line option to override that:

commit 635bc7fefc12cc1333ba6ec77e563b7c8af01265
Author: Peter Zotov <whitequark@whitequark.org>
Date: Wed Mar 9 05:18:16 2016 +0000

   Accept absolute paths in the -fuse-ld option.

   This patch extends the -fuse-ld option to accept a full path to an
executable
   and use it verbatim to invoke the linker. There are generally two
   reasons
   to desire this.

   The first reason relates to the sad truth is that Clang is
   retargetable,
   Binutils are not.

   While any Clang from a binary distribution is sufficient to compile
   code
   for a wide range of architectures and prefixed BFD linkers (e.g.
   installed as /usr/bin/arm-none-linux-gnueabi-ld) as well as
   cross-compiled
   libc's (for non-bare-metal targets) are widely available, including
   on all
   Debian derivatives, it is impossible to use them together because
   the -fuse-ld= option allows to specify neither a linker prefix nor
   a full path to one.

   The second reason is linker development, both when porting existing
   linkers
   to new architectures and when working on a new linker such as LLD.

   Differential Revision: http://reviews.llvm.org/D17952

   git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@262996
91177308-0d34-0410-b5e6-96231b3b80d8

Dmitri

Phew. Well I finally got everything ready to where I could bisect
binutils. I won't bore with the details of what it took to actually be
able to bisect binutils (unless someone actually wants to know).

I bisected from 71090e7a9dde8d28ff5c4b44d6d83e270d1088e4 to
2bd25930221dea4bf33c13a89c111514491440e2. 2bd259 was good and 71090 was
bad.

ca3fe95e469b9daec153caa2c90665f5daaec2b5 is the first bad commit
commit ca3fe95e469b9daec153caa2c90665f5daaec2b5
Author: H.J. Lu <hjl.tools@gmail.com <mailto:hjl.tools@gmail.com>>
Date: Thu Mar 5 06:34:39 2015 -0800

   Add extern_protected_data and set it for x86

   With copy relocation, address of protected data defined in the
   shared
   library may be external. This patch adds extern_protected_data and
   changes _bfd_elf_symbol_refs_local_p to return false for protected
   data
   if extern_protected_data is true.

   bfd/

     PR ld/pr15228
     PR ld/pr17709
     * elf-bfd.h (elf_backend_data): Add extern_protected_data.
     * elf32-i386.c (elf_backend_extern_protected_data): New.
     Defined to 1.
     * elf64-x86-64.c (elf_backend_extern_protected_data): Likewise.
     * elflink.c (_bfd_elf_adjust_dynamic_copy): Don't error on
     copy relocs against protected symbols if extern_protected_data
     is true.
     (_bfd_elf_symbol_refs_local_p): Don't return true on protected
     non-function symbols if extern_protected_data is true.
     * elfxx-target.h (elf_backend_extern_protected_data): New.
     Default to 0.
     (elfNN_bed): Initialize extern_protected_data with
     elf_backend_extern_protected_data.

   ld/testsuite/

     PR ld/pr15228
     PR ld/pr17709
     * ld-i386/i386.exp (i386tests): Add a test for PR ld/17709.
     * ld-i386/pr17709-nacl.rd: New file.
     * ld-i386/pr17709.rd: Likewise.
     * ld-i386/pr17709a.s: Likewise.
     * ld-i386/pr17709b.s: Likewise.
     * ld-i386/protected3.d: Updated.
     * ld-i386/protected3.s: Likewise.
     * ld-x86-64/pr17709-nacl.rd: New file.
     * ld-x86-64/pr17709.rd: Likewise.
     * ld-x86-64/pr17709a.s: Likewise.
     * ld-x86-64/pr17709b.s: Likewise.
     * ld-x86-64/protected3.d: Updated.
     * ld-x86-64/protected3.s: Likewise.
     * ld-x86-64/x86-64.exp (x86_64tests): Add a test for PR
     ld/17709.

:040000 040000 7fc861c288f9bed44c6444d1b04e2f6e688aeeaf
fca3f6ce979e7c00ed44c04f506880015235806d M bfd
:040000 040000 a5e358abb99b2b4089765f16904f9ebc496c0f12
7ccba1e77448a0155e56e8155073b40804b00daa M ld

I'd like to write this up as an issue, if one has not been created in
the 8-days it took me to get this working, is that a good idea or a bad
one?


(William Dillon) #16

Phew. Well I finally got everything ready to where I could bisect
binutils. I won't bore with the details of what it took to actually be
able to bisect binutils (unless someone actually wants to know).

I would be interested if it’s not too much trouble. I’ve been in a similar predicament with LLVM before. I’d love to have another tool in the chest in case I have to do such a thing again.

- Will


(Ryan Lovelett) #17

Thank you for the hard work! It is definitely a good idea to create a
bug, I am not aware of one existing on bugs.swift.org.

Great, I've reported this as: https://bugs.swift.org/browse/SR-1023.


(Ryan Lovelett) #18

Phew. Well I finally got everything ready to where I could bisect
binutils. I won't bore with the details of what it took to actually be
able to bisect binutils (unless someone actually wants to know).

I would be interested if it’s not too much trouble. I’ve been in a similar predicament with LLVM before. I’d love to have another tool in the chest in case I have to do such a thing again.

- Will

Not too much trouble at all. In fact, I kind of just want to tell
someone; so I'm glad you asked!

The biggest challenge of the whole thing was just getting reproducable
builds using binutils. For the better part of a week I was unable to
build a linker that could link even the most basic "Hello World" source.
Eventually I ended up contacting the binutils package maintainer, Allan
McRae.

He pointed out that in order to build binutils on Arch there was a
specific order of operations. You have to build the
linux-api-headers->glibc->binutils->gcc->binutils->glibc. This was
especially true because of the change in the way that relocations occurs
in the newer binutils. Since Arch is "rolling release" distribution I
ended up having to make a VM that had a configuration of Arch from early
February, when I knew all this stuff worked. Once I could compile newer
versions of binutils and that made all the difference.

Now that I had a stable process for building binutils the next parts
were relatively straight forward. When I set out to do this I had
_thought_ what I am about to describe would be all I'd need to do.

Effectively I went back to a version/commit of binutils that I new
worked. I new the last version that worked was package 2.25.1-3; so I
got the commit hash from the PKGBUILD script [1] that was used to make
that package. Which turned out to be 2bd25930. Next I wanted the commit
hash of the package that I knew didn't work 2.26-3; so I got the commit
hash from PKGBUILD script [2] that was used to make that package. Which
turned out to be 71090e7a.

Generally speaking that would have been enough to actually do the
bisecting. However since I knew the compilation loops were going to be
long I realized this could take a few hours of compiling and rinse and
repeat to actually find the script. Luckily `git bisect run` exists.

`git bisect run` is effectively like manual `git bisect` except you can
write a script that performs the action you'd perform at each step of
the bisect.

From there I wrote a script that provided the steps to build binutils

from source. Then install that new binutils and compile Swift with that
updated version. The actual script that I used is right here [3].

The script ensures that the system gets back to a "pristine" state
between bisection steps and then compiles and installs binutils. It then
compiles Swift.

If anything goes wrong compiling binutils then its considered a
skippable commit and the script returns 125. If Swift fails to compile
it was considered a "bad" commit and the script returns 1. If Swift
compiled cleanly it was a "good" commit and the script returns 0.

The most important advice I can give you is don't forget the 125 return
code. If something goes wrong and you prematurely mark a commit as "bad"
then the entire bisect can complete but erroneously tell you which
commit was actually the first bad commit. In my opinion its better to
skip than prematurely mark as bad.

Once the script was written the hard part was over.

$ cd /directory/with/binutils/source
$ git bisect start 71090e7a9dde8d28ff5c4b44d6d83e270d1088e4
2bd25930221dea4bf33c13a89c111514491440e2
$ git bisect run ~/binutils-bisect.sh |& tee -a ~/bisect-output.log

I put all of the logs and scripts in a Gist just in-case I ever need
them again in the future [4].

Hope that was enough information Will.

[1]
https://projects.archlinux.org/svntogit/packages.git/diff/trunk/?h=packages%2Fbinutils&id=d5de6c456d1391bf802b675b1b6227f5dd3b7073&context=40&ignorews=0&dt=0
[2]
https://projects.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/binutils&id=97e6d52b49ff3882a3ec7a5833ceb4569ac5d914
[3]
https://gist.github.com/RLovelett/f73fb5f701035b33417c#file-binutils-bisect-sh
[4] https://gist.github.com/RLovelett/f73fb5f701035b33417c

···

On Tue, Mar 22, 2016, at 01:26 PM, William Dillon wrote:


(Dmitri Gribenko) #19

There's clang-3.7 in the repo, could you try that?

Dmitri

···

On Wed, Mar 2, 2016 at 12:25 PM, Michael Buckley <michael@buckleyisms.com> wrote:

Thanks. Apt-get reports that my clang is up to date (Ubuntu clang version
3.6.2-1), but I'm happy to wait for a newer version. I was just attempting
to compile it to make sure a change I made worked on Linux, but it's not
urgent.

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/


(William Dillon) #20

WOW! That is fantastic! Thanks for sharing.

- Will

···

On Mar 22, 2016, at 1:36 PM, Ryan Lovelett <swift-dev@ryan.lovelett.me> wrote:

On Tue, Mar 22, 2016, at 01:26 PM, William Dillon wrote:

Phew. Well I finally got everything ready to where I could bisect
binutils. I won't bore with the details of what it took to actually be
able to bisect binutils (unless someone actually wants to know).

I would be interested if it’s not too much trouble. I’ve been in a similar predicament with LLVM before. I’d love to have another tool in the chest in case I have to do such a thing again.

- Will

Not too much trouble at all. In fact, I kind of just want to tell
someone; so I'm glad you asked!

The biggest challenge of the whole thing was just getting reproducable
builds using binutils. For the better part of a week I was unable to
build a linker that could link even the most basic "Hello World" source.
Eventually I ended up contacting the binutils package maintainer, Allan
McRae.

He pointed out that in order to build binutils on Arch there was a
specific order of operations. You have to build the
linux-api-headers->glibc->binutils->gcc->binutils->glibc. This was
especially true because of the change in the way that relocations occurs
in the newer binutils. Since Arch is "rolling release" distribution I
ended up having to make a VM that had a configuration of Arch from early
February, when I knew all this stuff worked. Once I could compile newer
versions of binutils and that made all the difference.

Now that I had a stable process for building binutils the next parts
were relatively straight forward. When I set out to do this I had
_thought_ what I am about to describe would be all I'd need to do.

Effectively I went back to a version/commit of binutils that I new
worked. I new the last version that worked was package 2.25.1-3; so I
got the commit hash from the PKGBUILD script [1] that was used to make
that package. Which turned out to be 2bd25930. Next I wanted the commit
hash of the package that I knew didn't work 2.26-3; so I got the commit
hash from PKGBUILD script [2] that was used to make that package. Which
turned out to be 71090e7a.

Generally speaking that would have been enough to actually do the
bisecting. However since I knew the compilation loops were going to be
long I realized this could take a few hours of compiling and rinse and
repeat to actually find the script. Luckily `git bisect run` exists.

`git bisect run` is effectively like manual `git bisect` except you can
write a script that performs the action you'd perform at each step of
the bisect.

From there I wrote a script that provided the steps to build binutils
from source. Then install that new binutils and compile Swift with that
updated version. The actual script that I used is right here [3].

The script ensures that the system gets back to a "pristine" state
between bisection steps and then compiles and installs binutils. It then
compiles Swift.

If anything goes wrong compiling binutils then its considered a
skippable commit and the script returns 125. If Swift fails to compile
it was considered a "bad" commit and the script returns 1. If Swift
compiled cleanly it was a "good" commit and the script returns 0.

The most important advice I can give you is don't forget the 125 return
code. If something goes wrong and you prematurely mark a commit as "bad"
then the entire bisect can complete but erroneously tell you which
commit was actually the first bad commit. In my opinion its better to
skip than prematurely mark as bad.

Once the script was written the hard part was over.

$ cd /directory/with/binutils/source
$ git bisect start 71090e7a9dde8d28ff5c4b44d6d83e270d1088e4
2bd25930221dea4bf33c13a89c111514491440e2
$ git bisect run ~/binutils-bisect.sh |& tee -a ~/bisect-output.log

I put all of the logs and scripts in a Gist just in-case I ever need
them again in the future [4].

Hope that was enough information Will.

[1]
https://projects.archlinux.org/svntogit/packages.git/diff/trunk/?h=packages%2Fbinutils&id=d5de6c456d1391bf802b675b1b6227f5dd3b7073&context=40&ignorews=0&dt=0
[2]
https://projects.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/binutils&id=97e6d52b49ff3882a3ec7a5833ceb4569ac5d914
[3]
https://gist.github.com/RLovelett/f73fb5f701035b33417c#file-binutils-bisect-sh
[4] https://gist.github.com/RLovelett/f73fb5f701035b33417c