Building swift-3.1-branch from source

Ian_Partridge2 · 2017-04-25T12:26:06.000Z

Hi, I'm trying to build a toolchain from swift-3.1-branch on Linux.

I cloned GitHub - apple/swift: The Swift Programming Language and ran
"./swift/utils/update-checkout --clone" successfully.

Then, for each subdirectory (swift, swift-corelibs-foundation, lldb,
llvm etc.) I switched to swift-3.1-branch via "git checkout
swift-3.1-branch".

Then I ran:

./swift/utils/build-script --preset=buildbot_linux
install_destdir=/home/ipartrid/swift-3.1
installable_package=/home/ipartrid/swift-3.1.tar.gz

But this fails with:

Traceback (most recent call last):
  File "./swift/utils/build-script", line 27, in <module>
    from SwiftBuildSupport import (
  File "/home/ipartrid/swift-source/swift/utils/SwiftBuildSupport.py",
line 26, in <module>
    from swift_build_support import diagnostics # noqa (E402)
ImportError: cannot import name diagnostics

Can anyone advise on how to build swift-3.1-branch? I'm pretty sure
my procedure above used to work, until recently.

Many thanks,

···

--
Ian Partridge

Alex_Blewitt · 2017-04-25T12:30:41.000Z

You can use ./swift/utils/update-checkout --scheme swift-3.1-branch which will switch the branches to the right place for you. Some of the git repositories aren't necessarily the same; using the update-checkout script to select the right ones will be more successful. You can also use --reset-to-remote if you've made local changes that you want to throw away or want to ensure that it does the right thing from a clean update.

Alex

···

On 25 Apr 2017, at 13:26, Ian Partridge via swift-dev <swift-dev@swift.org> wrote:

Hi, I'm trying to build a toolchain from swift-3.1-branch on Linux.

I cloned GitHub - apple/swift: The Swift Programming Language and ran
"./swift/utils/update-checkout --clone" successfully.

Then, for each subdirectory (swift, swift-corelibs-foundation, lldb,
llvm etc.) I switched to swift-3.1-branch via "git checkout
swift-3.1-branch".

Then I ran:

./swift/utils/build-script --preset=buildbot_linux
install_destdir=/home/ipartrid/swift-3.1
installable_package=/home/ipartrid/swift-3.1.tar.gz

But this fails with:

Traceback (most recent call last):
File "./swift/utils/build-script", line 27, in <module>
from SwiftBuildSupport import (
File "/home/ipartrid/swift-source/swift/utils/SwiftBuildSupport.py",
line 26, in <module>
from swift_build_support import diagnostics # noqa (E402)
ImportError: cannot import name diagnostics

Can anyone advise on how to build swift-3.1-branch? I'm pretty sure
my procedure above used to work, until recently.

Many thanks,

--
Ian Partridge
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Ian_Partridge2 · 2017-04-25T12:40:42.000Z

Thanks Alex, I wasn't aware of --scheme.

Unfortunately the same problem is present after using that though:

$ git clone GitHub - apple/swift: The Swift Programming Language # fine
$ ./swift/utils/update-checkout --clone # fine
$ ./swift/utils/update-checkout --scheme swift-3.1-branch # fine
$ ./swift/utils/build-script --preset=buildbot_linux
install_destdir=/home/ipartrid/swift-3.1
installable_package=/home/ipartrid/swift-3.1.tar.gz
Traceback (most recent call last):
  File "./swift/utils/build-script", line 27, in <module>
    from SwiftBuildSupport import (
  File "/home/ipartrid/swift-source/swift/utils/SwiftBuildSupport.py",
line 26, in <module>
    from swift_build_support import diagnostics # noqa (E402)
ImportError: cannot import name diagnostics

···

On 25 April 2017 at 13:30, Alex Blewitt <alblue@apple.com> wrote:

You can use ./swift/utils/update-checkout --scheme swift-3.1-branch which will switch the branches to the right place for you. Some of the git repositories aren't necessarily the same; using the update-checkout script to select the right ones will be more successful. You can also use --reset-to-remote if you've made local changes that you want to throw away or want to ensure that it does the right thing from a clean update.

--
Ian Partridge

vivkong · 2017-04-25T14:15:23.000Z

I had to remove swift/utils/swift_build_support/__init__.pyc if it is there
(it got generated when I use the checkout script from the master branch).
Wonder if that might help you.

Regards,

Vivian Kong
Linux on z Systems Open Source Ecosystem
IBM Canada Toronto Lab

···

From: Ian Partridge via swift-dev <swift-dev@swift.org>
To: Alex Blewitt <alblue@apple.com>
Cc: swift-dev <swift-dev@swift.org>
Date: 2017/04/25 08:40 AM
Subject: Re: [swift-dev] Building swift-3.1-branch from source
Sent by: swift-dev-bounces@swift.org

On 25 April 2017 at 13:30, Alex Blewitt <alblue@apple.com> wrote:

You can use ./swift/utils/update-checkout --scheme swift-3.1-branch which

will switch the branches to the right place for you. Some of the git
repositories aren't necessarily the same; using the update-checkout script
to select the right ones will be more successful. You can also use
--reset-to-remote if you've made local changes that you want to throw away
or want to ensure that it does the right thing from a clean update.

Thanks Alex, I wasn't aware of --scheme.

Unfortunately the same problem is present after using that though:

$ git clone GitHub - apple/swift: The Swift Programming Language # fine
$ ./swift/utils/update-checkout --clone # fine
$ ./swift/utils/update-checkout --scheme swift-3.1-branch # fine
$ ./swift/utils/build-script --preset=buildbot_linux
install_destdir=/home/ipartrid/swift-3.1
installable_package=/home/ipartrid/swift-3.1.tar.gz
Traceback (most recent call last):
  File "./swift/utils/build-script", line 27, in <module>
    from SwiftBuildSupport import (
  File "/home/ipartrid/swift-source/swift/utils/SwiftBuildSupport.py",
line 26, in <module>
    from swift_build_support import diagnostics # noqa (E402)
ImportError: cannot import name diagnostics

--
Ian Partridge
_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

Ian_Partridge2 · 2017-04-25T14:27:23.000Z

Yes it works! Thanks!

···

On 25 April 2017 at 15:15, Vivian Kong <vivkong@ca.ibm.com> wrote:

I had to remove swift/utils/swift_build_support/__init__.pyc if it is there (it got generated when I use the checkout script from the master branch). Wonder if that might help you.

--
Ian Partridge

vivkong · 2017-04-25T14:35:03.000Z

I actually have a question regarding build as well. I haven't been able to
build 3.1 branch from source on Ubuntu 16.04 x86_64. I hit this error:

./swift/utils/build-script -r

...

swift: using gold linker
+ /usr/bin/cmake
--build /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64
-- -j2 all swift-test-stdlib-linux-x86_64
[5/63] Linking CXX executable bin/swift-reflection-dump
FAILED: : && /usr/bin/clang++ -fno-stack-protector -fPIC
-fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings
-Wcast-qual -Wmissing-field-initializers -Wcovered-switch-default
-Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Werror=date-time -std=c++11
-fcolor-diagnostics -ffunction-sections -fdata-sections -Wdocumentation
-Wimplicit-fallthrough -Wunreachable-code -Woverloaded-virtual -O2
-L/home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/./lib/swift/linux
-target x86_64-unknown-linux-gnu -lpthread -ldl -fuse-ld=gold
tools/swift-reflection-dump/CMakeFiles/swift-reflection-dump.dir/swift-reflection-dump.cpp.o
-o bin/swift-reflection-dump
-L/home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/./lib

-lswiftReflection /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMObject.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMSupport.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMBitReader.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMCore.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMMCParser.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMMC.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMSupport.a
-lrt -ldl -ltinfo -lpthread -lz -lm
-Wl,-rpath,/home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/./lib
&& :
/home/vivkong/swift31/swift/include/swift/Remote/MetadataReader.h:92:
error: undefined reference to 'swift::Demangle::mangleNode
(std::shared_ptr<swift::Demangle::Node> const&)'
/home/vivkong/swift31/swift/include/swift/Remote/MetadataReader.h:152:
error: undefined reference to 'swift::Demangle::mangleNode
(std::shared_ptr<swift::Demangle::Node> const&)'
/home/vivkong/swift31/swift/include/swift/Reflection/TypeRefBuilder.h:171:
error: undefined reference to 'swift::Demangle::mangleNode
(std::shared_ptr<swift::Demangle::Node> const&)'
clang: error: linker command failed with exit code 1 (use -v to see
invocation)
[5/63] Linking CXX static library lib/libswiftBasic.a
ninja: build stopped: subcommand failed.
./swift/utils/build-script: fatal error: command terminated with a non-zero
exit status 1, aborting

I saw [SR-4147] swift-reflection-dump error while building swift · Issue #46730 · apple/swift · GitHub opened for the master branch
(and the workaround is to move to Ubuntu 16.04). I'm actually able to
build the 3.1 branch on s390x so I'm wondering if I'm missing some pre-reqs
or setting? Any clues would be much appreciated!

Regards,

Vivian Kong
Linux on z Systems Open Source Ecosystem
IBM Canada Toronto Lab

···

From: Ian Partridge <ipartridge@gmail.com>
To: Vivian Kong/Toronto/IBM@IBMCA
Cc: Alex Blewitt <alblue@apple.com>, swift-dev
<swift-dev@swift.org>
Date: 2017/04/25 10:27 AM
Subject: Re: [swift-dev] Building swift-3.1-branch from source

On 25 April 2017 at 15:15, Vivian Kong <vivkong@ca.ibm.com> wrote:

I had to remove swift/utils/swift_build_support/__init__.pyc if it is

there (it got generated when I use the checkout script from the master
branch). Wonder if that might help you.

Yes it works! Thanks!

--
Ian Partridge

ffried · 2017-04-25T14:43:14.000Z

What version of clang are you using? I’ve had the same problem with clang 3.9 when building on Ubuntu 16.04 on RasPi: https://twitter.com/ffried/status/833985445516165121
Using clang 3.8 worked fine.

best regards,

Florian Friedrich
(CTO / CIO, Senior Developer @ ser.soft GmbH)

···

On 25. Apr 2017, at 16:35, Vivian Kong via swift-dev <swift-dev@swift.org> wrote:

I actually have a question regarding build as well. I haven't been able to build 3.1 branch from source on Ubuntu 16.04 x86_64. I hit this error:

./swift/utils/build-script -r

...

swift: using gold linker
+ /usr/bin/cmake --build /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64 -- -j2 all swift-test-stdlib-linux-x86_64
[5/63] Linking CXX executable bin/swift-reflection-dump
FAILED: : && /usr/bin/clang++ -fno-stack-protector -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Werror=date-time -std=c++11 -fcolor-diagnostics -ffunction-sections -fdata-sections -Wdocumentation -Wimplicit-fallthrough -Wunreachable-code -Woverloaded-virtual -O2 -L/home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/./lib/swift/linux -target x86_64-unknown-linux-gnu -lpthread -ldl -fuse-ld=gold tools/swift-reflection-dump/CMakeFiles/swift-reflection-dump.dir/swift-reflection-dump.cpp.o -o bin/swift-reflection-dump -L/home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/./lib -lswiftReflection /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMObject.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMSupport.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMBitReader.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMCore.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMMCParser.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMMC.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMSupport.a -lrt -ldl -ltinfo -lpthread -lz -lm -Wl,-rpath,/home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/./lib && :
/home/vivkong/swift31/swift/include/swift/Remote/MetadataReader.h:92: error: undefined reference to 'swift::Demangle::mangleNode(std::shared_ptr<swift::Demangle::Node> const&)'
/home/vivkong/swift31/swift/include/swift/Remote/MetadataReader.h:152: error: undefined reference to 'swift::Demangle::mangleNode(std::shared_ptr<swift::Demangle::Node> const&)'
/home/vivkong/swift31/swift/include/swift/Reflection/TypeRefBuilder.h:171: error: undefined reference to 'swift::Demangle::mangleNode(std::shared_ptr<swift::Demangle::Node> const&)'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
[5/63] Linking CXX static library lib/libswiftBasic.a
ninja: build stopped: subcommand failed.
./swift/utils/build-script: fatal error: command terminated with a non-zero exit status 1, aborting

I saw [SR-4147] swift-reflection-dump error while building swift · Issue #46730 · apple/swift · GitHub opened for the master branch (and the workaround is to move to Ubuntu 16.04). I'm actually able to build the 3.1 branch on s390x so I'm wondering if I'm missing some pre-reqs or setting? Any clues would be much appreciated!

Regards,

Vivian Kong
Linux on z Systems Open Source Ecosystem
IBM Canada Toronto Lab

<graycol.gif>Ian Partridge ---2017/04/25 10:27:39 AM---On 25 April 2017 at 15:15, Vivian Kong <vivkong@ca.ibm.com> wrote: > I had to remove swift/utils/swi

From: Ian Partridge <ipartridge@gmail.com>
To: Vivian Kong/Toronto/IBM@IBMCA
Cc: Alex Blewitt <alblue@apple.com>, swift-dev <swift-dev@swift.org>
Date: 2017/04/25 10:27 AM
Subject: Re: [swift-dev] Building swift-3.1-branch from source

On 25 April 2017 at 15:15, Vivian Kong <vivkong@ca.ibm.com> wrote:
> I had to remove swift/utils/swift_build_support/__init__.pyc if it is there (it got generated when I use the checkout script from the master branch). Wonder if that might help you.

Yes it works! Thanks!

--
Ian Partridge

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev

vivkong · 2017-04-25T14:50:52.000Z

Thanks! However, it looks like I am using clang 3.8 already:

$ clang -v
clang version 3.8.0-2ubuntu3 (tags/RELEASE_380/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/i686-linux-gnu/5.4.0
Found candidate GCC installation: /usr/bin/../lib/gcc/i686-linux-gnu/6.0.0
Found candidate GCC
installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0
Found candidate GCC
installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/6.0.0
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/5.4.0
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/6.0.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.4.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6.0.0
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0
Candidate multilib: .;@m64
Selected multilib: .;@m64

$ apt list clang -a

Listing... Done
clang/xenial-updates,now 1:3.8-33ubuntu3.1 amd64 [installed]
clang/xenial 1:3.8-33ubuntu3 amd64

Regards,

Vivian Kong
Linux on z Systems Open Source Ecosystem
IBM Canada Toronto Lab

···

From: Florian Friedrich <ffried@me.com>
To: Vivian Kong/Toronto/IBM@IBMCA
Cc: swift-dev <swift-dev@swift.org>
Date: 2017/04/25 10:43 AM
Subject: Re: [swift-dev] Building swift-3.1-branch from source

What version of clang are you using? I’ve had the same problem with clang
3.9 when building on Ubuntu 16.04 on RasPi:

twitter.com

Florian Friedrich

@ffried

Building Swift (master) on Ubuntu 16.04 results in a linking error. But CI says buildbot_linux_1604 is fine. Ideas? /cc @jckarter https://t.co/VsBoDblFj5

10:23 AM - 21 Feb 2017

Using clang 3.8 worked fine.

best regards,

Florian Friedrich
(CTO / CIO, Senior Developer @ ser.soft GmbH)

On 25. Apr 2017, at 16:35, Vivian Kong via swift-dev < swift-dev@swift.org> wrote:

      I actually have a question regarding build as well. I haven't been
      able to build 3.1 branch from source on Ubuntu 16.04 x86_64. I hit
      this error:

./swift/utils/build-script -r

...

      swift: using gold linker
      + /usr/bin/cmake
      --build /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64
-- -j2 all swift-test-stdlib-linux-x86_64
      [5/63] Linking CXX executable bin/swift-reflection-dump
      FAILED: : && /usr/bin/clang++ -fno-stack-protector -fPIC
      -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter
      -Wwrite-strings -Wcast-qual -Wmissing-field-initializers
      -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
      -Werror=date-time -std=c++11 -fcolor-diagnostics -ffunction-sections
      -fdata-sections -Wdocumentation -Wimplicit-fallthrough
      -Wunreachable-code -Woverloaded-virtual -O2
      -L/home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/swift-linux-x86_64/./lib/swift/linux
-target x86_64-unknown-linux-gnu -lpthread -ldl -fuse-ld=gold
      tools/swift-reflection-dump/CMakeFiles/swift-reflection-dump.dir/swift-reflection-dump.cpp.o
-o bin/swift-reflection-dump
      -L/home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/./lib
-lswiftReflection /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMObject.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMSupport.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMBitReader.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMCore.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMMCParser.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMMC.a /home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/lib/libLLVMSupport.a
-lrt -ldl -ltinfo -lpthread -lz -lm
      -Wl,-rpath,/home/vivkong/swift31/build/Ninja-RelWithDebInfoAssert/llvm-linux-x86_64/./lib
&& :
      /home/vivkong/swift31/swift/include/swift/Remote/MetadataReader.h:92:
      error: undefined reference to 'swift::Demangle::mangleNode
      (std::shared_ptr<swift::Demangle::Node> const&)'
      /home/vivkong/swift31/swift/include/swift/Remote/MetadataReader.h:152:
error: undefined reference to 'swift::Demangle::mangleNode
      (std::shared_ptr<swift::Demangle::Node> const&)'
      /home/vivkong/swift31/swift/include/swift/Reflection/TypeRefBuilder.h:171:
error: undefined reference to 'swift::Demangle::mangleNode
      (std::shared_ptr<swift::Demangle::Node> const&)'
      clang: error: linker command failed with exit code 1 (use -v to see
      invocation)
      [5/63] Linking CXX static library lib/libswiftBasic.a
      ninja: build stopped: subcommand failed.
      ./swift/utils/build-script: fatal error: command terminated with a
      non-zero exit status 1, aborting

      I saw [SR-4147] swift-reflection-dump error while building swift · Issue #46730 · apple/swift · GitHub opened for the master
      branch (and the workaround is to move to Ubuntu 16.04). I'm actually
      able to build the 3.1 branch on s390x so I'm wondering if I'm missing
      some pre-reqs or setting? Any clues would be much appreciated!

Regards,

      Vivian Kong
      Linux on z Systems Open Source Ecosystem
      IBM Canada Toronto Lab

      <graycol.gif>Ian Partridge ---2017/04/25 10:27:39 AM---On 25 April
      2017 at 15:15, Vivian Kong <vivkong@ca.ibm.com> wrote: > I had to
      remove swift/utils/swi

      From: Ian Partridge <ipartridge@gmail.com>
      To: Vivian Kong/Toronto/IBM@IBMCA
      Cc: Alex Blewitt <alblue@apple.com>, swift-dev <swift-dev@swift.org>
      Date: 2017/04/25 10:27 AM
      Subject: Re: [swift-dev] Building swift-3.1-branch from source

      On 25 April 2017 at 15:15, Vivian Kong <vivkong@ca.ibm.com> wrote:
      > I had to remove swift/utils/swift_build_support/__init__.pyc if it
      is there (it got generated when I use the checkout script from the
      master branch). Wonder if that might help you.

Yes it works! Thanks!

--
Ian Partridge

      _______________________________________________
      swift-dev mailing list
      swift-dev@swift.org
      https://lists.swift.org/mailman/listinfo/swift-dev