Libdispatch on Linux, or C++ std::async?

Brian_Gesiak · May 30, 2016, 4:28am

Hello all!

I'm working on [SR-710] Generate XCTestCaseProvider entries on Linux · Issue #5320 · apple/swift-package-manager · GitHub, which involves porting
SourceKit to Linux. SourceKit contains a handful of references to
libdispatch symbols.

I have two questions:

1. Could someone share a set of instructions to build
swift-corelibs-libdispatch on Linux? I'm using Ubuntu 15.10 and have
installed [the dependencies listed here](
https://lists.swift.org/pipermail/swift-corelibs-dev/Week-of-Mon-20151207/000122.html\).
I have tried using both `swift/utils/build-script --libdispatch` as well as
the instructions in the swift-corelibs-libdispatch `INSTALL` file. Both
fail with the [same errors](
https://gist.github.com/modocache/8389d5d64e39ca5a4281500f84e9ce11\). Any
tips?

2. I don't know much about C++, but from what I understand the C++ stdlib
provides tools for asynchronous programming. I list SourceKit's uses of
libdispatch in a comment on https://bugs.swift.org/browse/SR-1639\. For
those familiar with both C++ and libdispatch: do you think it's a good idea
to migrate to C++, at least for now?

Any advice would be appreciated. Thanks!!

- Brian Gesiak

Rob_Allen · May 30, 2016, 7:05am

Hi,

For what it's worth, I can't get the master branch to build either, but I can get the experimental/foundation branch to build.

Regards,

Rob....

···

On 30 May 2016, at 05:28, Brian Gesiak via swift-corelibs-dev <swift-corelibs-dev@swift.org> wrote:

Hello all!

I'm working on [SR-710] Generate XCTestCaseProvider entries on Linux · Issue #5320 · apple/swift-package-manager · GitHub, which involves porting SourceKit to Linux. SourceKit contains a handful of references to libdispatch symbols.

I have two questions:

1. Could someone share a set of instructions to build swift-corelibs-libdispatch on Linux? I'm using Ubuntu 15.10 and have installed [the dependencies listed here](https://lists.swift.org/pipermail/swift-corelibs-dev/Week-of-Mon-20151207/000122.html\). I have tried using both `swift/utils/build-script --libdispatch` as well as the instructions in the swift-corelibs-libdispatch `INSTALL` file. Both fail with the [same errors](https://gist.github.com/modocache/8389d5d64e39ca5a4281500f84e9ce11\). Any tips?

2. I don't know much about C++, but from what I understand the C++ stdlib provides tools for asynchronous programming. I list SourceKit's uses of libdispatch in a comment on https://bugs.swift.org/browse/SR-1639\. For those familiar with both C++ and libdispatch: do you think it's a good idea to migrate to C++, at least for now?

Any advice would be appreciated. Thanks!!

- Brian Gesiak

--
Development thoughts at http://akrabat.com
Daily Jotter for Mac OS X at http://dailyjotter.com

Brian_Gesiak · May 30, 2016, 3:50pm

Thanks for the quick responses!

I had tried the experimental/foundation branch as well, with no luck.
Here's the full output from a clean build I just attempted:

gist.github.com

https://gist.github.com/modocache/fcf0c78908b39834448c73ac3f50c110

gistfile1.txt

modocache@ubuntu:~/GitHub/apple/swift-corelibs-libdispatch$ ../swift/utils/build-script -R --libdispatch
+ mkdir -p /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert
Building the standard library for: swift-test-stdlib-linux-x86_64

cmark: using gold linker
+ mkdir -p /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/cmark-linux-x86_64
+ [[ -n '' ]]
+ cd /home/modocache/GitHub/apple/build/Ninja-ReleaseAssert/cmark-linux-x86_64
+ env /usr/bin/cmake -G Ninja -DCMAKE_C_COMPILER:PATH=/usr/bin/clang -DCMAKE_CXX_COMPILER:PATH=/usr/bin/clang++ -DCMAKE_EXE_LINKER_FLAGS:STRING=-fuse-ld=gold -DCMAKE_SHARED_LINKER_FLAGS:STRING=-fuse-ld=gold -DCMAKE_BUILD_TYPE:STRING=Release /home/modocache/GitHub/apple/cmark
-- The C compiler identification is Clang 3.6.2

This file has been truncated. show original

Prior to the build I removed my entire build directory (which included
both libdispatch-linux-x86_64 and swift-linux-x86_64) and removed all build
artifacts from my libdispatch directory (I used `git clean -fxd` -- Make
tells me there's no target `distclean`).

Looking at the build failure, it appears to be the same one that was
reported in https://bugs.swift.org/browse/SR-1242\. The author of that
report indicates that
Another fix for libpwq configuration by dgrove-oss · Pull Request #64 · apple/swift-corelibs-libdispatch · GitHub fixes the
problem. My checkout includes the commits from that pull request, but the
problem remains.

In any case, thanks for all your help! I'll keep trying. Any and all advice
would be much appreciated!

- Brian Gesiak

···

On Mon, May 30, 2016 at 10:22 AM, Chris Bailey <BAILEYC@uk.ibm.com> wrote:

I'd certainly recommend picking up the experimental/foundation branch as
Rob recommends. Its where we've been doing some additional work to get
properly built into the Swift toolchain and to open up some internal APIs
needed by Foundation.

Whilst I've been building Dispatch either independently or as part of a
full toolchain using the buildbot_linux preset (with dispatch and
install-dispatch added), I've just checked your build line and its working
fine for me. It might be worth cleaning out any partial build you might
have of libdispatch in your build directory, and running 'make distclean'
from your libdispatch source directory.

FYI, we've got some further changes to the experimental/foundation branch
of libdispatch with should land in the next couple of days. This fixes some
issues with the Swift overlay, and resolves a crash that we've found in the
underlying libkqueue implementation.

Chris

From: Rob Allen via swift-corelibs-dev <
swift-corelibs-dev@swift.org>
To: Brian Gesiak <modocache@gmail.com>
Cc: Swift Core Libs <swift-corelibs-dev@swift.org>
Date: 30/05/2016 08:08
Subject: Re: [swift-corelibs-dev] libdispatch on Linux, or C++
std::async?
Sent by: swift-corelibs-dev-bounces@swift.org
------------------------------

Hi,

For what it's worth, I can't get the master branch to build either, but I
can get the experimental/foundation branch to build.

Regards,

Rob....

On 30 May 2016, at 05:28, Brian Gesiak via swift-corelibs-dev < > *swift-corelibs-dev@swift.org* <swift-corelibs-dev@swift.org>> wrote:

Hello all!

I'm working on *[SR-710] Generate XCTestCaseProvider entries on Linux · Issue #5320 · apple/swift-package-manager · GitHub*
<Issues · apple/swift-issues · GitHub, which involves porting SourceKit
to Linux. SourceKit contains a handful of references to libdispatch symbols.

I have two questions:

1. Could someone share a set of instructions to build
swift-corelibs-libdispatch on Linux? I'm using Ubuntu 15.10 and have
installed [the dependencies listed here](
*[swift-corelibs-dev] libdispatch build issue*
<https://lists.swift.org/pipermail/swift-corelibs-dev/Week-of-Mon-20151207/000122.html>\).
I have tried using both `swift/utils/build-script --libdispatch` as well as
the instructions in the swift-corelibs-libdispatch `INSTALL` file. Both
fail with the [same errors](
*gist:8389d5d64e39ca5a4281500f84e9ce11 · GitHub*
<https://gist.github.com/modocache/8389d5d64e39ca5a4281500f84e9ce11>\).
Any tips?

2. I don't know much about C++, but from what I understand the C++ stdlib
provides tools for asynchronous programming. I list SourceKit's uses of
libdispatch in a comment on *[SR-1639] [SourceKit] Add in-process implementations of sourcekitd functions only defined for XPC · Issue #5268 · apple/swift-package-manager · GitHub*
<https://bugs.swift.org/browse/SR-1639>\. For those familiar with both C++
and libdispatch: do you think it's a good idea to migrate to C++, at least
for now?

Any advice would be appreciated. Thanks!!

- Brian Gesiak

--
Development thoughts at *http://akrabat.com* <http://akrabat.com/>
Daily Jotter for Mac OS X at *http://dailyjotter.com*
<Daily Jotter;
_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Chris_Bailey · May 30, 2016, 2:22pm

I'd certainly recommend picking up the experimental/foundation branch as
Rob recommends. Its where we've been doing some additional work to get
properly built into the Swift toolchain and to open up some internal APIs
needed by Foundation.

Whilst I've been building Dispatch either independently or as part of a
full toolchain using the buildbot_linux preset (with dispatch and
install-dispatch added), I've just checked your build line and its working
fine for me. It might be worth cleaning out any partial build you might
have of libdispatch in your build directory, and running 'make distclean'
from your libdispatch source directory.

FYI, we've got some further changes to the experimental/foundation branch
of libdispatch with should land in the next couple of days. This fixes
some issues with the Swift overlay, and resolves a crash that we've found
in the underlying libkqueue implementation.

Chris

···

From: Rob Allen via swift-corelibs-dev <swift-corelibs-dev@swift.org>
To: Brian Gesiak <modocache@gmail.com>
Cc: Swift Core Libs <swift-corelibs-dev@swift.org>
Date: 30/05/2016 08:08
Subject: Re: [swift-corelibs-dev] libdispatch on Linux, or C++
std::async?
Sent by: swift-corelibs-dev-bounces@swift.org

Hi,

For what it's worth, I can't get the master branch to build either, but I
can get the experimental/foundation branch to build.

Regards,

Rob....

On 30 May 2016, at 05:28, Brian Gesiak via swift-corelibs-dev < swift-corelibs-dev@swift.org> wrote:

Hello all!

I'm working on [SR-710] Generate XCTestCaseProvider entries on Linux · Issue #5320 · apple/swift-package-manager · GitHub, which involves
porting SourceKit to Linux. SourceKit contains a handful of references to
libdispatch symbols.

I have two questions:

1. Could someone share a set of instructions to build
swift-corelibs-libdispatch on Linux? I'm using Ubuntu 15.10 and have
installed [the dependencies listed here](
https://lists.swift.org/pipermail/swift-corelibs-dev/Week-of-Mon-20151207/000122.html
). I have tried using both `swift/utils/build-script --libdispatch` as
well as the instructions in the swift-corelibs-libdispatch `INSTALL` file.
Both fail with the [same errors](
https://gist.github.com/modocache/8389d5d64e39ca5a4281500f84e9ce11\). Any
tips?

2. I don't know much about C++, but from what I understand the C++ stdlib
provides tools for asynchronous programming. I list SourceKit's uses of
libdispatch in a comment on https://bugs.swift.org/browse/SR-1639\. For
those familiar with both C++ and libdispatch: do you think it's a good
idea to migrate to C++, at least for now?

Any advice would be appreciated. Thanks!!

- Brian Gesiak

--
Development thoughts at http://akrabat.com
Daily Jotter for Mac OS X at http://dailyjotter.com
_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Brian_Croom · May 30, 2016, 4:08pm

I've noticed that there is a submodule reference to libpwq (see
https://github.com/apple/swift-corelibs-libdispatch/blob/master/.gitmodules\).
Is it possible that the submodule hasn't been initialized in your clone?

···

måndag 30 maj 2016 skrev Brian Gesiak via swift-corelibs-dev < swift-corelibs-dev@swift.org>:

Thanks for the quick responses!

I had tried the experimental/foundation branch as well, with no luck.
Here's the full output from a clean build I just attempted:
gist:fcf0c78908b39834448c73ac3f50c110 · GitHub

Prior to the build I removed my entire build directory (which included
both libdispatch-linux-x86_64 and swift-linux-x86_64) and removed all build
artifacts from my libdispatch directory (I used `git clean -fxd` -- Make
tells me there's no target `distclean`).

Looking at the build failure, it appears to be the same one that was
reported in https://bugs.swift.org/browse/SR-1242\. The author of that
report indicates that
Another fix for libpwq configuration by dgrove-oss · Pull Request #64 · apple/swift-corelibs-libdispatch · GitHub fixes the
problem. My checkout includes the commits from that pull request, but the
problem remains.

In any case, thanks for all your help! I'll keep trying. Any and all
advice would be much appreciated!

- Brian Gesiak

On Mon, May 30, 2016 at 10:22 AM, Chris Bailey <BAILEYC@uk.ibm.com > <javascript:_e(%7B%7D,'cvml','BAILEYC@uk.ibm.com');>> wrote:

I'd certainly recommend picking up the experimental/foundation branch as
Rob recommends. Its where we've been doing some additional work to get
properly built into the Swift toolchain and to open up some internal APIs
needed by Foundation.

Whilst I've been building Dispatch either independently or as part of a
full toolchain using the buildbot_linux preset (with dispatch and
install-dispatch added), I've just checked your build line and its working
fine for me. It might be worth cleaning out any partial build you might
have of libdispatch in your build directory, and running 'make distclean'
from your libdispatch source directory.

FYI, we've got some further changes to the experimental/foundation branch
of libdispatch with should land in the next couple of days. This fixes some
issues with the Swift overlay, and resolves a crash that we've found in the
underlying libkqueue implementation.

Chris

From: Rob Allen via swift-corelibs-dev <
swift-corelibs-dev@swift.org
<javascript:_e(%7B%7D,'cvml','swift-corelibs-dev@swift.org');>>
To: Brian Gesiak <modocache@gmail.com
<javascript:_e(%7B%7D,'cvml','modocache@gmail.com');>>
Cc: Swift Core Libs <swift-corelibs-dev@swift.org
<javascript:_e(%7B%7D,'cvml','swift-corelibs-dev@swift.org');>>
Date: 30/05/2016 08:08
Subject: Re: [swift-corelibs-dev] libdispatch on Linux, or C++
std::async?
Sent by: swift-corelibs-dev-bounces@swift.org
<javascript:_e(%7B%7D,'cvml','swift-corelibs-dev-bounces@swift.org');>
------------------------------

Hi,

For what it's worth, I can't get the master branch to build either, but I
can get the experimental/foundation branch to build.

Regards,

Rob....

On 30 May 2016, at 05:28, Brian Gesiak via swift-corelibs-dev < >> *swift-corelibs-dev@swift.org* >> <javascript:_e(%7B%7D,'cvml','swift-corelibs-dev@swift.org');>> wrote:

Hello all!

I'm working on *[SR-710] Generate XCTestCaseProvider entries on Linux · Issue #5320 · apple/swift-package-manager · GitHub*
<Issues · apple/swift-issues · GitHub, which involves porting SourceKit
to Linux. SourceKit contains a handful of references to libdispatch symbols.

I have two questions:

1. Could someone share a set of instructions to build
swift-corelibs-libdispatch on Linux? I'm using Ubuntu 15.10 and have
installed [the dependencies listed here](
*[swift-corelibs-dev] libdispatch build issue*
<https://lists.swift.org/pipermail/swift-corelibs-dev/Week-of-Mon-20151207/000122.html>\).
I have tried using both `swift/utils/build-script --libdispatch` as well as
the instructions in the swift-corelibs-libdispatch `INSTALL` file. Both
fail with the [same errors](
*gist:8389d5d64e39ca5a4281500f84e9ce11 · GitHub*
<https://gist.github.com/modocache/8389d5d64e39ca5a4281500f84e9ce11>\).
Any tips?

2. I don't know much about C++, but from what I understand the C++ stdlib
provides tools for asynchronous programming. I list SourceKit's uses of
libdispatch in a comment on *[SR-1639] [SourceKit] Add in-process implementations of sourcekitd functions only defined for XPC · Issue #5268 · apple/swift-package-manager · GitHub*
<https://bugs.swift.org/browse/SR-1639>\. For those familiar with both
C++ and libdispatch: do you think it's a good idea to migrate to C++, at
least for now?

Any advice would be appreciated. Thanks!!

- Brian Gesiak

--
Development thoughts at *http://akrabat.com* <http://akrabat.com/>
Daily Jotter for Mac OS X at *http://dailyjotter.com*
<Daily Jotter;
_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
<javascript:_e(%7B%7D,'cvml','swift-corelibs-dev@swift.org');>
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

dgrove-oss · May 30, 2016, 4:24pm

Hi,

my guess is you missed the git submodule init and update step in
setting up your libdispatch checkout. If the libpwq directory is empty,
this is certainly what happened (and is why it isn't invoking auotconf
recursively on libpwq). Buried at the bottom of the INSTALL file:

Initialize git submodules:
  We are using git submodules to incorporate a specific revision of the
  upstream pthread_workqueue library into the build.
    git submodule init
    git submodule update

--dave

<swift-corelibs-dev@swift.org>

···

From: Brian Gesiak via swift-corelibs-dev
To: Chris Bailey <BAILEYC@uk.ibm.com>
Cc: Swift Core Libs <swift-corelibs-dev@swift.org>
Date: 05/30/2016 11:51 AM
Subject: Re: [swift-corelibs-dev] libdispatch on Linux, or C++
std::async?
Sent by: swift-corelibs-dev-bounces@swift.org

Thanks for the quick responses!

I had tried the experimental/foundation branch as well, with no luck.
Here's the full output from a clean build I just attempted:

Prior to the build I removed my entire build directory (which included
both libdispatch-linux-x86_64 and swift-linux-x86_64) and removed all build
artifacts from my libdispatch directory (I used `git clean -fxd` -- Make
tells me there's no target `distclean`).

Looking at the build failure, it appears to be the same one that was
reported in https://bugs.swift.org/browse/SR-1242\. The author of that
report indicates that
Another fix for libpwq configuration by dgrove-oss · Pull Request #64 · apple/swift-corelibs-libdispatch · GitHub fixes the
problem. My checkout includes the commits from that pull request, but the
problem remains.

In any case, thanks for all your help! I'll keep trying. Any and all advice
would be much appreciated!

- Brian Gesiak

On Mon, May 30, 2016 at 10:22 AM, Chris Bailey <BAILEYC@uk.ibm.com> wrote:
  I'd certainly recommend picking up the experimental/foundation branch as
  Rob recommends. Its where we've been doing some additional work to get
  properly built into the Swift toolchain and to open up some internal APIs
  needed by Foundation.

  Whilst I've been building Dispatch either independently or as part of a
  full toolchain using the buildbot_linux preset (with dispatch and
  install-dispatch added), I've just checked your build line and its
  working fine for me. It might be worth cleaning out any partial build you
  might have of libdispatch in your build directory, and running 'make
  distclean' from your libdispatch source directory.

  FYI, we've got some further changes to the experimental/foundation branch
  of libdispatch with should land in the next couple of days. This fixes
  some issues with the Swift overlay, and resolves a crash that we've found
  in the underlying libkqueue implementation.

Chris

  From: Rob Allen via swift-corelibs-dev <
  swift-corelibs-dev@swift.org>
  To: Brian Gesiak <modocache@gmail.com>
  Cc: Swift Core Libs <swift-corelibs-dev@swift.org>
  Date: 30/05/2016 08:08
  Subject: Re: [swift-corelibs-dev] libdispatch on Linux, or C++
  std::async?
  Sent by: swift-corelibs-dev-bounces@swift.org

Hi,

For what it's worth, I can't get the master branch to build either, but I
can get the experimental/foundation branch to build.

Regards,

Rob....

On 30 May 2016, at 05:28, Brian Gesiak via swift-corelibs-dev < swift-corelibs-dev@swift.org> wrote:

Hello all!

  I'm working on [SR-710] Generate XCTestCaseProvider entries on Linux · Issue #5320 · apple/swift-package-manager · GitHub, which involves
  porting SourceKit to Linux. SourceKit contains a handful of references to
  libdispatch symbols.

I have two questions:

  1. Could someone share a set of instructions to build
  swift-corelibs-libdispatch on Linux? I'm using Ubuntu 15.10 and have
  installed [the dependencies listed here](
  [swift-corelibs-dev] libdispatch build issue
  ). I have tried using both `swift/utils/build-script --libdispatch` as
  well as the instructions in the swift-corelibs-libdispatch `INSTALL`
  file. Both fail with the [same errors](
  https://gist.github.com/modocache/8389d5d64e39ca5a4281500f84e9ce11\). Any
  tips?

  2. I don't know much about C++, but from what I understand the C++ stdlib
  provides tools for asynchronous programming. I list SourceKit's uses of
  libdispatch in a comment on https://bugs.swift.org/browse/SR-1639\. For
  those familiar with both C++ and libdispatch: do you think it's a good
  idea to migrate to C++, at least for now?

Any advice would be appreciated. Thanks!!

- Brian Gesiak

  --
  Development thoughts at http://akrabat.com
  Daily Jotter for Mac OS X at http://dailyjotter.com
  _______________________________________________
  swift-corelibs-dev mailing list
  swift-corelibs-dev@swift.org
  https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Brian_Gesiak · May 30, 2016, 4:26pm

Gah, thanks Brian and Chris -- yes, I had forgotten. I initialized it once
on a separate development environment and forgot to do so again.

Thanks everyone! I managed to build the project. :)

I'll try getting SourceKit to link against my built libdispatch. Does
anyone have any thoughts on my second question? Would it make sense to move
SourceKit over to C++ async features? SourceKit only uses the following
libdispatch symbols:

- dispatch_after
- dispatch_release
- dispatch_semaphore_create
- dispatch_semaphore_signal
- dispatch_semaphore_wait
- dispatch_time

As far as I understand, having SourceKit use libdispatch means:

1. Having libdispatch built as part of all Swift builds that build
SourceKit. On OS X, most buildbot presets build SourceKit. I'd like to do
the same for Linux.
2. Building libdispatch prior to building Swift, or at least prior to
building Swift tools like SourceKit.

Moving these build dependencies around seems like more work than using C++
async, but I barely know C++, so I'm not sure. Thoughts?

- Brian Gesiak

···

On Mon, May 30, 2016 at 12:19 PM, Chris Bailey <BAILEYC@uk.ibm.com> wrote:

The missing reference is to 'pthread_workqueue_signal_np()' which was
added to libpwq via PR #10:
        thread manager signaling interface to allow libdispatch to force thre… by frankeh · Pull Request #10 · mheily/libpwq · GitHub

I suspect that you don't have the libpwq submodule checked out in
swift-corelibs-libdispatch. Is there any contents in
swift-corelibs-libdispatch/libpwq?

If not, running:
        git submodule init
        git submodule update
from the swift-corelibs-libdispatch directory should fix it.

Chris

From: Brian Gesiak <modocache@gmail.com>
To: Chris Bailey/UK/IBM@IBMGB
Cc: Rob Allen <rob@akrabat.com>, Swift Core Libs <
swift-corelibs-dev@swift.org>
Date: 30/05/2016 16:51
Subject: Re: [swift-corelibs-dev] libdispatch on Linux, or C++
std::async?
------------------------------

Thanks for the quick responses!

I had tried the experimental/foundation branch as well, with no luck.
Here's the full output from a clean build I just attempted:
*gist:fcf0c78908b39834448c73ac3f50c110 · GitHub*
<https://gist.github.com/modocache/fcf0c78908b39834448c73ac3f50c110>

Prior to the build I removed my entire build directory (which included
both libdispatch-linux-x86_64 and swift-linux-x86_64) and removed all build
artifacts from my libdispatch directory (I used `git clean -fxd` -- Make
tells me there's no target `distclean`).

Looking at the build failure, it appears to be the same one that was
reported in *[SR-1242] undefined reference to `pthread_workqueue_signal_np' · Issue #747 · apple/swift-corelibs-libdispatch · GitHub*
<https://bugs.swift.org/browse/SR-1242>\. The author of that report
indicates that
*Another fix for libpwq configuration by dgrove-oss · Pull Request #64 · apple/swift-corelibs-libdispatch · GitHub*
<https://github.com/apple/swift-corelibs-libdispatch/pull/64> fixes the
problem. My checkout includes the commits from that pull request, but the
problem remains.

In any case, thanks for all your help! I'll keep trying. Any and all
advice would be much appreciated!

- Brian Gesiak

On Mon, May 30, 2016 at 10:22 AM, Chris Bailey <*BAILEYC@uk.ibm.com* > <BAILEYC@uk.ibm.com>> wrote:
I'd certainly recommend picking up the experimental/foundation branch as
Rob recommends. Its where we've been doing some additional work to get
properly built into the Swift toolchain and to open up some internal APIs
needed by Foundation.

Whilst I've been building Dispatch either independently or as part of a
full toolchain using the buildbot_linux preset (with dispatch and
install-dispatch added), I've just checked your build line and its working
fine for me. It might be worth cleaning out any partial build you might
have of libdispatch in your build directory, and running 'make distclean'
from your libdispatch source directory.

FYI, we've got some further changes to the experimental/foundation branch
of libdispatch with should land in the next couple of days. This fixes some
issues with the Swift overlay, and resolves a crash that we've found in the
underlying libkqueue implementation.

Chris

From: Rob Allen via swift-corelibs-dev <
*swift-corelibs-dev@swift.org* <swift-corelibs-dev@swift.org>>
To: Brian Gesiak <*modocache@gmail.com* <modocache@gmail.com>>
Cc: Swift Core Libs <*swift-corelibs-dev@swift.org*
<swift-corelibs-dev@swift.org>>
Date: 30/05/2016 08:08
Subject: Re: [swift-corelibs-dev] libdispatch on Linux, or C++
std::async?
Sent by: *swift-corelibs-dev-bounces@swift.org*
<swift-corelibs-dev-bounces@swift.org>
------------------------------

Hi,

For what it's worth, I can't get the master branch to build either, but I
can get the experimental/foundation branch to build.

Regards,

Rob....

On 30 May 2016, at 05:28, Brian Gesiak via swift-corelibs-dev < > *swift-corelibs-dev@swift.org* <swift-corelibs-dev@swift.org>> wrote:

Hello all!

I'm working on *[SR-710] Generate XCTestCaseProvider entries on Linux · Issue #5320 · apple/swift-package-manager · GitHub*
<Issues · apple/swift-issues · GitHub, which involves porting SourceKit
to Linux. SourceKit contains a handful of references to libdispatch symbols.

I have two questions:

1. Could someone share a set of instructions to build
swift-corelibs-libdispatch on Linux? I'm using Ubuntu 15.10 and have
installed [the dependencies listed here](
*[swift-corelibs-dev] libdispatch build issue*
<https://lists.swift.org/pipermail/swift-corelibs-dev/Week-of-Mon-20151207/000122.html>\).
I have tried using both `swift/utils/build-script --libdispatch` as well as
the instructions in the swift-corelibs-libdispatch `INSTALL` file. Both
fail with the [same errors](
*gist:8389d5d64e39ca5a4281500f84e9ce11 · GitHub*
<https://gist.github.com/modocache/8389d5d64e39ca5a4281500f84e9ce11>\).
Any tips?

2. I don't know much about C++, but from what I understand the C++ stdlib
provides tools for asynchronous programming. I list SourceKit's uses of
libdispatch in a comment on *[SR-1639] [SourceKit] Add in-process implementations of sourcekitd functions only defined for XPC · Issue #5268 · apple/swift-package-manager · GitHub*
<https://bugs.swift.org/browse/SR-1639>\. For those familiar with both C++
and libdispatch: do you think it's a good idea to migrate to C++, at least
for now?

Any advice would be appreciated. Thanks!!

- Brian Gesiak

--
Development thoughts at *http://akrabat.com* <http://akrabat.com/>
Daily Jotter for Mac OS X at *http://dailyjotter.com*
<Daily Jotter;
_______________________________________________
swift-corelibs-dev mailing list
*swift-corelibs-dev@swift.org* <swift-corelibs-dev@swift.org>
*https://lists.swift.org/mailman/listinfo/swift-corelibs-dev*
<https://lists.swift.org/mailman/listinfo/swift-corelibs-dev>

Rob_Allen · May 30, 2016, 4:13pm

The steps that I use are:

$ cd /vagrant
$ git clone --recursive -b experimental/foundation GitHub - apple/swift-corelibs-libdispatch: The libdispatch Project, (a.k.a. Grand Central Dispatch), for concurrency on multicore hardware
$ cd swift-corelibs-libdispatch
$ sh ./autogen.sh
$ ./configure --with-swift-toolchain=/home/vagrant/swiftenv/versions/DEVELOPMENT-SNAPSHOT-2016-05-03-a/usr --prefix=/home/vagrant/swiftenv/versions/DEVELOPMENT-SNAPSHOT-2016-05-03-a/usr
$ make
$ make install

Regards,

Rob...

···

On 30 May 2016, at 16:50, Brian Gesiak <modocache@gmail.com> wrote:

Thanks for the quick responses!

I had tried the experimental/foundation branch as well, with no luck. Here's the full output from a clean build I just attempted: gist:fcf0c78908b39834448c73ac3f50c110 · GitHub

Prior to the build I removed my entire build directory (which included both libdispatch-linux-x86_64 and swift-linux-x86_64) and removed all build artifacts from my libdispatch directory (I used `git clean -fxd` -- Make tells me there's no target `distclean`).

Looking at the build failure, it appears to be the same one that was reported in https://bugs.swift.org/browse/SR-1242\. The author of that report indicates that Another fix for libpwq configuration by dgrove-oss · Pull Request #64 · apple/swift-corelibs-libdispatch · GitHub fixes the problem. My checkout includes the commits from that pull request, but the problem remains.

In any case, thanks for all your help! I'll keep trying. Any and all advice would be much appreciated!

- Brian Gesiak

On Mon, May 30, 2016 at 10:22 AM, Chris Bailey <BAILEYC@uk.ibm.com <mailto:BAILEYC@uk.ibm.com>> wrote:
I'd certainly recommend picking up the experimental/foundation branch as Rob recommends. Its where we've been doing some additional work to get properly built into the Swift toolchain and to open up some internal APIs needed by Foundation.

Whilst I've been building Dispatch either independently or as part of a full toolchain using the buildbot_linux preset (with dispatch and install-dispatch added), I've just checked your build line and its working fine for me. It might be worth cleaning out any partial build you might have of libdispatch in your build directory, and running 'make distclean' from your libdispatch source directory.

FYI, we've got some further changes to the experimental/foundation branch of libdispatch with should land in the next couple of days. This fixes some issues with the Swift overlay, and resolves a crash that we've found in the underlying libkqueue implementation.

Chris

From: Rob Allen via swift-corelibs-dev <swift-corelibs-dev@swift.org <mailto:swift-corelibs-dev@swift.org>>
To: Brian Gesiak <modocache@gmail.com <mailto:modocache@gmail.com>>
Cc: Swift Core Libs <swift-corelibs-dev@swift.org <mailto:swift-corelibs-dev@swift.org>>
Date: 30/05/2016 08:08
Subject: Re: [swift-corelibs-dev] libdispatch on Linux, or C++ std::async?
Sent by: swift-corelibs-dev-bounces@swift.org <mailto:swift-corelibs-dev-bounces@swift.org>

Hi,

For what it's worth, I can't get the master branch to build either, but I can get the experimental/foundation branch to build.

Regards,

Rob....

On 30 May 2016, at 05:28, Brian Gesiak via swift-corelibs-dev <swift-corelibs-dev@swift.org <mailto:swift-corelibs-dev@swift.org>> wrote:

Hello all!

I'm working on [SR-710] Generate XCTestCaseProvider entries on Linux · Issue #5320 · apple/swift-package-manager · GitHub, which involves porting SourceKit to Linux. SourceKit contains a handful of references to libdispatch symbols.

I have two questions:

1. Could someone share a set of instructions to build swift-corelibs-libdispatch on Linux? I'm using Ubuntu 15.10 and have installed [the dependencies listed here](https://lists.swift.org/pipermail/swift-corelibs-dev/Week-of-Mon-20151207/000122.html\). I have tried using both `swift/utils/build-script --libdispatch` as well as the instructions in the swift-corelibs-libdispatch `INSTALL` file. Both fail with the [same errors](https://gist.github.com/modocache/8389d5d64e39ca5a4281500f84e9ce11\). Any tips?

2. I don't know much about C++, but from what I understand the C++ stdlib provides tools for asynchronous programming. I list SourceKit's uses of libdispatch in a comment on https://bugs.swift.org/browse/SR-1639\. For those familiar with both C++ and libdispatch: do you think it's a good idea to migrate to C++, at least for now?

Any advice would be appreciated. Thanks!!

- Brian Gesiak

--
Development thoughts at http://akrabat.com <http://akrabat.com/>
Daily Jotter for Mac OS X at http://dailyjotter.com <Daily Jotter;
_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org <mailto:swift-corelibs-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

--
Development thoughts at http://akrabat.com
Daily Jotter for Mac OS X at http://dailyjotter.com

Chris_Bailey · May 30, 2016, 4:19pm

The missing reference is to 'pthread_workqueue_signal_np()' which was
added to libpwq via PR #10:
thread manager signaling interface to allow libdispatch to force thre… by frankeh · Pull Request #10 · mheily/libpwq · GitHub

I suspect that you don't have the libpwq submodule checked out in
swift-corelibs-libdispatch. Is there any contents in
swift-corelibs-libdispatch/libpwq?

If not, running:
git submodule init
git submodule update
from the swift-corelibs-libdispatch directory should fix it.

Chris

···

From: Brian Gesiak <modocache@gmail.com>
To: Chris Bailey/UK/IBM@IBMGB
Cc: Rob Allen <rob@akrabat.com>, Swift Core Libs
<swift-corelibs-dev@swift.org>
Date: 30/05/2016 16:51
Subject: Re: [swift-corelibs-dev] libdispatch on Linux, or C++
std::async?

Thanks for the quick responses!

I had tried the experimental/foundation branch as well, with no luck.
Here's the full output from a clean build I just attempted:

Prior to the build I removed my entire build directory (which included
both libdispatch-linux-x86_64 and swift-linux-x86_64) and removed all
build artifacts from my libdispatch directory (I used `git clean -fxd` --
Make tells me there's no target `distclean`).

Looking at the build failure, it appears to be the same one that was
reported in https://bugs.swift.org/browse/SR-1242\. The author of that
report indicates that
Another fix for libpwq configuration by dgrove-oss · Pull Request #64 · apple/swift-corelibs-libdispatch · GitHub fixes the
problem. My checkout includes the commits from that pull request, but the
problem remains.

In any case, thanks for all your help! I'll keep trying. Any and all
advice would be much appreciated!

- Brian Gesiak

On Mon, May 30, 2016 at 10:22 AM, Chris Bailey <BAILEYC@uk.ibm.com> wrote:
I'd certainly recommend picking up the experimental/foundation branch as
Rob recommends. Its where we've been doing some additional work to get
properly built into the Swift toolchain and to open up some internal APIs
needed by Foundation.

Whilst I've been building Dispatch either independently or as part of a
full toolchain using the buildbot_linux preset (with dispatch and
install-dispatch added), I've just checked your build line and its working
fine for me. It might be worth cleaning out any partial build you might
have of libdispatch in your build directory, and running 'make distclean'
from your libdispatch source directory.

FYI, we've got some further changes to the experimental/foundation branch
of libdispatch with should land in the next couple of days. This fixes
some issues with the Swift overlay, and resolves a crash that we've found
in the underlying libkqueue implementation.

Chris

From: Rob Allen via swift-corelibs-dev <
swift-corelibs-dev@swift.org>
To: Brian Gesiak <modocache@gmail.com>
Cc: Swift Core Libs <swift-corelibs-dev@swift.org>
Date: 30/05/2016 08:08
Subject: Re: [swift-corelibs-dev] libdispatch on Linux, or C++
std::async?
Sent by: swift-corelibs-dev-bounces@swift.org

Hi,

For what it's worth, I can't get the master branch to build either, but I
can get the experimental/foundation branch to build.

Regards,

Rob....

On 30 May 2016, at 05:28, Brian Gesiak via swift-corelibs-dev < swift-corelibs-dev@swift.org> wrote:

Hello all!

I'm working on [SR-710] Generate XCTestCaseProvider entries on Linux · Issue #5320 · apple/swift-package-manager · GitHub, which involves
porting SourceKit to Linux. SourceKit contains a handful of references to
libdispatch symbols.

I have two questions:

1. Could someone share a set of instructions to build
swift-corelibs-libdispatch on Linux? I'm using Ubuntu 15.10 and have
installed [the dependencies listed here](
https://lists.swift.org/pipermail/swift-corelibs-dev/Week-of-Mon-20151207/000122.html
). I have tried using both `swift/utils/build-script --libdispatch` as
well as the instructions in the swift-corelibs-libdispatch `INSTALL` file.
Both fail with the [same errors](
https://gist.github.com/modocache/8389d5d64e39ca5a4281500f84e9ce11\). Any
tips?

2. I don't know much about C++, but from what I understand the C++ stdlib
provides tools for asynchronous programming. I list SourceKit's uses of
libdispatch in a comment on https://bugs.swift.org/browse/SR-1639\. For
those familiar with both C++ and libdispatch: do you think it's a good
idea to migrate to C++, at least for now?

Any advice would be appreciated. Thanks!!

- Brian Gesiak

--
Development thoughts at http://akrabat.com
Daily Jotter for Mac OS X at http://dailyjotter.com
_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev

Brian_Croom · May 31, 2016, 12:39am

I have no hands-on experience with it itself, but my understanding is that
the std::async API in the current standards is pretty severely flawed and
makes it awfully easy to shoot yourself in the foot. On the other hand, the
libdispatch usage here does appear to be pretty basic.

Hopefully someone with more expertise can weigh in still!

···

måndag 30 maj 2016 skrev Brian Gesiak via swift-corelibs-dev < swift-corelibs-dev@swift.org>:

Gah, thanks Brian and Chris -- yes, I had forgotten. I initialized it once
on a separate development environment and forgot to do so again.

Thanks everyone! I managed to build the project. :)

I'll try getting SourceKit to link against my built libdispatch. Does
anyone have any thoughts on my second question? Would it make sense to move
SourceKit over to C++ async features? SourceKit only uses the following
libdispatch symbols:

- dispatch_after
- dispatch_release
- dispatch_semaphore_create
- dispatch_semaphore_signal
- dispatch_semaphore_wait
- dispatch_time

As far as I understand, having SourceKit use libdispatch means:

1. Having libdispatch built as part of all Swift builds that build
SourceKit. On OS X, most buildbot presets build SourceKit. I'd like to do
the same for Linux.
2. Building libdispatch prior to building Swift, or at least prior to
building Swift tools like SourceKit.

Moving these build dependencies around seems like more work than using C++
async, but I barely know C++, so I'm not sure. Thoughts?

- Brian Gesiak

On Mon, May 30, 2016 at 12:19 PM, Chris Bailey <BAILEYC@uk.ibm.com > <javascript:_e(%7B%7D,'cvml','BAILEYC@uk.ibm.com');>> wrote:

The missing reference is to 'pthread_workqueue_signal_np()' which was
added to libpwq via PR #10:
        thread manager signaling interface to allow libdispatch to force thre… by frankeh · Pull Request #10 · mheily/libpwq · GitHub

I suspect that you don't have the libpwq submodule checked out in
swift-corelibs-libdispatch. Is there any contents in
swift-corelibs-libdispatch/libpwq?

If not, running:
        git submodule init
        git submodule update
from the swift-corelibs-libdispatch directory should fix it.

Chris

From: Brian Gesiak <modocache@gmail.com
<javascript:_e(%7B%7D,'cvml','modocache@gmail.com');>>
To: Chris Bailey/UK/IBM@IBMGB
Cc: Rob Allen <rob@akrabat.com
<javascript:_e(%7B%7D,'cvml','rob@akrabat.com');>>, Swift Core Libs <
swift-corelibs-dev@swift.org
<javascript:_e(%7B%7D,'cvml','swift-corelibs-dev@swift.org');>>
Date: 30/05/2016 16:51
Subject: Re: [swift-corelibs-dev] libdispatch on Linux, or C++
std::async?
------------------------------

Thanks for the quick responses!

I had tried the experimental/foundation branch as well, with no luck.
Here's the full output from a clean build I just attempted:
*gist:fcf0c78908b39834448c73ac3f50c110 · GitHub*
<https://gist.github.com/modocache/fcf0c78908b39834448c73ac3f50c110>

Prior to the build I removed my entire build directory (which included
both libdispatch-linux-x86_64 and swift-linux-x86_64) and removed all build
artifacts from my libdispatch directory (I used `git clean -fxd` -- Make
tells me there's no target `distclean`).

Looking at the build failure, it appears to be the same one that was
reported in *[SR-1242] undefined reference to `pthread_workqueue_signal_np' · Issue #747 · apple/swift-corelibs-libdispatch · GitHub*
<https://bugs.swift.org/browse/SR-1242>\. The author of that report
indicates that
*Another fix for libpwq configuration by dgrove-oss · Pull Request #64 · apple/swift-corelibs-libdispatch · GitHub*
<https://github.com/apple/swift-corelibs-libdispatch/pull/64> fixes the
problem. My checkout includes the commits from that pull request, but the
problem remains.

In any case, thanks for all your help! I'll keep trying. Any and all
advice would be much appreciated!

- Brian Gesiak

On Mon, May 30, 2016 at 10:22 AM, Chris Bailey <*BAILEYC@uk.ibm.com* >> <javascript:_e(%7B%7D,'cvml','BAILEYC@uk.ibm.com');>> wrote:
I'd certainly recommend picking up the experimental/foundation branch as
Rob recommends. Its where we've been doing some additional work to get
properly built into the Swift toolchain and to open up some internal APIs
needed by Foundation.

Whilst I've been building Dispatch either independently or as part of a
full toolchain using the buildbot_linux preset (with dispatch and
install-dispatch added), I've just checked your build line and its working
fine for me. It might be worth cleaning out any partial build you might
have of libdispatch in your build directory, and running 'make distclean'
from your libdispatch source directory.

FYI, we've got some further changes to the experimental/foundation branch
of libdispatch with should land in the next couple of days. This fixes some
issues with the Swift overlay, and resolves a crash that we've found in the
underlying libkqueue implementation.

Chris

From: Rob Allen via swift-corelibs-dev <
*swift-corelibs-dev@swift.org*
<javascript:_e(%7B%7D,'cvml','swift-corelibs-dev@swift.org');>>
To: Brian Gesiak <*modocache@gmail.com*
<javascript:_e(%7B%7D,'cvml','modocache@gmail.com');>>
Cc: Swift Core Libs <*swift-corelibs-dev@swift.org*
<javascript:_e(%7B%7D,'cvml','swift-corelibs-dev@swift.org');>>
Date: 30/05/2016 08:08
Subject: Re: [swift-corelibs-dev] libdispatch on Linux, or C++
std::async?
Sent by: *swift-corelibs-dev-bounces@swift.org*
<javascript:_e(%7B%7D,'cvml','swift-corelibs-dev-bounces@swift.org');>
------------------------------

Hi,

For what it's worth, I can't get the master branch to build either, but I
can get the experimental/foundation branch to build.

Regards,

Rob....

On 30 May 2016, at 05:28, Brian Gesiak via swift-corelibs-dev < >> *swift-corelibs-dev@swift.org* >> <javascript:_e(%7B%7D,'cvml','swift-corelibs-dev@swift.org');>> wrote:

Hello all!

I'm working on *[SR-710] Generate XCTestCaseProvider entries on Linux · Issue #5320 · apple/swift-package-manager · GitHub*
<Issues · apple/swift-issues · GitHub, which involves porting SourceKit
to Linux. SourceKit contains a handful of references to libdispatch symbols.

I have two questions:

1. Could someone share a set of instructions to build
swift-corelibs-libdispatch on Linux? I'm using Ubuntu 15.10 and have
installed [the dependencies listed here](
*[swift-corelibs-dev] libdispatch build issue*
<https://lists.swift.org/pipermail/swift-corelibs-dev/Week-of-Mon-20151207/000122.html>\).
I have tried using both `swift/utils/build-script --libdispatch` as well as
the instructions in the swift-corelibs-libdispatch `INSTALL` file. Both
fail with the [same errors](
*gist:8389d5d64e39ca5a4281500f84e9ce11 · GitHub*
<https://gist.github.com/modocache/8389d5d64e39ca5a4281500f84e9ce11>\).
Any tips?

2. I don't know much about C++, but from what I understand the C++ stdlib
provides tools for asynchronous programming. I list SourceKit's uses of
libdispatch in a comment on *[SR-1639] [SourceKit] Add in-process implementations of sourcekitd functions only defined for XPC · Issue #5268 · apple/swift-package-manager · GitHub*
<https://bugs.swift.org/browse/SR-1639>\. For those familiar with both
C++ and libdispatch: do you think it's a good idea to migrate to C++, at
least for now?

Any advice would be appreciated. Thanks!!

- Brian Gesiak

--
Development thoughts at *http://akrabat.com* <http://akrabat.com/>
Daily Jotter for Mac OS X at *http://dailyjotter.com*
<Daily Jotter;
_______________________________________________
swift-corelibs-dev mailing list
*swift-corelibs-dev@swift.org*
<javascript:_e(%7B%7D,'cvml','swift-corelibs-dev@swift.org');>
*https://lists.swift.org/mailman/listinfo/swift-corelibs-dev*
<https://lists.swift.org/mailman/listinfo/swift-corelibs-dev>