Good day, Swift package manager development folks.
(There are at least two separate issues being inquired about, but with the same introductory context.)
"Cloudy" deployment options derived from or akin to CloudFoundry are agonizingly locked-down environments. Essentially Swift and all of its dependencies and one's project's dependencies must be stuffed into an arbitrary directory (henceforth referred to as "the hole," but usually /app/ ) and build processes performed without any root-user privileges. One consequence is that one cannot use the OS' package-management system to install dependencies, but one must obtain them and wrestle them into "the hole," instead. The strategy seems rather silly.
While developing a so-called "buildpack" for Swift 3 projects to be deployed via CloudFoundryish options and utilizing the 'swift build' command, I have come across a few issues.
One issue is that 'swift build' invokes and 'swift-build' command, who then invokes a 'swiftc' command, but the environment provided to this last is insufficient for the dynamic loader to gain knowledge of libraries present in "the hole" and specified by the LD_LIBRARY_PATH variable. This variable and other environment-variables present for the parent processes are not provided to the 'swiftc' command. This means that libraries are searched for in the usual OS locations, but due to the locked-down CloudFoundry environment, they won't be found. At one point I was able to find a 'popen' in an older version of Swift that seemed to be responsible, but I cannot find it in the Swift 3 [prototype] source-code.
Is there a good reason to discard these environment-variables, or would it be reasonable to introduce some logic for 'swift-build' to pass its environment-variables to its child 'swiftc' process(es)?
I have extracted it and its dependencies into the /app/.delta/ directory. I am issuing the following command:
swift build -Xcc -I/app/.delta/ -Xswiftc -I/app/.delta/ -Xlinker -L/app/.delta/ -v
I observe the following error:
/home/vcap/app/.delta/usr/bin/clang++: error while loading shared libraries: libLLVM-3.4.so.1: cannot open shared object file: No such file or directory
The library is present at /app/.delta/usr/lib/x86_64-linux-gnu/libLLVM-3.4.so.1
The environment for BASh includes both LIBRARY_PATH and LD_LIBRARY_PATH variables, which both include the
/app/.delta/usr/lib/x86_64-linux-gnu/ directory. These variables are both exported. Once again, the environment that clang++ is created withdoes not include these critical environment variables. The -Xlinker doesn't seem to be helping, either.
My questions are:
Is my expectation for -Xlinker incorrect? Is there a good reason to discard these environment-variables, or would it be reasonable to introduce some logic for 'swift-build' to pass its inherited environment-variables to its child process(es)?
On 6/8/2016 23:03, Shao Miller via swift-build-dev wrote:
Good day, Swift package manager development folks.
(There are at least two separate issues being inquired about, but with
the same introductory context.)
"Cloudy" deployment options derived from or akin to CloudFoundry are
agonizingly locked-down environments. Essentially Swift and all of
its dependencies and one's project's dependencies must be stuffed into
an arbitrary directory (henceforth referred to as "the hole," but
usually /app/ ) and build processes performed without any root-user
privileges. One consequence is that one cannot use the OS'
package-management system to install dependencies, but one must obtain
them and wrestle them into "the hole," instead. The strategy seems
rather silly.
While developing a so-called "buildpack" for Swift 3 projects to be
deployed via CloudFoundryish options and utilizing the 'swift build'
command, I have come across a few issues.
One issue is that 'swift build' invokes and 'swift-build' command, who
then invokes a 'swiftc' command, but the environment provided to this
last is insufficient for the dynamic loader to gain knowledge of
libraries present in "the hole" and specified by the LD_LIBRARY_PATH
variable. This variable and other environment-variables present for
the parent processes are not provided to the 'swiftc' command. This
means that libraries are searched for in the usual OS locations, but
due to the locked-down CloudFoundry environment, they won't be found.
At one point I was able to find a 'popen' in an older version of Swift
that seemed to be responsible, but I cannot find it in the Swift 3
[prototype] source-code.
Is there a good reason to discard these environment-variables, or
would it be reasonable to introduce some logic for 'swift-build' to
pass its environment-variables to its child 'swiftc' process(es)?
Please disregard my mention of -Xlinker, as I was temporarily confused by the error-message.
If I rename clang to clang.orig and introduce a fake clang as a script that establishes LD_LIBRARY_PATH and invokes clang.orig, all is well. This allows the following question to remain:
Is there a good reason to discard these environment-variables, or would it be reasonable to introduce some logic for 'swift-build' to pass its inherited environment-variables to its child process(es)?
I have extracted it and its dependencies into the /app/.delta/
directory. I am issuing the following command:
swift build -Xcc -I/app/.delta/ -Xswiftc -I/app/.delta/ -Xlinker
-L/app/.delta/ -v
I observe the following error:
/home/vcap/app/.delta/usr/bin/clang++: error while loading shared
libraries: libLLVM-3.4.so.1: cannot open shared object file: No such
file or directory
The library is present at
/app/.delta/usr/lib/x86_64-linux-gnu/libLLVM-3.4.so.1
The environment for BASh includes both LIBRARY_PATH and
LD_LIBRARY_PATH variables, which both include the
/app/.delta/usr/lib/x86_64-linux-gnu/ directory. These variables are
both exported. Once again, the environment that clang++ is created
withdoes not include these critical environment variables. The
-Xlinker doesn't seem to be helping, either.
My questions are:
Is my expectation for -Xlinker incorrect? Is there a good reason to
discard these environment-variables, or would it be reasonable to
introduce some logic for 'swift-build' to pass its inherited
environment-variables to its child process(es)?
On 6/8/2016 23:03, Shao Miller via swift-build-dev wrote:
Good day, Swift package manager development folks.
(There are at least two separate issues being inquired about, but with
the same introductory context.)
"Cloudy" deployment options derived from or akin to CloudFoundry are
agonizingly locked-down environments. Essentially Swift and all of
its dependencies and one's project's dependencies must be stuffed into
an arbitrary directory (henceforth referred to as "the hole," but
usually /app/ ) and build processes performed without any root-user
privileges. One consequence is that one cannot use the OS'
package-management system to install dependencies, but one must obtain
them and wrestle them into "the hole," instead. The strategy seems
rather silly.
While developing a so-called "buildpack" for Swift 3 projects to be
deployed via CloudFoundryish options and utilizing the 'swift build'
command, I have come across a few issues.
One issue is that 'swift build' invokes and 'swift-build' command, who
then invokes a 'swiftc' command, but the environment provided to this
last is insufficient for the dynamic loader to gain knowledge of
libraries present in "the hole" and specified by the LD_LIBRARY_PATH
variable. This variable and other environment-variables present for
the parent processes are not provided to the 'swiftc' command. This
means that libraries are searched for in the usual OS locations, but
due to the locked-down CloudFoundry environment, they won't be found.
At one point I was able to find a 'popen' in an older version of Swift
that seemed to be responsible, but I cannot find it in the Swift 3
[prototype] source-code.
Is there a good reason to discard these environment-variables, or
would it be reasonable to introduce some logic for 'swift-build' to
pass its environment-variables to its child 'swiftc' process(es)?
Given that nobody has volunteered an answer, does anyone have any suggestions for how I should best pursue the answer to this question? Should I track down where the sub-process is spawned, then 'git blame' and contact the author, directly?
Thanks for reading.
- Shao Miller
···
On 7/4/2016 22:45, Shao Miller via swift-build-dev wrote:
Please disregard my mention of -Xlinker, as I was temporarily confused by the error-message.
If I rename clang to clang.orig and introduce a fake clang as a script that establishes LD_LIBRARY_PATH and invokes clang.orig, all is well. This allows the following question to remain:
Is there a good reason to discard these environment-variables, or would it be reasonable to introduce some logic for 'swift-build' to pass its inherited environment-variables to its child process(es)?
I have extracted it and its dependencies into the /app/.delta/
directory. I am issuing the following command:
swift build -Xcc -I/app/.delta/ -Xswiftc -I/app/.delta/ -Xlinker
-L/app/.delta/ -v
I observe the following error:
/home/vcap/app/.delta/usr/bin/clang++: error while loading shared
libraries: libLLVM-3.4.so.1: cannot open shared object file: No such
file or directory
The library is present at
/app/.delta/usr/lib/x86_64-linux-gnu/libLLVM-3.4.so.1
The environment for BASh includes both LIBRARY_PATH and
LD_LIBRARY_PATH variables, which both include the
/app/.delta/usr/lib/x86_64-linux-gnu/ directory. These variables are
both exported. Once again, the environment that clang++ is created
withdoes not include these critical environment variables. The
-Xlinker doesn't seem to be helping, either.
My questions are:
Is my expectation for -Xlinker incorrect? Is there a good reason to
discard these environment-variables, or would it be reasonable to
introduce some logic for 'swift-build' to pass its inherited
environment-variables to its child process(es)?
On 6/8/2016 23:03, Shao Miller via swift-build-dev wrote:
Good day, Swift package manager development folks.
(There are at least two separate issues being inquired about, but with
the same introductory context.)
"Cloudy" deployment options derived from or akin to CloudFoundry are
agonizingly locked-down environments. Essentially Swift and all of
its dependencies and one's project's dependencies must be stuffed into
an arbitrary directory (henceforth referred to as "the hole," but
usually /app/ ) and build processes performed without any root-user
privileges. One consequence is that one cannot use the OS'
package-management system to install dependencies, but one must obtain
them and wrestle them into "the hole," instead. The strategy seems
rather silly.
While developing a so-called "buildpack" for Swift 3 projects to be
deployed via CloudFoundryish options and utilizing the 'swift build'
command, I have come across a few issues.
One issue is that 'swift build' invokes and 'swift-build' command, who
then invokes a 'swiftc' command, but the environment provided to this
last is insufficient for the dynamic loader to gain knowledge of
libraries present in "the hole" and specified by the LD_LIBRARY_PATH
variable. This variable and other environment-variables present for
the parent processes are not provided to the 'swiftc' command. This
means that libraries are searched for in the usual OS locations, but
due to the locked-down CloudFoundry environment, they won't be found.
At one point I was able to find a 'popen' in an older version of Swift
that seemed to be responsible, but I cannot find it in the Swift 3
[prototype] source-code.
Is there a good reason to discard these environment-variables, or
would it be reasonable to introduce some logic for 'swift-build' to
pass its environment-variables to its child 'swiftc' process(es)?
Does this issue not appear to block Swift from being a successful back-end choice with CloudFoundry-based application-deployment options, including Pivotal, Heroku and IBM Bluemix? I'm surprised and disappointed by the lack of discussion. I would guess that it would be unfortunate to release Swift 3 and find that no CloudFoundry people can use it because it (Swift) doesn't co-operate with the limitations of those environments.
On 7/12/2016 20:43, Shao Miller via swift-build-dev wrote:
Good day, Swift build folks.
Given that nobody has volunteered an answer, does anyone have any suggestions for how I should best pursue the answer to this question? Should I track down where the sub-process is spawned, then 'git blame' and contact the author, directly?
Thanks for reading.
- Shao Miller
On 7/4/2016 22:45, Shao Miller via swift-build-dev wrote:
Please disregard my mention of -Xlinker, as I was temporarily confused by the error-message.
If I rename clang to clang.orig and introduce a fake clang as a script that establishes LD_LIBRARY_PATH and invokes clang.orig, all is well. This allows the following question to remain:
Is there a good reason to discard these environment-variables, or would it be reasonable to introduce some logic for 'swift-build' to pass its inherited environment-variables to its child process(es)?
I have extracted it and its dependencies into the /app/.delta/
directory. I am issuing the following command:
swift build -Xcc -I/app/.delta/ -Xswiftc -I/app/.delta/ -Xlinker
-L/app/.delta/ -v
I observe the following error:
/home/vcap/app/.delta/usr/bin/clang++: error while loading shared
libraries: libLLVM-3.4.so.1: cannot open shared object file: No such
file or directory
The library is present at
/app/.delta/usr/lib/x86_64-linux-gnu/libLLVM-3.4.so.1
The environment for BASh includes both LIBRARY_PATH and
LD_LIBRARY_PATH variables, which both include the
/app/.delta/usr/lib/x86_64-linux-gnu/ directory. These variables are
both exported. Once again, the environment that clang++ is created
withdoes not include these critical environment variables. The
-Xlinker doesn't seem to be helping, either.
My questions are:
Is my expectation for -Xlinker incorrect? Is there a good reason to
discard these environment-variables, or would it be reasonable to
introduce some logic for 'swift-build' to pass its inherited
environment-variables to its child process(es)?
On 6/8/2016 23:03, Shao Miller via swift-build-dev wrote:
Good day, Swift package manager development folks.
(There are at least two separate issues being inquired about, but with
the same introductory context.)
"Cloudy" deployment options derived from or akin to CloudFoundry are
agonizingly locked-down environments. Essentially Swift and all of
its dependencies and one's project's dependencies must be stuffed into
an arbitrary directory (henceforth referred to as "the hole," but
usually /app/ ) and build processes performed without any root-user
privileges. One consequence is that one cannot use the OS'
package-management system to install dependencies, but one must obtain
them and wrestle them into "the hole," instead. The strategy seems
rather silly.
While developing a so-called "buildpack" for Swift 3 projects to be
deployed via CloudFoundryish options and utilizing the 'swift build'
command, I have come across a few issues.
One issue is that 'swift build' invokes and 'swift-build' command, who
then invokes a 'swiftc' command, but the environment provided to this
last is insufficient for the dynamic loader to gain knowledge of
libraries present in "the hole" and specified by the LD_LIBRARY_PATH
variable. This variable and other environment-variables present for
the parent processes are not provided to the 'swiftc' command. This
means that libraries are searched for in the usual OS locations, but
due to the locked-down CloudFoundry environment, they won't be found.
At one point I was able to find a 'popen' in an older version of Swift
that seemed to be responsible, but I cannot find it in the Swift 3
[prototype] source-code.
Is there a good reason to discard these environment-variables, or
would it be reasonable to introduce some logic for 'swift-build' to
pass its environment-variables to its child 'swiftc' process(es)?
You are right, this is a very big issue currently. The unfortunate reality of SwiftPM is that it is still very early days, and while people are doing all sorts of interesting cool things with it, there are also a long list of major limitations.
That said, I know many people are able to get some things running on various PaaS platforms (including Heroku and CloudFoundry-based things) so it is not completely blocking use in those environments.
- Daniel
···
On Jul 21, 2016, at 11:07 AM, Shao Miller via swift-build-dev <swift-build-dev@swift.org> wrote:
Does this issue not appear to block Swift from being a successful back-end choice with CloudFoundry-based application-deployment options, including Pivotal, Heroku and IBM Bluemix? I'm surprised and disappointed by the lack of discussion. I would guess that it would be unfortunate to release Swift 3 and find that no CloudFoundry people can use it because it (Swift) doesn't co-operate with the limitations of those environments.
On 7/12/2016 20:43, Shao Miller via swift-build-dev wrote:
Good day, Swift build folks.
Given that nobody has volunteered an answer, does anyone have any suggestions for how I should best pursue the answer to this question? Should I track down where the sub-process is spawned, then 'git blame' and contact the author, directly?
Thanks for reading.
- Shao Miller
On 7/4/2016 22:45, Shao Miller via swift-build-dev wrote:
Please disregard my mention of -Xlinker, as I was temporarily confused by the error-message.
If I rename clang to clang.orig and introduce a fake clang as a script that establishes LD_LIBRARY_PATH and invokes clang.orig, all is well. This allows the following question to remain:
Is there a good reason to discard these environment-variables, or would it be reasonable to introduce some logic for 'swift-build' to pass its inherited environment-variables to its child process(es)?
I have extracted it and its dependencies into the /app/.delta/
directory. I am issuing the following command:
swift build -Xcc -I/app/.delta/ -Xswiftc -I/app/.delta/ -Xlinker
-L/app/.delta/ -v
I observe the following error:
/home/vcap/app/.delta/usr/bin/clang++: error while loading shared
libraries: libLLVM-3.4.so.1: cannot open shared object file: No such
file or directory
The library is present at
/app/.delta/usr/lib/x86_64-linux-gnu/libLLVM-3.4.so.1
The environment for BASh includes both LIBRARY_PATH and
LD_LIBRARY_PATH variables, which both include the
/app/.delta/usr/lib/x86_64-linux-gnu/ directory. These variables are
both exported. Once again, the environment that clang++ is created
withdoes not include these critical environment variables. The
-Xlinker doesn't seem to be helping, either.
My questions are:
Is my expectation for -Xlinker incorrect? Is there a good reason to
discard these environment-variables, or would it be reasonable to
introduce some logic for 'swift-build' to pass its inherited
environment-variables to its child process(es)?
On 6/8/2016 23:03, Shao Miller via swift-build-dev wrote:
Good day, Swift package manager development folks.
(There are at least two separate issues being inquired about, but with
the same introductory context.)
"Cloudy" deployment options derived from or akin to CloudFoundry are
agonizingly locked-down environments. Essentially Swift and all of
its dependencies and one's project's dependencies must be stuffed into
an arbitrary directory (henceforth referred to as "the hole," but
usually /app/ ) and build processes performed without any root-user
privileges. One consequence is that one cannot use the OS'
package-management system to install dependencies, but one must obtain
them and wrestle them into "the hole," instead. The strategy seems
rather silly.
While developing a so-called "buildpack" for Swift 3 projects to be
deployed via CloudFoundryish options and utilizing the 'swift build'
command, I have come across a few issues.
One issue is that 'swift build' invokes and 'swift-build' command, who
then invokes a 'swiftc' command, but the environment provided to this
last is insufficient for the dynamic loader to gain knowledge of
libraries present in "the hole" and specified by the LD_LIBRARY_PATH
variable. This variable and other environment-variables present for
the parent processes are not provided to the 'swiftc' command. This
means that libraries are searched for in the usual OS locations, but
due to the locked-down CloudFoundry environment, they won't be found.
At one point I was able to find a 'popen' in an older version of Swift
that seemed to be responsible, but I cannot find it in the Swift 3
[prototype] source-code.
Is there a good reason to discard these environment-variables, or
would it be reasonable to introduce some logic for 'swift-build' to
pass its environment-variables to its child 'swiftc' process(es)?
I suppose my concern is that the currently working CloudFoundry options might be using a version of Swift prior to the June 20th snapshot and that Swift 3 (once released) will have had this issue slip past. If anyone is able to point out which source-code file for the swift-build program is responsible for spawning the swiftc or clang programs, then perhaps I can submit a patch.
You are right, this is a very big issue currently. The unfortunate reality of SwiftPM is that it is still very early days, and while people are doing all sorts of interesting cool things with it, there are also a long list of major limitations.
That said, I know many people are able to get some things running on various PaaS platforms (including Heroku and CloudFoundry-based things) so it is not completely blocking use in those environments.
- Daniel
On Jul 21, 2016, at 11:07 AM, Shao Miller via swift-build-dev <swift-build-dev@swift.org> wrote:
Does this issue not appear to block Swift from being a successful back-end choice with CloudFoundry-based application-deployment options, including Pivotal, Heroku and IBM Bluemix? I'm surprised and disappointed by the lack of discussion. I would guess that it would be unfortunate to release Swift 3 and find that no CloudFoundry people can use it because it (Swift) doesn't co-operate with the limitations of those environments.
On 7/12/2016 20:43, Shao Miller via swift-build-dev wrote:
Good day, Swift build folks.
Given that nobody has volunteered an answer, does anyone have any suggestions for how I should best pursue the answer to this question? Should I track down where the sub-process is spawned, then 'git blame' and contact the author, directly?
Thanks for reading.
- Shao Miller
On 7/4/2016 22:45, Shao Miller via swift-build-dev wrote:
Please disregard my mention of -Xlinker, as I was temporarily confused by the error-message.
If I rename clang to clang.orig and introduce a fake clang as a script that establishes LD_LIBRARY_PATH and invokes clang.orig, all is well. This allows the following question to remain:
Is there a good reason to discard these environment-variables, or would it be reasonable to introduce some logic for 'swift-build' to pass its inherited environment-variables to its child process(es)?
I have extracted it and its dependencies into the /app/.delta/
directory. I am issuing the following command:
swift build -Xcc -I/app/.delta/ -Xswiftc -I/app/.delta/ -Xlinker
-L/app/.delta/ -v
I observe the following error:
/home/vcap/app/.delta/usr/bin/clang++: error while loading shared
libraries: libLLVM-3.4.so.1: cannot open shared object file: No such
file or directory
The library is present at
/app/.delta/usr/lib/x86_64-linux-gnu/libLLVM-3.4.so.1
The environment for BASh includes both LIBRARY_PATH and
LD_LIBRARY_PATH variables, which both include the
/app/.delta/usr/lib/x86_64-linux-gnu/ directory. These variables are
both exported. Once again, the environment that clang++ is created
withdoes not include these critical environment variables. The
-Xlinker doesn't seem to be helping, either.
My questions are:
Is my expectation for -Xlinker incorrect? Is there a good reason to
discard these environment-variables, or would it be reasonable to
introduce some logic for 'swift-build' to pass its inherited
environment-variables to its child process(es)?
On 6/8/2016 23:03, Shao Miller via swift-build-dev wrote:
Good day, Swift package manager development folks.
(There are at least two separate issues being inquired about, but with
the same introductory context.)
"Cloudy" deployment options derived from or akin to CloudFoundry are
agonizingly locked-down environments. Essentially Swift and all of
its dependencies and one's project's dependencies must be stuffed into
an arbitrary directory (henceforth referred to as "the hole," but
usually /app/ ) and build processes performed without any root-user
privileges. One consequence is that one cannot use the OS'
package-management system to install dependencies, but one must obtain
them and wrestle them into "the hole," instead. The strategy seems
rather silly.
While developing a so-called "buildpack" for Swift 3 projects to be
deployed via CloudFoundryish options and utilizing the 'swift build'
command, I have come across a few issues.
One issue is that 'swift build' invokes and 'swift-build' command, who
then invokes a 'swiftc' command, but the environment provided to this
last is insufficient for the dynamic loader to gain knowledge of
libraries present in "the hole" and specified by the LD_LIBRARY_PATH
variable. This variable and other environment-variables present for
the parent processes are not provided to the 'swiftc' command. This
means that libraries are searched for in the usual OS locations, but
due to the locked-down CloudFoundry environment, they won't be found.
At one point I was able to find a 'popen' in an older version of Swift
that seemed to be responsible, but I cannot find it in the Swift 3
[prototype] source-code.
Is there a good reason to discard these environment-variables, or
would it be reasonable to introduce some logic for 'swift-build' to
pass its environment-variables to its child 'swiftc' process(es)?