Linux: Build process not finding my ICU


(Eric Wing) #1

I'm trying to build Swift 3 on a Linux system that doesn't come with a
usable libICU.
I've built ICU myself and placed it in a non-root directory (along
with a lot of other dependencies not available on my system that Swift
needs).

I've had to reimplement my patch because it looks like a rebase was
done which broke the merge. (https://bugs.swift.org/browse/SR-1358)

Anyway, I think I've gotten through building the swift compiler
itself, but the build process is now in swift-corelibs-foundation. The
build process fails at #include <unicode/ucal.h> for
CoreFoundation/Locale.subproj/CFCalendar.c

Looking at the build flags in my console, I see no -I flags to the
location of my ICU headers (or any other of my extra dependencies).

How do I fix the build process to make sure these extra paths are searched?

Thanks,
Eric


(Will Stanton) #2

Hello Eric,

Not sure if this will help, but perhaps you can feed something in build.py?
https://github.com/apple/swift-corelibs-foundation/blob/master/build.py

You seem to be working under some tight constraints! Is installing ICU in /usr/local/ not possible?

Regards,
Will Stanton

···

On Sep 13, 2016, at 6:21 PM, Eric Wing via swift-dev <swift-dev@swift.org> wrote:

I'm trying to build Swift 3 on a Linux system that doesn't come with a
usable libICU.
I've built ICU myself and placed it in a non-root directory (along
with a lot of other dependencies not available on my system that Swift
needs).

I've had to reimplement my patch because it looks like a rebase was
done which broke the merge. (https://bugs.swift.org/browse/SR-1358)

Anyway, I think I've gotten through building the swift compiler
itself, but the build process is now in swift-corelibs-foundation. The
build process fails at #include <unicode/ucal.h> for
CoreFoundation/Locale.subproj/CFCalendar.c

Looking at the build flags in my console, I see no -I flags to the
location of my ICU headers (or any other of my extra dependencies).

How do I fix the build process to make sure these extra paths are searched?

Thanks,
Eric


(Eric Wing) #3

Thanks for the reply. That was helpful in giving me a place to start.

So I'm trying to make it possible to build Swift binaries that can be
run on most modern Linux's. Steam-Runtime is my base as they are
heavily focused on binary distribution. I suppose I could put stuff
into my /usr/local since they do not use that directory for their
stuff. However, as a general principle, I try to avoid putting stuff
into installed areas because it makes auditing dependencies for binary
distribution harder later. The CMake build process is mostly aware
that components can be elsewhere, but this part of the build process
is not using CMake.

So I hacked up build.py, but it looks like I have additional problems.

Foundation is using newer features in libxml and libcurl than what I
have available. xmlXPathNodeEval and CURLOPT_XFERINFOFUNCTION seem to
be the biggest roadblocks at the moment.

But my bigger problem at the moment is build.py doesn't put the Link
flags in the right place. When it builds foundation, it puts the link
flags first, and then all the .o filles from Foundation. The link
flags need to be after all the .o files. Some of my dependencies are
static libraries, and I think if the link process tries to use a
library before the symbol is actually needed, it gets dropped early.
Then I get a massive amount of unresolved symbol errors. (I've
verified by manually invoking the command on the terminal myself.)

I would appreciate any thoughts/help on what I should look at changing
next. And I would like to do this in a way that might be accepted back
into the main source tree.

Thanks,
Eric

···

On 9/13/16, Will Stanton <willstanton1@yahoo.com> wrote:

Hello Eric,

Not sure if this will help, but perhaps you can feed something in build.py?
https://github.com/apple/swift-corelibs-foundation/blob/master/build.py

You seem to be working under some tight constraints! Is installing ICU in
/usr/local/ not possible?

Regards,
Will Stanton


(Tony Parker) #4

Hi Eric,

Hello Eric,

Not sure if this will help, but perhaps you can feed something in build.py?
https://github.com/apple/swift-corelibs-foundation/blob/master/build.py

You seem to be working under some tight constraints! Is installing ICU in
/usr/local/ not possible?

Regards,
Will Stanton

Thanks for the reply. That was helpful in giving me a place to start.

So I'm trying to make it possible to build Swift binaries that can be
run on most modern Linux's. Steam-Runtime is my base as they are
heavily focused on binary distribution. I suppose I could put stuff
into my /usr/local since they do not use that directory for their
stuff. However, as a general principle, I try to avoid putting stuff
into installed areas because it makes auditing dependencies for binary
distribution harder later. The CMake build process is mostly aware
that components can be elsewhere, but this part of the build process
is not using CMake.

So I hacked up build.py, but it looks like I have additional problems.

Foundation is using newer features in libxml and libcurl than what I
have available. xmlXPathNodeEval and CURLOPT_XFERINFOFUNCTION seem to
be the biggest roadblocks at the moment.

But my bigger problem at the moment is build.py doesn't put the Link
flags in the right place. When it builds foundation, it puts the link
flags first, and then all the .o filles from Foundation. The link
flags need to be after all the .o files. Some of my dependencies are
static libraries, and I think if the link process tries to use a
library before the symbol is actually needed, it gets dropped early.
Then I get a massive amount of unresolved symbol errors. (I've
verified by manually invoking the command on the terminal myself.)

I think a patch which fixed just this part would probably be fine.

In general we’re probably going to just have to specify a list of minimum dependencies for Foundation. We don’t really have a great story for missing features from ICU or libxml.

- Tony

···

On Sep 13, 2016, at 10:30 PM, Eric Wing via swift-corelibs-dev <swift-corelibs-dev@swift.org> wrote:
On 9/13/16, Will Stanton <willstanton1@yahoo.com> wrote:

I would appreciate any thoughts/help on what I should look at changing
next. And I would like to do this in a way that might be accepted back
into the main source tree.

Thanks,
Eric
_______________________________________________
swift-corelibs-dev mailing list
swift-corelibs-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-corelibs-dev


(Eric Wing) #5

I think a patch which fixed just this part would probably be fine.

Can you point me to where/how the build invocation is constructed? I
see how the foundation.LDFLAGS are set, but I don't understand how the
system takes that value and constructs the build command. (This is
where I need to reorder flags so they come last.)

In general we’re probably going to just have to specify a list of minimum
dependencies for Foundation. We don’t really have a great story for missing
features from ICU or libxml.

- Tony

So for XFERINFOFUNCTION, there is an older PROGRESSFUNCTION that can
be used. They give an example of how it works and how both of them can
be put into your code for legacy/migration concerns. With versions
that have XFERINFOFUNCTION, if both callbacks are set,
XFERINFOFUNCTION is always preferred.
https://curl.haxx.se/libcurl/c/progressfunc.html

I don't know anything about about xmlXPathNodeEval though.

Thanks,
Eric