stdlib tests under FreeBSD

Hi.
I've eventually tracked down all the problems that prevented swift
standard library and related tests to run correctly under FreeBSD but
one.When I run the testsuite as ./utils/build-script -R -t each single
test requiring SwiftGlibc fails because of missing headers.
e.g.

<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/usr/include/complex.h"
         ^
/usr/include/complex.h:32:10: error: 'sys/cdefs.h' file not found with
<angled> include; use "quotes" instead
#include <sys/cdefs.h>
         ^
<module-includes>:2:10: note: in file included from <module-includes>:2:
#include "/usr/include/ctype.h"
         ^

In a different thread Jordan suggested to pass -sdk / to see if that
fixed the problem, and indeed it did. Digging into the code I noticed
that the Driver checks for SDKROOT environment variable being set as
fallback if -sdk argument isn't passed on the cmdline.

If I set that environment variable and I run the tests standalone
(directly from cmdline), everything seems to be fine.

% /exps/swift/build/Ninja-ReleaseAssert/swift-freebsd-x86_64/bin/swiftc
-target x86_64-unknown-freebsd11.0-CURRENT -module-cache-path
'/tmp/swift-testsuite-clang-module-cache3VHSof'
/exps/swift/swift/test/1_stdlib/Builtins.swift -parse-stdlib
-Xfrontend -disable-access-control -Onone -o
/exps/swift/build/Ninja-ReleaseAssert/swift-freebsd-x86_64/test-freebsd-x86_64/1_stdlib/Output/Builtins.swift.tmp/Builtins
/exps/swift/swift/test/1_stdlib/Builtins.swift:113:3: warning: result
of call to 'unsafeUnwrap' is unused
  unsafeUnwrap(empty)
  ^
davide@rabbit1:/exps/swift/swift % echo $SDKROOT
/

But, if I run the whole testsuite, it fails (with the error above).
I'm no expert about swift build but it seems to me SDKROOT isn't
honored when the testsuite is invoked (maybe it doesn't share the
enviroment with the calling process?)

In any case, I'm interested in getting this fixed as it seems to be
the only missing bit to get the whole testsuite to run (and setup a
buildbot to catch regressions).

I'm not sure if it's better forcing the user to pass SDKROOT (and then
understanding why it's not honoured), or forcing -sdk / to be passed
to each test. I'm not sure what Linux does (and wasn't entirely
obvious to me where to look).
Any suggestion on how to handle this case is greatly appreciated.

Thanks,

···

--
Davide

Hi, Davide. On OS X, there are no headers in /, so it's pretty important that Swift doesn't try to use what's in /. (This was doubly true during the transition period where many people had stale headers in /.) It's also common to use swiftc to cross-compile on OS X, and we didn't want inscrutable errors about using OS X headers while compiling for iOS.

That said, on OS X either Xcode or the "xcrun" tool takes care of setting up SDKROOT appropriately. I'd be fine with it defaulting to / on FreeBSD (and Linux), like C compilers do. If you're interested in implementing this, I would put it into the Driver <https://github.com/apple/swift/blob/master/lib/Driver/Driver.cpp#L1080&gt;; we already do this for "swift" (the interpreter) on OS X.

The test suite deliberately blocks out all environment variables from your environment except for a select few, to minimize the chance that something is passing (or failing!) based solely on your configuration. For the test suite specifically, I would suggest adding "-sdk" to the %target-swift-frontend substitution the way we do on OS X.

Best,
Jordan

···

On Jan 8, 2016, at 15:12, Davide Italiano <dccitaliano@gmail.com> wrote:

Hi.
I've eventually tracked down all the problems that prevented swift
standard library and related tests to run correctly under FreeBSD but
one.When I run the testsuite as ./utils/build-script -R -t each single
test requiring SwiftGlibc fails because of missing headers.
e.g.

<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "/usr/include/complex.h"
        ^
/usr/include/complex.h:32:10: error: 'sys/cdefs.h' file not found with
<angled> include; use "quotes" instead
#include <sys/cdefs.h>
        ^
<module-includes>:2:10: note: in file included from <module-includes>:2:
#include "/usr/include/ctype.h"
        ^

In a different thread Jordan suggested to pass -sdk / to see if that
fixed the problem, and indeed it did. Digging into the code I noticed
that the Driver checks for SDKROOT environment variable being set as
fallback if -sdk argument isn't passed on the cmdline.

If I set that environment variable and I run the tests standalone
(directly from cmdline), everything seems to be fine.

% /exps/swift/build/Ninja-ReleaseAssert/swift-freebsd-x86_64/bin/swiftc
-target x86_64-unknown-freebsd11.0-CURRENT -module-cache-path
'/tmp/swift-testsuite-clang-module-cache3VHSof'
/exps/swift/swift/test/1_stdlib/Builtins.swift -parse-stdlib
-Xfrontend -disable-access-control -Onone -o
/exps/swift/build/Ninja-ReleaseAssert/swift-freebsd-x86_64/test-freebsd-x86_64/1_stdlib/Output/Builtins.swift.tmp/Builtins
/exps/swift/swift/test/1_stdlib/Builtins.swift:113:3: warning: result
of call to 'unsafeUnwrap' is unused
unsafeUnwrap(empty)
^
davide@rabbit1:/exps/swift/swift % echo $SDKROOT
/

But, if I run the whole testsuite, it fails (with the error above).
I'm no expert about swift build but it seems to me SDKROOT isn't
honored when the testsuite is invoked (maybe it doesn't share the
enviroment with the calling process?)

In any case, I'm interested in getting this fixed as it seems to be
the only missing bit to get the whole testsuite to run (and setup a
buildbot to catch regressions).

I'm not sure if it's better forcing the user to pass SDKROOT (and then
understanding why it's not honoured), or forcing -sdk / to be passed
to each test. I'm not sure what Linux does (and wasn't entirely
obvious to me where to look).
Any suggestion on how to handle this case is greatly appreciated.

Thanks,

--
Davide

I agree with your proposal. I'll implement the patches and submit pull
requests accordingly. Thank you for the time spent explaining me the
details/the rationale behind choices made.

Thanks,

···

On Fri, Jan 8, 2016 at 4:30 PM, Jordan Rose <jordan_rose@apple.com> wrote:

Hi, Davide. On OS X, there are no headers in /, so it's pretty important
that Swift doesn't try to use what's in /. (This was doubly true during the
transition period where many people had stale headers in /.) It's also
common to use swiftc to cross-compile on OS X, and we didn't want
inscrutable errors about using OS X headers while compiling for iOS.

That said, on OS X either Xcode or the "xcrun" tool takes care of setting up
SDKROOT appropriately. I'd be fine with it defaulting to / on FreeBSD (and
Linux), like C compilers do. If you're interested in implementing this, I
would put it into the Driver; we already do this for "swift" (the
interpreter) on OS X.

The test suite deliberately blocks out all environment variables from your
environment except for a select few, to minimize the chance that something
is passing (or failing!) based solely on your configuration. For the test
suite specifically, I would suggest adding "-sdk" to the
%target-swift-frontend substitution the way we do on OS X.

Best,
Jordan

--
Davide