Swift POSIX tests versus newer Linux distros

Development Compiler
1

So it looks like newer Linux distros cause the test suite to fail. What's the right way to fix this and not break older distros?

FAIL: Swift(linux-x86_64) :: stdlib/POSIX.swift (600 of 12185)
******************** TEST 'Swift(linux-x86_64) :: stdlib/POSIX.swift' FAILED ********************
Script:
--
: 'RUN: at line 1';   rm -rf "/home/dave/b/u/t/tools/swift/test-linux-x86_64/stdlib/Output/POSIX.swift.tmp" && mkdir -p "/home/dave/b/u/t/tools/swift/test-linux-x86_64/stdlib/Output/POSIX.swift.tmp" && /home/dave/b/u/t/bin/swiftc -target x86_64-unknown-linux-gnu -toolchain-stdlib-rpath  -module-cache-path '/home/dave/b/u/t/swift-test-results/x86_64-unknown-linux-gnu/clang-module-cache' -swift-version 4   -module-cache-path '/home/dave/b/u/t/swift-test-results/x86_64-unknown-linux-gnu/clang-module-cache' /home/dave/s/u/swift/test/stdlib/POSIX.swift -o /home/dave/b/u/t/tools/swift/test-linux-x86_64/stdlib/Output/POSIX.swift.tmp/a.out -module-name main  && echo /home/dave/b/u/t/tools/swift/test-linux-x86_64/stdlib/Output/POSIX.swift.tmp/a.out &&/usr/bin/env DYLD_LIBRARY_PATH='/home/dave/b/u/t/lib/swift/linux/x86_64' LD_LIBRARY_PATH='/home/dave/b/u/t/lib/swift/linux/x86_64:' SIMCTL_CHILD_DYLD_LIBRARY_PATH='/home/dave/b/u/t/lib/swift/linux/x86_64'  /home/dave/b/u/t/tools/swift/test-linux-x86_64/stdlib/Output/POSIX.swift.tmp/a.out /home/dave/b/u/t/tools/swift/test-linux-x86_64/stdlib/Output/POSIX.swift.tmp
--
Exit Code: 1

Command Output (stderr):
--
/home/dave/s/u/swift/test/stdlib/POSIX.swift:72:23: error: value of optional type 'UnsafeMutablePointer<sem_t>?' must be unwrapped to a value of type 'UnsafeMutablePointer<sem_t>'
  let res = sem_close(sem)
                      ^
/home/dave/s/u/swift/test/stdlib/POSIX.swift:72:23: note: coalesce using '??' to provide a default when the optional value contains 'nil'
  let res = sem_close(sem)
                      ^
                          ?? <#default value#>
/home/dave/s/u/swift/test/stdlib/POSIX.swift:72:23: note: force-unwrap using '!' to abort execution if the optional value contains 'nil'
  let res = sem_close(sem)
                      ^
                         !
/home/dave/s/u/swift/test/stdlib/POSIX.swift:86:23: error: value of optional type 'UnsafeMutablePointer<sem_t>?' must be unwrapped to a value of type 'UnsafeMutablePointer<sem_t>'
  let res = sem_close(sem)
                      ^
/home/dave/s/u/swift/test/stdlib/POSIX.swift:86:23: note: coalesce using '??' to provide a default when the optional value contains 'nil'
  let res = sem_close(sem)
                      ^
                          ?? <#default value#>
/home/dave/s/u/swift/test/stdlib/POSIX.swift:86:23: note: force-unwrap using '!' to abort execution if the optional value contains 'nil'
  let res = sem_close(sem)
                      ^
                         !
/home/dave/s/u/swift/test/stdlib/POSIX.swift:105:23: error: value of optional type 'UnsafeMutablePointer<sem_t>?' must be unwrapped to a value of type 'UnsafeMutablePointer<sem_t>'
  let res = sem_close(sem)
                      ^
/home/dave/s/u/swift/test/stdlib/POSIX.swift:105:23: note: coalesce using '??' to provide a default when the optional value contains 'nil'
  let res = sem_close(sem)
                      ^
                          ?? <#default value#>
/home/dave/s/u/swift/test/stdlib/POSIX.swift:105:23: note: force-unwrap using '!' to abort execution if the optional value contains 'nil'
  let res = sem_close(sem)
                      ^
                         !
/home/dave/s/u/swift/test/stdlib/POSIX.swift:123:23: error: value of optional type 'UnsafeMutablePointer<sem_t>?' must be unwrapped to a value of type 'UnsafeMutablePointer<sem_t>'
  let res = sem_close(sem)
                      ^
/home/dave/s/u/swift/test/stdlib/POSIX.swift:123:23: note: coalesce using '??' to provide a default when the optional value contains 'nil'
  let res = sem_close(sem)
                      ^
                          ?? <#default value#>
/home/dave/s/u/swift/test/stdlib/POSIX.swift:123:23: note: force-unwrap using '!' to abort execution if the optional value contains 'nil'
  let res = sem_close(sem)
                      ^
                         !
2

Is the sem_t type alias controlled by a macro for compatibility? We might be able to shove a -Xcc -D... to make things compatible if it is. Which distro/libc/version? It may be help to be able to look at the header to possible come up with an approach.

3

One less pretty way: create an overloaded unwrapping function. If it is already unwrapped, it just forwards the argument, but if it is an optional, it could unwrap it.

4

This seems like a Swift policy problem. In commit f492fc99f48e1a4fe99d9014f92b91fd3fc18fb0, glibc added non-null attributes to semaphore APIs, thus changing how the clang importer presents the APIs to Swift code.

5

Please forgive me for putting my "pedantic curmudgeon" hat on. I feel bad for anybody trying to write portable POSIX code via a language as ambitious as Swift. The problem is that POSIX often doesn't specify how types should be implemented. For example, one POSIX compliant platform could implement "something_t" as a pointer type (because it's implemented in user-space) and a second platform might use a C typedef to an integer (because it's implemented in the kernel and the integer is just a handle). These differences are tolerable for C code, but far less so for Swift code due to type optionality and how type extensions work.

6

POSIX is generally a minefield anyways. There are plenty of options in implementing POSIX compliance for the OS. It’s surprisingly variable as to what is implemented and how much of it is implemented. That’s certainly a huge challenge for people to write portable code. I would generally be in favor of reducing the content of the POSIX module.

Removing the semaphore surface IMO is perfectly legitimate. If someone wishes to use a semaphore, I would say that we want documentation instead. They could/should be redirected to DispatchSemaphore. Now, I realize that many people may have an immediate reaction of that being too draconian. While I would agree with the assessment, the fact is that dispatch is required more or less as that is the only reasonable threading approach which is currently viable with Swift. It is also something which is already ported to all the supported environments under active development. I don’t think it is too unreasonable to request developers to adjust some of their code. It would provide the necessary semantics with a better supported interface which yields a better experience and easier porting of the code.

The biggest downside to this I think is that it adds an additional dependency on the test suite. Currently, the toolchain build does not build the Swift overlay for libdispatch, but I think that we should consider changing that. This would be somewhat challenging, but it would improve things too I think. It would allow for a more uniform test suite execution across platforms as currently the test suite uses the system provided libdispatch for Darwin and disables the tests on other platforms. We could build the Swift overlay and then use the build uniformly across all platforms, enabling the test coverage everywhere.