importer questions

Development Compiler
1

I'm porting a codebase to Linux. As part of this work, I have a large class of problems of the form "Darwin defines this constant to be Int32 but Glibc defines it to be Int16".

Are these considered bugs, from Swift POV? They mean I have to `#if os(Linux)` a lot of things, but one interpretation is that Darwin and Glibc headers are just different, get over it.
If they are bugs, should they be resolved in favor of making Glibc follow the Darwin types?
If they are bugs, perhaps someone could link me to the place where I can override the header import, so I can fix these directly? Filing them separately would probably be inefficient, as I have a codebase that can identify them at some scale.

Thanks

Drew

2

I'm porting a codebase to Linux. As part of this work, I have a large class
of problems of the form "Darwin defines this constant to be Int32 but Glibc
defines it to be Int16".

Are these considered bugs, from Swift POV?

It depends on the API. Could you give some examples?

If they are bugs, should they be resolved in favor of making Glibc follow
the Darwin types?

Blanket rules like that don't make sense to me, since the correct
import could be neither Int32, nor Int16, but a plain Int. And there
are other cases that we won't even imagine before we have a list of
problematic APIs in front of us.

If they are bugs, perhaps someone could link me to the place where I can
override the header import, so I can fix these directly?

So far, we have been fixing suboptimal API imports in the overlays.

Dmitri

···

On Mon, Feb 15, 2016 at 9:22 PM, Drew Crawford via swift-dev <swift-dev@swift.org> wrote:

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/

3

Here are some that I #ifdefed on Linux just today:

SOCK_STREAM
IPPROTO_TCP
timeval. This is a tricky case, as there are actually 2 types on Darwin (timeval / timeval64), that map to 1 (as far as I know) on Linux.
timeval[64] members, which are in some platforms Int64, in some platforms __darwin_xx and in other platforms Int
SHUT_RDWR
POLLIN etc. is wrong IMO even on Darwin. I opened SR-178 for this (and previously I filed a radar), but since that bug has sat for 2 months (and the radar for longer) without any comment it motivated me to investigate the DIY approach to these simple "importer bugs", as perhaps the core team is too busy.

Obviously I don't expect 100% portability (these are different OSes afterall), but there are several cases where the type differences feel more by accident than by design.

I will probably have a new list in a few days if interested, as I am still battling Linux-specific compile errors :-)

···

On Feb 16, 2016, at 2:06 AM, Dmitri Gribenko <gribozavr@gmail.com> wrote:

Could you give some examples?

4

Could you give some examples?

Here are some that I #ifdefed on Linux just today:

Thank you for the list!

Unfortunately, it is not clear what is wrong about the import of these
APIs. Could you elaborate?

Dmitri

···

On Tue, Feb 16, 2016 at 12:36 AM, Drew Crawford <drew@sealedabstract.com> wrote:

On Feb 16, 2016, at 2:06 AM, Dmitri Gribenko <gribozavr@gmail.com> wrote:

SOCK_STREAM
IPPROTO_TCP
timeval. This is a tricky case, as there are actually 2 types on Darwin
(timeval / timeval64), that map to 1 (as far as I know) on Linux.
timeval[64] members, which are in some platforms Int64, in some platforms
__darwin_xx and in other platforms Int
SHUT_RDWR
POLLIN etc. is wrong IMO even on Darwin. I opened SR-178 for this (and
previously I filed a radar), but since that bug has sat for 2 months (and
the radar for longer) without any comment it motivated me to investigate the
DIY approach to these simple "importer bugs", as perhaps the core team is
too busy.

Obviously I don't expect 100% portability (these are different OSes
afterall), but there are several cases where the type differences feel more
by accident than by design.

I will probably have a new list in a few days if interested, as I am still
battling Linux-specific compile errors :-)

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/

5

Sorry, I wasn't sure how detailed to be.

BTW, I have not determined how to print the interface for e.g. Glibc, if you know how to do that, it might help gather more items.

1. SOCK_STREAM is a __socket_type struct on Glibc, with an inner member rawValue : UInt32. On Darwin it is Int32 without any wrapper struct.
     a. However APIs where one would use this value (such as socket() ) take Int32 in both cases, this requires applications to do a member access + cast on Linux that is not necessary or allowed on Darwin
2. IPPROTO_TCP is Int on Glibc, while Int32 on Darwin
    a. APIs (e.g. socket() ) expect Int32 for this argument in practice
3. timeval is a royal pain.
     a. Darwin defines both timeval and timeval64, but at runtime, Darwin generally expects timeval64 on 64-bit platforms and timeval on 32-bit platforms
    b. Linux seems only to define timeval, although I have not tried 32-bit Linux so I cannot speak to that case
4. timeval's members are (e.g. to construct one, you need to use)
    a. Int64 if using timeval64/Darwin64,
    b. __darwin_time_t and __darwin_suseconds_t if using 32-bit Darwin,
    c. and Int if on Linux
5. SHUT_RDWR is Int on Linux and Int32 on Darwin,
    a. whereas function parameters (such as shutdown() ) expect Int32 in both cases
6. The issues with POLLIN etc. I have documented in some detail on SR-178

Please let me know if I can provide more information.

···

On Feb 16, 2016, at 2:42 AM, Dmitri Gribenko <gribozavr@gmail.com> wrote:

On Tue, Feb 16, 2016 at 12:36 AM, Drew Crawford <drew@sealedabstract.com> wrote:

On Feb 16, 2016, at 2:06 AM, Dmitri Gribenko <gribozavr@gmail.com> wrote:

Could you give some examples?

Here are some that I #ifdefed on Linux just today:

Thank you for the list!

Unfortunately, it is not clear what is wrong about the import of these
APIs. Could you elaborate?

Dmitri

SOCK_STREAM
IPPROTO_TCP
timeval. This is a tricky case, as there are actually 2 types on Darwin
(timeval / timeval64), that map to 1 (as far as I know) on Linux.
timeval[64] members, which are in some platforms Int64, in some platforms
__darwin_xx and in other platforms Int
SHUT_RDWR
POLLIN etc. is wrong IMO even on Darwin. I opened SR-178 for this (and
previously I filed a radar), but since that bug has sat for 2 months (and
the radar for longer) without any comment it motivated me to investigate the
DIY approach to these simple "importer bugs", as perhaps the core team is
too busy.

Obviously I don't expect 100% portability (these are different OSes
afterall), but there are several cases where the type differences feel more
by accident than by design.

I will probably have a new list in a few days if interested, as I am still
battling Linux-specific compile errors :-)

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/

6

Sorry, I wasn't sure how detailed to be.

BTW, I have not determined how to print the interface for e.g. Glibc, if you know how to do that, it might help gather more items.

1. SOCK_STREAM is a __socket_type struct on Glibc, with an inner member rawValue : UInt32. On Darwin it is Int32 without any wrapper struct.
     a. However APIs where one would use this value (such as socket() ) take Int32 in both cases, this requires applications to do a member access + cast on Linux that is not necessary or allowed on Darwin

That's because __socket_type is an enum. This issue can be easily
fixed with an addition to the Glibc overlay that defines these
constants and shadows imported ones.

2. IPPROTO_TCP is Int on Glibc, while Int32 on Darwin
    a. APIs (e.g. socket() ) expect Int32 for this argument in practice

Again, I think overlay would fix this.

3. timeval is a royal pain.
     a. Darwin defines both timeval and timeval64, but at runtime, Darwin generally expects timeval64 on 64-bit platforms and timeval on 32-bit platforms
    b. Linux seems only to define timeval, although I have not tried 32-bit Linux so I cannot speak to that case

typealias timeval = timeval64 in the Darwin overlay?

4. timeval's members are (e.g. to construct one, you need to use)
    a. Int64 if using timeval64/Darwin64,
    b. __darwin_time_t and __darwin_suseconds_t if using 32-bit Darwin,
    c. and Int if on Linux

The overlay should add a sane initializer, timeval(tv_sec: tv_usec:),
on both platforms.

5. SHUT_RDWR is Int on Linux and Int32 on Darwin,
    a. whereas function parameters (such as shutdown() ) expect Int32 in both cases

Another overlay fix.

6. The issues with POLLIN etc. I have documented in some detail on SR-178

I think this would be another overlay fix.

By the way, I think everywhere where you mention Int32, you actually
meant 'CInt'.

Another approach that was considered, and sounds promising, is to
import more of C's constants as untyped integers, so that they would
infer the type from context. Unfortunately, this feature requires
design and compiler work.

Overlay fixes, I think, don't require special skills to prepare.
Here's an example: Added an an overlay for IOKit that adds the kIOReturnError* constants by mattrajca · Pull Request #1096 · apple/swift · GitHub

Dmitri

···

On Tue, Feb 16, 2016 at 1:04 AM, Drew Crawford <drew@sealedabstract.com> wrote:

--
main(i,j){for(i=2;;i++){for(j=2;j<i;j++){if(!(i%j)){j=0;break;}}if
(j){printf("%d\n",i);}}} /*Dmitri Gribenko <gribozavr@gmail.com>*/

7

Sorry; I tried to ask for this at the outset, but I think I used the wrong words :-)

···

On Feb 16, 2016, at 3:20 AM, Dmitri Gribenko <gribozavr@gmail.com> wrote:

Overlay fixes, I think, don't require special skills to prepare.
Here's an example: Added an an overlay for IOKit that adds the kIOReturnError* constants by mattrajca · Pull Request #1096 · apple/swift · GitHub

8

Another approach that was considered, and sounds promising, is to
import more of C's constants as untyped integers, so that they would
infer the type from context. Unfortunately, this feature requires
design and compiler work.

Dmitri

Yes, please! That would improve my life so much.

- Will