[RFC] Finer grained OS checks


(Saleem Abdulrasool) #1

Hi,

Id like to revive the discussion around OS "variants". I've been doing
some work to bring up Windows without any emulation layer (MSVCRT based) as
a viable host environment. This work is bringing to light the need for
more finer grained OS checks.

Currently, we have the `os` compilation condition. However, this doesn't
provide sufficiently detailed information for Windows. On Windows, we have
at least 4 different "variants":

- "msvc" (Microsoft's environment)
- "itanium" (MS ABI for C, Itanium ABI for C++)
- "gnu" (MinGW)
- "cygnus" (cygwin)

Each one of these is slightly different and requires particular handling in
the runtime. However, the OS for each one of these is windows, and so
`os(Windows)` yields true on all of them.

This is not a problem strictly limited to Windows. It also appears in
other OSes. As a concrete example, Linux has traditionally had the "gnu"
environment (libc). However, there is also "uclibc" which is pretty
common, and these days, "musl" as different targets. Swift also supports
android, which is a Linux environment variant.

As deeper system integration occurs with swift, the need for finer grained
os detection logic is probably going to be needed.

To keep things simple, I would like to propose the "environment"
conditional compilation directive. It would yield true for the appropriate
environment with one of the previously mentioned values for windows and
true for "android" on Android. It would sit as a peer to `os` and allow
for finer grained querying of the host environment.

···

--
Saleem Abdulrasool
compnerd (at) compnerd (dot) org


(Jordan Rose) #2

Thanks for sending this out, Saleem!

I’m not convinced that gnu/uclibc/musl are environment variants worth testing for, nor do I think we actually want to model Android as a kind of Linux. It’s unclear whether “environment” is another set of mutually-exclusive options (enum-like) or a way to check several things that might all be true (option-set-like). For example, is “itanium” really distinct from all the others, or can you have itanium-gnu?

I imagine some people might like to distinguish Linux distros as well, but since those follow the “like Gecko <https://www.nczonline.net/blog/2010/01/12/history-of-the-user-agent-string/>” policy, I’m not sure we’d actually want to encourage people to use such a thing. On the other hand, we could easily have ‘environment(simulator)’ for the Apple device OSs.

It’s also not clear whether we always want ‘environment' to be a subset of ‘os', i.e. it’s a mistake to test for “simulator” before checking for an OS because it might mean different things on different platforms. (iOS simulator or Android simulator?)

I’ll defer to all of you as to whether “gnu” and “cygnus” are the right names for MinGW and Cygwin. I kind of thought Cygnus was named as such because it was also a GNU environment.

A related use is something that’s above `os(…)`, so that we can have all the Apple platforms grouped together. That could be “environment”, or it could be something different.

Finally, in the spirit of “question everything”, is “environment” the right name for this setting? :slight_smile:

The goal of this thread should be for you/us to converge on a design, and then for someone to write it up and take to swift-evolution.

Jordan

···

On Jul 7, 2016, at 18:17, Saleem Abdulrasool <compnerd@compnerd.org> wrote:

Hi,

Id like to revive the discussion around OS "variants". I've been doing some work to bring up Windows without any emulation layer (MSVCRT based) as a viable host environment. This work is bringing to light the need for more finer grained OS checks.

Currently, we have the `os` compilation condition. However, this doesn't provide sufficiently detailed information for Windows. On Windows, we have at least 4 different "variants":

- "msvc" (Microsoft's environment)
- "itanium" (MS ABI for C, Itanium ABI for C++)
- "gnu" (MinGW)
- "cygnus" (cygwin)

Each one of these is slightly different and requires particular handling in the runtime. However, the OS for each one of these is windows, and so `os(Windows)` yields true on all of them.

This is not a problem strictly limited to Windows. It also appears in other OSes. As a concrete example, Linux has traditionally had the "gnu" environment (libc). However, there is also "uclibc" which is pretty common, and these days, "musl" as different targets. Swift also supports android, which is a Linux environment variant.

As deeper system integration occurs with swift, the need for finer grained os detection logic is probably going to be needed.

To keep things simple, I would like to propose the "environment" conditional compilation directive. It would yield true for the appropriate environment with one of the previously mentioned values for windows and true for "android" on Android. It would sit as a peer to `os` and allow for finer grained querying of the host environment.

--
Saleem Abdulrasool
compnerd (at) compnerd (dot) org


(Karl) #3

My thoughts:
I don't like the idea of additional qualifiers after OS. Perhaps we could rename it something more generic like "SDK" or split the non-common stuff in to a seprarate module?
As for the proposal (I know it's too late for Swift 3, but I read it so I might as well say what i thought about it):- I don't like "vendor"; it's too vague to give useful guarantees. Even the Apple platforms can be wildly different. I would prefer a group SDK target instead, like "mobileDarwin", or even something better than that (the "canImport" idea is even better - so you can test for the presence of frameworks such as UIKit directly, and the compiler will/should use @availability attributes to ensure it is safe for all of your deployment targets)- Also don't like the simulator condition variable. The iOS simulator is literally x86 iOS. If there was an x86 iPhone, theoretically your binaries would be compatible. The fact that it runs on a simulator instead of a real device is not such a vital distinction (or shouldn't be) that we need integrate it in the language. What would we do in the future if there ever was a real x86 iOS target?
Sorry to give a list of stuff I don't like, but it's easier that way because the rest of it is good. Endianness, word-size and Interop availability are useful things to know and really are compile-time options, and as I said canImport is also a very good idea.
Karl

···

On Fri, Jul 8, 2016 at 3:17 AM +0200, "Saleem Abdulrasool via swift-dev" <swift-dev@swift.org> wrote:

Hi,
Id like to revive the discussion around OS "variants". I've been doing some work to bring up Windows without any emulation layer (MSVCRT based) as a viable host environment. This work is bringing to light the need for more finer grained OS checks.
Currently, we have the `os` compilation condition. However, this doesn't provide sufficiently detailed information for Windows. On Windows, we have at least 4 different "variants":
- "msvc" (Microsoft's environment)- "itanium" (MS ABI for C, Itanium ABI for C++)- "gnu" (MinGW)- "cygnus" (cygwin)
Each one of these is slightly different and requires particular handling in the runtime. However, the OS for each one of these is windows, and so `os(Windows)` yields true on all of them.
This is not a problem strictly limited to Windows. It also appears in other OSes. As a concrete example, Linux has traditionally had the "gnu" environment (libc). However, there is also "uclibc" which is pretty common, and these days, "musl" as different targets. Swift also supports android, which is a Linux environment variant.

As deeper system integration occurs with swift, the need for finer grained os detection logic is probably going to be needed.
To keep things simple, I would like to propose the "environment" conditional compilation directive. It would yield true for the appropriate environment with one of the previously mentioned values for windows and true for "android" on Android. It would sit as a peer to `os` and allow for finer grained querying of the host environment.
--
Saleem Abdulrasool
compnerd (at) compnerd (dot) org


(William Dillon) #4

Hi all,

I Agree with Saleem that this is a worthwhile discussion, and I think I agree with Jordan that word environment seems a little bit wrong to me.

Following the discussion, it feels to me like what we're really concerned about is libc and ABI. Most of the examples that Saleem gave seems to have that in common. Jordan's example about environment(simulator) feels like a more natural fit (i.e. using it as a way to talk about the physical/virtual environment). I think that libc(gnu), etc., would be perhaps workable, but doesn't express the differences in ABI for the Itanium example.

I do like Erica's contribution of the Evolution proposal from a bit ago. Adding libc to it with a combined check for arch should get is substantially there.

I know that I'm rambling bit; my thoughts on the subject are not fully formed. I haven't been bitten by this particular issue yet, so it's of academic interest so far.

Thanks to all for their work,
- Will

···

On Jul 7, 2016, at 7:19 PM, Jordan Rose <jordan_rose@apple.com> wrote:

Thanks for sending this out, Saleem!

I’m not convinced that gnu/uclibc/musl are environment variants worth testing for, nor do I think we actually want to model Android as a kind of Linux. It’s unclear whether “environment” is another set of mutually-exclusive options (enum-like) or a way to check several things that might all be true (option-set-like). For example, is “itanium” really distinct from all the others, or can you have itanium-gnu?

I imagine some people might like to distinguish Linux distros as well, but since those follow the “like Gecko <https://www.nczonline.net/blog/2010/01/12/history-of-the-user-agent-string/>” policy, I’m not sure we’d actually want to encourage people to use such a thing. On the other hand, we could easily have ‘environment(simulator)’ for the Apple device OSs.

It’s also not clear whether we always want ‘environment' to be a subset of ‘os', i.e. it’s a mistake to test for “simulator” before checking for an OS because it might mean different things on different platforms. (iOS simulator or Android simulator?)

I’ll defer to all of you as to whether “gnu” and “cygnus” are the right names for MinGW and Cygwin. I kind of thought Cygnus was named as such because it was also a GNU environment.

A related use is something that’s above `os(…)`, so that we can have all the Apple platforms grouped together. That could be “environment”, or it could be something different.

Finally, in the spirit of “question everything”, is “environment” the right name for this setting? :slight_smile:

The goal of this thread should be for you/us to converge on a design, and then for someone to write it up and take to swift-evolution.

Jordan

On Jul 7, 2016, at 18:17, Saleem Abdulrasool <compnerd@compnerd.org <mailto:compnerd@compnerd.org>> wrote:

Hi,

Id like to revive the discussion around OS "variants". I've been doing some work to bring up Windows without any emulation layer (MSVCRT based) as a viable host environment. This work is bringing to light the need for more finer grained OS checks.

Currently, we have the `os` compilation condition. However, this doesn't provide sufficiently detailed information for Windows. On Windows, we have at least 4 different "variants":

- "msvc" (Microsoft's environment)
- "itanium" (MS ABI for C, Itanium ABI for C++)
- "gnu" (MinGW)
- "cygnus" (cygwin)

Each one of these is slightly different and requires particular handling in the runtime. However, the OS for each one of these is windows, and so `os(Windows)` yields true on all of them.

This is not a problem strictly limited to Windows. It also appears in other OSes. As a concrete example, Linux has traditionally had the "gnu" environment (libc). However, there is also "uclibc" which is pretty common, and these days, "musl" as different targets. Swift also supports android, which is a Linux environment variant.

As deeper system integration occurs with swift, the need for finer grained os detection logic is probably going to be needed.

To keep things simple, I would like to propose the "environment" conditional compilation directive. It would yield true for the appropriate environment with one of the previously mentioned values for windows and true for "android" on Android. It would sit as a peer to `os` and allow for finer grained querying of the host environment.

--
Saleem Abdulrasool
compnerd (at) compnerd (dot) org


(Erica Sadun) #5

Just want to throw this into the discussion:

* https://github.com/apple/swift-evolution/pull/369

Also discussions on -evolution, which do touch on "Apple-like", etc.

* http://thread.gmane.org/gmane.comp.lang.swift.evolution/7516
* http://thread.gmane.org/gmane.comp.lang.swift.evolution/12065

-- E

···

On Jul 7, 2016, at 8:19 PM, Jordan Rose via swift-dev <swift-dev@swift.org> wrote:

Thanks for sending this out, Saleem!

I’m not convinced that gnu/uclibc/musl are environment variants worth testing for, nor do I think we actually want to model Android as a kind of Linux. It’s unclear whether “environment” is another set of mutually-exclusive options (enum-like) or a way to check several things that might all be true (option-set-like). For example, is “itanium” really distinct from all the others, or can you have itanium-gnu?

I imagine some people might like to distinguish Linux distros as well, but since those follow the “like Gecko <https://www.nczonline.net/blog/2010/01/12/history-of-the-user-agent-string/>” policy, I’m not sure we’d actually want to encourage people to use such a thing. On the other hand, we could easily have ‘environment(simulator)’ for the Apple device OSs.

It’s also not clear whether we always want ‘environment' to be a subset of ‘os', i.e. it’s a mistake to test for “simulator” before checking for an OS because it might mean different things on different platforms. (iOS simulator or Android simulator?)

I’ll defer to all of you as to whether “gnu” and “cygnus” are the right names for MinGW and Cygwin. I kind of thought Cygnus was named as such because it was also a GNU environment.

A related use is something that’s above `os(…)`, so that we can have all the Apple platforms grouped together. That could be “environment”, or it could be something different.

Finally, in the spirit of “question everything”, is “environment” the right name for this setting? :slight_smile:

The goal of this thread should be for you/us to converge on a design, and then for someone to write it up and take to swift-evolution.

Jordan

On Jul 7, 2016, at 18:17, Saleem Abdulrasool <compnerd@compnerd.org <mailto:compnerd@compnerd.org>> wrote:

Hi,

Id like to revive the discussion around OS "variants". I've been doing some work to bring up Windows without any emulation layer (MSVCRT based) as a viable host environment. This work is bringing to light the need for more finer grained OS checks.

Currently, we have the `os` compilation condition. However, this doesn't provide sufficiently detailed information for Windows. On Windows, we have at least 4 different "variants":

- "msvc" (Microsoft's environment)
- "itanium" (MS ABI for C, Itanium ABI for C++)
- "gnu" (MinGW)
- "cygnus" (cygwin)

Each one of these is slightly different and requires particular handling in the runtime. However, the OS for each one of these is windows, and so `os(Windows)` yields true on all of them.

This is not a problem strictly limited to Windows. It also appears in other OSes. As a concrete example, Linux has traditionally had the "gnu" environment (libc). However, there is also "uclibc" which is pretty common, and these days, "musl" as different targets. Swift also supports android, which is a Linux environment variant.

As deeper system integration occurs with swift, the need for finer grained os detection logic is probably going to be needed.

To keep things simple, I would like to propose the "environment" conditional compilation directive. It would yield true for the appropriate environment with one of the previously mentioned values for windows and true for "android" on Android. It would sit as a peer to `os` and allow for finer grained querying of the host environment.

--
Saleem Abdulrasool
compnerd (at) compnerd (dot) org

_______________________________________________
swift-dev mailing list
swift-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-dev


(Brent Royal-Gordon) #6

Also, is "environment" a name we want to take for this particular feature? "Environment" is a broad and general word, and this proposal uses it in a fairly narrow and esoteric context.

···

On Jul 7, 2016, at 7:19 PM, Jordan Rose via swift-dev <swift-dev@swift.org> wrote:

Finally, in the spirit of “question everything”, is “environment” the right name for this setting? :slight_smile:

--
Brent Royal-Gordon
Architechies


(Erica Sadun) #7

`canImport` is already adopted. Feel free to implement:

https://github.com/apple/swift-evolution/blob/master/proposals/0075-import-test.md

-- E

···

On Jul 11, 2016, at 10:50 AM, Karl Wagner via swift-dev <swift-dev@swift.org> wrote:

My thoughts:

I don't like the idea of additional qualifiers after OS. Perhaps we could rename it something more generic like "SDK" or split the non-common stuff in to a seprarate module?

As for the proposal (I know it's too late for Swift 3, but I read it so I might as well say what i thought about it):
- I don't like "vendor"; it's too vague to give useful guarantees. Even the Apple platforms can be wildly different. I would prefer a group SDK target instead, like "mobileDarwin", or even something better than that (the "canImport" idea is even better - so you can test for the presence of frameworks such as UIKit directly, and the compiler will/should use @availability attributes to ensure it is safe for all of your deployment targets)
- Also don't like the simulator condition variable. The iOS simulator is literally x86 iOS. If there was an x86 iPhone, theoretically your binaries would be compatible. The fact that it runs on a simulator instead of a real device is not such a vital distinction (or shouldn't be) that we need integrate it in the language. What would we do in the future if there ever was a real x86 iOS target?

Sorry to give a list of stuff I don't like, but it's easier that way because the rest of it is good. Endianness, word-size and Interop availability are useful things to know and really are compile-time options, and as I said canImport is also a very good idea.

Karl


(Greg Parker) #8

The iOS simulator is not literally x86 iOS. It has changed ABI in incompatible ways in the past and reserves the right to do so in the future. Any real x86 iOS would have a real ABI which would likely differ from today's simulator.

···

On Jul 11, 2016, at 9:50 AM, Karl Wagner via swift-dev <swift-dev@swift.org> wrote:

- Also don't like the simulator condition variable. The iOS simulator is literally x86 iOS. If there was an x86 iPhone, theoretically your binaries would be compatible. The fact that it runs on a simulator instead of a real device is not such a vital distinction (or shouldn't be) that we need integrate it in the language. What would we do in the future if there ever was a real x86 iOS target?

--
Greg Parker gparker@apple.com <mailto:gparker@apple.com> Runtime Wrangler


(Karl) #9

I remember somebody telling me it was, but it was years ago and I'm probably remembering it wrong. Fair enough though; I got told on that one :no_mouth:
  
I'm standing by the principle - it shouldn't matter if you're running in a simulator or not. Use a compile flag if you must know, but in general I disagree with a compiler flag for determining the runtime platform for two platforms with the same API and triple.
  
Karl

···

  
On Jul 11, 2016 at 11:26 PM, <Greg Parker (mailto:gparker@apple.com)> wrote:
  
>
> On Jul 11, 2016, at 9:50 AM, Karl Wagner via swift-dev <swift-dev@swift.org (mailto:swift-dev@swift.org)> wrote:
>
>
>
>
> - Also don't like the simulator condition variable. The iOS simulator is literally x86 iOS. If there was an x86 iPhone, theoretically your binaries would be compatible. The fact that it runs on a simulator instead of a real device is not such a vital distinction (or shouldn't be) that we need integrate it in the language. What would we do in the future if there ever was a real x86 iOS target?
>
>
>
  
The iOS simulator is not literally x86 iOS. It has changed ABI in incompatible ways in the past and reserves the right to do so in the future. Any real x86 iOS would have a real ABI which would likely differ from today's simulator.
  
--
  
Greg Parker gparker@apple.com (mailto:gparker@apple.com) Runtime Wrangler
  


(Saleem Abdulrasool) #10

Well, the name that I went with was environment was derived from the fact
that it was based on the "environment" component of the "triple". That
said, an alternative idea is to make a more invasive change:

`host(...)`: where we can actually do a more target specific component
matching. Something like:

#if host(Windows, msvc)
#elseif host(Linux, musl)
#elseif host(iOS, ARM)
#endif

···

On Mon, Jul 11, 2016 at 2:33 PM, Karl Wagner <razielim@gmail.com> wrote:

I remember somebody telling me it was, but it was years ago and I'm
probably remembering it wrong. Fair enough though; I got told on that one
:no_mouth:

I'm standing by the principle - it shouldn't matter if you're running in a
simulator or not. Use a compile flag if you must know, but in general I
disagree with a compiler flag for determining the runtime platform for two
platforms with the same API and triple.

Karl

On Jul 11, 2016 at 11:26 PM, <Greg Parker <gparker@apple.com>> wrote:

On Jul 11, 2016, at 9:50 AM, Karl Wagner via swift-dev < > swift-dev@swift.org> wrote:

- Also don't like the simulator condition variable. The iOS simulator is
literally x86 iOS. If there was an x86 iPhone, theoretically your binaries
would be compatible. The fact that it runs on a simulator instead of a real
device is not such a vital distinction (or shouldn't be) that we need
integrate it in the language. What would we do in the future if there ever
was a real x86 iOS target?

The iOS simulator is not literally x86 iOS. It has changed ABI in
incompatible ways in the past and reserves the right to do so in the
future. Any real x86 iOS would have a real ABI which would likely differ
from today's simulator.

--
Greg Parker gparker@apple.com Runtime Wrangler

--
Saleem Abdulrasool
compnerd (at) compnerd (dot) org