pitch: Unified libc import (again)

I know this has come up before without any action, but having the standard
C library be packaged under `Darwin` on OSX and `Glibc` under Linux is
something that’s really becoming an issue as the Swift package ecosystem
matures. Right now a lot of packages are a lot less portable than they
could be because somewhere along the dependency line, there is a package
that only imports the C library for one platform. Unifying it under one
import would allow people to write packages that are portable by default.

Since I think this got hung up in the past over “what constitutes” a
universal libc, I propose a unified package should just consist of the
functionality that is common between Darwin and Glibc right now, since
those are the only two supported platforms anyway.

Alternatively, Swift could make it a priority to get basic functionality
like file IO and math functions shipped with the compiler, which would
probably cover 98% of cases where people currently import Darwin/Glibc. A
large portion of the standard C libraries are redundant to the Swift
standard library anyway.

I know this has come up before without any action, but having the standard C library be packaged under `Darwin` on OSX and `Glibc` under Linux is something that’s really becoming an issue as the Swift package ecosystem matures. Right now a lot of packages are a lot less portable than they could be because somewhere along the dependency line, there is a package that only imports the C library for one platform. Unifying it under one import would allow people to write packages that are portable by default.

What we (SwiftPM) have done for now is use a `libc` target to start by normalizing the name:
  https://github.com/apple/swift-package-manager/tree/master/Sources/libc
(and in the past, when we find missing things in Glibc getting them added to the compiler). Also, this name is technically a misnomer, but we couldn’t think of a better one (“os” might have been a good one).

Unfortunately, I think this change alone is really only the tip of the iceberg. It would be nice to not have it the difference, but we found we very quickly needed several other layers on top to get to having a relatively stable cross-platform base:
  https://github.com/apple/swift-package-manager/tree/master/Sources/POSIX
  https://github.com/apple/swift-package-manager/tree/master/Sources/Basic

My hope is that one minimal improvement we can get soon is multi-package repo support in SwiftPM, which will at least allow us to share those targets & functionality with other packages.

Since I think this got hung up in the past over “what constitutes” a universal libc, I propose a unified package should just consist of the functionality that is common between Darwin and Glibc right now, since those are the only two supported platforms anyway.

What would the concrete proposal be? It isn’t trivial to determine that subset and make it well-defined what the exact API is. Is the proposal to just to pick a standard name, and reexport Darwin and Glibc from it?

Alternatively, Swift could make it a priority to get basic functionality like file IO and math functions shipped with the compiler, which would probably cover 98% of cases where people currently import Darwin/Glibc. A large portion of the standard C libraries are redundant to the Swift standard library anyway.

I’m not sure I agree with these statements about the percentages. For some clients (I’m biased, the areas I work in tend to be in this boat), what we largely need is good platform-agnostic access to the POSIX APIs. This is a hard problem to make a good cross-platform API for (esp. if Windows support is in your head), and which many projects struggle with (Netscape :: NSPR, LLVM :: libSupport, many many more).

The sticking point I see is this: if the proposal is just to unify the name & that doesn’t actually buy us all that much (still need standard layers on top), then have we really solved enough of a problem to be worth standardizing on?

+1 in general agreement with the meta-issue being an important one to discuss.

- Daniel

···

On Aug 17, 2017, at 9:26 AM, Taylor Swift via swift-evolution <swift-evolution@swift.org> wrote:

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

>
> I know this has come up before without any action, but having the
standard C library be packaged under `Darwin` on OSX and `Glibc` under
Linux is something that’s really becoming an issue as the Swift package
ecosystem matures. Right now a lot of packages are a lot less portable than
they could be because somewhere along the dependency line, there is a
package that only imports the C library for one platform. Unifying it under
one import would allow people to write packages that are portable by
default.

What we (SwiftPM) have done for now is use a `libc` target to start by
normalizing the name:
  https://github.com/apple/swift-package-manager/tree/master/Sources/libc
(and in the past, when we find missing things in Glibc getting them added
to the compiler). Also, this name is technically a misnomer, but we
couldn’t think of a better one (“os” might have been a good one).

Unfortunately, I think this change alone is really only the tip of the
iceberg. It would be nice to not have it the difference, but we found we
very quickly needed several other layers on top to get to having a
relatively stable cross-platform base:
  https://github.com/apple/swift-package-manager/tree/master/Sources/POSIX
  https://github.com/apple/swift-package-manager/tree/master/Sources/Basic

My hope is that one minimal improvement we can get soon is multi-package
repo support in SwiftPM, which will at least allow us to share those
targets & functionality with other packages.

> Since I think this got hung up in the past over “what constitutes” a
universal libc, I propose a unified package should just consist of the
functionality that is common between Darwin and Glibc right now, since
those are the only two supported platforms anyway.

What would the concrete proposal be? It isn’t trivial to determine that
subset and make it well-defined what the exact API is. Is the proposal to
just to pick a standard name, and reexport Darwin and Glibc from it?

I don’t know if it’s actually this simple, but could it just be the symbols
that are defined in both modules?

> Alternatively, Swift could make it a priority to get basic functionality
like file IO and math functions shipped with the compiler, which would
probably cover 98% of cases where people currently import Darwin/Glibc. A
large portion of the standard C libraries are redundant to the Swift
standard library anyway.

I’m not sure I agree with these statements about the percentages. For some
clients (I’m biased, the areas I work in tend to be in this boat), what we
largely need is good platform-agnostic access to the POSIX APIs. This is a
hard problem to make a good cross-platform API for (esp. if Windows support
is in your head), and which many projects struggle with (Netscape :: NSPR,
LLVM :: libSupport, many many more).

The sticking point I see is this: if the proposal is just to unify the
name & that doesn’t actually buy us all that much (still need standard
layers on top), then have we really solved enough of a problem to be worth
standardizing on?

+1 in general agreement with the meta-issue being an important one to
discuss.

There probably is an XY issue at play here; what we *really* need is a way
to access the file system built into the standard library. (Math functions
are a separate, beleaguered topic for a different thread.) Having support
for outputting to `stderr` is also something I’d really like. Going through
Glibc/Darwin is just one way to solve this.

···

On Thu, Aug 17, 2017 at 3:50 PM, Daniel Dunbar <daniel_dunbar@apple.com> wrote:

> On Aug 17, 2017, at 9:26 AM, Taylor Swift via swift-evolution < > swift-evolution@swift.org> wrote:

Python’s os.path <https://docs.python.org/2/library/os.path.html&gt; is a nice
abstract model for doing path manipulations. Maybe Swift could get a struct
like `Filepath` or something on which these operations could be done.

···

On Thu, Aug 17, 2017 at 6:47 PM, Taylor Swift <kelvin13ma@gmail.com> wrote:

On Thu, Aug 17, 2017 at 3:50 PM, Daniel Dunbar <daniel_dunbar@apple.com> > wrote:

> On Aug 17, 2017, at 9:26 AM, Taylor Swift via swift-evolution < >> swift-evolution@swift.org> wrote:
>
> I know this has come up before without any action, but having the
standard C library be packaged under `Darwin` on OSX and `Glibc` under
Linux is something that’s really becoming an issue as the Swift package
ecosystem matures. Right now a lot of packages are a lot less portable than
they could be because somewhere along the dependency line, there is a
package that only imports the C library for one platform. Unifying it under
one import would allow people to write packages that are portable by
default.

What we (SwiftPM) have done for now is use a `libc` target to start by
normalizing the name:
  https://github.com/apple/swift-package-manager/tree/master/Sources/libc
(and in the past, when we find missing things in Glibc getting them added
to the compiler). Also, this name is technically a misnomer, but we
couldn’t think of a better one (“os” might have been a good one).

Unfortunately, I think this change alone is really only the tip of the
iceberg. It would be nice to not have it the difference, but we found we
very quickly needed several other layers on top to get to having a
relatively stable cross-platform base:
  GitHub - apple/swift-package-manager: The Package Manager for the Swift Programming Language
Sources/POSIX
  GitHub - apple/swift-package-manager: The Package Manager for the Swift Programming Language
Sources/Basic

My hope is that one minimal improvement we can get soon is multi-package
repo support in SwiftPM, which will at least allow us to share those
targets & functionality with other packages.

> Since I think this got hung up in the past over “what constitutes” a
universal libc, I propose a unified package should just consist of the
functionality that is common between Darwin and Glibc right now, since
those are the only two supported platforms anyway.

What would the concrete proposal be? It isn’t trivial to determine that
subset and make it well-defined what the exact API is. Is the proposal to
just to pick a standard name, and reexport Darwin and Glibc from it?

I don’t know if it’s actually this simple, but could it just be the
symbols that are defined in both modules?

> Alternatively, Swift could make it a priority to get basic
functionality like file IO and math functions shipped with the compiler,
which would probably cover 98% of cases where people currently import
Darwin/Glibc. A large portion of the standard C libraries are redundant to
the Swift standard library anyway.

I’m not sure I agree with these statements about the percentages. For
some clients (I’m biased, the areas I work in tend to be in this boat),
what we largely need is good platform-agnostic access to the POSIX APIs.
This is a hard problem to make a good cross-platform API for (esp. if
Windows support is in your head), and which many projects struggle with
(Netscape :: NSPR, LLVM :: libSupport, many many more).

The sticking point I see is this: if the proposal is just to unify the
name & that doesn’t actually buy us all that much (still need standard
layers on top), then have we really solved enough of a problem to be worth
standardizing on?

+1 in general agreement with the meta-issue being an important one to
discuss.

There probably is an XY issue at play here; what we *really* need is a
way to access the file system built into the standard library. (Math
functions are a separate, beleaguered topic for a different thread.) Having
support for outputting to `stderr` is also something I’d really like. Going
through Glibc/Darwin is just one way to solve this.

>
> I know this has come up before without any action, but having the standard C library be packaged under `Darwin` on OSX and `Glibc` under Linux is something that’s really becoming an issue as the Swift package ecosystem matures. Right now a lot of packages are a lot less portable than they could be because somewhere along the dependency line, there is a package that only imports the C library for one platform. Unifying it under one import would allow people to write packages that are portable by default.

What we (SwiftPM) have done for now is use a `libc` target to start by normalizing the name:
  https://github.com/apple/swift-package-manager/tree/master/Sources/libc
(and in the past, when we find missing things in Glibc getting them added to the compiler). Also, this name is technically a misnomer, but we couldn’t think of a better one (“os” might have been a good one).

Unfortunately, I think this change alone is really only the tip of the iceberg. It would be nice to not have it the difference, but we found we very quickly needed several other layers on top to get to having a relatively stable cross-platform base:
  https://github.com/apple/swift-package-manager/tree/master/Sources/POSIX
  https://github.com/apple/swift-package-manager/tree/master/Sources/Basic

My hope is that one minimal improvement we can get soon is multi-package repo support in SwiftPM, which will at least allow us to share those targets & functionality with other packages.

> Since I think this got hung up in the past over “what constitutes” a universal libc, I propose a unified package should just consist of the functionality that is common between Darwin and Glibc right now, since those are the only two supported platforms anyway.

What would the concrete proposal be? It isn’t trivial to determine that subset and make it well-defined what the exact API is. Is the proposal to just to pick a standard name, and reexport Darwin and Glibc from it?

I don’t know if it’s actually this simple, but could it just be the symbols that are defined in both modules?

I think this would be hard to implement in practice. There are also symbols which are defined the same, but behave differently (or accept slightly different types). Reconciling all of that would be a large project.

> Alternatively, Swift could make it a priority to get basic functionality like file IO and math functions shipped with the compiler, which would probably cover 98% of cases where people currently import Darwin/Glibc. A large portion of the standard C libraries are redundant to the Swift standard library anyway.

I’m not sure I agree with these statements about the percentages. For some clients (I’m biased, the areas I work in tend to be in this boat), what we largely need is good platform-agnostic access to the POSIX APIs. This is a hard problem to make a good cross-platform API for (esp. if Windows support is in your head), and which many projects struggle with (Netscape :: NSPR, LLVM :: libSupport, many many more).

The sticking point I see is this: if the proposal is just to unify the name & that doesn’t actually buy us all that much (still need standard layers on top), then have we really solved enough of a problem to be worth standardizing on?

+1 in general agreement with the meta-issue being an important one to discuss.

There probably is an XY issue at play here; what we really need is a way to access the file system built into the standard library. (Math functions are a separate, beleaguered topic for a different thread.) Having support for outputting to `stderr` is also something I’d really like. Going through Glibc/Darwin is just one way to solve this.

Right, this is exactly the kind of thing SwiftPM’s Basic has.

- Daniel

···

On Aug 17, 2017, at 3:47 PM, Taylor Swift <kelvin13ma@gmail.com> wrote:
On Thu, Aug 17, 2017 at 3:50 PM, Daniel Dunbar <daniel_dunbar@apple.com <mailto:daniel_dunbar@apple.com>> wrote:
> On Aug 17, 2017, at 9:26 AM, Taylor Swift via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

IMO, you’re touching on at least three or four separate topics here. Daniel
touched on several, but just some comments/questions:

* Standard error output is something that’s been discussed here previously.
I believe the last pitch had something like StandardError being added to
the standard library as a TextOutputStream.

* Foundation is supposed to be the core library that provides file system
access facilities. What’s missing, and can we add it to Foundation?

···

On Thu, Aug 17, 2017 at 17:50 Taylor Swift via swift-evolution < swift-evolution@swift.org> wrote:

Python’s os.path <https://docs.python.org/2/library/os.path.html&gt; is a
nice abstract model for doing path manipulations. Maybe Swift could get a
struct like `Filepath` or something on which these operations could be done.

On Thu, Aug 17, 2017 at 6:47 PM, Taylor Swift <kelvin13ma@gmail.com> > wrote:

On Thu, Aug 17, 2017 at 3:50 PM, Daniel Dunbar <daniel_dunbar@apple.com> >> wrote:

> On Aug 17, 2017, at 9:26 AM, Taylor Swift via swift-evolution < >>> swift-evolution@swift.org> wrote:
>
> I know this has come up before without any action, but having the
standard C library be packaged under `Darwin` on OSX and `Glibc` under
Linux is something that’s really becoming an issue as the Swift package
ecosystem matures. Right now a lot of packages are a lot less portable than
they could be because somewhere along the dependency line, there is a
package that only imports the C library for one platform. Unifying it under
one import would allow people to write packages that are portable by
default.

What we (SwiftPM) have done for now is use a `libc` target to start by
normalizing the name:

https://github.com/apple/swift-package-manager/tree/master/Sources/libc
(and in the past, when we find missing things in Glibc getting them
added to the compiler). Also, this name is technically a misnomer, but we
couldn’t think of a better one (“os” might have been a good one).

Unfortunately, I think this change alone is really only the tip of the
iceberg. It would be nice to not have it the difference, but we found we
very quickly needed several other layers on top to get to having a
relatively stable cross-platform base:

https://github.com/apple/swift-package-manager/tree/master/Sources/POSIX

https://github.com/apple/swift-package-manager/tree/master/Sources/Basic

My hope is that one minimal improvement we can get soon is multi-package
repo support in SwiftPM, which will at least allow us to share those
targets & functionality with other packages.

> Since I think this got hung up in the past over “what constitutes” a
universal libc, I propose a unified package should just consist of the
functionality that is common between Darwin and Glibc right now, since
those are the only two supported platforms anyway.

What would the concrete proposal be? It isn’t trivial to determine that
subset and make it well-defined what the exact API is. Is the proposal to
just to pick a standard name, and reexport Darwin and Glibc from it?

I don’t know if it’s actually this simple, but could it just be the
symbols that are defined in both modules?

> Alternatively, Swift could make it a priority to get basic
functionality like file IO and math functions shipped with the compiler,
which would probably cover 98% of cases where people currently import
Darwin/Glibc. A large portion of the standard C libraries are redundant to
the Swift standard library anyway.

I’m not sure I agree with these statements about the percentages. For
some clients (I’m biased, the areas I work in tend to be in this boat),
what we largely need is good platform-agnostic access to the POSIX APIs.
This is a hard problem to make a good cross-platform API for (esp. if
Windows support is in your head), and which many projects struggle with
(Netscape :: NSPR, LLVM :: libSupport, many many more).

The sticking point I see is this: if the proposal is just to unify the
name & that doesn’t actually buy us all that much (still need standard
layers on top), then have we really solved enough of a problem to be worth
standardizing on?

+1 in general agreement with the meta-issue being an important one to
discuss.

There probably is an XY issue at play here; what we *really* need is a
way to access the file system built into the standard library. (Math
functions are a separate, beleaguered topic for a different thread.) Having
support for outputting to `stderr` is also something I’d really like. Going
through Glibc/Darwin is just one way to solve this.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I don’t think the “is this library functionality or standard library
functionality” argument is worth having, but if stdout and stdin are
first-class citizens in the Swift world, so should stderr.

As for bringing Foundation into the discussion, you can’t really talk
about Foundation without also talking about the mountains of problems that
come with the monolith pattern. But that’s a completely different
conversation to be had.

I'm not sure what you're getting at here, but I don't believe you've
addressed my question, which is: it's been firmly decided that I/O belongs
in Foundation, and Foundation does in fact offer such facilities--what is
missing from those facilities, and how can we fill it out?

···

On Thu, Aug 17, 2017 at 6:46 PM, Taylor Swift <kelvin13ma@gmail.com> wrote:

On Thu, Aug 17, 2017 at 7:13 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

IMO, you’re touching on at least three or four separate topics here.
Daniel touched on several, but just some comments/questions:

* Standard error output is something that’s been discussed here
previously. I believe the last pitch had something like StandardError being
added to the standard library as a TextOutputStream.

* Foundation is supposed to be the core library that provides file system
access facilities. What’s missing, and can we add it to Foundation?

On Thu, Aug 17, 2017 at 17:50 Taylor Swift via swift-evolution < >> swift-evolution@swift.org> wrote:

Python’s os.path <https://docs.python.org/2/library/os.path.html&gt; is a
nice abstract model for doing path manipulations. Maybe Swift could get a
struct like `Filepath` or something on which these operations could be done.

On Thu, Aug 17, 2017 at 6:47 PM, Taylor Swift <kelvin13ma@gmail.com> >>> wrote:

On Thu, Aug 17, 2017 at 3:50 PM, Daniel Dunbar <daniel_dunbar@apple.com >>>> > wrote:

> On Aug 17, 2017, at 9:26 AM, Taylor Swift via swift-evolution < >>>>> swift-evolution@swift.org> wrote:
>
> I know this has come up before without any action, but having the
standard C library be packaged under `Darwin` on OSX and `Glibc` under
Linux is something that’s really becoming an issue as the Swift package
ecosystem matures. Right now a lot of packages are a lot less portable than
they could be because somewhere along the dependency line, there is a
package that only imports the C library for one platform. Unifying it under
one import would allow people to write packages that are portable by
default.

What we (SwiftPM) have done for now is use a `libc` target to start by
normalizing the name:
  GitHub - apple/swift-package-manager: The Package Manager for the Swift Programming Language
Sources/libc
(and in the past, when we find missing things in Glibc getting them
added to the compiler). Also, this name is technically a misnomer, but we
couldn’t think of a better one (“os” might have been a good one).

Unfortunately, I think this change alone is really only the tip of the
iceberg. It would be nice to not have it the difference, but we found we
very quickly needed several other layers on top to get to having a
relatively stable cross-platform base:
  GitHub - apple/swift-package-manager: The Package Manager for the Swift Programming Language
Sources/POSIX
  GitHub - apple/swift-package-manager: The Package Manager for the Swift Programming Language
Sources/Basic

My hope is that one minimal improvement we can get soon is
multi-package repo support in SwiftPM, which will at least allow us to
share those targets & functionality with other packages.

> Since I think this got hung up in the past over “what constitutes” a
universal libc, I propose a unified package should just consist of the
functionality that is common between Darwin and Glibc right now, since
those are the only two supported platforms anyway.

What would the concrete proposal be? It isn’t trivial to determine
that subset and make it well-defined what the exact API is. Is the proposal
to just to pick a standard name, and reexport Darwin and Glibc from it?

I don’t know if it’s actually this simple, but could it just be the
symbols that are defined in both modules?

> Alternatively, Swift could make it a priority to get basic
functionality like file IO and math functions shipped with the compiler,
which would probably cover 98% of cases where people currently import
Darwin/Glibc. A large portion of the standard C libraries are redundant to
the Swift standard library anyway.

I’m not sure I agree with these statements about the percentages. For
some clients (I’m biased, the areas I work in tend to be in this boat),
what we largely need is good platform-agnostic access to the POSIX APIs.
This is a hard problem to make a good cross-platform API for (esp. if
Windows support is in your head), and which many projects struggle with
(Netscape :: NSPR, LLVM :: libSupport, many many more).

The sticking point I see is this: if the proposal is just to unify the
name & that doesn’t actually buy us all that much (still need standard
layers on top), then have we really solved enough of a problem to be worth
standardizing on?

+1 in general agreement with the meta-issue being an important one to
discuss.

There probably is an XY issue at play here; what we *really* need is a
way to access the file system built into the standard library. (Math
functions are a separate, beleaguered topic for a different thread.) Having
support for outputting to `stderr` is also something I’d really like. Going
through Glibc/Darwin is just one way to solve this.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

I don’t think the “is this library functionality or standard library
functionality” argument is worth having, but if stdout and stdin are
first-class citizens in the Swift world, so should stderr.

As for bringing Foundation into the discussion, you can’t really talk about
Foundation without also talking about the mountains of problems that come
with the monolith pattern. But that’s a completely different conversation
to be had.

···

On Thu, Aug 17, 2017 at 7:13 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

IMO, you’re touching on at least three or four separate topics here.
Daniel touched on several, but just some comments/questions:

* Standard error output is something that’s been discussed here
previously. I believe the last pitch had something like StandardError being
added to the standard library as a TextOutputStream.

* Foundation is supposed to be the core library that provides file system
access facilities. What’s missing, and can we add it to Foundation?

On Thu, Aug 17, 2017 at 17:50 Taylor Swift via swift-evolution < > swift-evolution@swift.org> wrote:

Python’s os.path <https://docs.python.org/2/library/os.path.html&gt; is a
nice abstract model for doing path manipulations. Maybe Swift could get a
struct like `Filepath` or something on which these operations could be done.

On Thu, Aug 17, 2017 at 6:47 PM, Taylor Swift <kelvin13ma@gmail.com> >> wrote:

On Thu, Aug 17, 2017 at 3:50 PM, Daniel Dunbar <daniel_dunbar@apple.com> >>> wrote:

> On Aug 17, 2017, at 9:26 AM, Taylor Swift via swift-evolution < >>>> swift-evolution@swift.org> wrote:
>
> I know this has come up before without any action, but having the
standard C library be packaged under `Darwin` on OSX and `Glibc` under
Linux is something that’s really becoming an issue as the Swift package
ecosystem matures. Right now a lot of packages are a lot less portable than
they could be because somewhere along the dependency line, there is a
package that only imports the C library for one platform. Unifying it under
one import would allow people to write packages that are portable by
default.

What we (SwiftPM) have done for now is use a `libc` target to start by
normalizing the name:
  https://github.com/apple/swift-package-manager/tree/
master/Sources/libc
(and in the past, when we find missing things in Glibc getting them
added to the compiler). Also, this name is technically a misnomer, but we
couldn’t think of a better one (“os” might have been a good one).

Unfortunately, I think this change alone is really only the tip of the
iceberg. It would be nice to not have it the difference, but we found we
very quickly needed several other layers on top to get to having a
relatively stable cross-platform base:
  https://github.com/apple/swift-package-manager/tree/
master/Sources/POSIX
  https://github.com/apple/swift-package-manager/tree/
master/Sources/Basic

My hope is that one minimal improvement we can get soon is
multi-package repo support in SwiftPM, which will at least allow us to
share those targets & functionality with other packages.

> Since I think this got hung up in the past over “what constitutes” a
universal libc, I propose a unified package should just consist of the
functionality that is common between Darwin and Glibc right now, since
those are the only two supported platforms anyway.

What would the concrete proposal be? It isn’t trivial to determine that
subset and make it well-defined what the exact API is. Is the proposal to
just to pick a standard name, and reexport Darwin and Glibc from it?

I don’t know if it’s actually this simple, but could it just be the
symbols that are defined in both modules?

> Alternatively, Swift could make it a priority to get basic
functionality like file IO and math functions shipped with the compiler,
which would probably cover 98% of cases where people currently import
Darwin/Glibc. A large portion of the standard C libraries are redundant to
the Swift standard library anyway.

I’m not sure I agree with these statements about the percentages. For
some clients (I’m biased, the areas I work in tend to be in this boat),
what we largely need is good platform-agnostic access to the POSIX APIs.
This is a hard problem to make a good cross-platform API for (esp. if
Windows support is in your head), and which many projects struggle with
(Netscape :: NSPR, LLVM :: libSupport, many many more).

The sticking point I see is this: if the proposal is just to unify the
name & that doesn’t actually buy us all that much (still need standard
layers on top), then have we really solved enough of a problem to be worth
standardizing on?

+1 in general agreement with the meta-issue being an important one to
discuss.

There probably is an XY issue at play here; what we *really* need is a
way to access the file system built into the standard library. (Math
functions are a separate, beleaguered topic for a different thread.) Having
support for outputting to `stderr` is also something I’d really like. Going
through Glibc/Darwin is just one way to solve this.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Okay I can sense this thread getting derailed so I’ll try to address your
comments one by one.

* stderr should go wherever stdin and stdout go. Since it’d be silly for a
function like `print(_:separator:terminator:)` or `
readLine(strippingNewline:)` to live anywhere but the standard library,
then it stands to reason that the stderr version should also live in the
standard library.

* Foundation is “supposed” to be the core library that does these things,
but this isn’t a good thing. Rationale:

    * Foundation is literally an Objective C framework that’s written with
Swift syntax. The classes extend *NSObject*. You use them by *subclassing*
them and overriding open methods. This is patently unswifty. Even parts of
Foundation that look swifty on the outside (like `URL`) are still backed by
NS reference types internally.

    * To fix this would require a complete rewrite from the ground up,
that’s more than a straight port from the Objective C framework. Many on
this list have also expressed desire to use this chance to redesign these
APIs. Since we’d basically have to rewrite all of it, any effort to
modernize Foundation is basically writing a new core library.

    * Even if a piece of Foundation was rewritten like a real Swift
library, because of the monolith pattern, you still wind up importing a lot
of legacy code that shouldn’t be floating around your project. Using the
file system tools shouldn’t change `String`.

* Foundation’s file system capabilities are really just a class wrapper
around Glibc/Darwin functions. So in a way, Foundation is already a sort of
unified import for C, except it brings in a whole load of Objective C cruft
with it.

    * Logically, this means in terms of the XY problem, Foundation is on
the Y side. Foundation is what happens if you rely on Swift’s C interop to
get the job done instead of building a true Swift solution, which would
involve Swift quarterbacking all the nasty system calls, instead of
Glibc/Darwin doing it.

In conclusion,

* What we need is a modern Swift solution for accessing file systems.

* What we have to do right now is leverage Swift’s C interop to use
Glibc/Darwin, while handing imports and platform inconsistencies manually.

* Eventually, the ideal would be for Swift to handle that in Swift, instead
of delegating to a platform-dependent C library, or at least standardize
things so that a swifty Swift library imports the right C library for you,
and exports a platform-independent set of symbols to the user, or a higher
level API.

* Foundation is in many ways the worst of both worlds. It handles the
Glibc/Darwin import for us, but at the cost of an aging framework that runs
against the grain of good Swift design idioms.

···

On Thu, Aug 17, 2017 at 8:16 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Thu, Aug 17, 2017 at 6:46 PM, Taylor Swift <kelvin13ma@gmail.com> > wrote:

I don’t think the “is this library functionality or standard library
functionality” argument is worth having, but if stdout and stdin are
first-class citizens in the Swift world, so should stderr.

As for bringing Foundation into the discussion, you can’t really talk
about Foundation without also talking about the mountains of problems that
come with the monolith pattern. But that’s a completely different
conversation to be had.

I'm not sure what you're getting at here, but I don't believe you've
addressed my question, which is: it's been firmly decided that I/O belongs
in Foundation, and Foundation does in fact offer such facilities--what is
missing from those facilities, and how can we fill it out?

On Thu, Aug 17, 2017 at 7:13 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

IMO, you’re touching on at least three or four separate topics here.
Daniel touched on several, but just some comments/questions:

* Standard error output is something that’s been discussed here
previously. I believe the last pitch had something like StandardError being
added to the standard library as a TextOutputStream.

* Foundation is supposed to be the core library that provides file
system access facilities. What’s missing, and can we add it to Foundation?

On Thu, Aug 17, 2017 at 17:50 Taylor Swift via swift-evolution < >>> swift-evolution@swift.org> wrote:

Python’s os.path <https://docs.python.org/2/library/os.path.html&gt; is a
nice abstract model for doing path manipulations. Maybe Swift could get a
struct like `Filepath` or something on which these operations could be done.

On Thu, Aug 17, 2017 at 6:47 PM, Taylor Swift <kelvin13ma@gmail.com> >>>> wrote:

On Thu, Aug 17, 2017 at 3:50 PM, Daniel Dunbar < >>>>> daniel_dunbar@apple.com> wrote:

> On Aug 17, 2017, at 9:26 AM, Taylor Swift via swift-evolution < >>>>>> swift-evolution@swift.org> wrote:
>
> I know this has come up before without any action, but having the
standard C library be packaged under `Darwin` on OSX and `Glibc` under
Linux is something that’s really becoming an issue as the Swift package
ecosystem matures. Right now a lot of packages are a lot less portable than
they could be because somewhere along the dependency line, there is a
package that only imports the C library for one platform. Unifying it under
one import would allow people to write packages that are portable by
default.

What we (SwiftPM) have done for now is use a `libc` target to start
by normalizing the name:
  https://github.com/apple/swift-package-manager/tree/master/S
ources/libc
(and in the past, when we find missing things in Glibc getting them
added to the compiler). Also, this name is technically a misnomer, but we
couldn’t think of a better one (“os” might have been a good one).

Unfortunately, I think this change alone is really only the tip of
the iceberg. It would be nice to not have it the difference, but we found
we very quickly needed several other layers on top to get to having a
relatively stable cross-platform base:
  https://github.com/apple/swift-package-manager/tree/master/S
ources/POSIX
  https://github.com/apple/swift-package-manager/tree/master/S
ources/Basic

My hope is that one minimal improvement we can get soon is
multi-package repo support in SwiftPM, which will at least allow us to
share those targets & functionality with other packages.

> Since I think this got hung up in the past over “what constitutes”
a universal libc, I propose a unified package should just consist of the
functionality that is common between Darwin and Glibc right now, since
those are the only two supported platforms anyway.

What would the concrete proposal be? It isn’t trivial to determine
that subset and make it well-defined what the exact API is. Is the proposal
to just to pick a standard name, and reexport Darwin and Glibc from it?

I don’t know if it’s actually this simple, but could it just be the
symbols that are defined in both modules?

> Alternatively, Swift could make it a priority to get basic
functionality like file IO and math functions shipped with the compiler,
which would probably cover 98% of cases where people currently import
Darwin/Glibc. A large portion of the standard C libraries are redundant to
the Swift standard library anyway.

I’m not sure I agree with these statements about the percentages. For
some clients (I’m biased, the areas I work in tend to be in this boat),
what we largely need is good platform-agnostic access to the POSIX APIs.
This is a hard problem to make a good cross-platform API for (esp. if
Windows support is in your head), and which many projects struggle with
(Netscape :: NSPR, LLVM :: libSupport, many many more).

The sticking point I see is this: if the proposal is just to unify
the name & that doesn’t actually buy us all that much (still need standard
layers on top), then have we really solved enough of a problem to be worth
standardizing on?

+1 in general agreement with the meta-issue being an important one to
discuss.

There probably is an XY issue at play here; what we *really* need is
a way to access the file system built into the standard library. (Math
functions are a separate, beleaguered topic for a different thread.) Having
support for outputting to `stderr` is also something I’d really like. Going
through Glibc/Darwin is just one way to solve this.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

1 Like

Okay I can sense this thread getting derailed so I’ll try to address your
comments one by one.

* stderr should go wherever stdin and stdout go. Since it’d be silly for
a function like `print(_:separator:terminator:)` or `
readLine(strippingNewline:)` to live anywhere but the standard library,
then it stands to reason that the stderr version should also live in the
standard library.

FWIW, FileHandle.standardInput, FileHandle.standardError,
FileHandle.standardOutput, and FileHandle.nullDevice all live in Foundation.

* Foundation is “supposed” to be the core library that does these things,
but this isn’t a good thing. Rationale:

    * Foundation is literally an Objective C framework that’s written with
Swift syntax. The classes extend *NSObject*. You use them by *subclassing*
them and overriding open methods. This is patently unswifty. Even parts
of Foundation that look swifty on the outside (like `URL`) are still
backed by NS reference types internally.

    * To fix this would require a complete rewrite from the ground up,
that’s more than a straight port from the Objective C framework. Many on
this list have also expressed desire to use this chance to redesign these
APIs. Since we’d basically have to rewrite all of it, any effort to
modernize Foundation is basically writing a new core library.

    * Even if a piece of Foundation was rewritten like a real Swift
library, because of the monolith pattern, you still wind up importing a lot
of legacy code that shouldn’t be floating around your project. Using the
file system tools shouldn’t change `String`.

* Foundation’s file system capabilities are really just a class wrapper
around Glibc/Darwin functions. So in a way, Foundation is already a sort of
unified import for C, except it brings in a whole load of Objective C cruft
with it.

    * Logically, this means in terms of the XY problem, Foundation is on
the Y side. Foundation is what happens if you rely on Swift’s C interop to
get the job done instead of building a true Swift solution, which would
involve Swift quarterbacking all the nasty system calls, instead of
Glibc/Darwin doing it.

In conclusion,

* What we need is a modern Swift solution for accessing file systems.

* What we have to do right now is leverage Swift’s C interop to use
Glibc/Darwin, while handing imports and platform inconsistencies manually.

* Eventually, the ideal would be for Swift to handle that in Swift,
instead of delegating to a platform-dependent C library, or at least
standardize things so that a swifty Swift library imports the right C
library for you, and exports a platform-independent set of symbols to the
user, or a higher level API.

* Foundation is in many ways the worst of both worlds. It handles the
Glibc/Darwin import for us, but at the cost of an aging framework that runs
against the grain of good Swift design idioms.

As has been stated on this list by the core team, replacing Foundation or
duplicating its functionality is a non-goal of the Swift project. The major
focus of several initial proposals, and a considerable amount of effort on
this list, was precisely to modernize Foundation so as to offer APIs that
conform to Swift guidelines. This work remains ongoing, as there are a
number of Foundation value types that have yet to be designed or
implemented. And there is a whole open source project devoted to Foundation
becoming a viable cross-platform library.

So given these parameters, let's return to what you are talking about: what
is deficient about Swift's core file system APIs, and how would you suggest
that we address these deficiencies, given that the core team has stated
that it is not up for debate that any such changes would be part of
Foundation?

···

On Thu, Aug 17, 2017 at 22:06 Taylor Swift <kelvin13ma@gmail.com> wrote:

On Thu, Aug 17, 2017 at 8:16 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Thu, Aug 17, 2017 at 6:46 PM, Taylor Swift <kelvin13ma@gmail.com> >> wrote:

I don’t think the “is this library functionality or standard library
functionality” argument is worth having, but if stdout and stdin are
first-class citizens in the Swift world, so should stderr.

As for bringing Foundation into the discussion, you can’t really talk
about Foundation without also talking about the mountains of problems that
come with the monolith pattern. But that’s a completely different
conversation to be had.

I'm not sure what you're getting at here, but I don't believe you've
addressed my question, which is: it's been firmly decided that I/O belongs
in Foundation, and Foundation does in fact offer such facilities--what is
missing from those facilities, and how can we fill it out?

On Thu, Aug 17, 2017 at 7:13 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

IMO, you’re touching on at least three or four separate topics here.
Daniel touched on several, but just some comments/questions:

* Standard error output is something that’s been discussed here
previously. I believe the last pitch had something like StandardError being
added to the standard library as a TextOutputStream.

* Foundation is supposed to be the core library that provides file
system access facilities. What’s missing, and can we add it to Foundation?

On Thu, Aug 17, 2017 at 17:50 Taylor Swift via swift-evolution < >>>> swift-evolution@swift.org> wrote:

Python’s os.path <https://docs.python.org/2/library/os.path.html&gt; is
a nice abstract model for doing path manipulations. Maybe Swift could get a
struct like `Filepath` or something on which these operations could be done.

On Thu, Aug 17, 2017 at 6:47 PM, Taylor Swift <kelvin13ma@gmail.com> >>>>> wrote:

On Thu, Aug 17, 2017 at 3:50 PM, Daniel Dunbar < >>>>>> daniel_dunbar@apple.com> wrote:

> On Aug 17, 2017, at 9:26 AM, Taylor Swift via swift-evolution < >>>>>>> swift-evolution@swift.org> wrote:
>
> I know this has come up before without any action, but having the
standard C library be packaged under `Darwin` on OSX and `Glibc` under
Linux is something that’s really becoming an issue as the Swift package
ecosystem matures. Right now a lot of packages are a lot less portable than
they could be because somewhere along the dependency line, there is a
package that only imports the C library for one platform. Unifying it under
one import would allow people to write packages that are portable by
default.

What we (SwiftPM) have done for now is use a `libc` target to start
by normalizing the name:

https://github.com/apple/swift-package-manager/tree/master/Sources/libc
(and in the past, when we find missing things in Glibc getting them
added to the compiler). Also, this name is technically a misnomer, but we
couldn’t think of a better one (“os” might have been a good one).

Unfortunately, I think this change alone is really only the tip of
the iceberg. It would be nice to not have it the difference, but we found
we very quickly needed several other layers on top to get to having a
relatively stable cross-platform base:

https://github.com/apple/swift-package-manager/tree/master/Sources/POSIX

https://github.com/apple/swift-package-manager/tree/master/Sources/Basic

My hope is that one minimal improvement we can get soon is
multi-package repo support in SwiftPM, which will at least allow us to
share those targets & functionality with other packages.

> Since I think this got hung up in the past over “what constitutes”
a universal libc, I propose a unified package should just consist of the
functionality that is common between Darwin and Glibc right now, since
those are the only two supported platforms anyway.

What would the concrete proposal be? It isn’t trivial to determine
that subset and make it well-defined what the exact API is. Is the proposal
to just to pick a standard name, and reexport Darwin and Glibc from it?

I don’t know if it’s actually this simple, but could it just be the
symbols that are defined in both modules?

> Alternatively, Swift could make it a priority to get basic
functionality like file IO and math functions shipped with the compiler,
which would probably cover 98% of cases where people currently import
Darwin/Glibc. A large portion of the standard C libraries are redundant to
the Swift standard library anyway.

I’m not sure I agree with these statements about the percentages.
For some clients (I’m biased, the areas I work in tend to be in this boat),
what we largely need is good platform-agnostic access to the POSIX APIs.
This is a hard problem to make a good cross-platform API for (esp. if
Windows support is in your head), and which many projects struggle with
(Netscape :: NSPR, LLVM :: libSupport, many many more).

The sticking point I see is this: if the proposal is just to unify
the name & that doesn’t actually buy us all that much (still need standard
layers on top), then have we really solved enough of a problem to be worth
standardizing on?

+1 in general agreement with the meta-issue being an important one
to discuss.

There probably is an XY issue at play here; what we *really* need is
a way to access the file system built into the standard library. (Math
functions are a separate, beleaguered topic for a different thread.) Having
support for outputting to `stderr` is also something I’d really like. Going
through Glibc/Darwin is just one way to solve this.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

Lots of I/O functionality is missing from Foundation. Foundation's design from time immemorial is that generally only relatively simple and high-level operations are available in Foundation itself, and if you want to do complicated or non-portable things then you are expected to drop down to POSIX or other C interfaces. That design works less well in Swift than it did in Objective-C because Swift's interface with C, especially low-level C, is often ugly.

Simple example: there is no way to access file information directly via NSFileHandle. You need to call NSFileHandle.fileDescriptor and pass that to fstat() or fcntl(). The NSFileHandle API in general is sparse and wholly inadequate for sophisticated I/O.

···

On Aug 17, 2017, at 5:16 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

On Thu, Aug 17, 2017 at 6:46 PM, Taylor Swift <kelvin13ma@gmail.com <mailto:kelvin13ma@gmail.com>> wrote:
I don’t think the “is this library functionality or standard library functionality” argument is worth having, but if stdout and stdin are first-class citizens in the Swift world, so should stderr.

As for bringing Foundation into the discussion, you can’t really talk about Foundation without also talking about the mountains of problems that come with the monolith pattern. But that’s a completely different conversation to be had.

I'm not sure what you're getting at here, but I don't believe you've addressed my question, which is: it's been firmly decided that I/O belongs in Foundation, and Foundation does in fact offer such facilities--what is missing from those facilities, and how can we fill it out?

--
Greg Parker gparker@apple.com <mailto:gparker@apple.com> Runtime Wrangler

And, since they're read-only, are fundamentally incompatible with `print(…to:)`, which requires its `output` parameter to be passed `inout`.

···

On Aug 17, 2017, at 8:20 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

* stderr should go wherever stdin and stdout go. Since it’d be silly for a function like `print(_:separator:terminator:)` or `readLine(strippingNewline:)` to live anywhere but the standard library, then it stands to reason that the stderr version should also live in the standard library.

FWIW, FileHandle.standardInput, FileHandle.standardError, FileHandle.standardOutput, and FileHandle.nullDevice all live in Foundation.

--
Brent Royal-Gordon
Architechies

Okay I can sense this thread getting derailed so I’ll try to address your
comments one by one.

* stderr should go wherever stdin and stdout go. Since it’d be silly for
a function like `print(_:separator:terminator:)` or `
readLine(strippingNewline:)` to live anywhere but the standard library,
then it stands to reason that the stderr version should also live in the
standard library.

FWIW, FileHandle.standardInput, FileHandle.standardError,
FileHandle.standardOutput, and FileHandle.nullDevice all live in Foundation.

Yet, we have print and readLine, but no printError.

* Foundation is “supposed” to be the core library that does these things,
but this isn’t a good thing. Rationale:

    * Foundation is literally an Objective C framework that’s written
with Swift syntax. The classes extend *NSObject*. You use them by
*subclassing* them and overriding open methods. This is patently
unswifty. Even parts of Foundation that look swifty on the outside (like `
URL`) are still backed by NS reference types internally.

    * To fix this would require a complete rewrite from the ground up,
that’s more than a straight port from the Objective C framework. Many on
this list have also expressed desire to use this chance to redesign these
APIs. Since we’d basically have to rewrite all of it, any effort to
modernize Foundation is basically writing a new core library.

    * Even if a piece of Foundation was rewritten like a real Swift
library, because of the monolith pattern, you still wind up importing a lot
of legacy code that shouldn’t be floating around your project. Using the
file system tools shouldn’t change `String`.

* Foundation’s file system capabilities are really just a class wrapper
around Glibc/Darwin functions. So in a way, Foundation is already a sort of
unified import for C, except it brings in a whole load of Objective C cruft
with it.

    * Logically, this means in terms of the XY problem, Foundation is on
the Y side. Foundation is what happens if you rely on Swift’s C interop to
get the job done instead of building a true Swift solution, which would
involve Swift quarterbacking all the nasty system calls, instead of
Glibc/Darwin doing it.

In conclusion,

* What we need is a modern Swift solution for accessing file systems.

* What we have to do right now is leverage Swift’s C interop to use
Glibc/Darwin, while handing imports and platform inconsistencies manually.

* Eventually, the ideal would be for Swift to handle that in Swift,
instead of delegating to a platform-dependent C library, or at least
standardize things so that a swifty Swift library imports the right C
library for you, and exports a platform-independent set of symbols to the
user, or a higher level API.

* Foundation is in many ways the worst of both worlds. It handles the
Glibc/Darwin import for us, but at the cost of an aging framework that runs
against the grain of good Swift design idioms.

As has been stated on this list by the core team, replacing Foundation or
duplicating its functionality is a non-goal of the Swift project. The major
focus of several initial proposals, and a considerable amount of effort on
this list, was precisely to modernize Foundation so as to offer APIs that
conform to Swift guidelines. This work remains ongoing, as there are a
number of Foundation value types that have yet to be designed or
implemented. And there is a whole open source project devoted to Foundation
becoming a viable cross-platform library.

So given these parameters, let's return to what you are talking about:
what is deficient about Swift's core file system APIs, and how would you
suggest that we address these deficiencies, given that the core team has
stated that it is not up for debate that any such changes would be part of
Foundation?

This probably isn’t the answer you were looking for, but, as explained,
*all* of it is deficient, so long as you still have to subclass FileManager
and catch CocoaErrors. The problem with Foundation is it tries to offer
core library features and support bridging to Cocoa types at the same time.
Those two goals aren’t very compatible. That’s why there’s such a day and
night difference between stdlib and “pure swift” code, and Foundation code.

So, within the parameters, I would suggest

* rewriting the Foundation file system API types as value and protocol
types, retire the delegate-driven pattern, and *drop* ReferenceConvertible
conformance.

    * value types should be trivial value types, with the possible
exception of file streams which need to be closed.

* separate it as much as possible from other Foundation types, so as to
help disentangle the library from NSNumber and Date.

    * inseparable dependencies like URL should also be rewritten as trivial
types. There is no reason for URL to have reference or copy on write
semantics.

* get submodules into Swift so that Foundation can be split into pieces
instead of being imported as a monolith.

···

On Thu, Aug 17, 2017 at 11:19 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Thu, Aug 17, 2017 at 22:06 Taylor Swift <kelvin13ma@gmail.com> wrote:

Not “fundamentally” incompatible:

var stderr = FileHandle.standardError
/* Conform FileHandle to TextOutputStream */
print("foo", to: &stderr)

···

On Fri, Aug 18, 2017 at 01:39 Brent Royal-Gordon <brent@architechies.com> wrote:

On Aug 17, 2017, at 8:20 PM, Xiaodi Wu via swift-evolution < > swift-evolution@swift.org> wrote:

* stderr should go wherever stdin and stdout go. Since it’d be silly for

a function like `print(_:separator:terminator:)` or `
readLine(strippingNewline:)` to live anywhere but the standard library,
then it stands to reason that the stderr version should also live in the
standard library.

FWIW, FileHandle.standardInput, FileHandle.standardError,
FileHandle.standardOutput, and FileHandle.nullDevice all live in Foundation.

And, since they're read-only, are fundamentally incompatible with
`print(…to:)`, which requires its `output` parameter to be passed `inout`.

--
Brent Royal-Gordon
Architechies

So that's a good starting point for the discussion.

What, in your opinion, should be the way forward in addressing this
situation? Is there a realistic chance of writing a single comprehensive,
cross-platform API that makes possible currently "complicated or
non-portable things" on both macOS/iOS/tvOS/watchOS and Linux, and
potentially Windows? If so, does that fit within the general category of
filling out the currently sparse Foundation APIs or would that be a matter
for a separate library? In the alternative, is it the right solution to
make dropping down to POSIX marginally easier by re-exporting C APIs under
a unified name, without attempting a single cross-platform API?

···

On Fri, Aug 18, 2017 at 6:55 PM, Greg Parker <gparker@apple.com> wrote:

On Aug 17, 2017, at 5:16 PM, Xiaodi Wu via swift-evolution < > swift-evolution@swift.org> wrote:

On Thu, Aug 17, 2017 at 6:46 PM, Taylor Swift <kelvin13ma@gmail.com> > wrote:

I don’t think the “is this library functionality or standard library
functionality” argument is worth having, but if stdout and stdin are
first-class citizens in the Swift world, so should stderr.

As for bringing Foundation into the discussion, you can’t really talk
about Foundation without also talking about the mountains of problems that
come with the monolith pattern. But that’s a completely different
conversation to be had.

I'm not sure what you're getting at here, but I don't believe you've
addressed my question, which is: it's been firmly decided that I/O belongs
in Foundation, and Foundation does in fact offer such facilities--what is
missing from those facilities, and how can we fill it out?

Lots of I/O functionality is missing from Foundation. Foundation's design
from time immemorial is that generally only relatively simple and
high-level operations are available in Foundation itself, and if you want
to do complicated or non-portable things then you are expected to drop down
to POSIX or other C interfaces. That design works less well in Swift than
it did in Objective-C because Swift's interface with C, especially
low-level C, is often ugly.

Simple example: there is no way to access file information directly via
NSFileHandle. You need to call NSFileHandle.fileDescriptor and pass that to
fstat() or fcntl(). The NSFileHandle API in general is sparse and wholly
inadequate for sophisticated I/O.

Just thought I’d link the previous thread on this topic: https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160704/023927.html\. I had promised to write a proposal for it back then, but never got around to doing it. Now that we’re out of the hectic Swift 3 timeframe, I can write a proposal and implementation for this, since it’s more likely to be accepted.

More broadly, however, I think a “Swiftication” of POSIX or libc is extremely necessary for anyone who’s not working with Foundation–and not everyone is! There are still parts of POSIX that are either 1. not a part of Foundation or 2. part of Foundation/CoreFoundation but not in the open source Swift Foundation or CFLite (and hence not available on Linux). A big part of attracting newer Swift developers a moving away from the “Swift is for apps only” and the in my experience the need to drop down to UnsafePointer or errno just to use this API has been a big turn-off for programmers.

Saagar Jha

···

On Aug 18, 2017, at 04:17, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:

Not “fundamentally” incompatible:

var stderr = FileHandle.standardError
/* Conform FileHandle to TextOutputStream */
print("foo", to: &stderr)

On Fri, Aug 18, 2017 at 01:39 Brent Royal-Gordon <brent@architechies.com <mailto:brent@architechies.com>> wrote:

On Aug 17, 2017, at 8:20 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

* stderr should go wherever stdin and stdout go. Since it’d be silly for a function like `print(_:separator:terminator:)` or `readLine(strippingNewline:)` to live anywhere but the standard library, then it stands to reason that the stderr version should also live in the standard library.

FWIW, FileHandle.standardInput, FileHandle.standardError, FileHandle.standardOutput, and FileHandle.nullDevice all live in Foundation.

And, since they're read-only, are fundamentally incompatible with `print(…to:)`, which requires its `output` parameter to be passed `inout`.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution

For what it’s worth, CoreFoundation appears to support windows
environments, at least for things like path manipulation. However atm it’s
not very relevant as Swift doesn’t run on Windows rn.

···

On Fri, Aug 18, 2017 at 8:09 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Fri, Aug 18, 2017 at 6:55 PM, Greg Parker <gparker@apple.com> wrote:

On Aug 17, 2017, at 5:16 PM, Xiaodi Wu via swift-evolution < >> swift-evolution@swift.org> wrote:

On Thu, Aug 17, 2017 at 6:46 PM, Taylor Swift <kelvin13ma@gmail.com> >> wrote:

I don’t think the “is this library functionality or standard library
functionality” argument is worth having, but if stdout and stdin are
first-class citizens in the Swift world, so should stderr.

As for bringing Foundation into the discussion, you can’t really talk
about Foundation without also talking about the mountains of problems that
come with the monolith pattern. But that’s a completely different
conversation to be had.

I'm not sure what you're getting at here, but I don't believe you've
addressed my question, which is: it's been firmly decided that I/O belongs
in Foundation, and Foundation does in fact offer such facilities--what is
missing from those facilities, and how can we fill it out?

Lots of I/O functionality is missing from Foundation. Foundation's design
from time immemorial is that generally only relatively simple and
high-level operations are available in Foundation itself, and if you want
to do complicated or non-portable things then you are expected to drop down
to POSIX or other C interfaces. That design works less well in Swift than
it did in Objective-C because Swift's interface with C, especially
low-level C, is often ugly.

Simple example: there is no way to access file information directly via
NSFileHandle. You need to call NSFileHandle.fileDescriptor and pass that to
fstat() or fcntl(). The NSFileHandle API in general is sparse and wholly
inadequate for sophisticated I/O.

So that's a good starting point for the discussion.

What, in your opinion, should be the way forward in addressing this
situation? Is there a realistic chance of writing a single comprehensive,
cross-platform API that makes possible currently "complicated or
non-portable things" on both macOS/iOS/tvOS/watchOS and Linux, and
potentially Windows? If so, does that fit within the general category of
filling out the currently sparse Foundation APIs or would that be a matter
for a separate library? In the alternative, is it the right solution to
make dropping down to POSIX marginally easier by re-exporting C APIs under
a unified name, without attempting a single cross-platform API?

If I recall, the CoreFoundation code for Windows is from a long-abandoned
port that no longer works, to the point that it is thought unsuitable as a
basis for a modern port.

···

On Fri, Aug 18, 2017 at 7:39 PM, Taylor Swift <kelvin13ma@gmail.com> wrote:

On Fri, Aug 18, 2017 at 8:09 PM, Xiaodi Wu <xiaodi.wu@gmail.com> wrote:

On Fri, Aug 18, 2017 at 6:55 PM, Greg Parker <gparker@apple.com> wrote:

On Aug 17, 2017, at 5:16 PM, Xiaodi Wu via swift-evolution < >>> swift-evolution@swift.org> wrote:

On Thu, Aug 17, 2017 at 6:46 PM, Taylor Swift <kelvin13ma@gmail.com> >>> wrote:

I don’t think the “is this library functionality or standard library
functionality” argument is worth having, but if stdout and stdin are
first-class citizens in the Swift world, so should stderr.

As for bringing Foundation into the discussion, you can’t really talk
about Foundation without also talking about the mountains of problems that
come with the monolith pattern. But that’s a completely different
conversation to be had.

I'm not sure what you're getting at here, but I don't believe you've
addressed my question, which is: it's been firmly decided that I/O belongs
in Foundation, and Foundation does in fact offer such facilities--what is
missing from those facilities, and how can we fill it out?

Lots of I/O functionality is missing from Foundation. Foundation's
design from time immemorial is that generally only relatively simple and
high-level operations are available in Foundation itself, and if you want
to do complicated or non-portable things then you are expected to drop down
to POSIX or other C interfaces. That design works less well in Swift than
it did in Objective-C because Swift's interface with C, especially
low-level C, is often ugly.

Simple example: there is no way to access file information directly via
NSFileHandle. You need to call NSFileHandle.fileDescriptor and pass that to
fstat() or fcntl(). The NSFileHandle API in general is sparse and wholly
inadequate for sophisticated I/O.

So that's a good starting point for the discussion.

What, in your opinion, should be the way forward in addressing this
situation? Is there a realistic chance of writing a single comprehensive,
cross-platform API that makes possible currently "complicated or
non-portable things" on both macOS/iOS/tvOS/watchOS and Linux, and
potentially Windows? If so, does that fit within the general category of
filling out the currently sparse Foundation APIs or would that be a matter
for a separate library? In the alternative, is it the right solution to
make dropping down to POSIX marginally easier by re-exporting C APIs under
a unified name, without attempting a single cross-platform API?

For what it’s worth, CoreFoundation appears to support windows
environments, at least for things like path manipulation. However atm it’s
not very relevant as Swift doesn’t run on Windows rn.

I don’t think the “is this library functionality or standard library functionality” argument is worth having, but if stdout and stdin are first-class citizens in the Swift world, so should stderr.

As for bringing Foundation into the discussion, you can’t really talk about Foundation without also talking about the mountains of problems that come with the monolith pattern. But that’s a completely different conversation to be had.

I'm not sure what you're getting at here, but I don't believe you've addressed my question, which is: it's been firmly decided that I/O belongs in Foundation, and Foundation does in fact offer such facilities--what is missing from those facilities, and how can we fill it out?

Lots of I/O functionality is missing from Foundation. Foundation's design from time immemorial is that generally only relatively simple and high-level operations are available in Foundation itself, and if you want to do complicated or non-portable things then you are expected to drop down to POSIX or other C interfaces. That design works less well in Swift than it did in Objective-C because Swift's interface with C, especially low-level C, is often ugly.

Simple example: there is no way to access file information directly via NSFileHandle. You need to call NSFileHandle.fileDescriptor and pass that to fstat() or fcntl(). The NSFileHandle API in general is sparse and wholly inadequate for sophisticated I/O.

So that's a good starting point for the discussion.

What, in your opinion, should be the way forward in addressing this situation? Is there a realistic chance of writing a single comprehensive, cross-platform API that makes possible currently "complicated or non-portable things" on both macOS/iOS/tvOS/watchOS and Linux, and potentially Windows? If so, does that fit within the general category of filling out the currently sparse Foundation APIs or would that be a matter for a separate library? In the alternative, is it the right solution to make dropping down to POSIX marginally easier by re-exporting C APIs under a unified name, without attempting a single cross-platform API?

For what it’s worth, CoreFoundation appears to support windows environments, at least for things like path manipulation. However atm it’s not very relevant as Swift doesn’t run on Windows rn.

If I recall, the CoreFoundation code for Windows is from a long-abandoned port that no longer works, to the point that it is thought unsuitable as a basis for a modern port.

I don’t think abandoned is the right term per-se. But the branch of CoreFoundation that is used for the CF inside of swift-corelibs-foundation has moved in a different direction than the one being used for the Windows support (might be better to think of it as a fork that diverged). I would definitely agree that anything of an update for windows support should definitely reach out and work together to see what we can and should update. But as it stands - I don’t think that is fully operational at the current point in time (modulo perhaps Cygwin; which recently had some contributors working on that front).

If a contributor wants to revitalize the windows portion I would be very willing to aide in that endeavor.

···

On Aug 18, 2017, at 5:42 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org> wrote:
On Fri, Aug 18, 2017 at 7:39 PM, Taylor Swift <kelvin13ma@gmail.com <mailto:kelvin13ma@gmail.com>> wrote:
On Fri, Aug 18, 2017 at 8:09 PM, Xiaodi Wu <xiaodi.wu@gmail.com <mailto:xiaodi.wu@gmail.com>> wrote:
On Fri, Aug 18, 2017 at 6:55 PM, Greg Parker <gparker@apple.com <mailto:gparker@apple.com>> wrote:

On Aug 17, 2017, at 5:16 PM, Xiaodi Wu via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:
On Thu, Aug 17, 2017 at 6:46 PM, Taylor Swift <kelvin13ma@gmail.com <mailto:kelvin13ma@gmail.com>> wrote:

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution