POSIX, Foundation, and layering


(Daniel Dunbar) #1

Hi all,

We have started taking some commits to swift-package-manager to incorporate Foundation. The original idea was that we would try and move a lot of our support facilities onto Foundation (https://bugs.swift.org/browse/SR-1005).

However, we have hit a couple issues that have made me want to take a different route:
1. In some cases, our needs (solved by existing code in the POSIX or Utilities modules) don't cleanly match the API Foundation provides. For example, we used the ability to pass arbitrary file descriptors to a subprocess in one place (see https://github.com/apple/swift-package-manager/pull/303) which is trivial with `posix_spawn` but difficult with `NSTask`.
2. In another case, we had to go to substantial effort to workaround the cross-platform bridging of Foundation. See https://github.com/apple/swift-package-manager/pull/333.

Based on this, I am proposing the following direction:

1. We keep our POSIX module. The intent of this module should be to provide "Swifty" APIs to access portable, POSIX facilities. "Swifty" here does not mean completely changing the API to work more conveniently, it just means translating the raw POSIX API into something which is easy to call from Swift and handles errors in a "native" fashion. Almost all APIs in this module should match the underlying POSIX name, and `man <name>` should describe the semantics even if the API is called differently. This module should only depend on `libc` and hopefully one day something like this module can be part of the standard library.

2. We always use `Foundation` when its APIs are the right tool for the job, but if not (because our behavior is in some places fairly low-level, with a neat mapping to POSIX) then we continue to build on the POSIX module. We shouldn't re-implement any significant system provided behavior, we should only be choosing between using a `POSIX` or a `Foundation` API for a task. If the `Foundation` API exists, but isn't implemented yet, then we should work to implement that API in `swift-corelibs-foundation`.

3. We introduce/rename a new `Basic` module out of `Utility`. This matches a convention used in Clang, Swift, and llbuild for the "base support module". This module will define the "convenient" APIs for system services, shared ADTs, etc. and will sit on top of `libc`, `POSIX`, and `Foundation`. We should try to limit cross-platform adaption code to this layer -- if we need a `Foundation` API, but it is not yet implemented in such a way that use of that API is *syntactically* cross-platform (e.g., it needs different bridging behavior) then we should provide an abstraction in `Basic` that hides that divergence. The goal here is that any code above `Basic` can be written without fear of cross-platform incompatibility (which is a serious burden on developer productivity).

4. We should incrementally audit our existing API usage to fit into this conceptual model.

Thoughts?

- Daniel


(Ankit Agarwal) #2

3. We introduce/rename a new `Basic` module out of `Utility`. This matches

a convention used in Clang, Swift, and llbuild for the "base support
module". This module will define the "convenient" APIs for system services,
shared ADTs, etc. and will sit on top of `libc`, `POSIX`, and `Foundation`.
We should try to limit cross-platform adaption code to this layer -- if we
need a `Foundation` API, but it is not yet implemented in such a way that
use of that API is *syntactically* cross-platform (e.g., it needs different
bridging behavior) then we should provide an abstraction in `Basic` that
hides that divergence. The goal here is that any code above `Basic` can be
written without fear of cross-platform incompatibility (which is a serious
burden on developer productivity).

I agree with all the points. Do you think its better to create new module
Basic off of Utility or just rename Utility to Basic? I think it might get
confusing to decide where something goes if there are two such modules.
Though if definition of Basic is to provide cross-platform I'd imagine
Utility contains things like handy extensions to stdlib types and Basic
also depends on Utility.

···

--
Ankit


(Daniel Dunbar) #3

3. We introduce/rename a new `Basic` module out of `Utility`. This matches a convention used in Clang, Swift, and llbuild for the "base support module". This module will define the "convenient" APIs for system services, shared ADTs, etc. and will sit on top of `libc`, `POSIX`, and `Foundation`. We should try to limit cross-platform adaption code to this layer -- if we need a `Foundation` API, but it is not yet implemented in such a way that use of that API is *syntactically* cross-platform (e.g., it needs different bridging behavior) then we should provide an abstraction in `Basic` that hides that divergence. The goal here is that any code above `Basic` can be written without fear of cross-platform incompatibility (which is a serious burden on developer productivity).

I agree with all the points. Do you think its better to create new module Basic off of Utility or just rename Utility to Basic? I think it might get confusing to decide where something goes if there are two such modules. Though if definition of Basic is to provide cross-platform I'd imagine Utility contains things like handy extensions to stdlib types and Basic also depends on Utility.

We should basically rename Utility to Basic. How we do that rename is an open question... my inclination is to move each API individually to the "right" place until Utility is empty, then delete it. As part of moving each API we should decide where the right home for it is (POSIX, Basic, or other as-yet-to-be-defined modules).

- Daniel

···

On May 12, 2016, at 9:15 AM, Ankit Agarwal <ankit@ankit.im> wrote:

--
Ankit