# Pitch: Move URLSession to new FoundationNetworking module **URL:** \ **Category:** Core Libraries\ **Created:** [June 26, 2018, 6:19pm UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002 "2018-06-26T18:19:24Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Tony\_Parker](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/tony_parker/32/273_2.png) [@Tony\_Parker](https://forums.swift.org/u/Tony_Parker)\ **Post date:** [June 26, 2018, 6:19pm UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/1 "2018-06-26T18:19:25Z")
Hi everyone, As we continue to push forward on completing swift-corelibs-foundation, I would like to propose a structural change to help us meet our goals: move `URLSession` and related types into a new library. Some of the feedback we've received from the SwiftNIO team is that Foundation's dependency on `libcurl` brings in too many other libraries, some of which conflict with the ones that SwiftNIO needs to use. Additionally, we think it might be interesting to explore implementation of Foundation networking on top of SwiftNIO itself. Therefore, I propose a moving the networking types (except `URL` and `URLComponents`) into another library, with a working name of `swift-corelibs-foundation-networking`. This library could still be part of the same git repository on GitHub or we could create a new one. This change means that there would be a clear dependency graph for the 3 projects: ```swift swift-corelibs-foundation-networking | SwiftNIO | swift-corelibs-foundation ``` Benefits: - SwiftNIO would be able to use core Foundation types. We are separately exploring improvements to types like `Data` with feedback from the SwiftNIO team. - `swift-corelibs-foundation-networking` could use SwiftPM to build itself. This is currently not possible for Foundation itself for a variety of dependency-related reasons. - Other projects that wish to maintain a minimum set of dependencies and do not need networking support can reduce their footprint. Challenges: - A new `import` statement would be needed for source code which uses `URLSession`. For example, `import Foundation.Networking` or `import FoundationNetworking`. We would need to decide if it is acceptable to deviate from the Darwin platform here, or if we need to introduce some kind of compatibility shim there as well. On Darwin Foundation must continue to include `URLSession` for source compatibility. - Due to the above, moving `URLSession` may introduce a source break for Linux clients. - Restructuring the projects will involve work for build scripts and infrastructure changes to CI. What are your thoughts? ---
**Author:** ![nuclearace](https://avatars.discourse-cdn.com/v4/letter/n/51bf81/32.png) [@nuclearace](https://forums.swift.org/u/nuclearace)\ **Post date:** [June 26, 2018, 7:21pm UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/2 "2018-06-26T19:21:43Z")
> [@Tony\_Parker](#): > > if it is acceptable to deviate from the Darwin platform I feel this is dangerous territory. This is going to set a precedent that open source Foundation doesn't have to evolve with Darwin Foundation. That being said, I think it's actually a good thing, I view the current connection as a bit of a burden on advancing Foundation and other "core" libraries. And if this is how we start having more "core" libraries that aren't tied to Darwin Foundation, I think it's a good idea. I think the name for the new module is a bit weird though. Current convention in Swift is to import the entire module. Something like `Foundation.Networking` looks like we're entering a new convention where we have submodules (again, I'm not opposed, it's just a new precedent.) I also don't much like the name `FoundationNetworking`, mostly because of the `Foundation` bit. It's a bit of a shame we can't use `Networking` now. ---
**Author:** ![Tony\_Parker](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/tony_parker/32/273_2.png) [@Tony\_Parker](https://forums.swift.org/u/Tony_Parker)\ **Post date:** [June 26, 2018, 7:43pm UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/3 "2018-06-26T19:43:27Z")
I agree about the deviation not being ideal, but I did want to raise it for discussion regardless. If we went this way, it may be possible for us to add `FoundationNetworking` as a module on Darwin as well, but like I said I think it would be empty since we can't break all the source code that currently just imports `Foundation` only. ---
**Author:** ![millenomi](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/millenomi/32/2922_2.png) [@millenomi](https://forums.swift.org/u/millenomi)\ **Post date:** [June 26, 2018, 8:23pm UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/4 "2018-06-26T20:23:37Z")
We discussed this, but do not have currently the support from the language to create submodules in this way. (Submodule syntax is probably something that would need extensive Evolution discussion first 🙁). **EDIT** : To be clear: `Foundation.Networking` as a name is what I'm referring to here; but we are open to figuring out how to make it happen if we feel strongly that's the way to go. ---
**Author:** ![spevans](https://avatars.discourse-cdn.com/v4/letter/s/59ef9b/32.png) [@spevans](https://forums.swift.org/u/spevans)\ **Post date:** [June 26, 2018, 8:27pm UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/5 "2018-06-26T20:27:19Z")
I can see a major benefit of moving it into its own repo and being built by SwiftPM is that updates could be released more frequently in the same way `swiftNIO` is updated. However I think that splitting up `Foundation` on one platform and making it work differently to Darwin is not ideal since its main benefit is in being the same everywhere. Would it be possible to change `URLSession` to use `SwiftNIO` instead of `curl` inside the current `swift-corelibs-foundation` and then once this is done see if it possible to make `Foundation` buildable by both its current method and also `SwiftPM`. This would hopefully make it possible to do frequent releases for `Foundation` on Linux in the same way `SwiftNIO` is released. If its found that its not possible to build in this way then it could be split out into its own repo. As to moving from `curl` to `SwiftNIO` I think this is a great idea as it also simplifies issues with static linking `Foundation` as `libcurl` requires a large number of extra libraries. ---
**Author:** ![benrimmington](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/benrimmington/32/695_2.png) [@benrimmington](https://forums.swift.org/u/benrimmington)\ **Post date:** [June 26, 2018, 9:55pm UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/6 "2018-06-26T21:55:23Z")
It might be confusing to have `Foundation.Networking` and a similarly named [`Network`](https://developer.apple.com/documentation/network) framework. ---
**Author:** ![pvieito](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/pvieito/32/33613_2.png) [@pvieito](https://forums.swift.org/u/pvieito)\ **Post date:** [June 26, 2018, 11:02pm UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/8 "2018-06-26T23:02:04Z")
I think we should not deviate from the Darwin implementation. What we could do is to have `Foundation.Base` (better name required) and `Foundation.Networking` modules and a shim `Foundation` module that would reexport both of them. This would maintain cross-platform source compatibility while splitting Foundation in sub-modules. ---
**Author:** ![jawbroken](https://avatars.discourse-cdn.com/v4/letter/j/cc9497/32.png) [@jawbroken](https://forums.swift.org/u/jawbroken)\ **Post date:** [June 27, 2018, 2:28am UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/9 "2018-06-27T02:28:51Z")
I can't judge if it's worth papering over this specific problem with this kind of workaround ahead of a solution to the general problem here. I hope there is a solution to the general problem soon, though (e.g. submodules). ---
**Author:** ![anon27714269](https://avatars.discourse-cdn.com/v4/letter/a/e8c25b/32.png) [@anon27714269](https://forums.swift.org/u/anon27714269)\ **Post date:** [June 27, 2018, 5:57am UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/10 "2018-06-27T05:57:14Z")
I'm strongly opposed to breaking Darwin compatibility. ---
**Author:** ![johannesweiss](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johannesweiss/32/38183_2.png) [@johannesweiss](https://forums.swift.org/u/johannesweiss)\ **Post date:** [June 27, 2018, 3:06pm UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/11 "2018-06-27T15:06:21Z")
Thanks Tony for the proposal and all the input received by others. As Tony points out, SwiftNIO and other server related projects struggle a lot with the fact that `Foundation` does quite a few dependencies through `libcurl` which depends on a `libssl` (usually OpenSSL). Another issue with using the platform's `libcurl` is that `URLSession` in `swift-corelibs-foundation` does support HTTP/2 if and only if the distribution's `libcurl` was compiled with HTTP/2 support (which it AFAIK isn't on Ubuntu 14.04 and 16.04). So from our point of view there's a number of concrete problems with the current approach: - impossible to link another (newer) libcurl/libssl because of 'symbol clash' on Linux - too many dependencies on the system in Foundation (so static binaries with anything that uses Foundation are not possible right now with the standard Swift distributions) - development velocity of `URLSession` rather slow because of the coupling with the Swift distribution - unclear path to supporting HTTP/2 on all platforms (as the system's libcurl is linked which might or might not be compiled with HTTP/2 support) Having `URLSession` in its own module that is distributed through SwiftPM would address most of those issues so I think it is a clear win. Yes, there's a similar issue with the dependency on `libxml2` for `XMLParser` but let's deal with one issue at the time. Obviously there are drawbacks too: Others have pointed out already that separating `URLSession` from Foundation comes source compatibility issues but also Data's [`init(contentsOf url: URL, ...)`](https://developer.apple.com/documentation/foundation/nsdata/1407864-init) might be a problem... @Tony_Parker is there any way we could move Darwin towards (soft) requiring a separate import for `URLSession` too? Of course we need to stay compatible but maybe we could have a warning issued if `URLSession` is used without also importing the new URLSession module? ---
**Author:** ![jrose](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/jrose/32/408_2.png) [@jrose](https://forums.swift.org/u/jrose)\ **Post date:** [June 27, 2018, 4:15pm UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/12 "2018-06-27T16:15:22Z")
I'd like to ask people to worry less about spelling for now, but `Foundation.Networking` is probably not a viable option. As noted by Lily, there would be a huge evolution discussion on what this syntax means and what we can do with it; apart from [submodules being a loaded term](https://forums.swift.org/t/submodule-is-an-ambiguous-term/12484), the closest thing we have today (Clang submodules) wouldn't allow the "FoundationNetworking-depends-on-SwiftNIO-depends-on-Foundation" setup that Tony floated. ---
**Author:** ![anon27714269](https://avatars.discourse-cdn.com/v4/letter/a/e8c25b/32.png) [@anon27714269](https://forums.swift.org/u/anon27714269)\ **Post date:** [June 27, 2018, 4:27pm UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/13 "2018-06-27T16:27:22Z")
It may be infamous but imo the right solution would be to fix the underlying problem in Linux library handling instead of breaking foundation for all macOS/iOS users... ---
**Author:** ![Tony\_Parker](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/tony_parker/32/273_2.png) [@Tony\_Parker](https://forums.swift.org/u/Tony_Parker)\ **Post date:** [June 27, 2018, 6:33pm UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/14 "2018-06-27T18:33:23Z")
To be clear, if we put URLSession in another module, then we would provide some form of that module for Darwin too so that source code written for iOS/macOS would remain compatible. However: - it would have to be introduced in some future OS update - On Darwin it would likely import nothing, since we can't remove URLSession from `import Foundation` ---
**Author:** ![anon27714269](https://avatars.discourse-cdn.com/v4/letter/a/e8c25b/32.png) [@anon27714269](https://forums.swift.org/u/anon27714269)\ **Post date:** [June 27, 2018, 6:51pm UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/15 "2018-06-27T18:51:21Z")
I'm very strongly against that kind of thinking. I don't want to get Linux dominated in a Apple centric language... ---
**Author:** ![nuclearace](https://avatars.discourse-cdn.com/v4/letter/n/51bf81/32.png) [@nuclearace](https://forums.swift.org/u/nuclearace)\ **Post date:** [June 27, 2018, 7:03pm UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/16 "2018-06-27T19:03:39Z")
> [@anon27714269](#): > > in a Apple centric language... Don't think of Swift as an Apple centric language. Swift has vast potential outside of Darwin platforms. > [@Tony\_Parker](#): > > To be clear, if we put URLSession in another module, then we would provide some form of that module for So is the idea that on Darwin `import Foundation` wouldn't import URLSession in Swift? Because if we have a disconnect between how we get URLSession on Darwin vs anything else, I'm firmly against that. ---
**Author:** ![amosavian](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/amosavian/32/2253_2.png) [@amosavian](https://forums.swift.org/u/amosavian)\ **Post date:** [June 28, 2018, 4:18am UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/17 "2018-06-28T04:18:26Z")
@Tony_Parker Swift’s Foundation lacks some classes implemented in Darwin’s Foundation (e.g. `NSFileCoordination` that I think can be implemented with `fcntl()` btw), thus moving `URLSession` is reasonable now that Swift’s Foundation did not become stable yet. Second, Swift’s `URLSession` lacks some critical functionalities like background session or resuming download tasks. I think no one expects complete compatibility between Swift’s and Darwin’s implementation. 😀 Third, `libcurl` dependency is an implementation detail. I think reasonable to think about switching to `SwiftNIO` in future. Having `URLSession` in a separate framework allows this switch without causing cyclic reference. The only missing part for that switch is FTP support I think, which you are free to reuse my ftp client implementation based on `URLSessionStreamTask` 😍. Finally I vote for submodule approach. 😜 ---
**Author:** ![Pushkar\_N\_Kulkarni](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/pushkar_n_kulkarni/32/1235_2.png) [@Pushkar\_N\_Kulkarni](https://forums.swift.org/u/Pushkar_N_Kulkarni)\ **Post date:** [June 28, 2018, 6:52am UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/18 "2018-06-28T06:52:16Z")
This is a great discussion. Thanks @Tony_Parker and @johannesweiss for clearly describing the rationale, merits and challenges of this proposal. Just adding my two cents here. I think URLSession will benefit a lot from this separation. I've had a decent amount of experience with `URLSession`, `libcurl` and `SwiftNIO` and I'm inclined towards thinking that a `SwiftNIO` based implementation of `URLSession` would be easier to develop/maintain, **look** better and probably perform better too. I'd like us to clearly define what classes from `Foundation` would be a part of the so called `FoundationNetworking`. Also regarding backward compatibility, if the shim approach does avoid breaking client code on Linux, I'd vote for that. That's the best case. Alternatively, we could start the shim approach and present a clear roadmap about the transition to something like `import FoundationNetworking`. ---
**Author:** ![anon27714269](https://avatars.discourse-cdn.com/v4/letter/a/e8c25b/32.png) [@anon27714269](https://forums.swift.org/u/anon27714269)\ **Post date:** [June 28, 2018, 7:01am UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/19 "2018-06-28T07:01:23Z")
Hello? The leading design for Foundation is Apple. You linux guys can't just go and decide to break it for us mac users... ---
**Author:** ![Pushkar\_N\_Kulkarni](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/pushkar_n_kulkarni/32/1235_2.png) [@Pushkar\_N\_Kulkarni](https://forums.swift.org/u/Pushkar_N_Kulkarni)\ **Post date:** [June 28, 2018, 7:09am UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/20 "2018-06-28T07:09:27Z")
I'm sure the compatibility problem can be and should be worked out. ---
**Author:** ![pvieito](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/pvieito/32/33613_2.png) [@pvieito](https://forums.swift.org/u/pvieito)\ **Post date:** [June 28, 2018, 7:12am UTC](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002/21 "2018-06-28T07:12:11Z")
> [@anon27714269](#): > > You linux guys can't just go and decide to break it for us mac users... I don’t understand what you mean by that. The Darwin (macOS/iOS) implementation of Foundation would be exactly the same as it is now. I mean, its even a completely different codebase. On Linux and other platforms, for me, the most important thing would be that `import Foundation` continues to import all of Foundation (base and networking) and if a given module wants only the base Foundation it could import specifically only that part (`import FoundationBase` / `import Foundation.Base`). [Next page](https://forums.swift.org/t/pitch-move-urlsession-to-new-foundationnetworking-module/14002.md?page=2)