# Announcing Swift HTTP Types **URL:** \ **Category:** Development\ **Tags:** http\ **Created:** [July 10, 2023, 6:23pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088 "2023-07-10T18:23:49Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Guoye](https://avatars.discourse-cdn.com/v4/letter/g/c77e96/32.png) [@Guoye](https://forums.swift.org/u/Guoye)\ **Post date:** [July 10, 2023, 6:23pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/1 "2023-07-10T18:23:49Z")
We're happy to introduce a Swift package for shared HTTP currency types. Details are on the [Swift.org](http://Swift.org) blog: [Introducing Swift HTTP Types](https://www.swift.org/blog/introducing-swift-http-types/) ---
**Author:** ![johannesweiss](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/johannesweiss/32/38183_2.png) [@johannesweiss](https://forums.swift.org/u/johannesweiss)\ **Post date:** [July 10, 2023, 7:51pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/2 "2023-07-10T19:51:22Z")
đź‘Ź super cool, thanks everybody who was involved!! ---
**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\ **Post date:** [July 10, 2023, 9:32pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/3 "2023-07-10T21:32:33Z")
re: Integrating with SwiftNIO (i wish these headings had anchors), is there any reason why the SwiftNIO integration is going in the [swift-nio-extras](https://github.com/apple/swift-nio-extras) package and not swift-nio itself? by my count there would now exist _three_ independently incrementing version tuples in an NIO-based build tree that uses this package, and that seems needlessly contrived. ---
**Author:** ![lukasa](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/lukasa/32/1232_2.png) [@lukasa](https://forums.swift.org/u/lukasa)\ **Post date:** [July 10, 2023, 9:54pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/4 "2023-07-10T21:54:02Z")
Yes, adding the dependency to swift-nio is unnecessary, forcing users to add this to their build graph when they only require NIOCore, for example. Keeping the core of SwiftNIO as dependency-free as possible is a goal here. Note that the long-term goal is that we will replace NIO's types entirely, at which point we'll revisit the dependency graph. ---
**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\ **Post date:** [July 10, 2023, 10:36pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/5 "2023-07-10T22:36:53Z")
> [@lukasa](#): > > Yes, adding the dependency to swift-nio is unnecessary, forcing users to add this to their build graph when they only require NIOCore, for example. Keeping the core of SwiftNIO as dependency-free as possible is a goal here. in my experience there exists a point at which “don’t include what you don’t use” becomes more harmful than it is beneficial. for _modules_ this threshold is rather distant — for example there are almost always huge dividends to be reaped by cleaving off a Foundation linkage even if it makes the intra-package dependency graph more complex. and another reason for this is because modules within the same package are unversioned with respect to one another, so the costs of breaking up large modules into many smaller modules is low. so, over time, i have gravitated towards package layouts that have many targets with many dependency edges between them. because this kind of atomized structure works well for modules. for _packages_ i have learned the opposite lesson. dividing up packages into many federated components is almost never worthwhile. benefits of federated packages: - SPM clones fewer repositories for users who are only using the “core” package - can upgrade small packages one-by-one instead of performing a huge migration - easier to document what a small package “does” drawbacks of federated packages: - nearly impossible to work on features that require changes to more than one package at a time - cannot possibly test every combination of pin resolutions - developers tend to waste time over-engineering upstream packages for imaginary use cases instead of allowing development to be driven by the needs of the downstream packages - developers end up having to coordinate “joint” PRs that all need to be merged separately into a constellation of source control repositories. - very hard to look back and understand the history of many related packages as a unit as for the first two benefits, i find that they are not really that valuable in practice. - _SPM clones fewer repositories for users who are only using the “core” package_ - _can upgrade small packages one-by-one instead of performing a huge migration_ at a higher level, i think that a big reason for this is because SPM uses the word _package_ to refer to what other languages think of as a _version-controlled repository_, and we do not have the ability to organize a project into many packages that still share a common repo and version sequence. and i think this is a problem unique to swift, because of a peculiar limitation of its preferred build system. ---
**Author:** ![stackotter](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/stackotter/32/20670_2.png) [@stackotter](https://forums.swift.org/u/stackotter)\ **Post date:** [July 11, 2023, 2:04am UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/6 "2023-07-11T02:04:27Z")
> [@taylorswift](#): > > for _packages_ i have learned the opposite lesson. dividing up packages into many federated components is almost never worthwhile. I completely agree. I have personally tried splitting a project into separate repos before, complete development nightmare, and that’s just when I was working on it alone. Changes that affect multiple packages are super annoying to do without ending up with some possible way someone could end up with incompatible versions. Unless the parts are actually just independent packages, such as a maths package and a graphics package that depends on the maths package, I just find the multi-repo architecture unwieldy. When a repo is called `project-extras`, imo it’s going to be getting developed in tandem with the base project and might as well be in the main repo, or split into independent projects with clear purposes. I can see that that’s probably just a matter of opinion, but I can say that I’ve always avoided NIO because of it feeling unwieldy (can’t exactly explain why in a concise way) ---
**Author:** ![Karl](https://avatars.discourse-cdn.com/v4/letter/k/34f0e0/32.png) [@Karl](https://forums.swift.org/u/Karl)\ **Post date:** [July 11, 2023, 8:19am UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/7 "2023-07-11T08:19:34Z")
This looks like a very interesting package! One thing: I noticed that this library [uses `CoreFoundation` directly](https://github.com/apple/swift-http-types/blob/39661f4ca82db8ad050cc060a471c8e2c542b24f/Sources/HTTPTypesFoundation/HTTPRequest%2BURL.swift#L15). IIRC that is not portable; CF is available on Darwin platforms and Linux, but is deliberately omitted from other platforms (e.g. Windows). > [@Swift 5.4 on Windows: error: missing required modules: 'CFXMLInterface', 'CoreFoundation'](https://forums.swift.org/t/swift-5-4-on-windows-error-missing-required-modules-cfxmlinterface-corefoundation/48575/2): > > CoreFoundation is not meant to be public, and the Windows port has generally been far more aggressive about preventing leakage of internal interfaces and encouraging cross-compilation. ---
**Author:** ![lukasa](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/lukasa/32/1232_2.png) [@lukasa](https://forums.swift.org/u/lukasa)\ **Post date:** [July 11, 2023, 11:46am UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/8 "2023-07-11T11:46:00Z")
To the core of your feedback, sure, we can retarget the PR towards core NIO. It'll delay our ability to merge until we drop 5.6 support, but that's potentially a worthwhile trade. As to your broader concerns, this is generally all accurate, but the reason SwiftNIO has historically had federated packages is solely about dependency management in older Swift versions. Specifically: - swift-nio-ssl originally had a dependency on the system OpenSSL. That was unacceptable in the core library, where users should not have build-time failures because of a missing system library dependency. It now carries a copy of BoringSSL. - swift-nio-http2 originally had a dependency on the system copy of nghttp2. - swift-nio-ssh has a dependency on swift-crypto, which forces a build and link of a copy of BoringSSL, which would be the second copy if we also had SSL in here. It also forces a breaking change as SSH has minimum deployment targets, which NIO core does not. - swift-nio-extras depends on the system zlib. All of this seems like overkill now for ssl, http2, and extars, but it's also worth adding the most current Swift version at the time of their release: - ssl: 4.0 - http2: 4.1 - extras: 4.1 None of these SwiftPM versions had support for conditional target dependencies (came in 5.3) or target based dependency resolution (came in 5.2), so adding them to the core NIO repo would have unnecessarily forced their system dependencies on all users. Our strong preference at the time (and now) was to keep all of these in a single repository, and if we were developing NIO from scratch in 2023 we would do so. But we cannot escape our past. Once we had split them out, it minimises churn on the ecosystem to keep them split. We could forcibly-deprecate the ecosystem repositories and push everyone to move to the core NIO repos, but we'd need to continue shipping bugfixes into the ecosystem repos for quite some time. It is possible that at our next breaking release we'll do exactly that, but for now we're trying to maximise compatibility and stability for the Swift on Server ecosystem. As a final note, I don't think NIO necessarily gets more approachable in a world where the satellite packages are all merged into the tree. NIO's `Sources` directory looks [like this](https://github.com/apple/swift-nio/tree/main/Sources). NIO has 11 products and 43 targets in the core project alone. Adding SSL, SSH, HTTP2, and Extras would push us up to 20 products and 75 targets. That's a pretty unwieldy project right there. ---
**Author:** ![taylorswift](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/taylorswift/32/43413_2.png) [@taylorswift](https://forums.swift.org/u/taylorswift)\ **Post date:** [July 11, 2023, 9:53pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/9 "2023-07-11T21:53:39Z")
> [@lukasa](#): > > Our strong preference at the time (and now) was to keep all of these in a single repository, and if we were developing NIO from scratch in 2023 we would do so. But we cannot escape our past. > > Once we had split them out, it minimises churn on the ecosystem to keep them split. We could forcibly-deprecate the ecosystem repositories and push everyone to move to the core NIO repos, but we'd need to continue shipping bugfixes into the ecosystem repos for quite some time. It is possible that at our next breaking release we'll do exactly that, but for now we're trying to maximise compatibility and stability for the Swift on Server ecosystem. thanks for all the explanation! in my view, this is something that needs to be done eventually, and the best time to do it is the present, while Swift on Server is still relatively small and doesn’t have much of an “ecosystem” grown around it yet. i think that if we are successful and Swift on Server gains traction beyond its current (and, let’s be honest, niche) user base, then compatibility concerns will be much greater in the future than they are now. > [@lukasa](#): > > As a final note, I don't think NIO necessarily gets more approachable in a world where the satellite packages are all merged into the tree. NIO's `Sources` directory looks [like this](https://github.com/apple/swift-nio/tree/main/Sources). NIO has 11 products and 43 targets in the core project alone. Adding SSL, SSH, HTTP2, and Extras would push us up to 20 products and 75 targets. That's a pretty unwieldy project right there. i wrote more about this topic [here](https://forums.swift.org/t/strategies-for-organizing-navigating-very-large-swiftpm-packages/66003) if you are interested, but to summarize: i personally find it amusing that a project with 20 products and 75 targets is considered “large” in the swift world. there are plenty of not-so-good reasons for this (swift developers’ penchant for writing very large modules, `@inlinable` fatigue, no SPM support for multiple package manifests, no multitarget DocC support, et cetera et cetera…) but ultimately i think that trying to adapt by splitting up packages (or keeping existing projects federalized) is not the right approach. it is better to tolerate large manifests in the medium term and wait for SPM to gain better support for large projects. because, as you yourself have observed, it is very, very difficult to stitch projects back together after they’ve been flashpointed into a dozen timelines. by the way, re: > [@lukasa](#): > > It'll delay our ability to merge until we drop 5.6 support, but that's potentially a worthwhile trade. my longstanding assumption (which may be wrong!) is that most people doing Swift on Server today are deploying to linux machines, and are probably manually installing the toolchain and runtime libraries on the cloud instances. after a while, i learned that maintaining a long toolchain support window for this type of project is a waste of time, it was way easier for me to just SSH into the cloud instance and upgrade the toolchain, which allows me to only target the latest swift release. i imagine this is quite different from swift on Apple platforms where long toolchain windows are important to a lot of people. but, different world, different incentives… ---
**Author:** ![Guoye](https://avatars.discourse-cdn.com/v4/letter/g/c77e96/32.png) [@Guoye](https://forums.swift.org/u/Guoye)\ **Post date:** [July 12, 2023, 12:24am UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/10 "2023-07-12T00:24:42Z")
The use of CFURL is unfortunate but necessary since Foundation URL / URLComponents implement the RFC and they are incompatible with WHATWG URL in subtle ways (e.g. percent encoding "|" characters). CFURL is more lenient and preserves the raw bytes given to it. The hope is that Foundation URL will one day gain WHATWG URL parsing mode. ---
**Author:** ![Karl](https://avatars.discourse-cdn.com/v4/letter/k/34f0e0/32.png) [@Karl](https://forums.swift.org/u/Karl)\ **Post date:** [July 12, 2023, 11:54am UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/11 "2023-07-12T11:54:24Z")
Ah, I see. We don't need WHATWG-compatible parsing to create HTTP requests from Foundation URLs, though. If you have a Foundation.URL value, the components will already be encoded to Foundation's satisfaction. All we need to create a request are the raw, percent-encoded values of the URL components (which is what is currently being extracted via CF). Fortunately, Foundation recently added APIs to provide that data, in the form of [the `path(percentEncoded: Bool)` family of methods](https://developer.apple.com/documentation/foundation/url/3988473-path). Unfortunately, `URLComponents` can't, in general, be used to get the percent-encoded path because of [[SR-15512] URLComponents percent-decodes paths which contain a semicolon · Issue #3352 · apple/swift-corelibs-foundation · GitHub](https://github.com/apple/swift-corelibs-foundation/issues/3352). This does however expose another issue, which is that `swift-corelibs-foundation` hasn't been updated in many, many years, so the `path(percentEncoded: Bool)` family of methods are only available on Darwin platforms. But I don't think we need a full codebase sync just to implement these particular methods - they should be fairly straightforward to implement on top of what's already in corelibs-foundation. If we implement those methods in corelibs, we can drop the use of CF in this package, making it portable to non Darwin/Linux systems. I think that's important for a library that exposes currency types. ---
**Author:** ![Guoye](https://avatars.discourse-cdn.com/v4/letter/g/c77e96/32.png) [@Guoye](https://forums.swift.org/u/Guoye)\ **Post date:** [July 12, 2023, 7:46pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/12 "2023-07-12T19:46:36Z")
> If you have a Foundation.URL value, the components will already be encoded to Foundation's satisfaction. This is not the case. `CFURLCreateAbsoluteURLWithBytes` and `CFURLGetBytes` are escape hatches which allow clients (e.g. WebKit) to sneak strings which aren't considered valid by URL past URL. > All we need to create a request are the raw, percent-encoded values of the URL components (which is what is currently being extracted via CF). We do need percent-encoded values, but not how URL currently does it. ```swift let cfURL = CFURLCreateAbsoluteURLWithBytes(kCFAllocatorDefault, "https://example.com?q=1|2", 25, kCFStringEncodingASCII, nil, false)! CFURLGetByteRangeForComponent(cfURL, .query, nil) // range of q=1|2 (cfURL as URL).query(percentEncoded: true) // q=1%7C2 ``` ```swift let url = URL(string: "https://example.com?q=1|2")! url.query(percentEncoded: true) // q=1%7C2 ``` We don't want the CF dependency either, but there are websites that depend on this today. ---
**Author:** ![Karl](https://avatars.discourse-cdn.com/v4/letter/k/34f0e0/32.png) [@Karl](https://forums.swift.org/u/Karl)\ **Post date:** [July 14, 2023, 8:45am UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/13 "2023-07-14T08:45:32Z")
I appreciate that you guys also don't want the CF dependency; I'm sure that was nobody's first choice. I've filed a GitHub issue to track it: [Drop CF dependency · Issue #10 · apple/swift-http-types · GitHub](https://github.com/apple/swift-http-types/issues/10) > [@Guoye](#): > > This is not the case. `CFURLCreateAbsoluteURLWithBytes` and `CFURLGetBytes` are escape hatches which allow clients (e.g. WebKit) to sneak strings which aren't considered valid by URL past URL. Wow! TIL. That's very subtle, and I don't think it is mentioned in [the documentation](https://developer.apple.com/documentation/corefoundation/1542795-cfurlcreateabsoluteurlwithbytes). For construction, it seems that we have an NSURL initialiser which forwards to CFURLCAUWB ([documentation](https://developer.apple.com/documentation/foundation/nsurl/1412404-absoluteurl) unfortunately is empty). NSURL is available on all platforms. ```swift NSURL(absoluteURLWithDataRepresentation: Data("https://example.com?q=1|2".utf8), relativeTo: nil) as URL $R34: Foundation.URL = "https://example.com?q=1|2" ^ ``` As for the component getters, it seems the behaviour can vary - `.host` does not perform any escaping but `.path` and `.query` do. ```swift let url = (NSURL(absoluteURLWithDataRepresentation: Data("https://ex ample.com?q=1 2".utf8), relativeTo: nil) as URL) url.host(percentEncoded: true) $R54: String? = "ex ample.com" ^^ url.query(percentEncoded: true) $R55: String? = "q=1%202" ^^^ ``` When I suggested that the Foundation team add these APIs, it was always the intention that they give the raw "unadulterated" components; I never imagined the getter would actually _add_ escaping and it is possible that it was unintentional. > [@Foundation URL Improvements](https://forums.swift.org/t/foundation-url-improvements/54057/7): > > I think the biggest, easiest improvements would be to: > > 1. Add `.percentEncodedX` properties to `URL`, so that people don't need to go through `URLComponents` to avoid URL's automatic percent-decoding. Automatic decoding is really, really, really bad and currently there is no way around it (even converting `URL` -\> `URLComponents` will automatically percent-decode under certain circumstances, and can corrupt the path). > > 2. Add a method which provides the URL's string buffer, with ranges of all of the components. This would be seriously useful for `URL` -\> `WebURL` conversion -- currently, because there are differences between the standards (as well as bugs in URL), we need to request each component individually to ensure the converted URL points to the same location, and each component potentially allocates a `String`. I would suggest that either: 1. We implement that behaviour in SCF (where `percentEncoded: true` returns the raw component), and Darwin changes its implementation to do the same. That means we can use official, public APIs everywhere 🙂 2. We add some kind of SPI which exposes these CF methods, whether via an "official" `@_spi` or just some underscored methods. We don't need this on Darwin Foundation, which _can_ fall back to CF, so it can be limited to the open-source SCF. Anyway, we can continue this discussion on the GitHub issue, if you prefer. ---
**Author:** ![yvsong](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/yvsong/32/30774_2.png) [@yvsong](https://forums.swift.org/u/yvsong)\ **Post date:** [July 14, 2023, 4:08pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/14 "2023-07-14T16:08:02Z")
Is the Retry-After header field missing in HTTPField.Name? ---
**Author:** ![Jon\_Shier](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/jon_shier/32/921_2.png) [@Jon\_Shier](https://forums.swift.org/u/Jon_Shier)\ **Post date:** [July 14, 2023, 5:04pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/15 "2023-07-14T17:04:29Z")
For things like that it's probably best to just create a PR to add it. ---
**Author:** ![Guoye](https://avatars.discourse-cdn.com/v4/letter/g/c77e96/32.png) [@Guoye](https://forums.swift.org/u/Guoye)\ **Post date:** [July 14, 2023, 8:10pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/16 "2023-07-14T20:10:17Z")
The list was compiled through the combination of 1. telemetry of most common fields used, and 2. internal use case (e.g. `Proxy-Authorization` even though it's uncommon on the internet). We don't have an exhaustive list of all fields. For example, `Content-Language` was omitted even though it's in RFC9110 since it's not seen used in the wild. That been said, if you have a use case for any of these fields defined in the RFC, we are happy to add them. ---
**Author:** ![dkz](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/dkz/32/29951_2.png) [@dkz](https://forums.swift.org/u/dkz)\ **Post date:** [July 21, 2023, 2:28pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/17 "2023-07-21T14:28:40Z")
Nice work! this is pretty cool! ---
**Author:** ![andre1sk](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/andre1sk/32/12560_2.png) [@andre1sk](https://forums.swift.org/u/andre1sk)\ **Post date:** [August 14, 2024, 8:20pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/18 "2024-08-14T20:20:31Z")
Sorry for a newb question so long term vision is to phase out SwiftNIO ? ---
**Author:** ![Guoye](https://avatars.discourse-cdn.com/v4/letter/g/c77e96/32.png) [@Guoye](https://forums.swift.org/u/Guoye)\ **Post date:** [August 14, 2024, 8:23pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/19 "2024-08-14T20:23:02Z")
No, there is no plan to phase out SwiftNIO ---
**Author:** ![andre1sk](https://sea2.discourse-cdn.com/swift/user_avatar/forums.swift.org/andre1sk/32/12560_2.png) [@andre1sk](https://forums.swift.org/u/andre1sk)\ **Post date:** [August 14, 2024, 8:37pm UTC](https://forums.swift.org/t/announcing-swift-http-types/66088/20 "2024-08-14T20:37:37Z")
Cool thank you for clarification!