Hello Swift community!
For a while now, I've been working on a new URL type for Swift. It has taken a long time, but it's finally at a stage where I'm happy to make a first release. So let me introduce you to WebURL
!
WebURL - A new URL type for Swift
WebURL
is a brand-new URL library. It is written entirely in Swift, implements the latest URL standard, and has a great API designed to take full advantage of Swift's features.
The repository is at https://github.com/karwa/swift-url
To use WebURL
in a SwiftPM project, add the following line to the dependencies in your Package.swift file:
.package(url: "https://github.com/karwa/swift-url", from: "0.1.0"),
And add a dependency on the "WebURL" product.
- The Getting Started guide contains an overview of how to use the
WebURL
type. - The Full documentation contains a detailed information about the API.
Additionally, I've developed a prototype port of async-http-client, which allows you to perform http(s) requests using WebURL
, and shows how easy it can be to adopt in your library.
Note: This project isn't limited to "web"-related URLs, it also has full support for "file" URLs, "data" URLs, and custom schemes like you might use for app-internal links.
It is called 'WebURL' because a) 'WebURL' is short, b) it has 'URL' in the name, and c) is different enough from Foundation's 'URL' that you hopefully won't confuse them.
A Modern Standard
The history of URLs is messy. There have been several standards over the years, working groups were established and closed due to lack of engagement, people came up with names like "URIs", "IRIs" and even a thing called a "URN"s in order to describe their various ideas for, uhm... URLs. As IBM's Sam Ruby and Adobe's Larry Masinter wrote in an IETF working draft in 2015:
The main problem is conflicting specifications that overlap but don't match each other.
...
From a user or developer point of view, it makes no sense for there to be a proliferation of definitions of URL nor for there to be a proliferation of incompatible implementations. This shouldn't be a competitive feature.
and as Daniel Stenberg (lead developer of CURL) explained on his blog:
A “URL” given in one place is certainly not certain to be accepted or understood as a “URL” in another place.
Not even curl follows any published spec very closely these days, as we’re slowly digressing for the sake of “web compatibility”. There’s no unified URL standard and there’s no work in progress towards that.
Well, the WHATWG's URL Living Standard is a step towards that. The WHATWG is an association of major browser developers, and is lead by representatives from Apple, Google, Mozilla, and Microsoft, who have agreed to align their browsers with the specifications developed at the WHATWG. In fact, the WebKit team have recently been rewriting their URL parser, and the latest Safari preview is now fully aligned with the URL Living Standard.
This library's parser is derived from the reference parser described in the standard, and should be fully compatible with it. It is tested using the same, common Web-Platform-Tests repository used by the major browser vendors to ensure compliance with the standard, as well as hundreds more tests covering the additional APIs.
The WebURL
library is entirely portable, having no dependencies other than the Swift standard library itself. Additionally, the parser and URL operations are free from OS-dependent behaviour; one benefit of the URL Living Standard is that all parser quirks, such as handling Windows drive letters or "/" vs "\" separators, work the same way on all platforms.
We will eventually add some OS-dependent behaviour, to convert file URLs to a local file-path, but it will be clearly apart from the rest of the API. We'll leave the actual manipulation of that path to OS libraries like swift-system
(e.g. I don't think it's our place to implement something like URL.resolveSymlinksInPath()
).
Speed
WebURL
is fast. Parsing speed depends a lot on the particular URL, but on my Intel Mac, compared to the existing URL
, I've seen improvements from 10%, all the way up to 67%. On a lower-end machine (a Raspberry Pi 4), I've seen orders of magnitude improvement - benchmarks which take 1.9 seconds with the existing URL
, but require only ~62 milliseconds with WebURL
. Common operations like hashing and checking for equality are consistently more than twice as fast as the existing URL
type.
Measuring parsing speed against the existing URL
is a bit of an apples-to-oranges comparison because the standards are so different, but it's a reference point people care about. There's still more that can be done to increase WebURL
's parsing speed; I have some ideas, and I'll be trying them in the coming months.
API
But perhaps the coolest thing about WebURL
is its API. Here are a couple of highlights:
-
In-place mutation
If you want to set a property on a
WebURL
, you don't need to create aURLComponents
and change the value and make another URL object; withWebURL
, you just set the value:var url = WebURL("http://github.com/karwa/swift-url/")! // Upgrade to https: url.scheme = "https" url.serialized // "https://github.com/karwa/swift-url/" // Change the path: url.path = "/apple/swift/" url.serialized // "https://github.com/apple/swift/"
Because
WebURL
is a copy-on-write value type, this replacement can happen in-place, without needing to allocate a bunch of intermediate objects. -
PathComponents view
WebURL
provides an efficientCollection
of the URL's path components, which shares storage with the URL object:var url = WebURL("http://github.com/karwa/swift-url/")! for component in url.pathComponents { // component = "karwa", "swift-url", "" } if url.pathComponents.dropLast().last == "swift-url" { // ... }
Not only that, but you can also mutate through this view. It's amazing, you have to try it:
var url = WebURL("file:///swift-url/Sources/WebURL/WebURL.swift")! url.pathComponents.removeLast() // url = "file:///swift-url/Sources/WebURL" url.pathComponents.append("My Folder") // url = "file:///swift-url/Sources/WebURL/My%20Folder" url.pathComponents.removeLast(3) url.pathComponents += ["Tests", "WebURLTests", "WebURLTests.swift"] // url = "file:///swift-url/Tests/WebURLTests/WebURLTests.swift"
It also has index-based mutations functions, including a full
replaceSubrange
, which you can use together with theCollection
implementation to perform more complex path manipulation. Again, all of this can happen in-place, so the days when URL operations were much slower than DIY string manipulation or regexes should be a thing of the past. -
Query parameters
WebURL
has a similar kind of view for the key-value pairs which are often encoded in a URL's query string. Here, we use@dynamicMemberLookup
to create a concise and convenient syntax:var url = WebURL("https://example.com/currency/convert?amount=20&from=EUR&to=USD")! url.formParams.amount // "20" url.formParams.to // "USD url.formParams.amount = "56" url.formParams.to = "Pound Sterling" // url = "https://example.com/currency/convert?amount=56&from=EUR&to=Pound+Sterling"
-
And more!
And there's a lot more than that!
WebURL
comes with a lot of utility functionality necessary for working with URLs, including:- pure-Swift IP address types, including parsing and serialisation compatible with
inet_aton/pton
andinet_ntoa/ntop
. - Percent-encoding and decoding (both eager and lazy)
Host
objects andOrigin
s- A UTF-8 view, allowing for arbitrary slices of URLs and modifying a URL's components using generic collections of UTF-8 bytes.
- pure-Swift IP address types, including parsing and serialisation compatible with
Roadmap
The implementation is extensively tested, but the interfaces have not had time to stabilise.
While the package is in its pre-1.0 state, it may be necessary to make source-breaking changes. I'll do my best to keep these to a minimum, and any such changes will be accompanied by clear documentation explaining how to update your code.
Aside from stabilising the API, the other priorities for v1.0 are:
- file URL <-> file path conversion.
- Converting to/from
Foundation.URL
. - Benchmarking and optimizing setters, including modifications via
pathComponents
andformParams
views.
For more information, see the project's README, where I go over some of the issues I'll expect to encounter along the way. I know that for a lot of macOS/iOS code, compatibility with URLSession
is critical; unfortunately I don't have an excellent answer for that right now. I go over some of the issues in the README, but the short answer is that even if we could convert reliably, round-tripping would break Hashable
and Equatable
because there's a good chance we'll need to add percent-encoding along the way. So the answer isn't obvious, but I'm going to be trying out a variety of approaches to see what's possible.
But yeah! That's an overview of WebURL
. Again, I'd encourage you to check out the Getting Started guide, and the API Reference. If you're using async-http-client
, consider experimenting with the WebURL port, or if you're developing another server library in Swift, consider trying out WebURL
in a branch, and letting me know how it goes for you.
I'd love to see this library adopted by as many other libraries and applications as possible, so if there's anything I can add to make that easier, please file a GitHub issue or write a post on the Swift forums. I’ll be requesting a sub-forum in the “related projects” section to discuss specific issues and ideas.
Thanks a lot for your time! I hope you're able to make use of WebURL
in your projects!