WebURL 0.3.1 released!

I'm happy to announce that WebURL 0.3.1 has been released!

What's Changed

:link: Foundation Integration

0.3.0 brought Foundation-to-WebURL conversion, and this release adds conversion in the opposite direction (WebURL-to-Foundation). This is a particularly important feature for developers on Apple platforms, as it means you can now use WebURL to make requests using URLSession! We now have full, bidirectional interop with Foundation's URL , which is a huge milestone and a big step towards v1.0.:partying_face:

WebURLFoundationExtras now adds a number of extensions to types such as URLRequest and URLSession to make that super easy:

import Foundation
import WebURL
import WebURLFoundationExtras

// ℹ️ Make URLSession requests using WebURL.
func makeRequest(to url: WebURL) -> URLSessionDataTask {
  return URLSession.shared.dataTask(with: url) {
    data, response, error in
    // ...
  }
}

// ℹ️ Also supports Swift concurrency.
func processData(from url: WebURL) async throws {
  let (data, _) = try await URLSession.shared.data(from: url)
  // ...
}

// ℹ️ For libraries: move to WebURL without breaking
// compatibility with clients using Foundation's URL.
public func processURL(_ url: Foundation.URL) throws {
  guard let webURL = WebURL(url) else {
    throw InvalidURLError()
  }
  // Internal code uses WebURL...
}

When you make a request using WebURL, you will benefit from its modern, web-compatible parser, which matches modern browsers and libraries in other languages:

// Using WebURL: Sends a request to "example.com". 
// Chrome, Safari, Firefox, Go, Python, NodeJS, Rust agree. βœ…
print( try String(contentsOf: WebURL("http://foo@evil.com:80@example.com/")!) )

// Using Foundation.URL: Sends a request to "evil.com"! 😡
print( try String(contentsOf: URL(string: "http://foo@evil.com:80@example.com/")!) )

Note that this only applies to the initial request; HTTP redirects continue to be processed by URLSession (it is not possible to override it universally), and so are not always web-compatible. As an alternative on non-Apple platforms, our fork of async-http-client uses WebURL for all of its internal URL processing, so it also provides web-compatible redirect handling.

For more information about why WebURL is a great choice even for applications and libraries using Foundation, and a discussion about how to safely work with multiple URL standards, we highly recommend reading: Using WebURL with Foundation.

URLSession extensions are only available on Apple platforms right now, due to a bug in swift-corelibs-foundation. I opened a PR to fix it, and once merged, we'll be able to make these extensions available to all platforms.

:zap: Performance improvements

I say it every time, and it's true every time :sweat_smile:. For this release, I noticed that, due to a quirk with how ManagedBuffer is implemented in the standard library, every access to the URL's header data required dynamic exclusivity enforcement. But that shouldn't be necessary - the URL storage uses COW to enforce non-local exclusivity, and local exclusivity can be enforced by the compiler if we wrap the ManagedBuffer in a struct with reference semantics. So that's what I did.

The result is ~5% faster parsing and 10-20% better performance when getting/setting URL components. For collection views like pathComponents, these enforcement checks affect basically every operation and amount to a consistent overhead that we're now able to eliminate.

Some benchmark results
benchmark                                          column     results/0_3_0 results/0_3_1      %
------------------------------------------------------------------------------------------------
Constructor.HTTP.AverageURLs                       time            23909.00      22665.00   5.20
Constructor.HTTP.AverageURLs.filtered              time            37826.50      36066.00   4.65
Constructor.HTTP.IPv4                              time            12205.00      11627.00   4.74
Constructor.HTTP.IPv4.filtered                     time            19164.00      17819.00   7.02
Constructor.HTTP.IPv6                              time            13677.00      13086.00   4.32
Constructor.HTTP.IPv6.filtered                     time            17614.00      16577.00   5.89
...
ComponentSetters.Unique.Username                   time              418.00        365.00  12.68
ComponentSetters.Unique.Username.PercentEncoding   time              767.00        632.00  17.60
ComponentSetters.Unique.Username.Long              time              636.00        527.00  17.14
...
ComponentSetters.Unique.Path.Simple                time             2525.00       2247.00  11.01
...
PathComponents.Iteration.Small.Forwards            time              705.00        602.00  14.61
PathComponents.Iteration.Small.Reverse             time              718.00        619.00  13.79
PathComponents.Iteration.Long.Reverse              time             3137.00       2752.00  12.27
PathComponents.Append.Single                       time             1362.00       1242.00   8.81

:earth_africa: Standard Update

This release also implements a recent change to the WHATWG URL Standard, which forbids C0 Control characters and U+007F delete from appearing in domains. Split forbidden host/domain code points, and add all C0 controls and add U+007F to the latter by karwa Β· Pull Request #685 Β· whatwg/url Β· GitHub

What's next

The plan for the next release is to take another look at the formParams view. I'd like to add a variant which supports modern percent-encoding, rather than requiring HTML form-encoding as it currently does.

The current design is essentially a port of JavaScript's URLSearchParams class - and while it works, there are some parts of its behaviour which I think could be tweaked a bit, and there are some APIs I'd like to add (e.g. sorting query parameters for better cache effectiveness).

That release is basically going to be the v1.0 preview. So it's a really good time to try WebURL and get your feedback in, before the API is stabilised!

15 Likes

Also worth pointing out: the WebURL port of async-http-client has been updated, and includes support for Swift concurrency.

WebURL is, of course, Sendable.

this is an very important development. i think this will be a pathway for users on Apple platforms to start exploring the wider Swift ecosystem, which is sorely needed.

by the way, i’m looking forward to seeing this project get to its first major release. i don’t know if anyone else has told you this, but i consider swift-url a model for other pure Swift libraries in terms of its testing, documentation, infrastructure, toolchain support, development methodology, and other things. it’s certainly gotten me to revisit my development process for swift-json, swift-png, and swift-jpeg.

10 Likes

:pleading_face: Aw, shucks - thanks very much. That's very nice of you to say. I try my best.

6 Likes

Proposal is now up - WebURL Proposal by karwa Β· Pull Request #66 Β· swift-server/sswg Β· GitHub

I'd appreciate any feedback. Also, if you've never used WebURL, that's perfect - what do you think? Does it make a compelling case?

2 Likes

i certainly am paying close attention to this review, as swift-json may be following in swift-url’s footsteps assuming this review goes well :slight_smile:

one bit of preliminary feedback: the Motivation and Detailed Design sections seem a bit too intricate for the audience it is targeting β€” I work a lot with URL routing and a lot of it went over my head still. a lot of the Motivation section could go into a blog post or an article, and a lot of Detailed Design belongs in a tutorial (or multiple tutorials), i feel.

1 Like