Designing an HTTP Client API for Swift

From the design documentation this stood out to me:

APIs should offer "careful hole-punching": well-defined customization points that don't expose unnecessary complexity. Simple operations stay simple with intuitive defaults and minimal boilerplate, while advanced capabilities become available as needed without forcing developers off a cliff when requirements grow.

High-level APIs handle common tasks at the usage site; complex specialization happens at configuration time. We prefer compile-time enforcement over runtime checks, and minimize implicit interactions that the compiler cannot validate.

Abstractions should not completely obscure the underlying reality though. Developers must be able to understand, debug, and reason about network behavior at a lower level when necessary.

I really love this notion, and while this may be true to some extent for past HTTP APIs, it is especially true for what app developers usually build on top of these system networking APIs.

In my experience developers intuitively build abstractions around the standard HTTP APIs to share common functionality like authentication, custom error handling or retry behavior, encoding and decoding to JSON or protobuf, adhering to more high-level standards like JSON-RPC and so on. So in reality I often do not find myself fighting the standard APIs, but the abstractions that developers (me included) build on top of them. While these abstractions usually serve a well-intentioned purpose, they often become inflexible and cumbersome to deal with as soon as exceptions to the rules come up, like unexpected status code handling, special-purpose HTTP headers or other calling conventions that the abstraction was not designed for.

So I wonder if we have the opportunity here to not only modernize the network stack, but also rethink how clients can derive HTTP clients tailor-made for their backend systems and calling conventions, that are built from primitive types and protocols that make the described abstractions unnecessary in the first place and that automatically adhere to the design considerations quoted above.

I would call this the problem of "sharing call patterns and request handling across a code base".

Typically we solve these problems of sharing functionality while keeping flexibility via some form of composable API, which I would like to see here as well.

One option would be to allow registration of middleware code client-side as well. In this case developers could just share pre-configured HTTPClient types with already registered middleware across the codebase and in special circumstances developers can add new middleware ad-hoc.

Another option would be to try to make HTTP clients or individual HTTP calls composable types by themselves, via a system that encourages writing new types that conform to a primitive protocol that can be used to perform HTTP calls – similar to how SwiftUI Views are composed from other views (Even though I would not propose to build something on top of ResultBuilders, but I think it is a good demonstration of the power of composability via a commonly shared protocol like View nevertheless)

In addition I would like to see ideas that include decoding and encoding of data via middleware or type composition as well, as this is usually part of every HTTP pipeline.

Long story short, independent of these rough ideas, I would like to see the use case of sharing call patterns and request handling across a code base reflected in the design considerations, because sub-optimal abstractions are way too common and I would like to make them unnecessary.