Pitch: `ContiguousBytes` protocol in the standard library

Evolution Pitches
1

Motivation:

Foundation has a ContiguousBytes protocol. It's a pretty simple protocol, having only one method, withUnsafeBytes. This method is extremely important for implementing APIs that need to work with raw data efficiently, without having to copy data into a buffer first. It is also extremely helpful when wrapping C APIs that take byte pointers.

As it turns out, the standard library has an almost identical protocol, named _HasContiguousBytes, which implements the same withUnsafeBytes method, and quite a few Standard Library types conform to it. As a result, these types can be made to conform to ContiguousBytes with minimal effort, and this conformance is defined in Foundation. Unfortunately, since _HasContiguousBytes is internal, there is no way to make a generic function that can take any contiguous byte sequence without involving Foundation.

There are workarounds for this, but none of them are great:

1. Link against Foundation

This works, and then you can easily define your generic function:

func twiddleBits(bytes: some ContiguousBytes) { ... }

Unfortunately, if done in a library, this now requires all consumers of this library to import Foundation, which can be a deal-breaker for many.

2. Add explicit overrides for each concrete contiguous bytes type in the Standard Library, and cause clients to have to call one of those

This isn't a great solution, because Foundation's Data struct is a really common container for raw bytes, and now your API can't be used with it without making the client jump through hoops.

3. Just make all your APIs take UnsafeRawBufferPointer arguments and let callers deal with turning whatever they have into bytes

I don't think I have to explain why this is non-ideal.

Proposed Solution:

Move the ContiguousBytes protocol, and the conformances for Standard Library types like Array and ContiguousArray, over to the Standard Library. Keep the conformances for Foundation types like Data in Foundation where they currently reside. Then, generic APIs can work equally well with both Foundation and non-Foundation callers and everything will just work.

Future Directions:

I'd also like to see the also relatively simple DataProtocol moved over to the Standard Library as well, but given how useful it would already be just to have ContiguousBytes, I'm not necessarily pushing my luck and demanding it. It seems to be something the Core Team has talked about in the past, though, and it would make it easy to also support things like DispatchData without necessarily needing to link against Foundation.

5 Likes
2

This was reviewed (with a slightly different name and design) first as part of SE-0237, then as SE-0256, and rejected by the core team.

It sounds like the concern that this would be an addition to the collection hierarchy for too narrow a purpose.

3

That proposal is for contiguous typed collections, not for byte buffers. The fact that its use case is not the same as ContiguousBytes was mentioned in the thread.

Byte buffers are far from a narrow purpose; it's impossible to do pretty much anything involving a disk, a network, compression, encryption, etc. without a byte buffer being involved at some level. This is fairly fundamental, and there is currently no "protocol-oriented" way to do it without involving the Foundation library.

6 Likes