I want to hear your ideas about "I/O" types as per my previous post.
Current Foundation has types that have properties taking I/O instances while I think their interfaces are inconsistent/inconvenient.
It may be surprising to hear, but I've actually benchmarked the byte-oriented AsyncBytes IO stream type that already exists in Foundation as faster than one that reads into chunked Data, because it gets to reuse its internal buffer repeatedly rather than calling malloc and free a lot.
Interesting.
So do you mean ReadableStream would be better to be just a typealias ReadableStream = AsyncSequence where Self.Element == UInt8 rather than a new protocol?
I wonder how about writing. What is better to define such types like Process's standardOutput...?
I'm also not convinced ReadableStream is a good name for that type (maybe ByteStream?), but yes, that's what I meant.
For writing I would expect to see methods that consume an async sequence instead of producing one. There's some tricky details about getting that to perform well though; I have some ideas, but experimentation will be needed.
This is an interesting topic that we are currently also exploring in the server ecosystem where we need to model both the read and the write side.
For us one of the most important types to bridge is a NIO Channel which is often an abstraction over a socket with some protocol transformation on top. A channel has both a read and write side to it so we had to come up with abstractions for both. Furthermore, both sides need to be fully async; hence, non-blocking. Lastly they both need to uphold back-pressure
The read side is relatively clear since we have a protocol in the standard library for this called AsyncSequence. It encapsulates the concepts of reads quite well. There are some open perf improvements like reducing executor hops or getting the whole buffer of elements instead of one but the shape of the protocol seems good. For NIO specifically we created the NIOAsyncSequenceProducer which is a root async sequence that allows us to bridge from sync (Channel) to async while upholding back-pressure on both the production and consumption edge.
The write side is more interesting since we don’t have a protocol in the standard library for it; however, we do see a pattern emerge in the server ecosystem. For NIO we created the NIOAsyncWriter which allows to bridge the write side of a channel to Concurrency.
The API shape for end-users is quite similar to your proposal and we could generalize it to this:
(We could and probably should add methods with default implementations for writing a sequence and an async sequence to it)
This protocol might be worth pitching but I am not sure if it clears the bar for inclusion in the standard library. Rust has something similar called a Writer.
IMO it is super important that the writer is not only based on consuming async sequences since it limits the usage of that API across more use-cases. I would love to see more exploration around the usage of writers especially in places like FileIO and server side libraries. The SSWG is currently exploring modeling of server interfaces and this has come up.
The Swift Async Algorithms package contains a very similar interface for the "sending" side of its "AsyncChannel" implementation(s) (albeit the send itself cannot fail).
Yes an AsyncChannel can be thought off as a non throwing async writer that backs an async sequence with a buffer of one. There has been some discussion around separating the types of the async channel into its atoms which I think is worth rediscussing before we tag 1.0.0 cc @Philippe_Hausler
Well, I guess it'd be also worth to discuss whether or not the function should be throwable, or when the error is thrown.
Concrete I/O behind the abstract interfaces may be in memory, on hard disk, or via network...