How to merge `NIOAsyncChannelInboundStream` into another stream without losing backpressure?

Related Projects SwiftNIO
3

merge and zip weren’t really the right tools here, because the injected events aren’t a second sequence. instead, i ended up using AsyncThrowingChannel against the advice of the other thread.

extension AsyncSequence where Element:Sendable
{
    consuming
    func forward<T>(to channel:AsyncThrowingChannel<T, any Error>,
        by transform:(Element) throws -> T ) async rethrows
    {
        do
        {
            for try await element:Element in self
            {
                await channel.send(try transform(element))
            }

            channel.finish()
        }
        catch let error
        {
            channel.fail(error)
        }
    }
}

this is a concerning statement to me. the swift compiler has always struggled to strip dead code, this is one of the oldest criticisms of the language. to give the library the benefit of the doubt, i did a small experiment with the Swiftinit server, a real open source server application from the swift-unidoc project.

i compared a version of the server that depends on a vendored copy of AsyncThrowingChannel from swift-async-algorithms to a version that imports the AsyncAlgorithms module from the canonical package.

/swift/swift-unidoc$ ls -l .build/release/ | grep SwiftinitServer
-rwxrwxr-x  1 ec2-user ec2-user  86283048 Feb 27 20:58 SwiftinitServer
drwxrwxr-x  9 ec2-user ec2-user      4096 Feb 22 20:03 SwiftinitServer.build
drwxrwxr-x  2 ec2-user ec2-user      4096 Feb 22 19:54 SwiftinitServer.product
-rw-rw-r--  1 ec2-user ec2-user       380 Feb 22 20:02 SwiftinitServer.swiftdoc
-rw-rw-r--  1 ec2-user ec2-user    213308 Feb 22 20:02 SwiftinitServer.swiftmodule
-rw-rw-r--  1 ec2-user ec2-user     28224 Feb 27 02:01 SwiftinitServer.swiftsourceinfo
/swift/swift-unidoc$ ls -l .build/release/ | grep SwiftinitServer
-rwxrwxr-x  1 ec2-user ec2-user  95346776 Feb 27 21:01 SwiftinitServer
drwxrwxr-x  9 ec2-user ec2-user      4096 Feb 22 20:03 SwiftinitServer.build
drwxrwxr-x  2 ec2-user ec2-user      4096 Feb 22 19:54 SwiftinitServer.product
-rw-rw-r--  1 ec2-user ec2-user       380 Feb 22 20:02 SwiftinitServer.swiftdoc
-rw-rw-r--  1 ec2-user ec2-user    213308 Feb 22 20:02 SwiftinitServer.swiftmodule
-rw-rw-r--  1 ec2-user ec2-user     28224 Feb 27 02:01 SwiftinitServer.swiftsourceinfo

compared to the version that uses the vendored copy of the type, adding a dependency on AsyncAlgorithms adds a whopping 9.1 MB to the compiled binary. perhaps i am not building it correctly. but all i did was follow the instructions in the README.

the changes to swift-async-algorithms that would pare down this binary bloat are not extensive. the AsyncAlgorithms module is already well-partitioned internally, and the only additional files i needed to copy to extract the AsyncThrowingChannel type were Locking.swift and Rethrow.swift.

it is understandable that an incubating package may not have had the time to address binary size concerns. but i would really encourage you to reconsider dismissing it as a “non-goal” for server libraries. swift-async-algorithms will need to be partitioned before it can be reasonably called production-ready. this work doesn’t have to happen today, but it will have to be done eventually.

1 Like