Swift NIO: noob question about reading all bytes

Out of curiosity I'm trying to build a simple socket client using Swift NIO. What I want to do is simply to send a message to the server, get the response and close the connection.

The server to which I'm connecting (I do not have access to its source code) writes back a response of 4152 bytes, in chunks of maximum 1024 bytes each. Therefor, what I am receiving is:

Chunk 1: 1024 bytes
Chunk 2: 1024 bytes
Chunk 3: 1024 bytes
Chunk 4: 1024 bytes
Chunk 5:   56 bytes

Now, since I want to handle the whole response at once, I am currently adding the chunks to a Data instance.

I noticed that channelReadComplete(context:) gets called once after the last full 1024 chunk and twice when there is no more data to read (after the last 56 bytes) so if I run the following code, I get this result:

channelRead: 1024
channelRead: 2048
channelRead: 3072
channelRead: 4096
channelReadComplete: 4096
channelRead: 4152
channelReadComplete: 4152
channelReadComplete: 4152
fileprivate final class SimpleClientHandler: ChannelInboundHandler {
    
    fileprivate typealias InboundIn = ByteBuffer
    
    private var receivedData = Data()
    
    private var lastCheckedSize = 0
    
    fileprivate func channelRead(context: ChannelHandlerContext, data: NIOAny) {
        var byteBuffer = unwrapInboundIn(data)
        if let newData = byteBuffer.readBytes(length: byteBuffer.readableBytes) {
            receivedData.append(contentsOf: newData)
            print("channelRead: \(receivedData.count)")
        }
    }

    func channelReadComplete(context: ChannelHandlerContext) {
        print("channelReadComplete: \(receivedData.count)")
        if lastCheckedSize == receivedData.count {
            // Seems to be done
        } else {
            lastCheckedSize = receivedData.count
        }
    }
}

I was wondering if there is any other approach that I am not aware of, that I can use to be notified once that the whole response has been received, or if this is actually the right way to handle this situation.

In general there is no way to define when a "whole response" has been received in the abstract. What indicates a "whole response" is entirely down to the way the protocol is implemented.

channelReadComplete is a very specific signalling mechanism that indicates that we are not going to attempt to read from the underlying socket until the next event loop tick occurs: that is, the event loop will temporarily be servicing other work. It's a signal to developers that if there is any work they've been delaying, now might be a good time to do it, as we won't be back to this specific connection for a little while. It has nothing to do with the framing of the data flowing through the connection.

So the question is: how are you supposed to know how long this response is? Are you expecting to get a connection close to signal the end of the response? Is the length included in the data somehow? Are you expected to know ahead of time? Depending on which of these is the answer I can give you better advice about how you could do this checking.

In the meantime, some other notes:

  1. Your channelReadComplete implementation is buggy, because it assumes you'll get a channelReadComplete with no channelRead before it (that's the only time you could hit the "seems to be done" block. That is by no means guaranteed to happen.

  2. Rather than use readBytes to create a [UInt8] and then passing that array to Data.append, you can use a ByteBufferView:

    receivedData.append(contentsOf: byteBuffer.readableBytesView)
    
2 Likes

Thank you for your answer and for the fixes you suggested me. As you can imagine, it is the first time I work with Swift NIO and I really appreciate the help.

As far as I can tell, again I do not have access to the source code of the server which I am connecting to, if I debugPrint the whole response as a utf8 encoded String, it always terminates with a "\0".

So I guess I have to check if a ByteBuffer contains a NUL?

Would something like this be correct or is there any more efficient way to do it?

fileprivate func channelRead(context: ChannelHandlerContext, data: NIOAny) {
        let byteBuffer = unwrapInboundIn(data)
        receivedData.append(contentsOf: byteBuffer.readableBytesView)
        if receivedData.last == 0 {
                // Ready to go
        }
    }

Sounds like a solid first guess, let's try that. The simplest way to do it is to use ByteBufferView again: buffer.readableBytesView.contains(0) would do it. Be careful to only do this to the new data: if you do it with the data you've been buffering this will exhibit O(n2) behaviour and make you really sad.

Alternatively, you can use firstIndex(of:) to get the specific index. This will help you to actually perform the transformation you need, as you can then use that index to slice the ByteBufferView and only copy the actual bytes you need.

Seems to work like a Swiss clock! :muscle:

Thank you again!

1 Like