Is there a length restriction on channelRead?

I created a server and a client using SwiftNIO. They communicate via TCP. To ensure a correct communication, I put every message from server to client and vice versa in a xml-string like this:
<packet ...>...</packet>.

As far as I could see, this works perfect and everytime channelRead was called, I had a complete packet. My question is now: Can there be a chance, that one packet will be split up in two calls of channelRead although the other side just sent on message?

By the way, my func looks like this:

func channelRead(context: ChannelHandlerContext, data: NIOAny) {
    var byteBuffer = self.unwrapInboundIn(data)

    if let string = byteBuffer.readString(length: byteBuffer.readableBytes) {
        self.messageHandler?(context.channel, string)
    }
}

Yes. TCP is stream-oriented: it does not expose message boundaries, and treats the data as a stream of bytes. Your code does need to guard against the possibility that messages will be split up across packets.

A good thing to do is to write a ByteToMessageDecoder that can process this framing. As an example (it's not super efficient but it's workable):

struct XMLFraming: ByteToMessageDecoder {
    typealias InboundOut = ByteBuffer

    mutating func decode(context: ChannelHandlerContext, buffer: inout ByteBuffer) throws -> DecodingState {
        var view = buffer.readableBytesView

        while view.count > 0, let index = view.firstIndex(of: UInt8(ascii: "<")) {
            view = view[index...]
            if view.starts(with: "</packet>".utf8) {
                // Got a match.
                view = view.dropFirst("</packet>".utf8.count)
                let packet = buffer.readSlice(length: buffer.readableBytes - view.count)!
                context.fireChannelRead(self.wrapInboundOut(packet))
                return .continue
            }
        }

        return .needMoreData
    }
}

This will chunk up the messages, sending one ByteBuffer that contains the full framing. If you wanted you could also arrange for this object to strip off the leading <packet ...> and trailing </packet>. You can then create a ByteToMessageHandler from this ByteToMessageDecoder, and insert it into the pipeline before your ChannelHandler.

This will guarantee that every channelRead contains one and only one packet that matches your framing.

Thank you very much for your answer and even for your example!

I checked my code again, because I was wondering, why it was working all the time and then I was a little bit surprised, because I realized, that I answered this question to myself a year ago, as I wrote that code. Actually I solved this problem in a different way:

let packetBegin = "<packet"
let packetEnd = "</packet>"
var dataRead = ""

func handleMessageString(messageString: String) {
    dataRead = dataRead + messageString

    if let startRange = dataRead.range(of: packetBegin), let endRange = dataRead.range(of: packetEnd) {
        let xmlString = String(dataRead[startRange.lowerBound..<endRange.upperBound])
        dataRead = String(dataRead[endRange.upperBound...])   // leave everything, that is behind </packet>
        ...
    }
}

This function is called every time from channelRead (see the self.messageHandler in my first post).

But I will have a closer look at your solution as well.

You can also use ByteToMessageDecoderVerifier from NIOTestUtils to test your ChannelHandler's ability to deal with incomplete messages. It tests different sizes of input for you to verify your handling of incomplete messages.

For example:

func testPingThenDisconnectDecoding() {
    // Given
    let channel = EmbeddedChannel()
    var disconnectInput = channel.allocator.buffer(capacity: 4)
    disconnectInput.writeBytes([0b11000000, 0b00000000, 0b11100000, 0b00000000])
    let expectedInOuts = [(disconnectInput, [Packet.ping(.init()), Packet.disconnect(.init())])]
    
    // When, Then
    XCTAssertNoThrow(try ByteToMessageDecoderVerifier.verifyDecoder(inputOutputPairs: expectedInOuts,
                                                                    decoderFactory: { PacketDecoder() }))
}
1 Like

That is at the end a very trick thing. I figured out a problem, when using an UTF8-String in this case. If there is a special character that is exactly splitted between two messages, then my solution is not working anymore (because appending those two Strings are different then if I would use the whole data). I am using:

func channelRead(context: ChannelHandlerContext, data: NIOAny) {
	var buffer = self.unwrapInboundIn(data)
	if let string = buffer.readString(length: buffer.readableBytes) {
		// pass the message to the messageHandler
		self.messageHandler(string)   // INFO: this will call my handleMessageString
	}
}

@lukasa: I don't understand your solution. Where can I put this?

In the meantime I implemented another solution using only Data so I will append Data and not Strings.

@Lupurus Do you get any length prefix before the actual string. Or do you get some end signal, that you know the string is now fully loaded? You could use those markers to know when you received the complete string and only then start reading the string.

@fabianfett: Yes of course, I do... as I wrote in my first post:

<packet ...>MYMESSAGE</packet>

But I will have a problem, when this one will be splitted in:

PART 1:
-----
<packet ...>.... some text \u00

AND PART 2:
----------
E4 rest of the message</packet>

This will result in a broken string, so handling it with data is working.

Terms of Service

Privacy Policy

Cookie Policy