Dynamic changes to the pipeline - pick a different decoder/handler depending on the following handler state

Hi. I am experimenting with the SwiftNIO, and I am writing a parser for a line based protocol that can occasionally have an optional body of fixed length, previously specified in the Content-Length header.

So, most of the time LineBasedFrameDecoder with ByteToMessageHandler is what I need, but sometimes I need to stop using it and get the next X bytes of input depending on some state (parsing headers or parsing body).

What is the recommended approach to solving this problem? Can I pick a different decoder/handler depending on the state of the following handler? Or do I have to write a custom handler that does everything in one step?

Thanks!

NIO gives you full freedom and you can add/removes handlers at any point in time. This is super useful for protocol up/downgrades (e.g. STARTTLS, STOPTLS, HTTP/1.1 -> WebSocket, ...) as well as diagnostic/telemetry/debugging (e.g. adding a WritePCAPHandler in a certain situation). So you could use this functionality to implement your protocol this way.

Said that, I would recommend you only mutate the pipeline if you're really switching protocols (or for diagnostic/telemetry/debugging as needed) and stay clear of pipeline mutation when simply switching states within your protocol. The reason I wouldn't recommend overusing pipeline mutation is because it very easy to get wrong, you need to be quite careful. It's also much harder to test.

Instead, I'd recommend to base your code on ContentLengthHeaderFrameDecoder. It doesn't do the line-by-line by default but I don't think it would be hard to extend that state machine to allow line-by-line if you're not expecting a content-length header. (The tests might be useful too)

Thanks. I now think so, too. It will be simpler to extend the decoder given my problem than mutating the pipeline. After all, the logic is pretty straightforward.

Out of curiosity and for future references, can you point at an example where a pipeline is mutated from inside a handler? (is that even a right spot to mutate it in the runtime?)

HTTPServerUpgradeHandler is a fairly complex example of doing this: https://github.com/apple/swift-nio/blob/5f542894dd8efbd766d8adf73ef2f947b0cd5e21/Sources/NIOHTTP1/HTTPServerUpgradeHandler.swift#L221-L261.

Note that it goes to enormous lengths to try to get this code to work correctly, which is why it's structured in such a strange way. Simpler examples can show up in usages of the ApplicationProtocolNegotiationHandler , though those are less common to see. For example, in HTTP/2: https://github.com/apple/swift-nio-http2/blob/044339de7bdffddb81cb77fc57fb77ff2e743088/Sources/NIOHTTP2/HTTP2PipelineHelpers.swift#L345-L363

1 Like

And here's a STARTTLS implementation example: https://github.com/apple/swift-nio-examples/blob/44ec8e1b353a592a93d6b38c56e17f078c9f0de3/NIOSMTP/NIOSMTP/SendEmailHandler.swift#L134