HTTPRequestDecoder does not forward request

I'm using SwiftNIO in my HTTP proxy application that does TLS MITM. It dynamically generates a SSLServerHandler after receiving the hostname from SNI in TLS ClientHello message. This requires adding handlers to the pipeline.

My application does seem to work without problems but I'm running into an issue where I have to remove and then add back HTTPRequestDecoder before any traffic is sent to the other handlers. I cannot figure out why this happens, and I would be very interested to learn why. I think there's a valuable learning hidden in all of this :slight_smile:

I made an example project at GitHub - PasiSalenius/NIOTest: Example SwiftNIO project to demonstrate an issue with handlers that reproduces this issue. Run the Swift package in Xcode, it starts listening on http://127.0.0.1:8080 and connect to it with curl using curl --proxy http://127.0.0.1:8080 --insecure --verbose https://127.0.0.1:8080. You get back the message hello from HelloHandler.

Now, if you uncomment the other part of func addSSLServer(context: ChannelHandlerContext) and try the curl command again HelloHandler does not receive a request. This is the issue I'm facing in my application.

This is designed behaviour, though it's not very clearly documented.

The TL;DR here is that once a CONNECT request is parsed, the HTTPRequestDecoder is essentially dead. This is a necessary safety decision: once we have parsed an HTTP CONNECT request, we cannot understand any of the bytes that follow that message on the connection. They are going to be something that is almost certainly not HTTP: in this case, the bytes of a TLS ClientHello. Parsing them would cause immediate parse errors.

What we don't have is a way for you to "reset" the handler. That's why you have to add a new one.

The way I think about this is not that you're "re-adding" the handler, but instead you're adding a new one for a nested protocol. For this case of HTTP CONNECT, your protocol stack is basically: IP <-> TCP <-> HTTP/1.1 <-> TLS <-> HTTP/1.1. That means you have a second set of HTTP handlers. You happen to want to remove the first set, but that's largely an implementation detail: it doesn't affect the logical spelling.

Ah okay that makes total sense. So I guess I'll just keep on doing it the way I have so far, then :slight_smile: Thanks for the explanation!