So, right now after successful connect, I am changing the state and sending the modified head, but after this it seems like nothing happens.
I think I should get another callback of the channelRead method right? Where I can consult the state and determine that I am already connected and I can just send the data along with context.fireChannelRead(self.wrapInboundOut(.head(head))).
Will there always be head to satisfy the first guard?
Nope, the GlueHandler is perhaps the most important part of the example: it's how the data gets from the inbound connection (where your proxy handler lives) to the outbound connection you just created. Without them, no data can flow.
Big oops on my part then. I thought even since you mentioned previously that the HTTP should be much simpler, that lot of the things from the connect proxy aren't needed.
I think I am still confused as to what the intended sequence is. I have tried having another detailed look at the Connext Proxy example but I am not sure why the glue method is called in two places?
Also the handleInitialMessage method which parses the uri from the head is called once, just for the connection to be made in the connectTo method but above in the steps, you said that I should repeat the steps 1-5 for each request. But it appears this only happens once.
Conceptually what I need to do is this (?):
Initial request comes to proxy -> Parse the head from the request -> Save the head (with modified uri) & connectTo to the host on port 80 -> connection succeeded -> Add the GlueHandler -> write the cached/buffered head with the context.fireChannelRead(self.wrapInboundOut(.head(head))) –> Done?
From the Connect proxy it appears that after it is "glued", there is nothing else to do?
I have tried creating another handler based on the ConnectHandler from Connect proxy but a lot of stuff there is quite different with the awaitingEnd and awaitingConnection + manipulating pendingBytes so I got lost.
glue is called in two places because the state machine needs to wait for two things to happen, and they could happen in either order:
Receive the .end of the CONNECT request
Receive a completed connection to the remote server
In your case you do not need to wait for (1) (as you are passing the request you received on), so you only need to call glue in the (2) case.
It only happens once per request. HTTP/1.1 uses connection keep-alive, where a connection will be re-used for subsequent requests to the same host. Your code should tolerate that possibility. Thus:
this is incorrect, as you are responsible to continually rewrite the HTTP requests as they come in.
If you want to tear the connection down each time, this is absolutely fine. You do have some extension points:
That clarifies some things.. So after the connectTo succeeded, I need to glue the channels which makes sense. But at what point am I supposed to send along the buffered head from the initial request?
Right, I meant more like what is the correct sequence.
I have tried calling it manually with completion handler which gets called after the glue method is done but I am getting a fatal error:
Fatal error: tried to decode as type IOData but found HTTPPart<HTTPRequestHead, ByteBuffer> with contents other(NIOHTTP1.HTTPPart<NIOHTTP1.HTTPRequestHead, NIO.ByteBuffer>.head(HTTPRequestHead
The glue method from the original Connect handler starts with these lines, which I commented out, because my handler is a bit different and I don't have access to these.
// Ok, upgrade has completed! We now need to begin the upgrade process.
// First, send the 200 message.
// This content-length header is MUST NOT, but we need to workaround NIO's insistence that we set one.
let headers = HTTPHeaders([("Content-Length", "0")])
let head = HTTPResponseHead(version: .init(major: 1, minor: 1), status: .ok, headers: headers)
context.write(self.wrapOutboundOut(.head(head)), promise: nil)
context.writeAndFlush(self.wrapOutboundOut(.end(nil)), promise: nil)
Ah, yes, you need HTTP encoders and decoders in the other pipeline.
Concretely, the problem you have is that you're sending HTTP structured data into the new channel you just created. That's fine, there's nothing wrong with that, but we need to re-encode them to binary data before we send them on.
In the channelInitializer for the ClientBootstrap you create, add a HTTPRequestEncoder and a HTTPResponseDecoder.
That makes sense, are you sure I need to add the HTTPResponseDecoder because this does not work, since this does not conform to the ChannelHandler protocol.
When I added just the HTTPRequestEncoder, I am getting slightly different crash:
NIO/NIOAny.swift:200: Fatal error: tried to decode as type HTTPPart<HTTPRequestHead, IOData> but found HTTPPart<HTTPRequestHead, ByteBuffer> with contents other(NIOHTTP1.HTTPPart<NIOHTTP1.HTTPRequestHead, NIO.ByteBuffer>.head(HTTPRequestHead
This looks like I am closer, since the only difference is now IOData vs ByteBuffer in the generic parameters list.
I have added the encoder like this:
let channelFuture = ClientBootstrap(group: context.eventLoop)
.channelInitializer { channel in
channel.pipeline.addHandler(HTTPRequestEncoder())
}
.connect(host: host, port: port)
HTTPResponseDecoder is wrapped in ByteToMessageHandler.
Concretely, we need to discuss what is happening here.
Your server channel is reading data from the network and decoding the data into HTTPServerRequestPart. The problem you have is that you need to transform these into HTTPClientRequestPart for the HTTPRequestEncoder to tolerate it (that's what the type mismatch you saw above was). You can do this my modifying your proxy handler to change its InboundOut type to HTTPClientRequestPart. In general this won't be a problem, but you will need to wrap each of your .body payloads in .body(.byteBuffer()).
Yeah, you only need to touch the body to get the types to line up. Essentially your guard probably wants to be a switch to make sure you make the appropriate transformations.
So after modifying the InboundOut to be HTTPClientRequestPart, I am no longer getting fatal errors. I see connect succeeded and then I am sending the buffered head, however nothing else happens after this point.
curl gets stuck on this output:
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET http://apple.com/ HTTP/1.1
> Host: apple.com
> User-Agent: curl/7.64.1
> Accept: */*
> Proxy-Connection: Keep-Alive
>
I have implemented the wrapping of .body stuff, like this for now to quickly try it out:
guard case .head(var head) = self.unwrapInboundIn(data) else {
let unwrapped = self.unwrapInboundIn(data)
switch unwrapped {
case .body(let buffer):
context.fireChannelRead(self.wrapInboundOut(.body(.byteBuffer(buffer))))
default:
context.fireChannelRead(data)
break
}
return
}
It is not super pretty code, but I think the basic logic should work. If I don't get head, I try to get .body and wrap it if it succeeds.
While we're waiting for the connection to succeed you may receive .body and .end. They all have to be buffered, and can be unbuffered together once you unbuffer the .head.
