So I would somehow "merge" these two handlers and based on the initial message conditionally either execute HTTPS handler or HTTP handler code?
Or you mean like a another handler that selects the appropriate handler first?
To get yourself started, either design works. Strictly speaking you can CONNECT after doing other proxying, so merging them into a single handler would be more correct, but you can get off the ground by having a simple handler that looks at the method and adds the right handler to the pipeline.
Not sure if starting new thread would be better, but I think I discovered new issue with my HTTP handler.
Some HTTP sites return HTTP/1.1 302 Found which makes the handler confused and it apparently tries to load the content over and over again.
In the Console I see that (apparently the first) connection with connectTo succeeded and my handler sends the buffered head. But then it tries to connect again and again until Safari fails with "too many redirects error".
When I open the same URL on a desktop it loads the page fine using HTTP (so it doensn't do redirect to HTTPS version). But when I try the exactly same URL over the HTTP handler, I get the loop and 302 status code.
How can I debug this find the issue? Should I detect this 302 from the .head and handle this request differently?
I have also tested this page with Charles Proxy turned on and it worked fine.
Below is output from Terminal using curl, I have redacted the URL because I am not sure if it is appropriate to share it.
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET http://[URL] HTTP/1.1
> Host: [URL]
> User-Agent: curl/7.64.1
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 302 Found
< Date: Wed, 07 Jul 2021 14:04:08 GMT
< Server: Apache
< Location: http://[URL]
< Cache-Control: max-age=0
< Expires: Wed, 07 Jul 2021 14:04:08 GMT
< Content-Length: 217
< Content-Type: text/html; charset=iso-8859-1
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="[URL]">here</a>.</p>
</body></html>
* Connection #0 to host 127.0.0.1 left intact
* Closing connection 0
What is curious to me is that the HTML points to the same URL I am trying to load 
Yea, my bad on "redacting" out the actual content. Apologies! It is indeed the hostname. It is on a subdomain but other pages on the same site seem to work fine with my HTTP handler.
You were spot on! It was dumb trailing slash 
I missed it in the logs and thought that URL would keep it since the default path is "/" and I had trailling slash in the address in Safari 
Now lets hope my appending of trailing slash doesn't break anything else
So I thought I would start building the "combined" HTTP & HTTPS handler, but realized, that there are obstacles:
Original connect handler:
class ConnectHandler: ChannelInboundHandler {
typealias InboundIn = HTTPServerRequestPart
typealias OutboundOut = HTTPServerResponsePart
}
And my HTTP:
class HTTPConnectHandler: ChannelDuplexHandler {
typealias InboundIn = HTTPServerRequestPart
typealias InboundOut = HTTPClientRequestPart
typealias OutboundIn = HTTPClientResponsePart
typealias OutboundOut = HTTPServerResponsePart
}
The original one is "just" ChannelInboundHandler but mine is ChannelDuplexHandler so I guess I cannot just mash their functionality together and use a few ifs right?
I think in principle it should still be fine: your handler will remove itself in the CONNECT case so its duplex nature shouldn't matter.
1 Like
Happy to report that "combined" handler actually works 
Hello, I miss the same error like this:
Fatal error: tried to decode as type HTTPPart<HTTPResponseHead, IOData> but found HTTPPart<HTTPResponseHead, ByteBuffer>
Can you tell me how you solved it at that time，here is my code:
final class HTTPConnectHandler: ChannelDuplexHandler, RemovableChannelHandler {
func removeHandler(context: ChannelHandlerContext, removalToken: ChannelHandlerContext.RemovalToken) {
if case let .pendingConnection(head) = self.state {
NSLog("[HTTPConnectHandler] head = \(type(of: head))")
self.state = .connected
context.fireChannelRead(self.wrapInboundOut(.head(head)))
if let bufferedBody = self.bufferedBody {
context.fireChannelRead(self.wrapInboundOut(.body(.byteBuffer(bufferedBody))))
self.bufferedBody = nil
}
if let bufferedEnd = self.bufferedEnd {
context.fireChannelRead(self.wrapInboundOut(.end(bufferedEnd)))
self.bufferedEnd = nil
}
NSLog("[HTTPConnectHandler] will fireChannelReadCoomplete")
context.fireChannelReadComplete()
}
context.leavePipeline(removalToken: removalToken)
}
enum State {
case idle
case pendingConnection(head: HTTPRequestHead)
case connected
}
enum ConnectError: Error {
case invalidURL
case wrongScheme
case wrongHost
}
// typealias InboundIn = HTTPServerRequestPart
// typealias InboundOut = HTTPClientRequestPart
typealias InboundIn = HTTPServerRequestPart
typealias InboundOut = HTTPClientRequestPart
typealias OutboundIn = HTTPClientResponsePart
typealias OutboundOut = HTTPServerResponsePart
private var state = State.idle
private var bufferedBody: ByteBuffer?
private var bufferedEnd: HTTPHeaders?
func channelRead(context: ChannelHandlerContext, data: NIOAny) {
guard case .head(var head) = self.unwrapInboundIn(data) else {
let unwrapped = self.unwrapInboundIn(data)
switch unwrapped {
case .body(let buffer):
switch state {
case .connected:
context.fireChannelRead(self.wrapInboundOut(.body(.byteBuffer(buffer))))
case .pendingConnection(_):
print("Buffering body")
self.bufferedBody = buffer
default:
// shouldnt happen
break
}
case .end(let headers):
switch state {
case .connected:
context.fireChannelRead(self.wrapInboundOut(.end(headers)))
case .pendingConnection(_):
print("Buffering end")
self.bufferedEnd = headers
default:
// shouldnt happen
break
}
case .head(_):
assertionFailure("Not possible")
break
}
return
}
NSLog("[HTTPConnectHandler] connecting to URI: \(head.uri)")
// os_log(.default, log: .default, "Connecting to URI: %{public}s", head.uri as NSString)
guard let parsedUrl = URL(string: head.uri) else {
context.fireErrorCaught(ConnectError.invalidURL)
return
}
NSLog("[HTTPConnectHandler] connecting to scheme: \(parsedUrl.scheme ?? "no scheme")")
// os_log(.default, log: .default, "Parsed scheme: %{public}s", (parsedUrl.scheme ?? "no scheme") as NSString)
guard parsedUrl.scheme == "http" else {
context.fireErrorCaught(ConnectError.wrongScheme)
return
}
guard let host = head.headers.first(name: "Host"), host == parsedUrl.host else {
// os_log(.default, log: .default, "Wrong host")
NSLog("[HTTPConnectHandler] wrong host")
context.fireErrorCaught(ConnectError.wrongHost)
return
}
var targetUrl = parsedUrl.path
if let query = parsedUrl.query {
targetUrl += "?\(query)"
}
head.uri = targetUrl
switch state {
case .idle:
state = .pendingConnection(head: head)
connectTo(host: host, port: 80, context: context)
case .pendingConnection(_):
break
// os_log(.default, log: .default, "Logic error fireChannelRead with incorrect state")
case .connected:
context.fireChannelRead(self.wrapInboundOut(.head(head)))
}
}
func write(context: ChannelHandlerContext, data: NIOAny, promise: EventLoopPromise<Void>?) {
// self.
switch self.unwrapOutboundIn(data) {
case .head(let head):
context.write(self.wrapOutboundOut(.head(head)), promise: nil)
case .body(let body):
// self.wrapOutboundOut(.body(.byteBuffer(body)))
context.write(self.wrapOutboundOut(.body(.byteBuffer(body))), promise: nil)
case .end(let trailers):
context.write(self.wrapOutboundOut(.end(trailers)), promise: nil)
}
}
private func connectTo(host: String, port: Int, context: ChannelHandlerContext) {
NSLog("[HTTPConnectHandler] start connect to \(host):\(port)")
let channelFuture = ClientBootstrap(group: context.eventLoop)
.channelInitializer { channel in
channel.pipeline.addHandler(HTTPRequestEncoder()).flatMap {
channel.pipeline.addHandler(ByteToMessageHandler(HTTPResponseDecoder(leftOverBytesStrategy: .forwardBytes)))
}
}
.connect(host: host, port: port)
channelFuture.whenSuccess { channel in
NSLog("[HTTPConnectHandler] connect success to \(host):\(port)")
self.connectSucceeded(channel: channel, context: context)
}
channelFuture.whenFailure { error in
NSLog("[HTTPConnectHandler] connect fail to \(host):\(port)")
self.connectFailed(error: error, context: context)
}
}
private func connectSucceeded(channel: Channel, context: ChannelHandlerContext) {
// os_log(.default, log: .default, "Connect succeeded")
self.glue(channel, context: context)
}
private func connectFailed(error: Error, context: ChannelHandlerContext) {
// os_log(.error, log: .default, "Connect failed: %@", error as NSError)
context.fireErrorCaught(error)
}
private func glue(_ peerChannel: Channel, context: ChannelHandlerContext) {
self.removeEncoder(context: context)
// Now we need to glue our channel and the peer channel together.
let (localGlue, peerGlue) = GlueHandler.matchedPair()
context.channel.pipeline.addHandler(localGlue).and(peerChannel.pipeline.addHandler(peerGlue)).whenComplete { result in
switch result {
case .success(_):
context.pipeline.removeHandler(self, promise: nil)
case .failure(_):
// Close connected peer channel before closing our channel.
peerChannel.close(mode: .all, promise: nil)
context.close(promise: nil)
}
}
}
private func removeEncoder(context: ChannelHandlerContext) {
context.pipeline.context(handlerType: HTTPResponseEncoder.self).whenSuccess {_ in
// context.pipeline.removeHandler(context: $0, promise: nil)
}
}
}
This post was also posted in the thread iOS HTTP traffic use SwiftNIO, so I'm going to respond there.