The fatal error doesn't happen in the glue handler, but it's caused by the glue handler forwarding data on. These types resolve to this:
Fatal error: tried to decode as type HTTPServerResponsePart but found HTTPClientResponsePart
There are two handlers that could be causing this: either your proxy handler, or the HTTPResponseEncoder. Can you show me your OutboundIn/OutboundOut declarations in your proxy handler?
Right, I missed the crash origin in the stack trace..
typealias InboundIn = HTTPServerRequestPart
typealias InboundOut = HTTPClientRequestPart
typealias OutboundIn = HTTPClientResponsePart
typealias OutboundOut = HTTPServerResponsePart
I have changed the InboundOut when we were fixing the fatal error here
Oh, right: we're still removing the proxy handler. We need to not remove the proxy handler at all. Can you replace the code that removes the handler with code that just forwards on the buffered data, and otherwise leaves the proxy handler in the pipeline? That is, in func glue replace the call to context.pipeline.removeHandler(self, promise: nil) with a call to a new functions that has the body of removeHandler, minus the context.leavePipeline call.
Holy cow 
It appears to work with these modifications.
But I guess I still need to solve the case when HTTPProxyHandler gets HTTP connection and server redirects/upgrades it to HTTPS.
Would it perhaps be possible to use just the original ConnectHandler in the server bootstrap and inside the initial channelRead detect http and somehow use the HTTProxyHandler? Or is this total nonsense?
Because currently I am creating one ServerBootstrap for HTTPS and another for HTTP. They just bind to different ports.
Hmm so far it appears to work fine. Safari opens Not secure websites without issues. I cannot believe it 
So the way you'd really do this is to have the handler be able to do both jobs: if the request head says CONNECT then it would run the CONNECT mode, and if it doesn't then it'd do plaintext proxying as we've implemented here.
So I would somehow "merge" these two handlers and based on the initial message conditionally either execute HTTPS handler or HTTP handler code?
Or you mean like a another handler that selects the appropriate handler first?
To get yourself started, either design works. Strictly speaking you can CONNECT after doing other proxying, so merging them into a single handler would be more correct, but you can get off the ground by having a simple handler that looks at the method and adds the right handler to the pipeline.
Not sure if starting new thread would be better, but I think I discovered new issue with my HTTP handler.
Some HTTP sites return HTTP/1.1 302 Found which makes the handler confused and it apparently tries to load the content over and over again.
In the Console I see that (apparently the first) connection with connectTo succeeded and my handler sends the buffered head. But then it tries to connect again and again until Safari fails with "too many redirects error".
When I open the same URL on a desktop it loads the page fine using HTTP (so it doensn't do redirect to HTTPS version). But when I try the exactly same URL over the HTTP handler, I get the loop and 302 status code.
How can I debug this find the issue? Should I detect this 302 from the .head and handle this request differently?
I have also tested this page with Charles Proxy turned on and it worked fine.
Below is output from Terminal using curl, I have redacted the URL because I am not sure if it is appropriate to share it.
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET http://[URL] HTTP/1.1
> Host: [URL]
> User-Agent: curl/7.64.1
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 302 Found
< Date: Wed, 07 Jul 2021 14:04:08 GMT
< Server: Apache
< Location: http://[URL]
< Cache-Control: max-age=0
< Expires: Wed, 07 Jul 2021 14:04:08 GMT
< Content-Length: 217
< Content-Type: text/html; charset=iso-8859-1
<
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>302 Found</title>
</head><body>
<h1>Found</h1>
<p>The document has moved <a href="[URL]">here</a>.</p>
</body></html>
* Connection #0 to host 127.0.0.1 left intact
* Closing connection 0
What is curious to me is that the HTML points to the same URL I am trying to load 
The Host header should not contain the URL but just the hostname.
Are you sure the URL is exactly the same? E.g. some sites do redirects for trailing slashes and such (e.g. http://blub.com/abc => http://blub.com/abc/).
Also when working with proxies, I highly recommend passing on the raw value coming in (after stripping the prefix). You wouldn't think how many invalid URLs are still flying around in the Interwebs.
Yea, my bad on "redacting" out the actual content. Apologies! It is indeed the hostname. It is on a subdomain but other pages on the same site seem to work fine with my HTTP handler.
You were spot on! It was dumb trailing slash 
I missed it in the logs and thought that URL would keep it since the default path is "/" and I had trailling slash in the address in Safari 
Now lets hope my appending of trailing slash doesn't break anything else
So I thought I would start building the "combined" HTTP & HTTPS handler, but realized, that there are obstacles:
Original connect handler:
class ConnectHandler: ChannelInboundHandler {
typealias InboundIn = HTTPServerRequestPart
typealias OutboundOut = HTTPServerResponsePart
}
And my HTTP:
class HTTPConnectHandler: ChannelDuplexHandler {
typealias InboundIn = HTTPServerRequestPart
typealias InboundOut = HTTPClientRequestPart
typealias OutboundIn = HTTPClientResponsePart
typealias OutboundOut = HTTPServerResponsePart
}
The original one is "just" ChannelInboundHandler but mine is ChannelDuplexHandler so I guess I cannot just mash their functionality together and use a few ifs right?
I think in principle it should still be fine: your handler will remove itself in the CONNECT case so its duplex nature shouldn't matter.
Happy to report that "combined" handler actually works 
Hello, I miss the same error like this:
Fatal error: tried to decode as type HTTPPart<HTTPResponseHead, IOData> but found HTTPPart<HTTPResponseHead, ByteBuffer>
Can you tell me how you solved it at that time，here is my code:
final class HTTPConnectHandler: ChannelDuplexHandler, RemovableChannelHandler {
func removeHandler(context: ChannelHandlerContext, removalToken: ChannelHandlerContext.RemovalToken) {
if case let .pendingConnection(head) = self.state {
NSLog("[HTTPConnectHandler] head = \(type(of: head))")
self.state = .connected
context.fireChannelRead(self.wrapInboundOut(.head(head)))
if let bufferedBody = self.bufferedBody {
context.fireChannelRead(self.wrapInboundOut(.body(.byteBuffer(bufferedBody))))
self.bufferedBody = nil
}
if let bufferedEnd = self.bufferedEnd {
context.fireChannelRead(self.wrapInboundOut(.end(bufferedEnd)))
self.bufferedEnd = nil
}
NSLog("[HTTPConnectHandler] will fireChannelReadCoomplete")
context.fireChannelReadComplete()
}
context.leavePipeline(removalToken: removalToken)
}
enum State {
case idle
case pendingConnection(head: HTTPRequestHead)
case connected
}
enum ConnectError: Error {
case invalidURL
case wrongScheme
case wrongHost
}
// typealias InboundIn = HTTPServerRequestPart
// typealias InboundOut = HTTPClientRequestPart
typealias InboundIn = HTTPServerRequestPart
typealias InboundOut = HTTPClientRequestPart
typealias OutboundIn = HTTPClientResponsePart
typealias OutboundOut = HTTPServerResponsePart
private var state = State.idle
private var bufferedBody: ByteBuffer?
private var bufferedEnd: HTTPHeaders?
func channelRead(context: ChannelHandlerContext, data: NIOAny) {
guard case .head(var head) = self.unwrapInboundIn(data) else {
let unwrapped = self.unwrapInboundIn(data)
switch unwrapped {
case .body(let buffer):
switch state {
case .connected:
context.fireChannelRead(self.wrapInboundOut(.body(.byteBuffer(buffer))))
case .pendingConnection(_):
print("Buffering body")
self.bufferedBody = buffer
default:
// shouldnt happen
break
}
case .end(let headers):
switch state {
case .connected:
context.fireChannelRead(self.wrapInboundOut(.end(headers)))
case .pendingConnection(_):
print("Buffering end")
self.bufferedEnd = headers
default:
// shouldnt happen
break
}
case .head(_):
assertionFailure("Not possible")
break
}
return
}
NSLog("[HTTPConnectHandler] connecting to URI: \(head.uri)")
// os_log(.default, log: .default, "Connecting to URI: %{public}s", head.uri as NSString)
guard let parsedUrl = URL(string: head.uri) else {
context.fireErrorCaught(ConnectError.invalidURL)
return
}
NSLog("[HTTPConnectHandler] connecting to scheme: \(parsedUrl.scheme ?? "no scheme")")
// os_log(.default, log: .default, "Parsed scheme: %{public}s", (parsedUrl.scheme ?? "no scheme") as NSString)
guard parsedUrl.scheme == "http" else {
context.fireErrorCaught(ConnectError.wrongScheme)
return
}
guard let host = head.headers.first(name: "Host"), host == parsedUrl.host else {
// os_log(.default, log: .default, "Wrong host")
NSLog("[HTTPConnectHandler] wrong host")
context.fireErrorCaught(ConnectError.wrongHost)
return
}
var targetUrl = parsedUrl.path
if let query = parsedUrl.query {
targetUrl += "?\(query)"
}
head.uri = targetUrl
switch state {
case .idle:
state = .pendingConnection(head: head)
connectTo(host: host, port: 80, context: context)
case .pendingConnection(_):
break
// os_log(.default, log: .default, "Logic error fireChannelRead with incorrect state")
case .connected:
context.fireChannelRead(self.wrapInboundOut(.head(head)))
}
}
func write(context: ChannelHandlerContext, data: NIOAny, promise: EventLoopPromise<Void>?) {
// self.
switch self.unwrapOutboundIn(data) {
case .head(let head):
context.write(self.wrapOutboundOut(.head(head)), promise: nil)
case .body(let body):
// self.wrapOutboundOut(.body(.byteBuffer(body)))
context.write(self.wrapOutboundOut(.body(.byteBuffer(body))), promise: nil)
case .end(let trailers):
context.write(self.wrapOutboundOut(.end(trailers)), promise: nil)
}
}
private func connectTo(host: String, port: Int, context: ChannelHandlerContext) {
NSLog("[HTTPConnectHandler] start connect to \(host):\(port)")
let channelFuture = ClientBootstrap(group: context.eventLoop)
.channelInitializer { channel in
channel.pipeline.addHandler(HTTPRequestEncoder()).flatMap {
channel.pipeline.addHandler(ByteToMessageHandler(HTTPResponseDecoder(leftOverBytesStrategy: .forwardBytes)))
}
}
.connect(host: host, port: port)
channelFuture.whenSuccess { channel in
NSLog("[HTTPConnectHandler] connect success to \(host):\(port)")
self.connectSucceeded(channel: channel, context: context)
}
channelFuture.whenFailure { error in
NSLog("[HTTPConnectHandler] connect fail to \(host):\(port)")
self.connectFailed(error: error, context: context)
}
}
private func connectSucceeded(channel: Channel, context: ChannelHandlerContext) {
// os_log(.default, log: .default, "Connect succeeded")
self.glue(channel, context: context)
}
private func connectFailed(error: Error, context: ChannelHandlerContext) {
// os_log(.error, log: .default, "Connect failed: %@", error as NSError)
context.fireErrorCaught(error)
}
private func glue(_ peerChannel: Channel, context: ChannelHandlerContext) {
self.removeEncoder(context: context)
// Now we need to glue our channel and the peer channel together.
let (localGlue, peerGlue) = GlueHandler.matchedPair()
context.channel.pipeline.addHandler(localGlue).and(peerChannel.pipeline.addHandler(peerGlue)).whenComplete { result in
switch result {
case .success(_):
context.pipeline.removeHandler(self, promise: nil)
case .failure(_):
// Close connected peer channel before closing our channel.
peerChannel.close(mode: .all, promise: nil)
context.close(promise: nil)
}
}
}
private func removeEncoder(context: ChannelHandlerContext) {
context.pipeline.context(handlerType: HTTPResponseEncoder.self).whenSuccess {_ in
// context.pipeline.removeHandler(context: $0, promise: nil)
}
}
}
This post was also posted in the thread iOS HTTP traffic use SwiftNIO, so I'm going to respond there.