Swift-NIO based proxy reconfigures pipeline on every incoming request (Solved)

I'm using Swift-NIO to create a Http/2 proxy with TLS, for iOS/tvOS. My proxy startup:

var tlsConfiguration = ...
tlsConfiguration.applicationProtocols = NIOHTTP2SupportedALPNProtocols

let bootstrap = NIOTSListenerBootstrap(group: loopGroup)
.serverChannelOption(ChannelOptions.socket(SocketOptionLevel(SOL_SOCKET), SO_REUSEADDR), value: 1)
.serverChannelOption(ChannelOptions.socket(SocketOptionLevel(SOL_SOCKET), SO_REUSEPORT), value: 1)
.childChannelInitializer { channel in
    let sslContext: NIOSSLContext
    let tlsHandler: NIOSSLServerHandler
    do {
         sslContext = try NIOSSLContext(configuration: tlsConfiguration)
         tlsHandler = NIOSSLServerHandler(context: sslContext)
    } catch {
         print("[HTTP2PROXY] Could not configure TLS")
         return channel.close(mode: .all)
    }
    
    return channel.pipeline.addHandler(tlsHandler, name: "TLS_Handler").flatMap {
        print("[HTTP2PROXY] TLSHandler added to pipeline")
        print("[HTTP2PROXY] Configuring pipeline for Http/1.1 and Http/2")
        return channel.configureCommonHTTPServerPipeline(h2ConnectionChannelConfigurator: nil) { streamChannel in
            return streamChannel.pipeline.addHandlers([DebugInboundEventsHandler(), DebugOutboundEventsHandler()]).flatMap {
                print("[HTTP2PROXY] Event debugger handlers added")
                return streamChannel.pipeline.addHandler(HTTPResponseCompressor(), name: "ResponseCompressor")
                }.flatMap {
                     print("[HTTP2PROXY] HTTPResponseCompressor added to pipeline")
                     return streamChannel.pipeline.addHandler(CustomHttp1Handler(hlsRequestHandler: self.hlsRequestHandler), name: "Custom_Http1")
                }.flatMap {
                     print("[HTTP2PROXY] Custom Http1Handler added to pipeline")
                     return streamChannel.pipeline.addHandler(ErrorHandler())
                }
            }
        }
    }
.childChannelOption(ChannelOptions.socket(IPPROTO_TCP, TCP_NODELAY), value: 1)
.childChannelOption(ChannelOptions.socket(SocketOptionLevel(SOL_SOCKET), SO_REUSEADDR), value: 1)
.childChannelOption(ChannelOptions.socket(SocketOptionLevel(SOL_SOCKET), SO_REUSEPORT), value: 1)
do {
    let serverChannel = try bootstrap.bind(host: Http2Proxy.host, port: Http2Proxy.port).wait()
    print("[HTTP2PROXY] Server Channel bound to: \(serverChannel.localAddress!)")
}
catch {
    try! loopGroup.syncShutdownGracefully()
    print("[HTTP2PROXY] Failed to start channel: \(error)")
}

The server works and handles requests correctly using Https over Http/2 and even gzips responses when asked. So that's great. But in the logs I see that for every incoming request, the pipeline is being reconfigured (i.e. escapes for attaching the handlers are being called over and over again). Is that the way it is supposed to work? As far as I know I'm not closing the context/channel anywhere. Is this Proxy setting up a new pipeline for every request and am I therefor missing out on the Http/2 approach of sending a lot of requests over the same channel (and pipeline)? Or is this actuallty the way it is supposed to work? It feels like a none optimal result...

LOGS:

10:21:15.760 [HTTP2PROXY] Server Channel bound to: [IPv4]127.0.0.1/127.0.0.1:50001
10:22:47.813 [HTTP2PROXY] TLSHandler added to pipeline
10:22:47.813 [HTTP2PROXY] Configuring pipeline for Http/1.1 and Http/2
10:22:47.837 [HTTP2PROXY] Event debugger handlers added
10:22:47.838 [HTTP2PROXY] HTTPResponseCompressor added to pipeline
10:22:47.838 [HTTP2PROXY] Custom Http1Handler added to pipeline
10:22:47.853 [HTTP2PROXY] Event debugger handlers added
10:22:47.853 [HTTP2PROXY] HTTPResponseCompressor added to pipeline
10:22:47.853 [HTTP2PROXY] Custom Http1Handler added to pipeline
10:22:47.854 [HTTP2PROXY] Event debugger handlers added
10:22:47.854 [HTTP2PROXY] HTTPResponseCompressor added to pipeline
10:22:47.854 [HTTP2PROXY] Custom Http1Handler added to pipeline
10:22:47.860 [HTTP2PROXY] Event debugger handlers added
10:22:47.860 [HTTP2PROXY] HTTPResponseCompressor added to pipeline
10:22:47.860 [HTTP2PROXY] Custom Http1Handler added to pipeline
10:22:47.861 [HTTP2PROXY] Event debugger handlers added
10:22:47.861 [HTTP2PROXY] HTTPResponseCompressor added to pipeline
10:22:47.861 [HTTP2PROXY] Custom Http1Handler added to pipeline
10:22:47.927 [HTTP2PROXY] Event debugger handlers added
10:22:47.927 [HTTP2PROXY] HTTPResponseCompressor added to pipeline
10:22:47.927 [HTTP2PROXY] Custom Http1Handler added to pipeline

Logging the context.channel.pipeline.debugDescription gives:

10:30:41.083 [HTTP2PROXY] Pipeline config: 
ChannelPipeline[ObjectIdentifier(0x00000002804d6fd0)]:
                                 [I] ↓↑ [O]
 HTTP2FramePayloadToHTTP1ServerCodec ↓↑ HTTP2FramePayloadToHTTP1ServerCodec [handler0]
              HTTPResponseCompressor ↓↑ HTTPResponseCompressor              [ResponseCompressor]
                  CustomHttp1Handler ↓↑                                     [Custom_Http1]
                        ErrorHandler ↓↑                                     [handler1]
10:30:41.087 [HTTP2PROXY] Pipeline config: 
ChannelPipeline[ObjectIdentifier(0x00000002804d7160)]:
                                 [I] ↓↑ [O]
 HTTP2FramePayloadToHTTP1ServerCodec ↓↑ HTTP2FramePayloadToHTTP1ServerCodec [handler0]
              HTTPResponseCompressor ↓↑ HTTPResponseCompressor              [ResponseCompressor]
                  CustomHttp1Handler ↓↑                                     [Custom_Http1]
                        ErrorHandler ↓↑                                     [handler1]
10:30:41.090 [HTTP2PROXY] Pipeline config: 
ChannelPipeline[ObjectIdentifier(0x00000002804d7610)]:
                                 [I] ↓↑ [O]
 HTTP2FramePayloadToHTTP1ServerCodec ↓↑ HTTP2FramePayloadToHTTP1ServerCodec [handler0]
              HTTPResponseCompressor ↓↑ HTTPResponseCompressor              [ResponseCompressor]
                  CustomHttp1Handler ↓↑                                     [Custom_Http1]
                        ErrorHandler ↓↑                                     [handler1]
10:30:41.100 [HTTP2PROXY] Pipeline config: 
ChannelPipeline[ObjectIdentifier(0x00000002804d71b0)]:
                                 [I] ↓↑ [O]
 HTTP2FramePayloadToHTTP1ServerCodec ↓↑ HTTP2FramePayloadToHTTP1ServerCodec [handler0]
              HTTPResponseCompressor ↓↑ HTTPResponseCompressor              [ResponseCompressor]
                  CustomHttp1Handler ↓↑                                     [Custom_Http1]
                        ErrorHandler ↓↑                                     [handler1]

So the objectIdentifier of the pipeline is different every time...

Yup, this is expected behaviour.

HTTP/2 is multiplexed: this means you can run multiple request/response sequences over the same TCP connection. This manifests in SwiftNIO HTTP/2 in the form of the "stream channel initializer": this is called once per stream creation. The stream channel initializer here is the trailing closure being passed to configureCommonHTTPServerPipeline.

If you would like to create the handlers only once, you can do that. But now your handlers need to support being involved in multiple concurrent requests and responses.

1 Like

Awesome! Thanks for the confirmation.

Another question related to the above: Is it possible to set up an http/2 channel, without TLS? Currently I'm setting up a NIOSSLServerHandler with a tlsConfiguration that has:

tlsConfiguration.applicationProtocols = NIOHTTP2SupportedALPNProtocols

That way, during the TLS negotiation, http/2 can be established as the chosen protocol.
Is there a way to make this happen without the TLS setup?

Yes, you can use HTTP/2 using what's called "prior knowledge". In this instance, you can simply remove the SSL handler.

But this means the client should immediately (and only) make http/2 requests? Even without knowing the server can/will handle them. That sounds like a feature you would use when you are in charge of both the client and the server application and have full control over the data traffic.
In my case the client can send both http and http/2 requests. But I would like it to talk http/2. That's why I chose the above setup where the protocol negotiation decides the outcome. The thing is that only during a TLS negotiation the protocol is being discussed, right?

Plaintext HTTP/2 is no longer negotiated. RFC 7540 originally defined a mechanism to negotiate plaintext HTTP/2 using the Upgrade header, but this has been removed in the current draft of the revision to the HTTP/2 specification because it was never widely deployed. SwiftNIO HTTP/2 doesn't provide any support for doing this, and cannot be used in the HTTP/1.1 Upgrade mode.

2 Likes