Difference between NIOHTTPServerUpgradeConfiguration.completionHandler and upgradePipelineHandler

when setting up a websocket server connection, the original HTTP handler can be removed in the completionHandler of the NIOHTTPServerUpgradeConfiguration or the upgradePipelineHandler of NIOWebSocketServerUpgrader. is there a difference between the two? is it safe to write to the channel before the HTTP handler removal future completes?

Looking at the code, it looks to me like the HTTPEncoder, HTTPDecoder are already removed for you by the NIOHTTPClientUpgradeHandler or the HTTPServerUpgradeHandler (depending on your use case). For this reason you should not need to worry about removing these. The NIOWebSocketServerUpgrader seems to implement the protocol needed to actually perform an update.

    public func configureHTTPServerPipeline(position: ChannelPipeline.Position = .last,
                                            withPipeliningAssistance pipelining: Bool = true,
                                            withServerUpgrade upgrade: NIOHTTPServerUpgradeConfiguration? = nil,
                                            withErrorHandling errorHandling: Bool = true) throws {
        self.eventLoop.assertInEventLoop()

        let responseEncoder = HTTPResponseEncoder()
        let requestDecoder = HTTPRequestDecoder(leftOverBytesStrategy: upgrade == nil ? .dropBytes : .forwardBytes)

        var handlers: [RemovableChannelHandler] = [responseEncoder, ByteToMessageHandler(requestDecoder)]

        if pipelining {
            handlers.append(HTTPServerPipelineHandler())
        }

        if errorHandling {
            handlers.append(HTTPServerProtocolErrorHandler())
        }

        if let (upgraders, completionHandler) = upgrade {
            let upgrader = HTTPServerUpgradeHandler(upgraders: upgraders,
                                                    httpEncoder: responseEncoder,
                                                    extraHTTPHandlers: Array(handlers.dropFirst()),
                                                    upgradeCompletionHandler: completionHandler)
            handlers.append(upgrader)
        }

        try self.addHandlers(handlers, position: position)
    }

Does that solve your question?

I’m talking about the final HTTP handler, like the HTTPHandler in the websocket server example

Okay, I had to look around quite a bit, but from the API standpoint it seems to be WAY safer, to mutate your ChannelPipeline in the NIOWebSocketServerUpgrader's upgradePipelineHandler.

The reason for this is as follows: NIOWebSocketServerUpgrader implements the HTTPServerProtocolUpgrader protocol. According to the protocol documentation an upgrader should modify the ChannelPipeline in the upgrade function. While the upgrade functions returned future has not completed, all incoming data will be buffered.

/// Called when the upgrade response has been flushed. At this time it is safe to mutate the channel pipeline
/// to add whatever channel handlers are required. Until the returned `EventLoopFuture` succeeds, all received
/// data will be buffered.
func upgrade(context: ChannelHandlerContext, upgradeRequest: HTTPRequestHead) -> EventLoopFuture<Void>

The NIOWebSocketServerUpgrader calls its upgradePipelineHandler after having modified the ChannelPipeline by adding websocket encoder and decoder. The important bit here is, that it waits for the upgradePipelineHandler's returned EventLoopFuture before advancing in its callback chain in the upgrade method. Because of this you can be sure no data will be forwarded to your new handlers until your handler removal has succeeded.

The NIOHTTPServerUpgradeConfiguration's completionHandler will not wait until your previous HTTPHandler has been removed. It is actually called after the HTTPEncoder and HTTPDecoder were removed, but before the NIOHTTPClientProtocolUpgrader upgrade method was called. The important bit here is, that the completionHandler does not let you return a future that is waited on until your handler is removed. If your HTTPHandler takes a long time to remove, you might run into scheduling problems here, as the upgrade is marked as succeeded, before you actually removed your HTTPHandler.

Generally, no.

I hope that makes some sense. If you have any further questions please reach out.

i’m creating the websocket handlers outside of the childChannelInitializer (because they need to be given a serial number, among other things). so the upgradePipelineHandler asks for a websocket handler and a promise to fulfill once the HTTP handler is removed:

typealias Connection = 
(
    // the websocket handler
    endpoint:Endpoint<T, Server>, 
    // tells concurrent code that websocket is ready for use
    ready:EventLoopPromise<Void>
)
upgradePipelineHandler: 
{ 
    (channel:Channel, _:HTTPRequestHead) -> EventLoopFuture<Void> in

    let connection:EventLoopPromise<Connection> = channel.eventLoop.makePromise(of: Connection.self)
    // yields into an AsyncStream.Continuation, asks for 
    // the websocket handler, and a ‘ready’ promise
    us.yield((request: connection, channel: channel))
    
    return connection.futureResult.flatMap 
    {
        (connection:Connection) -> EventLoopFuture<Void> in 
        // wait for external code to create the websocket handler, 
        // and adds it to the pipeline
        channel.pipeline.addHandler(connection.endpoint).whenComplete 
        {
            switch $0 
            {
            case .success:
                // remove the original HTTP endpoint, succeeds the ‘ready’ 
                // promise on completion
                channel.pipeline.removeHandler(initializer, promise: connection.ready)
            case .failure(let error):
                // fails the ‘ready’ promise if the websocket handler could 
                // not be added
                connection.ready.fail(error)
            }
        }
        // waits to report completion at the same time as the 
        // ‘ready’ promise
        return connection.ready.futureResult
    }
}

how is this approach?