Capture WebSocket packets using man-in-the-middle

Related Projects SwiftNIO
1

I want to use the man-in-the-middle to capture and decrypt webSocket protocol traffic on an iOS App. This App is an independent packet capture tool running in iOS devices, which acts as both the Server and Client. My idea is that the middleman intercepts the request initiated by the Client and plays the role of the Server to complete the connection with the Client. Then the middleman initiates a real request to the Server based on the intercepted data.

I know I should use NIOWebSocketServerUpgrader and call configureHTTPServerPipeline method for Server side, and use NIOWebSocketClientUpgrader and call addHTTPClientHandlers method for Client side, but I ran into a few problems, After I configure http server pipeline, I add a custom handler to receive http request, But I can't get the channelRead method being called. and in Client side, I can't the the response, and don't know how to handle websocket frames.

Can someone give me a demo? I really need your help!

2

This is not going to be straightforward. You can either implement a HTTP CONNECT proxy (if the app knows you're there and you appear like a proxy), or you have to be a transparent proxy. In either case, you will have to on-the-fly decrypt the network traffic. This will also require a NIOSSLServerHandler to handle the TLS, as well as appropriate creation and deployment of TLS certificates. This is before you get anywhere near having a web socket connection.

My advice is to start small, one protocol at a time. Write something that does TCP, just plain TCP (no channel handlers), and prints to the log whatever it receives. Get that working. Then add TLS and get that working. Then add HTTP and get that working. And then add web sockets. Right now you're trying to do a fairly complex protocol stack all in one go, and that's going to make your life much harder than it needs to be.

1 Like
3

@lukasa
Yes, you are right, I have implemented a proxy using Network Extension framework, and I captured all packets by creating a local VPN. Now I have succeeded to capture and decrypt HTTPS packets, I know how to implement TLS handshake and ALPN by using NIOSSLServerHandler、NIOSSLClientHandler and ApplicationProtocolNegotiationHandler. But when I use SwiftNIO to implement WebSocket capturing, I faced some problems I mentioned above. I need your help!

4

So it seems like after you add HTTP handlers to the pipeline you stop seeing channel reads. Can you remove the HTTP handlers and copy over to here the data you get from channelRead?

5

When I remove the WebSocket related handlers, I get the following data from channelRead (I add a test handler After TLS handshake succeed):

NIOAny { head(HTTPRequestHead { method: GET, uri: "/ws/v2?aid=13&device_id=2410572936917666&access_key=671d0788ad0bbc7ab4b5e3f92de9d96d&fpid=1&sdk_version=3&iid=3255087214874926&pl=1&ne=1&version_code=80301&is_background=-1", version: HTTP/1.1, headers: [("Host", "ws-test.company.com"), ("Connection", "Upgrade"), ("Pragma", "no-cache"), ("Cache-Control", "no-cache"), ("x-support-ack", "1"), ("Upgrade", "websocket"), ("Origin", "wss://ws-test.company.com"), ("Sec-WebSocket-Version", "13"), ("x-tt-trace-id", "00-c1147c7c0d890673f9c31c8b761a000d-c1147c7c0d890673-01"), ("User-Agent", "Mozilla/5.0 (iPhone; CPU iPhone OS 14_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Cronet Mobile/15E148 Safari/605.1"), ("Accept-Encoding", "gzip, deflate, br"), ("Cookie", "odin_tt=d46f16cbbdff09c4f628b8dc1dht8f66fa7737e7db7e7ee34c7f61e54b00e9f8900e1625483fe132c5db711eaaf3aae9; passport_csrf_token=82572faeb8d92882ea423a31266fc8dc; passport_csrf_token_default= 82572faeb8d92882ea423a31266fc8dc; install_id=3266027214874926; ttreq=1$c917312dc491edc3f6e09740ccce7e93342b854a"), ("Sec-WebSocket-Key", "r8s6WZhuOxB1UN+b8nteyA=="), ("Sec-WebSocket-Extensions", "permessage-deflate; client_max_window_bits"), ("Sec-WebSocket-Protocol", "pbbp2")] }) }

NIOAny { end(nil) }

This is WebSocket Upgrade request, and the data looks correct. But when I configure MITM as WebSocket Server, I can't receive channelRead message in my handler, and my code looks like as follows (I write these code with reference to the SwiftNIO WebSocket example Demo):

let upgrader = NIOWebSocketServerUpgrader(shouldUpgrade: { (channel: Channel, head: HTTPRequestHead) in channel.eventLoop.makeSucceededFuture(HTTPHeaders()) },
                                 upgradePipelineHandler: { (channel: Channel, _: HTTPRequestHead) in
                                    channel.pipeline.addHandler(WebSocketTimeHandler())
                                 })

let bootstrap = ServerBootstrap(group: group)
    // Specify backlog and enable SO_REUSEADDR for the server itself
    .serverChannelOption(ChannelOptions.backlog, value: 256)
    .serverChannelOption(ChannelOptions.socketOption(.so_reuseaddr), value: 1)

    // Set the handlers that are applied to the accepted Channels
    .childChannelInitializer { channel in
        let httpHandler = HTTPHandler()
        let config: NIOHTTPServerUpgradeConfiguration = (
                        upgraders: [ upgrader ],
                        completionHandler: { _ in
                            channel.pipeline.removeHandler(httpHandler, promise: nil)
                        }
                    )
        return channel.pipeline.configureHTTPServerPipeline(withServerUpgrade: config).flatMap {
            channel.pipeline.addHandler(httpHandler)
        }
    }

    // Enable SO_REUSEADDR for the accepted Channels
    .childChannelOption(ChannelOptions.socketOption(.so_reuseaddr), value: 1)

Theoretically, I should receive all request traffics from httpHandler'schannelRead , but httpHandler's been removed quickly before channelRead could be invoked.

But Why SwiftNIO's demo runs normally?

6

This is intentional: once the web socket upgrade succeeds, the HTTP handler must not be in the pipeline anymore. Any further data is not HTTP, but web socket, and the HTTP handler doesn't know what to do with that.

Where are you trying to observe channelRead from?

7

This is my code:

let upgrader = NIOWebSocketServerUpgrader(shouldUpgrade: { (_: Channel, _: HTTPRequestHead) in
    return context.eventLoop.makeSucceededFuture(HTTPHeaders())
}, upgradePipelineHandler: { (_: Channel, _: HTTPRequestHead) in
    return context.pipeline.addHandler(NTWebSocketMessageHandler(proxyContext: self.proxyContext))
})

let httpHandler = NTWebSocketHTTPHandler(proxyContext: self.proxyContext)
let config: NIOHTTPServerUpgradeConfiguration = (
    upgraders: [ upgrader ],
    completionHandler: { _ in
        context.pipeline.removeHandler(httpHandler, promise: nil)
    }
)
return context.pipeline.configureHTTPServerPipeline(withServerUpgrade: config).flatMap({
    context.pipeline.addHandler(httpHandler)
})

I observe channelRead in NTWebSocketHTTPHandler, but NTWebSocketHTTPHandler be removed after soon it was added to pipeline, so channelRead has no chance to be invoked. I think my code is correct, because I refer to the SwiftNIO demo for these codes.

8

After calling context.pipeline.configureHTTPServerPipeline(withServerUpgrade: config), the handlers in pipeline are:

                                                                                                          [I] ↓↑ [O]
                                                                                          NIOSSLServerHandler ↓↑ NIOSSLServerHandler            [NIOSSLServerHandler]
                                                                        ApplicationProtocolNegotiationHandler ↓↑                                [ApplicationProtocolNegotiationHandler]
                                                                                                              ↓↑ HTTPResponseEncoder            [handler0]
 ByteToMessageHandler<HTTPDecoder<HTTPPart<HTTPRequestHead, ByteBuffer>, HTTPPart<HTTPResponseHead, IOData>>> ↓↑                                [handler1]
                                                                                    HTTPServerPipelineHandler ↓↑ HTTPServerPipelineHandler      [handler2]
                                                                               HTTPServerProtocolErrorHandler ↓↑ HTTPServerProtocolErrorHandler [handler3]
                                                                                     HTTPServerUpgradeHandler ↓↑                                [handler4]

But when I add NTWebSocketHTTPHandler before HTTPServerUpgradeHandler , I can receive channelRead in NTWebSocketHTTPHandler. But as you might expect, this is not the right way to write the code.

9

What is NTWebSocketHTTPHandler trying to do?

10

The NTWebSocketHTTPHandler just makes a request to a real server and its code looks something like this (in channelActive):

for data in requestDatas {
    if let head = data as? HTTPRequestHead {
        let requestHead = HTTPRequestHead(version: head.version, method: head.method, uri: head.uri, headers: head.headers)
        proxyContext.clientChannel!.write(self.wrapOutboundOut(.head(requestHead)), promise: nil)
    }

    if let data = data as? ByteBuffer {
        let body = HTTPClientRequestPart.body(.byteBuffer(data))
        proxyContext.clientChannel!.write(self.wrapOutboundOut(body), promise: nil)
    }

    if let end = data as? HTTPHeaders {
        let promise = proxyContext.clientChannel?.eventLoop.makePromise(of: Void.self)
        proxyContext.clientChannel!.writeAndFlush(HTTPClientRequestPart.end(end), promise: promise)
        promise?.futureResult.whenComplete({ _ in
            // save request to local
        })
    }

    if let endstr = data as? String, endstr == "end" {
        let promise = proxyContext.clientChannel?.eventLoop.makePromise(of: Void.self)
        proxyContext.clientChannel!.writeAndFlush(HTTPClientRequestPart.end(nil), promise: promise)
        promise?.futureResult.whenComplete({ _ in
            // save request to local
        })
    }
}

After the request is sent to the real server, NTWebSocketRequestHandler will be removed quickly.

11

So right now you remove this handler after the web socket upgrade is perceived to succeed (this is what the NIOHTTPServerUpgradeConfiguration.completionHandler is doing). Is this what you want to achieve?

12

Yes, After upgrade is success, httpHandler is removed from the pipeline, but the rest of the work does not go as I expected. The following is a simple architecture diagram of the packet capture tool:

WechatIMG118
WechatIMG1181920×864 63.8 KB

When there is a traffic coming in, for example, the traffic is WebSocket upgrade request, I will use context.pipeline.configureHTTPServerPipeline(withServerUpgrade: config) to upgrade the fake server, and use outChannel.pipeline.addHTTPClientHandlers(withClientUpgrade: config) to upgrade the fake client, then I will use the fake client start the real request to the real server. But I realize that there is something wrong with my configure. I can't upgrade the fake server before response come back. Is that true? Should I upgrade the fake server after the response comes back?

13

Maybe I should config the fake server as normal http server using the following code:

let requestDecoder = HTTPRequestDecoder(leftOverBytesStrategy: .dropBytes)
return context.pipeline.addHandler(ByteToMessageHandler(requestDecoder), name: "ByteToMessageHandler").flatMap({
    context.pipeline.addHandler(HTTPResponseEncoder(), name: "HTTPResponseEncoder").flatMap({
        context.pipeline.addHandler(HTTPServerPipelineHandler(), name: "HTTPServerPipelineHandler").flatMap({
            context.pipeline.addHandler(HTTPHandler(proxyContext: self.proxyContext), name: "HTTPHandler")    
        })
    })
})

after receiving the http response, then upgrade the fake server to receive WebSocket Frame.

Is this the right idea?

Please help me, thanks.

14

Yes, I think your second idea is much more likely to work.

15

I config the fake server as normal http server, and I succeed to receive the correct response. but I don't know how to upgrade the normal http server to WebSocket server :sob: .
the normal http server pipeline handlers are:

[I] ↓↑ [O]
                                                                                          NIOSSLServerHandler ↓↑ NIOSSLServerHandler       [NIOSSLServerHandler]
 ByteToMessageHandler<HTTPDecoder<HTTPPart<HTTPRequestHead, ByteBuffer>, HTTPPart<HTTPResponseHead, IOData>>> ↓↑                           [ByteToMessageHandler]
                                                                                                              ↓↑ HTTPResponseEncoder       [HTTPResponseEncoder]
                                                                                    HTTPServerPipelineHandler ↓↑ HTTPServerPipelineHandler [HTTPServerPipelineHandler]
                                                                                       NTWebSocketHTTPHandler ↓↑ NTWebSocketHTTPHandler    [NTWebSocketHTTPHandler]

After receiving the response, I should upgrade the pipeline handlers to:

                         NIOSSLServerHandler ↓↑ NIOSSLServerHandler   [NIOSSLServerHandler]
                    HTTPServerUpgradeHandler ↓↑                       [handler4]
                                             ↓↑ WebSocketFrameEncoder [handler5]
 ByteToMessageHandler<WebSocketFrameDecoder> ↓↑                       [handler6]
               WebSocketProtocolErrorHandler ↓↑                       [handler7]
                     NTWebSocketFrameHandler ↓↑                       [handler8]

But I don't know how to config context or channel to realize this upgrade.
This is my code, it doesn't work as expected:

_ = context.pipeline.removeHandler(name: "ByteToMessageHandler")
_ = context.pipeline.removeHandler(name: "HTTPResponseEncoder")
_ = context.pipeline.removeHandler(name: "HTTPServerPipelineHandler")

let upgrader = NIOWebSocketServerUpgrader(shouldUpgrade: { (_: Channel, _: HTTPRequestHead) in
    return context.eventLoop.makeSucceededFuture(HTTPHeaders())
}, upgradePipelineHandler: { (_: Channel, _: HTTPRequestHead) in
    return context.pipeline.addHandler(NTWebSocketFrameHandler(proxyContext: self.proxyContext))
})

let config: NIOHTTPServerUpgradeConfiguration = (
    upgraders: [ upgrader ],
    completionHandler: { _ in
        
    }
)

_ = context.pipeline.configureHTTPServerPipeline(withServerUpgrade: config).flatMap {
    context.pipeline.removeHandler(name: "NTWebSocketHTTPHandler")
}

Do you know how to upgrade the http pipeline to WebSocket pipeline?

16

I would expect to see an HTTPServerUpgradeHandler in your original pipeline. This would normally drive the upgrade to WebSocket. You only seem to be adding it late: why is that happening?

17

Sorry, I have updated the code as follows:

let upgrader = NIOWebSocketServerUpgrader(shouldUpgrade: { (_: Channel, _: HTTPRequestHead) in
    return context.eventLoop.makeSucceededFuture(HTTPHeaders())
}, upgradePipelineHandler: { (_: Channel, _: HTTPRequestHead) in
    let future = context.pipeline.addHandler(NTWebSocketFrameHandler(proxyContext: self.proxyContext), name: "NTWebSocketFrameHandler")
    print("fake server pipeline handlers: \(context.pipeline)")
    return future
})

let httpHandler = NTWebSocketHTTPHandler(proxyContext: self.proxyContext)
let config: NIOHTTPServerUpgradeConfiguration = (
    upgraders: [ upgrader ],
    completionHandler: { _ in
        context.pipeline.removeHandler(httpHandler, promise: nil)
    }
)

return context.pipeline.configureHTTPServerPipeline(withServerUpgrade: config).flatMap({
    var future: EventLoopFuture<Void> = context.eventLoop.makeSucceededVoidFuture()
    context.pipeline.handler(type: HTTPServerUpgradeHandler.self).whenSuccess { serverUpgradeHandler in
        future = context.pipeline.addHandler(httpHandler, name: "NTWebSocketHTTPHandler", position: .before(serverUpgradeHandler))
    }
    return future
})

The handlers look right now:

                                         [I] ↓↑ [O]
                         NIOSSLServerHandler ↓↑ NIOSSLServerHandler   [NIOSSLServerHandler]
                    HTTPServerUpgradeHandler ↓↑                       [handler4]
                                             ↓↑ WebSocketFrameEncoder [handler5]
 ByteToMessageHandler<WebSocketFrameDecoder> ↓↑                       [handler6]
               WebSocketProtocolErrorHandler ↓↑                       [handler7]
                     NTWebSocketFrameHandler ↓↑                       [NTWebSocketFrameHandler]

But I run into another troubling problem, NTWebSocketFrameHandler is removed after being added to pipeline because a error, the logs are:

NTWebSocketFrameHandler handlerAdded called
NTWebSocketFrameHandler errorCaught called : uncleanShutdown
NTWebSocketFrameHandler handlerRemoved called

NTWebSocketFrameHandler just prints some logs, and does nothing else, the code as follows:

class NTWebSocketFrameHandler: ChannelInboundHandler {
    typealias InboundIn = WebSocketFrame

    var proxyContext: NTProxyContext

    init(proxyContext: NTProxyContext) {
        self.proxyContext = proxyContext
    }

    func handlerAdded(context: ChannelHandlerContext) {
        print("NTWebSocketFrameHandler handlerAdded called")
    }

    func handlerRemoved(context: ChannelHandlerContext) {
        print("NTWebSocketFrameHandler handlerRemoved called")
    }

    func channelActive(context: ChannelHandlerContext) {
        print(">>> NTWebSocketFrameHandler active")
    }

    func channelRead(context: ChannelHandlerContext, data: NIOAny) {
        let frame = self.unwrapInboundIn(data)
        print(">>> NTWebSocketFrameHandler frame.opcode : \(frame.opcode)")
        print(">>> NTWebSocketFrameHandler frame : \(frame)")
    }

    func errorCaught(context: ChannelHandlerContext, error: Error) {
        print("NTWebSocketFrameHandler errorCaught called : \(error)")
    }
}

Why is the uncleanShutdown error encountered?

18

uncleanShutdown is a TLS error, and is fired whenever we receive a TCP FIN packet (connection teardown) without having previously received a TLS CLOSE_NOTIFY alert. This can be a signal of a truncation attack. If you have alternative framing data (as HTTP and web sockets do) you can simply ignore it.

19

@yb_cherry I am struggling to intercept packets over HTTPS using a local proxy, would appreciate if you could chime in here: Capture packets using man-in-the-middle (MITM) proxy

20

But when upgrade succeed, NTWebSocketFrameHandler be removed quickly after being added to pipeline, so channelRead method has no chance to be called. This problem has been bothering me for a long time and I can't find any problems in my code.