Local NIO proxy server, started in NetworkExtension

YuriyNess · October 21, 2021, 3:15pm

Im new in SwiftNIO and NetworkExtension. I try to build proxy server which will be working in iOS Network extension. Thanks to some samples I had done this. But I can't understand how it is working((
Will be very thankful if you can answer some questions.

I use PacketTunnelProvider and on startTunnel

override func startTunnel(options: [String: NSObject]?, completionHandler: @escaping (Error?) -> Void) {
        // Add code here to start the process of connecting the tunnel.
        Logger.tunProvider.info("GGGG ---- startTunnel method called ----")
        
        pendingStartCompletion = completionHandler
        proxyService = ProxyService(host: defaultHost, port: defaultPort)
        
        proxyService?.start { result in
            switch result {
            case .success:
                self.setupSession()
                
            case .failure(let error):
                Logger.tunProvider.error("***************** Proxy Service Run Failed! \(error.localizedDescription, privacy: .public)")
                self.pendingStartCompletion(error)
            }
        }
    }

I create NIO server (on localhost)
(default example from swift-nio-examples/connect-proxy at main · apple/swift-nio-examples · GitHub)

public func start(_ completion: @escaping (Result<Void, Error>) -> Void) {
        Logger.proxyService.info("----- Starting Proxy Service -----")
        networkMonitorService.startMonitoring()
        
        bootstrap = ServerBootstrap(group: groupWorker)
            .serverChannelOption(ChannelOptions.socket(SOL_SOCKET, SO_REUSEADDR), value: 1)
            .childChannelOption(ChannelOptions.socket(SOL_SOCKET, SO_REUSEADDR), value: 1)
            .childChannelInitializer { channel in
                channel.pipeline.addHandler(ByteToMessageHandler(HTTPRequestDecoder(leftOverBytesStrategy: .forwardBytes))).flatMap {
                    channel.pipeline.addHandler(HTTPResponseEncoder()).flatMap {
                        channel.pipeline.addHandler(ConnectHandler(self.networkMonitorService.currentCunnection))
                    }
                }
            }
        
        do {
            bootstrap.bind(to: try SocketAddress(ipAddress: host, port: port)).whenComplete { result in
                // Need to create this here for thread-safety purposes
                switch result {
                case .success(let channel):
                    Logger.proxyService.info("Listening on \(String(describing: channel.localAddress), privacy: .public)")
                    completion(.success(()))
                case .failure(let error):
                    Logger.proxyService.error("Failed to bind \(self.host, privacy: .public):\(String(self.port), privacy: .public), \(error.localizedDescription, privacy: .public)")
                    completion(.failure(error))
                }
            }
        } catch {
            Logger.proxyService.error("Can't create address for \(self.host, privacy: .public):\(String(self.port), privacy: .public), \(error.localizedDescription, privacy: .public)")
        }
    }

ConnectHandler is main listener handler, it has

typealias InboundIn = HTTPServerRequestPart
    typealias OutboundOut = HTTPServerResponsePart

, and on channelRead it connect to - 2 GlueHandler

private func connectTo(host: String, port: Int, context: ChannelHandlerContext, throttle: Bool = false) {
        Logger.proxyService.info("Connecting to \(String(describing: host), privacy: .public):\(String(describing: port), privacy: .public)")
        
        let channelFuture = ClientBootstrap(group: context.eventLoop)
            .connect(host: String(host), port: port)
        
        channelFuture.whenSuccess { channel in
            self.connectSucceeded(channel: channel, context: context, throttle: throttle)
        }
        channelFuture.whenFailure { error in
            self.connectFailed(error: error, context: context)
        }
    }

private func connectSucceeded(channel: Channel, context: ChannelHandlerContext, throttle: Bool) {
        Logger.proxyService.info("Connected to \(String(describing: channel.remoteAddress), privacy: .public)")
        
        switch self.upgradeState {
        case .beganConnecting:
            // Ok, we have a channel, let's wait for end.
            self.upgradeState = .awaitingEnd(connectResult: channel)
            
        case .awaitingConnection(pendingBytes: let pendingBytes):
            // Upgrade complete! Begin gluing the connection together.
            self.upgradeState = .upgradeComplete(pendingBytes: pendingBytes)
            self.glue(channel, context: context, throttle: throttle)
            
        case .awaitingEnd(let peerChannel):
            // This case is a logic error, close already connected peer channel.
            peerChannel.close(mode: .all, promise: nil)
            context.close(promise: nil)
            
        case .idle, .upgradeFailed, .upgradeComplete:
            // These cases are logic errors, but let's be careful and just shut the connection.
            context.close(promise: nil)
        }
    }


 private func glue(_ peerChannel: Channel, context: ChannelHandlerContext, throttle: Bool = false) {
        Logger.proxyService.debug("Gluing together \(String(describing: ObjectIdentifier(context.channel)), privacy: .public) and \(String(describing: ObjectIdentifier(peerChannel)), privacy: .public)") // swiftlint:disable:this line_length
        
        // Ok, upgrade has completed! We now need to begin the upgrade process.
        // First, send the 200 message.
        // This content-length header is MUST NOT, but we need to workaround NIO's insistence that we set one.
        let headers = HTTPHeaders([("Content-Length", "0")])
        let head = HTTPResponseHead(version: .init(major: 1, minor: 1), status: .ok, headers: headers)
        context.write(self.wrapOutboundOut(.head(head)), promise: nil)
        context.writeAndFlush(self.wrapOutboundOut(.end(nil)), promise: nil)
        
        // Now remove the HTTP encoder.
        self.removeEncoder(context: context)
        
        // Now we need to glue our channel and the peer channel together.
        let (localGlue, peerGlue) = GlueHandler.matchedPair(throttle)
        context.channel.pipeline.addHandler(localGlue).and(peerChannel.pipeline.addHandler(peerGlue)).whenComplete { result in
            switch result {
            case .success(_):
                context.pipeline.removeHandler(self, promise: nil)
            case .failure(_):
                // Close connected peer channel before closing our channel.
                peerChannel.close(mode: .all, promise: nil)
                context.close(promise: nil)
            }
        }
    }

I don't clearly understand how data are going from device -> to proxy handlers -> to external resources, and how it goes back. I read How to understand InboundIn InboundOut OutboundOut OutboundIn names, but still cant imaging all the flow.
Why do we need GlueHandlers, and why do we remove Connect handler after GlueHandles has been added to the pipeline?

lukasa · October 21, 2021, 4:16pm

Your two questions are intimately related to each other, so I'll answer them together.

A Channel corresponds to a single connection. The Channel has only one "end" that is connected to the network: the head. The tail is dangling freely.

To build a proxy server, however, you need two connections: one from the client from you, and one from you to the destination. This means you must necessarily have two Channels: one represents the client connection to you, and one represents your connection to the destination. In the normal case, once the proxy is created, you're trying to forward the bytes from the client to the server and back again. This means that data you read from one Channel must be written to the other Channel, and the same in reverse. (Additionally, we need to pass some other events between the two Channels so that they do things like handle connection close or errors in graceful ways.)

Essentially what we want to do is to "glue" these two Channels together, tail to tail. This is what the GlueHandler does. They are always created in pairs, and know about their partner. Each GlueHandler will take the data it receives, and pass it to the pair for writing. This means that all reads from your first Channel turn into writes on your second Channel, and all reads from your second Channel turn into writes on your first. The result is that you have a "passthrough" connection.

This is what the GlueHandlers are for. As to why we remove the ConnectHandler once the GlueHandlers have been added: at that point, we're dealing with opaque data. The HTTP request at the start is used only to work out where the data is going to go: once we know that, we don't know anything about the rest of the data, and so that handler is just useless now. Only the GlueHandlers are going to do real work, so we remove the ConnectHandler.

This is also how the data goes from the device to the proxy handlers and then out to the network. It is read (via channelRead) on your first Channel, passed to the proxy handlers, and then to the GlueHandler in that channel. That GlueHandler passes the read to its partner, which turns it into a write on your second Channel. When data comes back from the network, the same thing happens, but in reverse.

YuriyNess · October 22, 2021, 10:58am

I try to write all the steps which I can imagine.

We create ServerSocketChannel, which manage inbound events. Add handlers and the last handler is ConnectHandler
When user start connecting to network, he connect to socket, on which work our ServerSocketChannel
In ConnectHandler we have Inbound connection from user. In ChannelRead we are handling initial message.
We get host and port ,from the (.head), to which user want to connect
1. Can we somehow get access to request params in this place? In some other place?
We create ClientChannel, add it to current eventLoop and connect it host and port which we saved previously when unwraping head.
We invoke Glue method and in it we make outBound write to our ServerChannel Context context.write(self.wrapOutboundOut(.head(head)), promise: nil) context.writeAndFlush(self.wrapOutboundOut(.end(nil)), promise: nil), than create twoGlueHandlers, one add to current channel pipline another to new ClientChannel pipeline. On success we remove our ConnectHandler from pipline.
1. One Gluehandler will be listening in ConnectHandler pipline for inbound events?
2. Second GlueHandler will be working on clinet pipeline and wiill be waiting when first glue handler invoke partnerWrite(data), to connect to remote resource?
3. If user will create another connection to new endpoit we will again have invoking of Connection handler method, but how it is possible? we have remove it from the pipeline?
When Second GlueHandler (which connected to remote resource) will get response, it will have inbound event, and in its method channelRead he invoke partnerWrite(data) and data will go to first GlueHandler, and thand in will create outBound write to user device?

YuriyNess · October 22, 2021, 3:09pm

@lukasa can you look at the previous post (UPDATE)

lukasa · October 22, 2021, 3:24pm

No, ServerSocketChannel manages accepting new inbound connections. You aren't adding the ConnectHandler to that, you're adding it to its child channels. Each of these corresponds to a new connection.

Yes, you can get access to request params here from the HTTPRequestHead object.

Yes.

If a user creates a new connection to a new endpoint they will also create a new connection to your proxy. This will cause a new channel to be created with a new ConnectHandler, and the dance will begin again.

Yes.