General Question Regarding ServerBootstrap

[tdotclare (Teague Clare)]

Greetings!

I've been reviewing the codebase to get a clearer idea of the flow of handling, and have a few questions regarding process flow that aren't fully clear to me in terms of what lifetime expectations are when setting up a server flow with ServerBootstrap:

1. I assume in that in all cases, childChannelInit is called for every established connection, and thus for y established connections, there will be exactly that many references to a corresponding Channel/Pipeline/Context combo (though they may be shared references should no changes to a pipeline occur on-the-fly). Correct me if my presumption is wrong.

2. Is the relationship of serverChannelInits to child channels thus 1 to y, x to y, or y to y?

   • My understanding of ServerBootstrap on a surface level suggests that, essentially, serverChannelInit would solely be responsible for Channel operations that will occur on the initial incoming call to the bound port (and listening for a possible user quiescing event), and childChannelInit effectively handles all the actual communications. From appearances, only one serverChannel (for a specific instance of ServerBootstap) will ever exist - EG, 1 to y.

     • Further, I assume in this case that while there is only one serverChannel, there may be numerous ChannelHandlerContexts that reference that one channel

   • ... Or will additional instances be generated by some mechanism when backlog triggers it, should the server channel that's "up" be actively in the process of handling a preceding connection? - EG, it might be 1 to y or it might be x to y

   • Final alternative, will server inits happen for every new incoming connection, regardless of duration, and essentially "shift" to the resultant child channel init, with a one-to-one correspondence? - EG, it will always be y to y

3. By comparison, DatagramBootstrap has no method of distinguishing between a "server" and "child" channel, so presumably every incoming envelope is handled by a distinct channel (which may or may not be reused? I'm unclear on this).

   • This makes me think the situation with ServerBootstrap is more likely the x to y ratio, where server channels may "come up" to be reused as they detach connections to childChannels, but I have no idea.

   • Possibly a Datagram-based server lacks the childChannelInit simply because it's assumed that such a server would be dealing with much smaller tasks and would be wasting overhead by switching to a child - EG, an NTP server?

4. Related; I notice that ServerBootstrap allows for distinctly binding the serverChannel(s) created to a EventLoopGroup separate from the one on which childChannels are created, but I can find no reference to any codebase either in NIO or elsewhere using this init.

   • I presume this is intended as a possible optimization tool, where one might try to tweak real-world performance by ensuring the listening/server channels have a prioritized/dedicated ELG on a higher QoS level?

---

For an arbitrary example, say this server is an http1.1 implementation and a client makes sixteen distinct requests from 4 remote ports on persistent connections; I would assume in this case that somewhere between 1 and 4 server channels would be init'd, and 4 child channels would be init'd.

Alternatively, if an http2 connection that takes sixteen requests on a single connection that's multiplexed , only created one server and one child channel will be created (though the child channel itself may indeed have additional child channels for multiplexing, but these, I don't believe, are functionally the same as the ServerBoostrap child channels, more like "grandchildren".)

In both cases, once a/the server channel has initially set up the connection and IP tuple, it is never in the path of incoming/outgoing traffic that's established, I believe - that goes directly to the child channels? Ergo only a childChannel will ever have a remoteAddress set, not a serverChannel?

[lukasa (Cory Benfield)]

@tdotclare

1. I assume in that in all cases, childChannelInit is called for every established connection, and thus for y established connections, there will be exactly that many references to a corresponding Channel/Pipeline/Context combo (though they may be shared references should no changes to a pipeline occur on-the-fly). Correct me if my presumption is wrong.

This is broadly right, but the allusion to "exactly that many references" is wrong. The Channel and Pipeline and ChannelHandlerContext objects all have mutual backreferences of various kinds.

@tdotclare

Is the relationship of serverChannelInits to child channels thus 1 to y, x to y, or y to y?

Strictly speaking the answer is x to y, where x is the number of times bind has been called. In most programs, however, x is 1.

The serverChannelInit is called exactly once per bind, and it sets up the server channel. The server channel then handles each new inbound connection, configuring it and setting up its channel, before relinquishing it. We never spawn new ones without user input, regardless of backlog or load.

@tdotclare

By comparison, DatagramBootstrap has no method of distinguishing between a "server" and "child" channel, so presumably every incoming envelope is handled by a distinct channel (which may or may not be reused? I'm unclear on this).

Not quite. DatagramBootstrap produces DatagramChannels. NIO's model of a DatagramChannel does not require it to be connected, and when it isn't it can send packets to and receive packets from any address. When operating in this mode, a single DatagramChannel can handle potentially all of the datagrams sent or received by an entire program.

Optionally, the DatagramChannel may be connected. In this case, it can only send and receive packets to the address to which it is connected. In that case, different Channels may be handling packets sent to the same local address/port combo.

You should not analogise DatagramBootstrap to ServerBootstrap: NIO does not have a built-in way to have a datagram "server" today. We could build such a thing though, by using connected mode.

@tdotclare

Related; I notice that ServerBootstrap allows for distinctly binding the serverChannel(s) created to a EventLoopGroup separate from the one on which childChannels are created, but I can find no reference to any codebase either in NIO or elsewhere using this init.

• I presume this is intended as a possible optimization tool, where one might try to tweak real-world performance by ensuring the listening/server channels have a prioritized/dedicated ELG on a higher QoS level?

Yes, that's the intention. This is typically required only in the absolutely highest load servers, where it is possible that you are receiving inbound connections so fast that accept burns an entire thread to process, or alternatively that you are so heavily slammed processing data that you can't accept new ones on your other threads.

In general, NIO is fast enough that the vast majority of programs using it only want to give it one thread. This is why you don't see it used anywhere: it's absolutely a last 0.1% solution.

@tdotclare

For an arbitrary example, say this server is an http1.1 implementation and a client makes sixteen distinct requests from 4 remote ports on persistent connections; I would assume in this case that somewhere between 1 and 4 server channels would be init'd, and 4 child channels would be init'd.

Assuming that bind was called only once, it's 1 server channel and 4 child channels.

@tdotclare

Alternatively, if an http2 connection that takes sixteen requests on a single connection that's multiplexed , only created one server and one child channel will be created (though the child channel itself may indeed have additional child channels for multiplexing, but these, I don't believe, are functionally the same as the ServerBoostrap child channels, more like "grandchildren".)

Correct: 1 server channel, 1 connection channel, 16 HTTP/2 stream channels, forming a tree. The parent of each stream channel is the connection channel, and the parent of the connection channel is the server channel.

(Sidebar: NIO's Channels have a parent property that you can use to walk this tree.)

@tdotclare

In both cases, once a/the server channel has initially set up the connection and IP tuple, it is never in the path of incoming/outgoing traffic that's established, I believe - that goes directly to the child channels? Ergo only a childChannel will ever have a remoteAddress set, not a serverChannel?

Correct.

[tdotclare (Teague Clare)]

This is broadly right, but the allusion to "exactly that many references" is wrong. The Channel and Pipeline and ChannelHandlerContext objects all have mutual backreferences of various kinds

Gotcha. should have said "at least y initializations" - 16 "forked" children from the server channel will equate to 16 calls to childChannelInit?

Strictly speaking the answer is x to y , where x is the number of times bind has been called. In most programs, however, x is 1.

OK - I get that distinction! So one might manually spool up multiple serverChannel instances through the same ServerBootstrap, even bound to the same socket, but it won't arbitrarily do so.

You should not analogise DatagramBootstrap to ServerBootstrap

Makes sense - I haven't really looked closely at it, just was skimming it to compare to ServerBootstrap

Yes, that's the intention. This is typically required only in the absolutely highest load servers, where it is possible that you are receiving inbound connections so fast that accept burns an entire thread to process, or alternatively that you are so heavily slammed processing data that you can't accept new ones on your other threads.

Gotcha. That's really the edge case I'm curious about though, hah - not trying to over optimize early, just want to have a sense of where I want to stay flexible.

As a real world example of why I'm confused and/or curious about this when reviewing public codebases for interpretation; consider Vapor's HTTP initializers, particularly this aspect inside the childChannelnitializer:

do {
  sslContext = try NIOSSLContext(configuration: tlsConfiguration)
  tlsHandler = NIOSSLServerHandler(context: sslContext, customVerifyCallback: configuration.customCertificateVerifyCallback)
} catch {
  configuration.logger.error("Could not configure TLS: \(error)")
  return channel.close(mode: .all)
}

By my understanding, this means all new childChannels (where HTTPS is configured) will independently call NIOSSLContext, which could have a pretty significant throttling effect on throughput since childChannelInit always is called on an EL. Is my interpretation amiss?

(Sidebar: NIO's Channels have a parent property that you can use to walk this tree.)

Yup! Noted that.

Thanks a ton!

[lukasa (Cory Benfield)]

@tdotclare

Gotcha. should have said "at least y initializations" - 16 "forked" children from the server channel will equate to 16 calls to childChannelInit?

Yes.

@tdotclare

So one might manually spool up multiple serverChannel instances through the same ServerBootstrap, even bound to the same socket, but it won't arbitrarily do so.

We have to be a bit terminologically careful here.

Yes, one may spool up multiple server channels through the same bootstrap. Those channels all have to listen on different ports, however, unless you specify SO_REUSEPORT. Then you can have multiple server channels listening on the same port. However, you can never have multiple server channels listening on the same socket. A socket is a single FD, and in NIO land these are always uniquely owned.

@tdotclare

By my understanding, this means all new childChannels (where HTTPS is configured) will independently call NIOSSLContext, which could have a pretty significant throttling effect on throughput since childChannelInit always is called on an EL. Is my interpretation amiss?

Your interpretation is correct. Unfortunately NIOSSLContext is frustratingly expensive to construct, so we'd typically recommend constructing it outside of the event loop. We'd also recommend caching it, as that enables TLS session resumption.

AsyncHTTPClient caches these in a dictionary to deal with the fact that users may use the same client with different TLSConfigurations.

[tdotclare (Teague Clare)]

We have to be a bit terminologically careful here.

Yes, one may spool up multiple server channels through the same bootstrap. Those channels all have to listen on different ports, however, unless you specify SO_REUSEPORT. Then you can have multiple server channels listening on the same port. However, you can never have multiple server channels listening on the same socket. A socket is a single FD, and in NIO land these are always uniquely owned.

Noted - yes, I'd meant port - was replying when I'd been up for far too long!

Your interpretation is correct. Unfortunately NIOSSLContext is frustratingly expensive to construct, so we'd typically recommend constructing it outside of the event loop. We'd also recommend caching it, as that enables TLS session resumption.

Got it!

Thanks again so much for the clarifications.