That's a great question! The overly simplistic answer is to accumulate all the bits of the HTTP request body you receive in say a var receivedBits: CircularBuffer<ByteBuffer> in your ChannelHandler. Once you receive them, you write them to disk one after the other using NonBlockingFileIO.
The reason I called this overly simplistic is because that would leave your server vulnerable to a fairly straightforward denial of service attack: If writing to disk is slower than the client sending you data, your server will eventually die because it runs out of memory.
To solve this problem, we need to exert backpressure from the file system APIs through the Channel, over TCP to the client sending this data.
[[ beginning of a few paragraphs copy & pasted from my example implementation ]]
First, let's establish what it means to propagate backpressure from the file system into the Channel: Let's assume we have a HTTP server that accepts arbitrary amounts of data and writes it to the file system. If data is received faster over the network than we can write it to the disk, then the server runs into trouble: It can now only either buffer the data in memory or (at least in theory) drop it on the floor. The former would easily be usable as a denial of service exploit, the latter means that the server isn't able to provide its core functioniality. Backpressure is the mechanism to the the buffering issue above. The idea is that the server stops accepting more data from the client than it can write to disk. Because HTTP runs over TCP which has flow-control built in, the TCP stacks will then lower the server's receive window size which means that the client gets slowed down of completely stopped from sending any data. Once the server finishes writing previously received data to disk, it starts draining the receive buffer which then make TCP's flow control raise the window sizes which allows the client to send further data.
Backpressure in SwiftNIO
In SwiftNIO, backpressure is propagated by stopping to call the outbound read event. By default, Channels in SwiftNIO have the autoRead ChannelOption enabled. When autoRead is enabled, SwiftNIO will automatically send a read (note, this is a very different event than the inbound channelRead event that is used to deliver data) event when the previous read burst has completed (signalled by the inbound channelReadComplete event).
Therefore, you may be unaware of the existance of the read event despite having used SwiftNIO before. Suppressing the read event is one of the key demonstrations of this example. The fundamental idea is that to start with we let read flow through the ChannelPipeline until we have an HTTP request and the first bits of its request body. Once we received the first bits of the HTTP request body, we will _ suppress _ read from flowing through the ChannelPipeline which means that SwiftNIO will stop reading further data from the network.
When SwiftNIO stops reading further data from the network, this means that TCP flow control will kick in and slow the client down sending more of the HTTP request body (once both the client's send and the server's receive buffer are full). Once the disk writes of the previously received chunks have completed, we will issue a read event (assuming we held up at least one). From then on, read events will flow freely until the next bit of the HTTP request body is received, when they'll be suppressed again.
This means however fast the client or however slow the disk is, we should be able to stream arbitarily size HTTP request bodies to disk in constant memory.
[[ c&p end ]]
Okay, how do we implement this
The implementation idea goes like this:
- Once we received the HTTP request head, we open the file
- Once we receive a body chunk, we write it to the file and start buffering the next ones that may arrive
- Whilst we're writing to disk, we're suppressing
readto slow the client down - When we're finishing writing the previously received chunks, we
readagain and continue - If there's any errors or we're done, we close the file and respond accordingly
How much code is this?
Quite a bit. As an example, I've implemented a simple HTTP server (don't use this in production!) that just saves anything it receives to a file in /tmp/uploaded_file_*.
Most of the code is fairly straightforward, but it does contain an almost 250 line long state machine that coordinates the whole receive/backpressure/write to disk spiel. It is totally possible to knock this up in an ad-hoc version directly inside a ChannelHandler but I feel this is complex enough that it warrants a completely I/O and side-effect free state machine that is super straightforward to test (and also debug if something goes wrong).
Why is this so hard?
This is so hard because we're combining two (incompatible) streaming pieces - a Channel and a NonBlockingFileIO - in a non-trivial way. Sure, they both come out of the swift-nio package but SwiftNIO's core mission is to deal with everything that happens inside a Channel. We only even ship NonBlockingFileIO because there's no real alternative thus far so we had to include it but it's definitely not anything that we consider the core of our API... So once you have to glue that Channel to say a NonBlockingFileIO, you've got to write the glue yourself.
And writing glue to glue two asynchronous pieces that can both deliver a variety of events and fail at any time isn't that easy. It's sometimes easy to forget that events can arrive in really inconvenient and unexpected orders 
. Therefore, we always suggest to use explicit state machines, that looks verbose but is the right call if you're building something that's more than just a toy. To learn more, my colleague @lukasa gave a wonderful talk about state machines that I can recommend to anybody who wants to retain their sanity whilst writing event-driven software .
IMHO, the right answer for the future here is to get the ReactiveStreams game going on all platforms and offer standard adapters for both SwiftNIO's ChannelPipeline as well as NonBlockingFileIO or hopefully something much better. Once we've got that going, it should be as straightforward as plugging two pieces together in a few lines of code.