Proposal: Structs in place of a request-handling function


(George) #1

I have been thinking recently about how Swift can be used to enable type-safe codebases for servers with high performance requirements. I’ve prototyped some of my ideas here: https://github.com/GeorgeLyon/Server (for an example see https://github.com/GeorgeLyon/Server/blob/master/Sources/Server/main.swift)

The main difference between my implementation and swift-server/http is that all of my request handlers are structs which the Server owns and mutates as data comes in. Eventually, this can enable a model with zero allocations per request. In addition, having the server own the memory that the request handlers use enable optimizations based on cache locality.

With the current design, we effectively guarantee at least one allocation per request because we process the body by returning a closure. If this function doesn’t capture anything it may be just a pointer but it is very likely it at least needs the ResponseWriter. My design doesn’t suffer from this limitation.

If this is something the community is interested in, I will gladly put together a PR,
George


(Cory Benfield) #2

I assume when you say “zero allocations per request” you mean “zero allocations of application data per request”, as we are presumably allocating some memory to process the HTTP.

Otherwise this design seems completely reasonable: using structures instead of closures is definitely a perfectly valid approach. I’m +0: there are benefits and costs to either model, and so don’t object to either design.

Cory

···

On 11 Dec 2017, at 01:27, George Leontiev via swift-server-dev <swift-server-dev@swift.org> wrote:

I have been thinking recently about how Swift can be used to enable type-safe codebases for servers with high performance requirements. I’ve prototyped some of my ideas here: https://github.com/GeorgeLyon/Server (for an example see https://github.com/GeorgeLyon/Server/blob/master/Sources/Server/main.swift)

The main difference between my implementation and swift-server/http is that all of my request handlers are structs which the Server owns and mutates as data comes in. Eventually, this can enable a model with zero allocations per request. In addition, having the server own the memory that the request handlers use enable optimizations based on cache locality.


(Helge Heß) #3

Well, while I like the premise, there are *so* many allocations in the current setup (just think about all the Strings in the HTTPRequestHead), this one should really be the last we care about :slight_smile:

As mentioned before I don’t particularly like the current API, but it is something people could agree on and which I think can do what higher level frameworks require. I’m also fine w/ using a protocol. But it should support the same functionality (or at least allow it at a higher level).

What I miss in your suggestion is a demo on how something complete would look like. I.e., what does the `echo` look like:

  https://github.com/ZeeZide/http-testserver/blob/master/Sources/http-testserver/main.swift#L54

and how would the async wait look like:

  https://github.com/ZeeZide/http-testserver/blob/master/Sources/http-testserver/main.swift#L111

A demo of an echo w/ back-pressure would be cool too.

Note: I’m not expecting working examples here, I would just like to see how you think those would look like in your API.

Thanks,
  Helge

···

On 11. Dec 2017, at 02:27, George Leontiev via swift-server-dev <swift-server-dev@swift.org> wrote:

With the current design, we effectively guarantee at least one allocation per request because we process the body by returning a closure.


(George) #4

@Cory
I do, in fact, mean “zero allocations per request”. I think this is a good thing to strive for, whether as a hard rule or as goal. With regard to your question about parsing, I’ve pushed a prototype of a zero-allocations wrapper for http_parser.c here <https://github.com/GeorgeLyon/SwiftHTTPParser>. There’s a few issues I still need to work through but I think it is enough to communicate the overall gist.

@Helge
I’m handwaving the actual HTTP writing part, but echo and async examples can be found in main.swift <https://github.com/GeorgeLyon/Server/blob/master/Sources/Server/main.swift> now. They are not meaningfully dissimilar from the ones you provided. I would argue avoid the return-a-closure semantic makes them cleaner, but that is just my opinion.

As for back pressure, could you provide a specific use case? It isn’t that I don’t believe back pressure is important, just that there are many places where it can be handled. For instance, the top-level Server object can have a maximum number of open connections of a particular type, or what is now the HTTPWriter can be something else which has a “applyBackpressure” method which synchronizes with the Server under the hood (think wrapping the DispatchQueue argument in your PR in a struct which does `queue.async{ applyBackpressure() }`) without revealing the queue to the handler.

···

On Dec 11, 2017, at 2:59 AM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

On 11. Dec 2017, at 02:27, George Leontiev via swift-server-dev <swift-server-dev@swift.org> wrote:

With the current design, we effectively guarantee at least one allocation per request because we process the body by returning a closure.

Well, while I like the premise, there are *so* many allocations in the current setup (just think about all the Strings in the HTTPRequestHead), this one should really be the last we care about :slight_smile:

As mentioned before I don’t particularly like the current API, but it is something people could agree on and which I think can do what higher level frameworks require. I’m also fine w/ using a protocol. But it should support the same functionality (or at least allow it at a higher level).

What I miss in your suggestion is a demo on how something complete would look like. I.e., what does the `echo` look like:

https://github.com/ZeeZide/http-testserver/blob/master/Sources/http-testserver/main.swift#L54

and how would the async wait look like:

https://github.com/ZeeZide/http-testserver/blob/master/Sources/http-testserver/main.swift#L111

A demo of an echo w/ back-pressure would be cool too.

Note: I’m not expecting working examples here, I would just like to see how you think those would look like in your API.

Thanks,
Helge

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev


(Helge Heß) #5

I do, in fact, mean “zero allocations per request”.

Well, I don’t see how this is possible with the current API design. You can propose a different API of course. Maybe just start w/ a complete API proposal, not an actual implementation, if you want to do this.
(You can’t expect people to browse your source code and dig for details :slight_smile:

I’ve pushed a prototype of a zero-allocations wrapper for http_parser.c here.

What is the point of that? The parser is internal to the API and already is zero-allocating? That part seems completely useless to me.

There is a *lot* of potential for optimisation in the use of the C HTTP parser. For example we could avoid a lot of allocs by matching header names and maybe even values at the C level, instead of always creating fresh String’s. I did some of that in my PR 96, but even more could be done.
But right now: Premature, the API matters, optimisations like that can be added later :slight_smile:

(As mentioned, I would prefer a lower level `HTTPMessage` object, which directly peeks into and reuses the receive buffer. But again, all this has been discussed quite actively, and people insisted that String is what we want to have here, and that it’ll be fast enough - so I guess we just stick to that.)

I’m handwaving the actual HTTP writing part, but echo and async examples can be found in main.swift now. They are not meaningfully dissimilar from the ones you provided.

In fact they are. They neither support pipelining nor back pressure. No offence - but it is a little like Tanner’s PR - if you choose to ignore what HTTP and Johannes' API provides, everything gets so much easier indeed :wink:

Also, your writes do not seem to be async at all? I mean I didn’t review your full code, but e.g. your `AsyncHandler` blocks the main queue for writes?!
That sounds completely unacceptable to me for sure. (Not unacceptable in general, of course Apache does the same, but unacceptable for an API which is specifically supposed to support async operation).

I would argue avoid the return-a-closure semantic makes them cleaner, but that is just my opinion.

I don’t disagree with that, but Johannes API is something people could agree on. As I said, I don’t think the API is particularly nice, but in its way it is “feature complete” :wink:
In the end it won’t matter (as long as it works properly), because this part will usually be hidden by a higher level framework.

I think something you are missing is that the server may need to maintain multiple instances of your struct - one for each in-flight request (as discussed in PR #106). Multiple requests per connection already exist in HTTP/1.1, and w/ HTTP/2 this only gets more prominent.
That could be done more efficiently than capturing a closure, but in the end it is the same thing, you need to reserve a block of memory for each incoming message.

As for back pressure, could you provide a specific use case?

You can easily research on the web what back pressure is and what it is good for.

And no, I don’t think that “just stick a proper server in front of your Swift server” is a valid answer here :slight_smile:

revealing the queue to the handler.

As mentioned the `queue` argument is just an optimisation to synchronise stream access. It is easy the hide this part in a higher level framework by various means. If we want, we could even provide a free threaded response writer as part of the API, but I don’t think it is worth it, because the various frameworks all have their own idea on how to deal with HTTP streams anyways.

Of course, if you just remove async I/O or threading, you don’t need to synchronise anything anymore ;->

hh

···

On 17. Dec 2017, at 01:49, George Leontiev via swift-server-dev <swift-server-dev@swift.org> wrote:

On Dec 11, 2017, at 2:59 AM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

On 11. Dec 2017, at 02:27, George Leontiev via swift-server-dev <swift-server-dev@swift.org> wrote:

With the current design, we effectively guarantee at least one allocation per request because we process the body by returning a closure.

Well, while I like the premise, there are *so* many allocations in the current setup (just think about all the Strings in the HTTPRequestHead), this one should really be the last we care about :slight_smile:

As mentioned before I don’t particularly like the current API, but it is something people could agree on and which I think can do what higher level frameworks require. I’m also fine w/ using a protocol. But it should support the same functionality (or at least allow it at a higher level).

What I miss in your suggestion is a demo on how something complete would look like. I.e., what does the `echo` look like:

https://github.com/ZeeZide/http-testserver/blob/master/Sources/http-testserver/main.swift#L54

and how would the async wait look like:

https://github.com/ZeeZide/http-testserver/blob/master/Sources/http-testserver/main.swift#L111

A demo of an echo w/ back-pressure would be cool too.

Note: I’m not expecting working examples here, I would just like to see how you think those would look like in your API.

Thanks,
Helge

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev


(Hasen Judy) #6

I for one think this kind of approach is very interesting.

Though I’m not sure how can it have zero allocations? How about allocating the space to hold the request body? What if the amount of incoming requests (or their bodies) exceed the pre-allocated space?


(Joannis Orlandos) #7

Instead of building up a Data-like buffer containing all of the data, you could see the body as an incoming stream of UnsafeBufferPointer<UInt8>.