Netty would be a good framework to look at


(Andrew Akira Toulouse) #1

I don't know to what degree I'm preaching to the choir, but I was
encouraged by an acquaintance on the Swift team to put my thoughts out
there whether or not I drop the mic and step away afterwards. So:

I've used a couple of networking/server/web frameworks over the years, and
the one that stands out for me is Netty. Netty is a highly asynchronous
Java framework used in very high-performance applications (including hedge
funds and Wall Stree, for example).

==aside==
I've looked at: Jersey (JAX-RS), JSR 356, Wangle (Facebook's Netty-ish C++
framework that only really runs on Linux), Finagle (Twitter's Scala layer

···

on top of Netty), writing my own with POSIX sockets + libdispatch, and writing my own with CFSockets (once each in ObjC and Swift). I also wrote
my own client-side networking framework at a previous job, modeling it very
roughly after the client bits of Netty (the product for which it was
implemented was never released however).

So, while I don't think I am the most qualified to suggest what a server
API's final form might look like (i've personally dabbled a bunch but I've
invested more heavily in client-side programming than server-side), I have
a couple of thoughts from an API consumer's side of things.
==end aside==

The common thread I've found to help with stability and reduce code churn
is the composability of layers of the stack. At a high level, the pattern I
found most useful was channels and pipelines. A pipeline would have a chain
of handlers to control the flow of data, and a channel would have the
specific instantiation of input->handler->handler->output. For example,
here's a sample pipeline:

==extended example==

* bitstream OR packet stream input (i.e. data sourced from disk, from
network, from a socket, or whatever)
* Frame Decoder, which buffers the bytes until a successful decode
* Text Decoder: decodes the frame into text. This is an example, but it
could otherwise some higher level message, could be some encoding other
than UTF8, Maybe it's a tightly packed set of bytes, maybe it's text
* JSON Decoder: parses JSON. Maybe it even goes directly to creating
structs, or maybe it creates a dictionary and passes it to...
* Model Decoder: inspects the dictionary and decides how to turn that into
an object or struct.
* Application logic
* Custom error handler (each handler decides if it can handle the message
passed down each stage, and errors can pass through or perhaps a previous
stage can be retried, with some context dictionary of metadata to track
previous failed attempts)
* Callback for output so that stateful things can be updated

Caching handlers would be insertable at will to bypass some pipeline
stages; statefulness would be possible to encapsulate within a given stage,
but by and large statefulness would express itself as a context parameter
rather than the handlers being stateful themselves. Clients would prefer
the above pipeline; servers would continue on to construct responses and
return, and none of these stages would be required to be run synchronously
(but pipeline allowing, they can) or on the same queue (they could draw on
thread/queue pools as executors).

Maybe the wire format changes from JSON to XML or to MessagePack or
whatever, but otherwise stays the same – swap that layer out. Maybe the
model framework changes from hand-rolled to autogenerated, or maybe an app
developer changed from an Objective-C model framework like Mantle to a
Swift one – swap that layer out. Maybe the developer created an
authentication layer that they insert for specific API calls that augments
the context metadata so API calls further down the pipeline can make use of
them (and maybe they support multiple concurrent users so they don't want
to initialize the handlers with auth data). And so on.

==end extended example==

This pattern has worked for me on servers, as well as on an iOS app (I
managed to get 0 frame drops in a very layout, network, and image-heavy app
with the approach, with transparent caching and guarantees of thread
safety), and from what I have seen, works also for API proxies, i.e.
Thrift-to-HTTP, or vice-versa, or protobuf, or whatever format you throw a
decoder or encoder on for.

So, that's a lot of background. Here's the TL;DR of my suggestions for the
network/server APIs:

1) Please highly prioritize composability. A good abstraction will allow
components to be composed upfront rather than creating sloppy asynchronous
calls that are very difficult to trace or debug.
2) Please heavily isolate state; or statelessness where reasonable.
3) It might spend some extra cycles, but being asynchronous (beyond just
nonblocking I/O) typically works better for apps and non-CPU-bound servers
– I think CPU-heavy synchronous workloads will tend to write their own
framework from low-level primitives anyways), so please build it around
that.

So, that's it for the lower-level server stuff. For the higher-level server
stuff, Dropwizard is a batteries-included framework (and the batteries are
good) that incorporates Jersey, and is an opinionated and incredibly fun
web/API framework. Dropwizard was built by Yammer, and has a number of good
ideas built in on top of the straightforward way of creating server
endpoints with it. The downside is that I didn't see anything like Jersey
which did an especially good job supporting WebSockets.

Hope you can take inspiration from those (Netty and Dropwizard) when
creating the Server APIs. I look forward to it!

Thanks,
Andy


(Alex Blewitt) #2

Yes, Netty is an excellent framework and there have been some discussions already with some of the Netty team. One open question is how to map the asynchronous behaviour into supported constructs in Swift, such as libdispatch, which means it isn't just a drop in API but needs further investigation.

Alex

···

On 30 Oct 2016, at 01:01, Andrew Akira Toulouse via swift-server-dev <swift-server-dev@swift.org> wrote:

I've used a couple of networking/server/web frameworks over the years, and the one that stands out for me is Netty. Netty is a highly asynchronous Java framework used in very high-performance applications (including hedge funds and Wall Street, for example).


(Helge Heß) #3

This goes back to my original question (https://lists.swift.org/pipermail/swift-server-dev/Week-of-Mon-20161024/000030.html):

  3) Do you really want just a Socket, or is what you really want
     a (byte) stream framework?

I’m not sure what people really want here (as part of this effort). There wasn’t a lot of feedback yet, but I think most of it was ‘just a basic socket’.

What you describe is similar to what I’m prototyping as Noze.io for Swift, it is described a little over here:

  http://noze.io/noze4nonnode/

It is not optimised for speed (yet) nor is the API geared towards Swift API style guides, but it does provide a pretty high level batch streaming framework that is completely asynchronous, provides piping and back pressure support. It has a type safe streaming API for arbitrary types (all your ‘frame decoder’, ‘text decoder’, ‘JSON decoder’ things). Maybe this is an OK demo, notice how the pipe input is UInt8 and the pipe output are RedisValue's:

  https://github.com/NozeIO/Noze.io/blob/master/Sources/redis/RedisParser.swift

Echo-Daemon with a user-level transform:

  https://github.com/NozeIO/Noze.io/blob/master/Samples/echozd/main.swift

Just to be clear: I’m _not_ suggesting to use this as the ‘socket’ for this project :slight_smile: But I think the basic concept of the streams may be applicable and worth considering IF the goal here is to actually provide something that high level.

hh

P.S.: A Swift Netty clone sounds excellent to me as well :slight_smile:

···

On 30 Oct 2016, at 02:01, Andrew Akira Toulouse via swift-server-dev <swift-server-dev@swift.org> wrote:

==extended example==


(Chris Bailey) #4

As Alex says, we actually have some of the contributors/committers for
Netty signed up to the work group - Tom Doron is one of them.

That means being able to build a highly performing async framework is very
much in our minds. We are also trying to build functional APIs though,
meaning we also have to make sure we support wider use cases - including
providing the ability to create a framework that does thread based
blocking I/O if you want to.

Chris

···

From: Alex Blewitt via swift-server-dev <swift-server-dev@swift.org>
To: Andrew Akira Toulouse <andrew@atoulou.se>
Cc: swift-server-dev <swift-server-dev@swift.org>
Date: 30/10/2016 08:40
Subject: Re: [swift-server-dev] Netty would be a good framework to
look at
Sent by: swift-server-dev-bounces@swift.org

On 30 Oct 2016, at 01:01, Andrew Akira Toulouse via swift-server-dev <swift-server-dev@swift.org> wrote:

I've used a couple of networking/server/web frameworks over the years,

and the one that stands out for me is Netty. Netty is a highly
asynchronous Java framework used in very high-performance applications
(including hedge funds and Wall Street, for example).

Yes, Netty is an excellent framework and there have been some discussions
already with some of the Netty team. One open question is how to map the
asynchronous behaviour into supported constructs in Swift, such as
libdispatch, which means it isn't just a drop in API but needs further
investigation.

Alex
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev