Keeping servers robust


(Andrew Akira Toulouse) #1

Has there been any thoughts to strategies for keeping servers written in
Swift running? People are certain to write bugs resulting in crashes from
time to time, and I think that natively and gracefully handling
process-death and relaunching workers would be something that probably gets
standardized – if not by the provided APIs, then by whatever project
springs up to fill that gap first.

Spitballing: Is there perhaps an IPC abstraction that can work together
with Swift and libdispatch? IIRC, XPC is dispatch-based, and I'm
particularly fond of its API, but I'm not knowledgeable enough to evaluate
it or how much of it can be made to work without the availability of Mach
ports. I recall that some part of it uses launchd, but I'm not certain what
it's used for or the degree to which that integration is required.


(Alex Blewitt) #2

Has there been any thoughts to strategies for keeping servers written in Swift running? People are certain to write bugs resulting in crashes from time to time, and I think that natively and gracefully handling process-death and relaunching workers would be something that probably gets standardized – if not by the provided APIs, then by whatever project springs up to fill that gap first.

At the moment, a force unwrap of a nil value results in triggering a ud2 instruction which brings the process crashing down, taking all dispatch queues with it. This is a sensible mechanism for single-user applications and (for example) preserves data from corruption, but having a multi-user server it's not as desirable.

The problem is that when a crash occurs, even if you could disable it by pausing instead, would result in leaked memory. Any reference types (or protocol backed value types which have been promoted to the heap) will now never be cleaned up. Since there isn't any form of background garbage collection, that memory just stays there until the process terminates.

As such, the only way to deal with this at the moment is to have a front end dispatcher/load balancer that spawns multiple children, and each child handles one request. You can use built-in CGI systems – Kitura has a FastCGI implementation https://github.com/IBM-Swift/Kitura-net/tree/master/Sources/KituraNet/FastCGI for example – and have the web server spawn off single-use processes each time. One process per request is potentially not as performant as re-using processes for subsequent requests though.

Of course you can use launchd or equivalent to spawn up a number of worker processes and have them statically/dynamically registered with the load balancer/front end, and has the advantage that the processes can then be distributed across multiple hosts – micro services are often deployed in this way.

Spitballing: Is there perhaps an IPC abstraction that can work together with Swift and libdispatch? IIRC, XPC is dispatch-based, and I'm particularly fond of its API, but I'm not knowledgeable enough to evaluate it or how much of it can be made to work without the availability of Mach ports. I recall that some part of it uses launchd, but I'm not certain what it's used for or the degree to which that integration is required.

It is possible to write your own to do this, but if you're fielding HTTP traffic then it makes just as much sense to use existing HTTP load balancer technology to be able to perform this for you, and gives you an advantage that you can distribute over multiple hosts instead of a single host. There are some downsides – you end up parsing the HTTP request twice, for example (though in practice load balancers are typically sufficiently finely tuned that their parsing isn't a significant overhead in the processing of a request). Of course you can use something like gRPC https://github.com/grpc/grpc-swift to perform a pre-parsed XPC request from a client to server.

Alex

···

On 23 Nov 2016, at 19:01, Andrew Akira Toulouse via swift-server-dev <swift-server-dev@swift.org> wrote:


(Andrew Akira Toulouse) #3

In a HTTP-centric universe, FastCGI or load balancer software are great
solutions, but ceding control to an opaque external system means any
somewhat stateful network traffic that isn't HTTP is going to have to build
an ad-hoc system for dealing with a process that can be brought down by a
logic error.

In a world where you're dealing with continuous byte streams, for example
some sort of video encoder or multiplexer (main point: dependent on a
finite hardware resource) where you can't just scale the worker processes
really high, tighter coupling between the server process and its recovery
mechanism seems like it'd be important.

···

On Thu, Nov 24, 2016 at 2:26 AM, Alex Blewitt <alblue@apple.com> wrote:

On 23 Nov 2016, at 19:01, Andrew Akira Toulouse via swift-server-dev < > swift-server-dev@swift.org> wrote:

Has there been any thoughts to strategies for keeping servers written in
Swift running? People are certain to write bugs resulting in crashes from
time to time, and I think that natively and gracefully handling
process-death and relaunching workers would be something that probably gets
standardized – if not by the provided APIs, then by whatever project
springs up to fill that gap first.

At the moment, a force unwrap of a nil value results in triggering a ud2
instruction which brings the process crashing down, taking all dispatch
queues with it. This is a sensible mechanism for single-user applications
and (for example) preserves data from corruption, but having a multi-user
server it's not as desirable.

The problem is that when a crash occurs, even if you could disable it by
pausing instead, would result in leaked memory. Any reference types (or
protocol backed value types which have been promoted to the heap) will now
never be cleaned up. Since there isn't any form of background garbage
collection, that memory just stays there until the process terminates.

As such, the only way to deal with this at the moment is to have a front
end dispatcher/load balancer that spawns multiple children, and each child
handles one request. You can use built-in CGI systems – Kitura has a
FastCGI implementation https://github.com/IBM-Swift/Kitura-net/tree/
master/Sources/KituraNet/FastCGI for example – and have the web server
spawn off single-use processes each time. One process per request is
potentially not as performant as re-using processes for subsequent requests
though.

Of course you can use launchd or equivalent to spawn up a number of worker
processes and have them statically/dynamically registered with the load
balancer/front end, and has the advantage that the processes can then be
distributed across multiple hosts – micro services are often deployed in
this way.

Spitballing: Is there perhaps an IPC abstraction that can work together
with Swift and libdispatch? IIRC, XPC is dispatch-based, and I'm
particularly fond of its API, but I'm not knowledgeable enough to evaluate
it or how much of it can be made to work without the availability of Mach
ports. I recall that some part of it uses launchd, but I'm not certain what
it's used for or the degree to which that integration is required.

It is possible to write your own to do this, but if you're fielding HTTP
traffic then it makes just as much sense to use existing HTTP load balancer
technology to be able to perform this for you, and gives you an advantage
that you can distribute over multiple hosts instead of a single host. There
are some downsides – you end up parsing the HTTP request twice, for example
(though in practice load balancers are typically sufficiently finely tuned
that their parsing isn't a significant overhead in the processing of a
request). Of course you can use something like gRPC
https://github.com/grpc/grpc-swift to perform a pre-parsed XPC request
from a client to server.

Alex