HTTP API v0.1.0

I like that :wink:

聽聽writeHeader(status:headers:)
聽聽writeHeader(method:target:headers:)

Look good to me, though I would prefer to have a distinct method for the continue and alike, so that the implementation can guard against usage errors.

hh

路路路

On 7. Nov 2017, at 17:02, Cory Benfield <cbenfield@apple.com> wrote:

On 7 Nov 2017, at 15:56, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:

On 7. Nov 2017, at 16:51, Georgios Moschovitis <george.moschovitis@icloud.com> wrote:

There is also

e) writeHeader => writeHead

The current API uses writeHeader for all headers. (writeHead sounds more like the one single response header). Though I agree it is in line with HTTPRequestHead :thinking:

If you want to be really pedantic, you are writing one response header section. The individual elements of the header section are the response line and zero or more header *fields*.

Of course, and I can not stress this enough, these are all details that should not particularly factor in to your decision. They鈥檙e absurdly pedantic.

Hi Carl,

b) A complete 鈥榪ueue free鈥 model. I鈥檓 doing this in my current approach. It is kinda lock free, but has a lot of async dispatching. The base performance overhead is/should-be pretty high, but scalability is kinda like to optimal (theoretically making use of as many CPUs as possible).

Not sure how well this goes in Linux. Are DispatchQueue鈥檚 also cheap on Linux or does the current implementation create a new thread for each?

From what I've seen, they're pretty expensive on Linux.

I had a previous implementation (prior to https://github.com/carlbrown/HTTPSketch/commit/fd083cd30 ) that created a new `DispatchQueue` for every `accept()` (i.e. each new TCP connection), and it performed horribly on Linux. It got much, much faster once I created a fixed pool of queues and reused/shared them between connections.

did you have anything that blocks on the queues? If yes, then that's probably why it performed badly (needed to spawn lots of threads).

The queues themselves should really be cheap but if there's fresh threads needed for lots of queues all the time, then performance will be bad. And probably terrible on Linux.

-- Johannes

路路路

On 2 Nov 2017, at 9:15 am, Carl Brown via swift-server-dev <swift-server-dev@swift.org> wrote:

On Nov 1, 2017, at 8:13 PM, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:

-Carl

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Hi Carl,

b) A complete 鈥榪ueue free鈥 model. I鈥檓 doing this in my current approach. It is kinda lock free, but has a lot of async dispatching. The base performance overhead is/should-be pretty high, but scalability is kinda like to optimal (theoretically making use of as many CPUs as possible).

Not sure how well this goes in Linux. Are DispatchQueue鈥檚 also cheap on Linux or does the current implementation create a new thread for each?

From what I've seen, they're pretty expensive on Linux.

I had a previous implementation (prior to https://github.com/carlbrown/HTTPSketch/commit/fd083cd30 ) that created a new `DispatchQueue` for every `accept()` (i.e. each new TCP connection), and it performed horribly on Linux. It got much, much faster once I created a fixed pool of queues and reused/shared them between connections.

did you have anything that blocks on the queues? If yes, then that's probably why it performed badly (needed to spawn lots of threads).

The queues themselves should really be cheap but if there's fresh threads needed for lots of queues all the time, then performance will be bad. And probably terrible on Linux.

They shouldn't have been blocked in the "waiting around with nothing to do" sense, but they should all have been really busy (my high load test suite uses 60-80 simultaneous curl processes hitting an 8-core server). There were lots of threads spawned, according to `top`. When I constrained the number of queues to be roughly the same as the number of cores, performance got much better.

I should say that this was with Swift 3.1.X. I haven't re-run that test since Swift 4.0 shipped. I don't know if there were any changes that would make for a different result now.

-Carl

路路路

On Nov 2, 2017, at 12:33 PM, Johannes Wei脽 <johannesweiss@apple.com> wrote:

On 2 Nov 2017, at 9:15 am, Carl Brown via swift-server-dev <swift-server-dev@swift.org> wrote:

On Nov 1, 2017, at 8:13 PM, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:

-- Johannes

-Carl

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev

b) A complete 鈥榪ueue free鈥 model. I鈥檓 doing this in my current approach. It is kinda lock free, but has a lot of async dispatching. The base performance overhead is/should-be pretty high, but scalability is kinda like to optimal (theoretically making use of as many CPUs as possible).

Not sure how well this goes in Linux. Are DispatchQueue鈥檚 also cheap on Linux or does the current implementation create a new thread for each?

From what I've seen, they're pretty expensive on Linux.

Do we know why? Is that a transient issue that is going to be fixed in the future? Do they actually create a new thread per queue?

I had a previous implementation (prior to https://github.com/carlbrown/HTTPSketch/commit/fd083cd30 ) that created a new `DispatchQueue` for every `accept()` (i.e. each new TCP connection), and it performed horribly on Linux. It got much, much faster once I created a fixed pool of queues and reused/shared them between connections.

This is what I wondered about. Yes, I also went with a DispatchQueue per connection first but a pool would be possible too. (though this really sounds kinda wrong and counter to the GCD idea).

In my a) and c) scenarios, you would just share one or a few top-level 鈥榞lobal鈥 queues, so that would be different too. Much less synchronisation overhead, but for c) you need to give the handler a way to dispatch back to the right queue.

If we get concurrency in Swift 5, and get actors, I would assume that the mailbox of an actor is a queue. If those don't scale on Linux 鈥 ;->

hh

P.S.: I think GCD may be fine for this project, I鈥檓 not yet convinced it is the right choice for like a bigger deployment. 炉\_(銉)_/炉

路路路

On 2. Nov 2017, at 17:15, Carl Brown <carl.brown.swift@linuxswift.com> wrote:

On Nov 1, 2017, at 8:13 PM, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:

I think if we really want async, we need a few API adjustments to make async efficient enough. E.g. maybe pass queues around (probably not a straight DispatchQueue if we don鈥檛 want to tie it to GCD, but a context which ensures synchronization - that would be efficient for sync too).

do you have suggestions how that could look?

Not really. I guess it would be sufficient if the handler gets it, like so:

聽聽func echo(request: .., response: .., queue: 鈥)

Though I was also wondering whether there should be a more general `WOContext` (ha) object which carries more details. Like a logging function to use, or other global (or HTTP transaction local) information.

But maybe that belongs into a higher level (and can be captured to the handler function).

What I would like to avoid is to make `queue:` a `queue: DispatchQueue`, but rather something like a simple

聽聽protocol SyncContext { func sync(_ cb: () -> ()) }

聽聽extension DispatchQueue {
聽聽聽聽func sync(_ cb: () -> ()) { async(execute: cb) }
聽聽}

Synchronous servers would immediately callback to the caller.

In our internal implementation I have bits of that but never got to the point to actually profiling stuff and I didn't go all the way.

Channels vs source and then doing manual read/write? Well, my basic assumption on this is that even if channels are slower today, they should be made as fast. Conceptually that should work.
I don鈥檛 remember what uv does, I think they are more like sources, but I鈥檓 not sure.

As mentioned, dispatch source has the little advantage (or not? I鈥檓 not convinced it is a good idea) that you can pass in those arbitrary buffer based objects. (And retain that original object).
The real *!?< is that Swift objects do not allow buffer access (as described in my previous mail). That results in a LOT of copying (IMO to a degree I consider it almost fair to say that Swift is simply not applicable for highperf implementations).

I didn鈥檛 do _any_ profiling yet. What I wanted to write up is a more realistic test for the implementation scalability. Ie something simple like

聽聽func scaleTestHandler(res) {
聽聽聽聽res.write(200)
聽聽聽聽setTimeout(0.150 * random-something) { // simulate database/other call
聽聽聽聽聽聽res.write(鈥渉appy?鈥)
聽聽聽聽聽聽res.end()
聽聽聽聽}
聽聽}

Then use ab to test it. Threaded/sync setups should fail that quickly while async ones presumably will just expose the scalability limits of GCD :->

My current async imp has a lot of dispatch overhead because the callbacks can be called from essentially any queue in the current API (affecting every read & write). There are a few models that can be used for doing async:

yes, we have that too

For me pipelining adds extra sync overhead. (you can pass writes directly to channel.write(), but if you need to spool because the previous request is not done, that takes another queue dispatch 鈥)

b) A complete 鈥榪ueue free鈥 model. I鈥檓 doing this in my current approach. It is kinda lock free, but has a lot of async dispatching. The base performance overhead is/should-be pretty high, but scalability is kinda like to optimal (theoretically making use of as many CPUs as possible).

there's indeed quite a few probably better models but I always thought of that as part of the 'networking/streams' track of the server APIs work group. We have a few ideas here, will follow up with that as soon as we can.

In my original Noze imp each stream also had (could have) its own queue, but I really thought that all the queuing will absolutely kill the performance. Don鈥檛 know.

For libdispatch I believe the following model should work very well:

- create a few number of 'base' queues, probably equal to the number of CPUs stored in an array 'baseQueues'
- for every request create a new DispatchQueue(label: "...", target: baseQueues[requestNo % baseQueues.count]) (where requestNo is a global atomic (oops) integer of the overall requests)

Sounds good. Note that the `accept`s also should run concurrently (I just put them on a concurrent queue).

the base queues will end up on different (kernel) threads and the request queues will be round-robin scheduled onto the base queues. That way we make sure we don't randomly spawn new threads which isn't good.

I鈥檓 not quite sure what 'randomly spawn new threads鈥 means. To be honest I expect GCD to do the work you describe above. That is, assign new queues to the optimal number of hosting threads.
The `baseQueues` things (while OK to implement) sounds entirely wrong to me. GCD is responsible for doing such stuff? But maybe I鈥檓 missing something.
(the `target` thing is a synchronisation stacking construct, not a threading one, IMO).

That model obviously only works iff the application code is either non-blocking or dispatches itself off the request queue if it needs to do blocking work. Needless to say we should aim for non-blocking but the reality of today's code in Swift doesn't entirely look like that :wink:

With this I don鈥檛 see an issue. If the higher level framework wants to work synchronously, it can dispatch its middleware functions and such to a worker queue.
We have back pressure, so the system should be able to deal with congestion.

But maybe this is just another hint that maybe there should be both options/implementations, a sync and a async one. Each has their application and merrits.

(And hey, Noze.io stuff is completely non-blocking, what are you talking about? :wink: )

Not sure how well this goes in Linux. Are DispatchQueue鈥檚 also cheap on Linux or does the current implementation create a new thread for each?

the queues themselves are cheap but the Linux implementation AFAIK behaves quite weirdly if it needs to spawn threads. IIRC there's one global thread which every 100ms evaluates if the existing threads are all blocked and if they are, it'll spawn a new thread. Haven't checked the code in a while, maybe someone knows better.

OK.

That's obviously not a great GCD implementation, the real GCD on macOS has kernel support to make that work much better. The same sadly applies to the eventing mechanism (DispatchSources) which are much more efficient and reduce thread hopping a lot on macOS.

Oh well.

But even on Linux I think not having many 'base' queues (which are queues that do not target other queues) should really give the best performance. Needless to say one has to be very careful not to ever block one of these base queues.

Sure.

c) Something like a), but with multiple worker queues. Kinda like the Node resolution, but w/o the different processes. This needs an API change, all the callbacks need get passed 鈥榯heir鈥 main queue (because it is not a global anymore).

Sorry, should've read the whole email before writing above. That sounds pretty much like what I wrote above, right? If you agree that sounds like the best model on GCD to me.

Yes. But unlike a) and b), this requires that the handler gets the queue it is running on, so that it can do:

聽聽聽func handler(req, res, queue httpQueue:鈥) {
聽聽聽聽聽bgQueue.async {
聽聽聽聽聽聽聽// very very very expensive work, like doing an Animoji
聽聽聽聽聽聽聽// done with it,
聽聽聽聽聽聽聽httpQueue.async(doneCallback)
聽聽聽聽聽}
聽聽聽}

If you get the point.

hh

路路路

On 2. Nov 2017, at 18:31, Johannes Wei脽 <johannesweiss@apple.com> wrote:

Hi,

b) A complete 鈥榪ueue free鈥 model. I鈥檓 doing this in my current approach. It is kinda lock free, but has a lot of async dispatching. The base performance overhead is/should-be pretty high, but scalability is kinda like to optimal (theoretically making use of as many CPUs as possible).

Not sure how well this goes in Linux. Are DispatchQueue鈥檚 also cheap on Linux or does the current implementation create a new thread for each?

From what I've seen, they're pretty expensive on Linux.

Do we know why? Is that a transient issue that is going to be fixed in the future? Do they actually create a new thread per queue?

good point. I reckon it's what I described (delay in spawning threads on Linux). But instead of guessing we should really have a minimal reproduction that shows the issue. No HTTP server but really only a thing that imitates the queue spawning behaviour. We could then analyse why it's slow and what to fix.

I had a previous implementation (prior to https://github.com/carlbrown/HTTPSketch/commit/fd083cd30 ) that created a new `DispatchQueue` for every `accept()` (i.e. each new TCP connection), and it performed horribly on Linux. It got much, much faster once I created a fixed pool of queues and reused/shared them between connections.

This is what I wondered about. Yes, I also went with a DispatchQueue per connection first but a pool would be possible too. (though this really sounds kinda wrong and counter to the GCD idea).

I don't think a pool of queues will help as I think they only get a thread spawned if there's work on them and the existing threads are all blocked. But again, we need a minimal benchmark/repro. The queue target hierarchy is very important to get the best performance out of GCD. This video has the latest information, can highly recommend: https://developer.apple.com/videos/play/wwdc2017/706/

-- Johannes

路路路

On 2 Nov 2017, at 9:54 am, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:
On 2. Nov 2017, at 17:15, Carl Brown <carl.brown.swift@linuxswift.com> wrote:

On Nov 1, 2017, at 8:13 PM, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:

In my a) and c) scenarios, you would just share one or a few top-level 鈥榞lobal鈥 queues, so that would be different too. Much less synchronisation overhead, but for c) you need to give the handler a way to dispatch back to the right queue.

If we get concurrency in Swift 5, and get actors, I would assume that the mailbox of an actor is a queue. If those don't scale on Linux 鈥 ;->

hh

P.S.: I think GCD may be fine for this project, I鈥檓 not yet convinced it is the right choice for like a bigger deployment. 炉\_(銉)_/炉

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Hi Carl,

Hi Carl,

b) A complete 鈥榪ueue free鈥 model. I鈥檓 doing this in my current approach. It is kinda lock free, but has a lot of async dispatching. The base performance overhead is/should-be pretty high, but scalability is kinda like to optimal (theoretically making use of as many CPUs as possible).

Not sure how well this goes in Linux. Are DispatchQueue鈥檚 also cheap on Linux or does the current implementation create a new thread for each?

From what I've seen, they're pretty expensive on Linux.

I had a previous implementation (prior to https://github.com/carlbrown/HTTPSketch/commit/fd083cd30 ) that created a new `DispatchQueue` for every `accept()` (i.e. each new TCP connection), and it performed horribly on Linux. It got much, much faster once I created a fixed pool of queues and reused/shared them between connections.

did you have anything that blocks on the queues? If yes, then that's probably why it performed badly (needed to spawn lots of threads).

The queues themselves should really be cheap but if there's fresh threads needed for lots of queues all the time, then performance will be bad. And probably terrible on Linux.

They shouldn't have been blocked in the "waiting around with nothing to do" sense, but they should all have been really busy (my high load test suite uses 60-80 simultaneous curl processes hitting an 8-core server). There were lots of threads spawned, according to `top`. When I constrained the number of queues to be roughly the same as the number of cores, performance got much better.

yes, constraining the number of base queues (ones that aren't targeting others) is a good idea. That way you can give every request its own queue. But that request queue should target a base queue. The basic idea could be:

- spawn a fixed number (best probably around number of CPUs but just using 16 or something should)
- create a new queue per request which targets one of the base queues (round robin should do)

that should give you much better performance as long as nothing blocks one queue for long.

I should say that this was with Swift 3.1.X. I haven't re-run that test since Swift 4.0 shipped. I don't know if there were any changes that would make for a different result now.

Definitely retry on Swift 4, the GCD implementation changed from using the kqueue adaptor on Linux to using epoll directly (think that was after Swift 3.1). Also bugs like https://bugs.swift.org/browse/SR-5759 have been fixed for Swift 4.0

-- Johannes

路路路

On 2 Nov 2017, at 10:53 am, Carl Brown <carl.brown.swift@linuxswift.com> wrote:

On Nov 2, 2017, at 12:33 PM, Johannes Wei脽 <johannesweiss@apple.com> wrote:

On 2 Nov 2017, at 9:15 am, Carl Brown via swift-server-dev <swift-server-dev@swift.org> wrote:

On Nov 1, 2017, at 8:13 PM, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:

-Carl

-- Johannes

-Carl

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Hi Helge,

I think if we really want async, we need a few API adjustments to make async efficient enough. E.g. maybe pass queues around (probably not a straight DispatchQueue if we don鈥檛 want to tie it to GCD, but a context which ensures synchronization - that would be efficient for sync too).

do you have suggestions how that could look?

Not really. I guess it would be sufficient if the handler gets it, like so:

func echo(request: .., response: .., queue: 鈥)

Though I was also wondering whether there should be a more general `WOContext` (ha) object which carries more details. Like a logging function to use, or other global (or HTTP transaction local) information.

But maybe that belongs into a higher level (and can be captured to the handler function).

What I would like to avoid is to make `queue:` a `queue: DispatchQueue`, but rather something like a simple

protocol SyncContext { func sync(_ cb: () -> ()) }

extension DispatchQueue {
聽聽聽func sync(_ cb: () -> ()) { async(execute: cb) }
}

Synchronous servers would immediately callback to the caller.

interesting. In our internal implementation we have an abstraction which has an API really similar to DispatchIO and two implementations of that. One is synchronous and one is DispatchIO. And at some point I had one which was DispatchSources.

And on these I do in fact have sync/async/notify (for DispatchGroup) methods. So basically the HTTPServer is generic over the IO mechanism it uses. And the IO mechanism has sync/async/notify methods that do the 'right' thing depending on if it's a sync or an async implementation.

In our internal implementation I have bits of that but never got to the point to actually profiling stuff and I didn't go all the way.

Channels vs source and then doing manual read/write? Well, my basic assumption on this is that even if channels are slower today, they should be made as fast. Conceptually that should work.

the only problems with the DispatchIO channels (you mean https://developer.apple.com/documentation/dispatch/dispatchio), right? is that they don't support back pressure directly. I created a layer on top of the IO which adds that on top of it with a _gross_ hack. By using DispatchSources one can implement that quite straightforwardly. What I should've done is in the aforementioned IO abstraction layer create the APIs I want to use (supporting back pressure there) and then implemented it where needed.

I don鈥檛 remember what uv does, I think they are more like sources, but I鈥檓 not sure.

yes, DispatchSources are just an eventing mechanism really. Libuv and friends are quite similar there.

As mentioned, dispatch source has the little advantage (or not? I鈥檓 not convinced it is a good idea) that you can pass in those arbitrary buffer based objects. (And retain that original object).

yes, DispatchSources are way more versatile, I just went for DispatchIO because I was lazy ;). You do the read(v)/write(v)/... yourself with DispatchSources so there's a lot that you can do that DispatchIO doesn't.

The real *!?< is that Swift objects do not allow buffer access (as described in my previous mail). That results in a LOT of copying (IMO to a degree I consider it almost fair to say that Swift is simply not applicable for highperf implementations).

I didn鈥檛 do _any_ profiling yet. What I wanted to write up is a more realistic test for the implementation scalability. Ie something simple like

func scaleTestHandler(res) {
聽聽聽res.write(200)
聽聽聽setTimeout(0.150 * random-something) { // simulate database/other call
聽聽聽聽聽res.write(鈥渉appy?鈥)
聽聽聽聽聽res.end()
聽聽聽}
}

Then use ab to test it. Threaded/sync setups should fail that quickly while async ones presumably will just expose the scalability limits of GCD :->

My current async imp has a lot of dispatch overhead because the callbacks can be called from essentially any queue in the current API (affecting every read & write). There are a few models that can be used for doing async:

yes, we have that too

For me pipelining adds extra sync overhead. (you can pass writes directly to channel.write(), but if you need to spool because the previous request is not done, that takes another queue dispatch 鈥)

b) A complete 鈥榪ueue free鈥 model. I鈥檓 doing this in my current approach. It is kinda lock free, but has a lot of async dispatching. The base performance overhead is/should-be pretty high, but scalability is kinda like to optimal (theoretically making use of as many CPUs as possible).

there's indeed quite a few probably better models but I always thought of that as part of the 'networking/streams' track of the server APIs work group. We have a few ideas here, will follow up with that as soon as we can.

In my original Noze imp each stream also had (could have) its own queue, but I really thought that all the queuing will absolutely kill the performance. Don鈥檛 know.

For libdispatch I believe the following model should work very well:

- create a few number of 'base' queues, probably equal to the number of CPUs stored in an array 'baseQueues'
- for every request create a new DispatchQueue(label: "...", target: baseQueues[requestNo % baseQueues.count]) (where requestNo is a global atomic (oops) integer of the overall requests)

Sounds good. Note that the `accept`s also should run concurrently (I just put them on a concurrent queue).

the base queues will end up on different (kernel) threads and the request queues will be round-robin scheduled onto the base queues. That way we make sure we don't randomly spawn new threads which isn't good.

I鈥檓 not quite sure what 'randomly spawn new threads鈥 means. To be honest I expect GCD to do the work you describe above. That is, assign new queues to the optimal number of hosting threads.

it tries but it really can't do it well and on Linux it's pretty terrible. The problem is that you need application knowledge to decide if it's better to spawn a new thread or not. GCD does (on macOS not Linux) have an upper thread level but it's 64 / 512 depending on your setup by default:

$ sysctl kern.wq_max_threads kern.wq_max_constrained_threads
kern.wq_max_threads: 512
kern.wq_max_constrained_threads: 64

but let's assume you have 4 cores, so for many high-performance networking needs, it'd be useful if GCD never spawned more than 4 threads (or whatever the best value would be for your workload). However that depends on the application. If your application might sometimes block a thread (sure, not great but real world etc) then it will be good if GCD spawned a few more threads. And that's exactly what it does. It tries to spawn new threads when it thinks it would be good for your app. But entirely without actual knowledge what's good for you.

With the base queues you can provide that knowledge to GCD and it will do the right thing. You can totally have 100k queues if they all target a very small number of base queues. You really shouldn't have 100k base queues, especially on Linux.

The `baseQueues` things (while OK to implement) sounds entirely wrong to me. GCD is responsible for doing such stuff? But maybe I鈥檓 missing something.
(the `target` thing is a synchronisation stacking construct, not a threading one, IMO).

I _think_ (guesswork) that was the idea a long while ago that GCD would just magically find the best number of threads to spawn. Unfortunately without knowledge of your application and what you want to optimise for that's hard. Then adding blocking system calls and potentially very long running loops into the mix makes it even harder. There's no silver bullet to that story so you'll need to tell GCD a little more about your queue hierarchies and then it can do a great job (on a good implementation, ie. Darwin).

In Erlang, Haskell & Go what you want is actually happening. But all those languages use green threads which stops blocking system calls from actually happening and long loops can be broken with yields. But C/ObjC/Swift are different.

That model obviously only works iff the application code is either non-blocking or dispatches itself off the request queue if it needs to do blocking work. Needless to say we should aim for non-blocking but the reality of today's code in Swift doesn't entirely look like that :wink:

With this I don鈥檛 see an issue. If the higher level framework wants to work synchronously, it can dispatch its middleware functions and such to a worker queue.
We have back pressure, so the system should be able to deal with congestion.

:+1:

But maybe this is just another hint that maybe there should be both options/implementations, a sync and a async one. Each has their application and merrits.

(And hey, Noze.io stuff is completely non-blocking, what are you talking about? :wink: )

Not sure how well this goes in Linux. Are DispatchQueue鈥檚 also cheap on Linux or does the current implementation create a new thread for each?

the queues themselves are cheap but the Linux implementation AFAIK behaves quite weirdly if it needs to spawn threads. IIRC there's one global thread which every 100ms evaluates if the existing threads are all blocked and if they are, it'll spawn a new thread. Haven't checked the code in a while, maybe someone knows better.

OK.

That's obviously not a great GCD implementation, the real GCD on macOS has kernel support to make that work much better. The same sadly applies to the eventing mechanism (DispatchSources) which are much more efficient and reduce thread hopping a lot on macOS.

Oh well.

But even on Linux I think not having many 'base' queues (which are queues that do not target other queues) should really give the best performance. Needless to say one has to be very careful not to ever block one of these base queues.

Sure.

c) Something like a), but with multiple worker queues. Kinda like the Node resolution, but w/o the different processes. This needs an API change, all the callbacks need get passed 鈥榯heir鈥 main queue (because it is not a global anymore).

Sorry, should've read the whole email before writing above. That sounds pretty much like what I wrote above, right? If you agree that sounds like the best model on GCD to me.

Yes. But unlike a) and b), this requires that the handler gets the queue it is running on, so that it can do:

聽聽func handler(req, res, queue httpQueue:鈥) {
聽聽聽聽bgQueue.async {
聽聽聽聽聽聽// very very very expensive work, like doing an Animoji
聽聽聽聽聽聽// done with it,
聽聽聽聽聽聽httpQueue.async(doneCallback)
聽聽聽聽}
聽聽}

If you get the point.

yes, agreed.

-- Johannes

路路路

On 2 Nov 2017, at 11:38 am, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:

On 2. Nov 2017, at 18:31, Johannes Wei脽 <johannesweiss@apple.com> wrote:

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

80 concurrent requests on a 8-core server is the high load test? :flushed:

I was thinking more about something like this:

聽聽ab -n 100000 -c 10000 http://localhost:1337

on a regular desktop machine and see where this goes.

But I don鈥檛 know what your handler does :slight_smile:

hh

路路路

On 2. Nov 2017, at 18:53, Carl Brown <carl.brown.swift@linuxswift.com> wrote:

my high load test suite uses 60-80 simultaneous curl processes hitting an 8-core server

Along those lines, when I tried to pause/suspend a channel a year back or so, that didn鈥檛 properly work. Is it supposed to work?

hh

路路路

On 2. Nov 2017, at 20:24, Johannes Wei脽 <johannesweiss@apple.com> wrote:

Definitely retry on Swift 4, the GCD implementation changed from using the kqueue adaptor on Linux to using epoll directly (think that was after Swift 3.1). Also bugs like https://bugs.swift.org/browse/SR-5759 have been fixed for Swift 4.0

I think if we really want async, we need a few API adjustments to make async efficient enough. E.g. maybe pass queues around (probably not a straight DispatchQueue if we don鈥檛 want to tie it to GCD, but a context which ensures synchronization - that would be efficient for sync too).

do you have suggestions how that could look?

Not really. I guess it would be sufficient if the handler gets it, like so:

func echo(request: .., response: .., queue: 鈥)

Though I was also wondering whether there should be a more general `WOContext` (ha) object which carries more details. Like a logging function to use, or other global (or HTTP transaction local) information.

But maybe that belongs into a higher level (and can be captured to the handler function).

What I would like to avoid is to make `queue:` a `queue: DispatchQueue`, but rather something like a simple

protocol SyncContext { func sync(_ cb: () -> ()) }

extension DispatchQueue {
聽聽func sync(_ cb: () -> ()) { async(execute: cb) }
}

Synchronous servers would immediately callback to the caller.

interesting. In our internal implementation we have an abstraction which has an API really similar to DispatchIO and two implementations of that. One is synchronous and one is DispatchIO. And at some point I had one which was DispatchSources.

脰hm, and why can鈥檛 we just use your stuff? :grimacing:

And on these I do in fact have sync/async/notify (for DispatchGroup) methods. So basically the HTTPServer is generic over the IO mechanism it uses. And the IO mechanism has sync/async/notify methods that do the 'right' thing depending on if it's a sync or an async implementation.

Along your lines, I also have an 鈥楨xpress鈥 like framework for Noze streams (completely asynchronous) and for Apache (completely synchronous). I initially copied the Noze one and then modified it to work with Apache which was rather trivial. BUT: For async code you very often have to do a lot of extra hops which make them slower (but more scalable). Doing this very often makes zero sense for sync variants (they just make them slower). So I ended up with pretty different implementations, though the surface API still *looks* very similar:

Compare those two 鈥榬oute evals鈥:
- async: https://github.com/NozeIO/Noze.io/blob/master/Sources/express/Route.swift#L75
- sync: https://github.com/modswift/ExExpress/blob/develop/Sources/ExExpress/express/Route.swift#L261

The first one needs those 鈥榥ext鈥 escaping closures which share-capture iterator state and all that. The second is just a plain loop and 鈥榥ext鈥 a simple flag.

I mentioned that a while back, but something I also like is this:

聽聽protocol AsyncAPI {
聽聽聽聽func doIt(_ cb: @escaping blah)
聽聽}
聽聽protocol SyncAPI {
聽聽聽聽func doIt(_ cb: blah)
聽聽}
And then depending on what you want:
聽聽typealias API = SyncAPI // = AsyncAPI

The removed @escaping has two significant benefits for the sync variant:
- it is waaaaayyyyy faster
- it makes it explicit that the callback _cannot_ escape,
聽聽it is guaranteed by the compiler (you cannot accidentally
聽聽use it in an async way)

Summary: While I agree that it is possible to share an async API and a sync one, I think it quite often doesn鈥檛 make sense performance wise.

In our internal implementation I have bits of that but never got to the point to actually profiling stuff and I didn't go all the way.

Channels vs source and then doing manual read/write? Well, my basic assumption on this is that even if channels are slower today, they should be made as fast. Conceptually that should work.

the only problems with the DispatchIO channels (you mean https://developer.apple.com/documentation/dispatch/dispatchio), right?

Yes

is that they don't support back pressure directly.

Well, API-wise they have suspend/resume? Which is quite something if that actually works now.

Say if you pipe, you could suspend the read channel and in the write channel you wait for the done and resume the read channel. I think last time I tried this, it didn鈥檛 work on Linux or something.

(Back-pressure is one of things I need to finish up in my async-imp)

I don鈥檛 remember what uv does, I think they are more like sources, but I鈥檓 not sure.

yes, DispatchSources are just an eventing mechanism really. Libuv and friends are quite similar there.

I once started to rework Noze to use libuv, but then didn鈥檛 have time or some Lego was more interesting :wink:
But it is another reason to maybe not tie the `Queue` to a `DispatchQueue` (could be uv queue something).

As mentioned, dispatch source has the little advantage (or not? I鈥檓 not convinced it is a good idea) that you can pass in those arbitrary buffer based objects. (And retain that original object).

yes, DispatchSources are way more versatile, I just went for DispatchIO because I was lazy ;). You do the read(v)/write(v)/... yourself with DispatchSources so there's a lot that you can do that DispatchIO doesn鈥檛.

There are more reasons to use channels. For example a `channel.write(DispatchData)` hopefully does a `writev`. We have no API to do this w/ sources.
I wouldn鈥檛 write off channels so easily. They are like tiny little Noze streams :->

the base queues will end up on different (kernel) threads and the request queues will be round-robin scheduled onto the base queues. That way we make sure we don't randomly spawn new threads which isn't good.

I鈥檓 not quite sure what 'randomly spawn new threads鈥 means. To be honest I expect GCD to do the work you describe above. That is, assign new queues to the optimal number of hosting threads.

it tries but it really can't do it well and on Linux it's pretty terrible.

Just make it better then :wink:

The problem is that you need application knowledge to decide if it's better to spawn a new thread or not.

I鈥檓 not entirely convinced of that. If an app really needs an own thread, it can still create it and communicate with it.
BTW: is there a way to assign a queue to an existing thread?

GCD does (on macOS not Linux) have an upper thread level but it's 64 / 512 depending on your setup by default:

$ sysctl kern.wq_max_threads kern.wq_max_constrained_threads
kern.wq_max_threads: 512
kern.wq_max_constrained_threads: 64

but let's assume you have 4 cores, so for many high-performance networking needs, it'd be useful if GCD never spawned more than 4 threads (or whatever the best value would be for your workload). However that depends on the application. If your application might sometimes block a thread (sure, not great but real world etc) then it will be good if GCD spawned a few more threads. And that's exactly what it does. It tries to spawn new threads when it thinks it would be good for your app. But entirely without actual knowledge what's good for you.

With the base queues you can provide that knowledge to GCD and it will do the right thing. You can totally have 100k queues if they all target a very small number of base queues. You really shouldn't have 100k base queues, especially on Linux.

I鈥檓 not entirely convinced by all that, but I admit it makes my head hurt :->

My takeaway is that base queues make sense at least today. Fair enough :wink:

c) Something like a), but with multiple worker queues. Kinda like the Node resolution, but w/o the different processes. This needs an API change, all the callbacks need get passed 鈥榯heir鈥 main queue (because it is not a global anymore).

Sorry, should've read the whole email before writing above. That sounds pretty much like what I wrote above, right? If you agree that sounds like the best model on GCD to me.

Yes. But unlike a) and b), this requires that the handler gets the queue it is running on, so that it can do:

func handler(req, res, queue httpQueue:鈥) {
聽聽聽bgQueue.async {
聽聽聽聽聽// very very very expensive work, like doing an Animoji
聽聽聽聽聽// done with it,
聽聽聽聽聽httpQueue.async(doneCallback)
聽聽聽}
}

If you get the point.

yes, agreed.

Just to be clear, the above is no different to

聽聽doneCallback = {
聽聽聽聽internalQueue.async {
聽聽聽聽聽聽// do the done work
聽聽聽聽}
聽聽}

(i.e. the server could pass in a `done` which synchronises itself, which is what I plan to do in my demo free threaded imp).

The gain is the common case where you don鈥檛 have that, but just call `done` straight from the same queue and do not dispatch somewhere else.

hh

路路路

On 2. Nov 2017, at 21:23, Johannes Wei脽 <johannesweiss@apple.com> wrote:

On 2. Nov 2017, at 18:31, Johannes Wei脽 <johannesweiss@apple.com> wrote:

I pushed this:

聽聽https://github.com/swift-server/http/pull/86

Take this as a point for discussion. I chose to pass in an optional DispatchQueue (sync servers would pass in nil). It could also be a non-optional protocol which DispatchQueue implements.

I also attached specific semantics:
- the handler is called on the queue which is passed in
- hence callbacks don鈥檛 need synchronisation if they stay on the queue

Maybe the protocol variant is better, but I wanted to avoid an additional HTTPSynchronizationContext protocol if possible. And my proposal works OK for both sync and async (but not for other async options like uv).

hh

路路路

On 2. Nov 2017, at 21:23, Johannes Wei脽 <johannesweiss@apple.com> wrote:

Yes. But unlike a) and b), this requires that the handler gets the queue it is running on, so that it can do:

func handler(req, res, queue httpQueue:鈥) {
聽聽聽bgQueue.async {
聽聽聽聽聽// very very very expensive work, like doing an Animoji
聽聽聽聽聽// done with it,
聽聽聽聽聽httpQueue.async(doneCallback)
聽聽聽}
}

If you get the point.

yes, agreed.

my high load test suite uses 60-80 simultaneous curl processes hitting an 8-core server

80 concurrent requests on a 8-core server is the high load test? :flushed:

:slight_smile:

I was thinking more about something like this:

ab -n 100000 -c 10000 http://localhost:1337

on a regular desktop machine and see where this goes.

you probably want to add a -k (keep alive) to the mix, otherwise you'll run out of ephemeral ports pretty quickly. Also I find `wrk` much better than `ab`, have also used `siege` before.

Btw. I always need to do http://127.0.0.1:1337 for ab as name resolution doesn't seem to work properly in ab :grimacing:

路路路

On 2 Nov 2017, at 11:51 am, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:
On 2. Nov 2017, at 18:53, Carl Brown <carl.brown.swift@linuxswift.com> wrote:

But I don鈥檛 know what your handler does :slight_smile:

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Hi,

Definitely retry on Swift 4, the GCD implementation changed from using the kqueue adaptor on Linux to using epoll directly (think that was after Swift 3.1). Also bugs like https://bugs.swift.org/browse/SR-5759 have been fixed for Swift 4.0

Along those lines, when I tried to pause/suspend a channel a year back or so, that didn鈥檛 properly work. Is it supposed to work?

all API is supposed to work :). But I find the suspend/resume APIs are rarely the right tool, still they're supposed to work. Mind filing a bug on bugs.swift.org or is that on macOS?

-- J

路路路

On 2 Nov 2017, at 12:30 pm, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:
On 2. Nov 2017, at 20:24, Johannes Wei脽 <johannesweiss@apple.com> wrote:

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Hi,

I think if we really want async, we need a few API adjustments to make async efficient enough. E.g. maybe pass queues around (probably not a straight DispatchQueue if we don鈥檛 want to tie it to GCD, but a context which ensures synchronization - that would be efficient for sync too).

do you have suggestions how that could look?

Not really. I guess it would be sufficient if the handler gets it, like so:

func echo(request: .., response: .., queue: 鈥)

Though I was also wondering whether there should be a more general `WOContext` (ha) object which carries more details. Like a logging function to use, or other global (or HTTP transaction local) information.

But maybe that belongs into a higher level (and can be captured to the handler function).

What I would like to avoid is to make `queue:` a `queue: DispatchQueue`, but rather something like a simple

protocol SyncContext { func sync(_ cb: () -> ()) }

extension DispatchQueue {
func sync(_ cb: () -> ()) { async(execute: cb) }
}

Synchronous servers would immediately callback to the caller.

interesting. In our internal implementation we have an abstraction which has an API really similar to DispatchIO and two implementations of that. One is synchronous and one is DispatchIO. And at some point I had one which was DispatchSources.

脰hm, and why can鈥檛 we just use your stuff? :grimacing:

And on these I do in fact have sync/async/notify (for DispatchGroup) methods. So basically the HTTPServer is generic over the IO mechanism it uses. And the IO mechanism has sync/async/notify methods that do the 'right' thing depending on if it's a sync or an async implementation.

Along your lines, I also have an 鈥楨xpress鈥 like framework for Noze streams (completely asynchronous) and for Apache (completely synchronous). I initially copied the Noze one and then modified it to work with Apache which was rather trivial. BUT: For async code you very often have to do a lot of extra hops which make them slower (but more scalable). Doing this very often makes zero sense for sync variants (they just make them slower). So I ended up with pretty different implementations, though the surface API still *looks* very similar:

Compare those two 鈥榬oute evals鈥:
- async: https://github.com/NozeIO/Noze.io/blob/master/Sources/express/Route.swift#L75
- sync: https://github.com/modswift/ExExpress/blob/develop/Sources/ExExpress/express/Route.swift#L261

The first one needs those 鈥榥ext鈥 escaping closures which share-capture iterator state and all that. The second is just a plain loop and 鈥榥ext鈥 a simple flag.

I mentioned that a while back, but something I also like is this:

protocol AsyncAPI {
聽聽聽func doIt(_ cb: @escaping blah)
}
protocol SyncAPI {
聽聽聽func doIt(_ cb: blah)
}
And then depending on what you want:
typealias API = SyncAPI // = AsyncAPI

The removed @escaping has two significant benefits for the sync variant:
- it is waaaaayyyyy faster
- it makes it explicit that the callback _cannot_ escape,
it is guaranteed by the compiler (you cannot accidentally
use it in an async way)

Summary: While I agree that it is possible to share an async API and a sync one, I think it quite often doesn鈥檛 make sense performance wise.

Agreed but whilst I would like to make a synchronous API possible (for legacy and 'special use' systems) I don't think we should try to optimise for it. Most of the Swift/Cocoa ecosystem on macOS/iOS is already asynchronous.

In our internal implementation I have bits of that but never got to the point to actually profiling stuff and I didn't go all the way.

Channels vs source and then doing manual read/write? Well, my basic assumption on this is that even if channels are slower today, they should be made as fast. Conceptually that should work.

the only problems with the DispatchIO channels (you mean https://developer.apple.com/documentation/dispatch/dispatchio), right?

Yes

is that they don't support back pressure directly.

Well, API-wise they have suspend/resume? Which is quite something if that actually works now.

Say if you pipe, you could suspend the read channel and in the write channel you wait for the done and resume the read channel. I think last time I tried this, it didn鈥檛 work on Linux or something.

I _think_ I've tried suspend/resume on the DispatchIO channels but I don't think it actually stopped the underlying abstraction to read bytes off the file descriptor. It merely stops delivering bytes to you. But I'm not 100% sure anymore.

(Back-pressure is one of things I need to finish up in my async-imp)

I don鈥檛 remember what uv does, I think they are more like sources, but I鈥檓 not sure.

yes, DispatchSources are just an eventing mechanism really. Libuv and friends are quite similar there.

I once started to rework Noze to use libuv, but then didn鈥檛 have time or some Lego was more interesting :wink:
But it is another reason to maybe not tie the `Queue` to a `DispatchQueue` (could be uv queue something).

:+1:, maybe we could make it generic over some 'context'?

As mentioned, dispatch source has the little advantage (or not? I鈥檓 not convinced it is a good idea) that you can pass in those arbitrary buffer based objects. (And retain that original object).

yes, DispatchSources are way more versatile, I just went for DispatchIO because I was lazy ;). You do the read(v)/write(v)/... yourself with DispatchSources so there's a lot that you can do that DispatchIO doesn鈥檛.

There are more reasons to use channels. For example a `channel.write(DispatchData)` hopefully does a `writev`.

it does not today (AFAIK) but it _really_ should. But it's open-source :wink:

We have no API to do this w/ sources.
I wouldn鈥檛 write off channels so easily. They are like tiny little Noze streams :->

the base queues will end up on different (kernel) threads and the request queues will be round-robin scheduled onto the base queues. That way we make sure we don't randomly spawn new threads which isn't good.

I鈥檓 not quite sure what 'randomly spawn new threads鈥 means. To be honest I expect GCD to do the work you describe above. That is, assign new queues to the optimal number of hosting threads.

it tries but it really can't do it well and on Linux it's pretty terrible.

Just make it better then :wink:

well unfortunately certain things in GCD need kernel support to be implemented as efficiently as on Darwin. I don't know what the way forward on that front really is :frowning:

The problem is that you need application knowledge to decide if it's better to spawn a new thread or not.

I鈥檓 not entirely convinced of that. If an app really needs an own thread, it can still create it and communicate with it.

right, that's not how GCD has been 'marketed' so far. If you want this behaviour I think the best way is to create those base queues which isn't too hard. I reckon in real-world software there's just too many places where something blocks even if that's discouraged. And kicking off another thread for all those places is probably not the programming model anyone really wants. I'm sure the GCD team would make certain decisions quite a bit differently today if they could but I can't speak for them at all.

BTW: is there a way to assign a queue to an existing thread?

no

GCD does (on macOS not Linux) have an upper thread level but it's 64 / 512 depending on your setup by default:

$ sysctl kern.wq_max_threads kern.wq_max_constrained_threads
kern.wq_max_threads: 512
kern.wq_max_constrained_threads: 64

but let's assume you have 4 cores, so for many high-performance networking needs, it'd be useful if GCD never spawned more than 4 threads (or whatever the best value would be for your workload). However that depends on the application. If your application might sometimes block a thread (sure, not great but real world etc) then it will be good if GCD spawned a few more threads. And that's exactly what it does. It tries to spawn new threads when it thinks it would be good for your app. But entirely without actual knowledge what's good for you.

With the base queues you can provide that knowledge to GCD and it will do the right thing. You can totally have 100k queues if they all target a very small number of base queues. You really shouldn't have 100k base queues, especially on Linux.

I鈥檓 not entirely convinced by all that, but I admit it makes my head hurt :->

My takeaway is that base queues make sense at least today. Fair enough :wink:

that is my understanding but I'm not on the GCD team. And the importance of the queue hierarchy (ie. have very few base queues that other queues/sources target) has been stressed a lot in this year's WWDC, see https://developer.apple.com/videos/play/wwdc2017/706/

路路路

On 2 Nov 2017, at 2:02 pm, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:
On 2. Nov 2017, at 21:23, Johannes Wei脽 <johannesweiss@apple.com> wrote:

On 2. Nov 2017, at 18:31, Johannes Wei脽 <johannesweiss@apple.com> wrote:

c) Something like a), but with multiple worker queues. Kinda like the Node resolution, but w/o the different processes. This needs an API change, all the callbacks need get passed 鈥榯heir鈥 main queue (because it is not a global anymore).

Sorry, should've read the whole email before writing above. That sounds pretty much like what I wrote above, right? If you agree that sounds like the best model on GCD to me.

Yes. But unlike a) and b), this requires that the handler gets the queue it is running on, so that it can do:

func handler(req, res, queue httpQueue:鈥) {
聽聽bgQueue.async {
聽聽聽聽// very very very expensive work, like doing an Animoji
聽聽聽聽// done with it,
聽聽聽聽httpQueue.async(doneCallback)
聽聽}
}

If you get the point.

yes, agreed.

Just to be clear, the above is no different to

doneCallback = {
聽聽聽internalQueue.async {
聽聽聽聽聽// do the done work
聽聽聽}
}

(i.e. the server could pass in a `done` which synchronises itself, which is what I plan to do in my demo free threaded imp).

The gain is the common case where you don鈥檛 have that, but just call `done` straight from the same queue and do not dispatch somewhere else.

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Definitely retry on Swift 4, the GCD implementation changed from using the kqueue adaptor on Linux to using epoll directly (think that was after Swift 3.1). Also bugs like https://bugs.swift.org/browse/SR-5759 have been fixed for Swift 4.0

Along those lines, when I tried to pause/suspend a channel a year back or so, that didn鈥檛 properly work. Is it supposed to work?

all API is supposed to work :). But I find the suspend/resume APIs are rarely the right tool, still they're supposed to work.

Not sure why you say so. This is essentially how back pressure works in Node/Noze. If the stream buffers are full, they stop reading and return don鈥檛-want-no-more on writing. If space gets available, they start filling the buffer again.

Mind filing a bug on bugs.swift.org or is that on macOS?

Jira? Really? What did I do that you suggest that?

I don鈥檛 remember whether that was Linux only or macOS too. It must have been Swift 2.x when I tried that. I think I resorted to just doing max-buffer reads instead of a single big one which gets paused. To file a bug report, this would need to be re-evaluated. I won鈥檛 :slight_smile:

hh

路路路

On 2. Nov 2017, at 21:27, Johannes Wei脽 <johannesweiss@apple.com> wrote:

On 2 Nov 2017, at 12:30 pm, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:
On 2. Nov 2017, at 20:24, Johannes Wei脽 <johannesweiss@apple.com> wrote:

my high load test suite uses 60-80 simultaneous curl processes hitting an 8-core server

80 concurrent requests on a 8-core server is the high load test? :flushed:

I was thinking more about something like this:

ab -n 100000 -c 10000 http://localhost:1337

on a regular desktop machine and see where this goes.

But I don鈥檛 know what your handler does :slight_smile:

It鈥檚 here: https://github.com/carlbrown/SwiftServerComparison.git

It鈥檚 a real-world mix of static content of various sizes with reused connections. It鈥檚 a lot less synthetic than `ab` or `wrk`.

-Carl

路路路

On Nov 2, 2017, at 1:51 PM, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:
On 2. Nov 2017, at 18:53, Carl Brown <carl.brown.swift@linuxswift.com> wrote:

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Hi Helge,

Why not a

protocol HTTPRequestHandling {
聽聽聽聽associatedtype Context
}

then an implementation that uses DispatchQueues can make `typealias Context = DispatchQueue` or whatever it feels like?

-- Johannes

路路路

On 5 Nov 2017, at 3:26 pm, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:

On 2. Nov 2017, at 21:23, Johannes Wei脽 <johannesweiss@apple.com> wrote:

Yes. But unlike a) and b), this requires that the handler gets the queue it is running on, so that it can do:

func handler(req, res, queue httpQueue:鈥) {
聽聽bgQueue.async {
聽聽聽聽// very very very expensive work, like doing an Animoji
聽聽聽聽// done with it,
聽聽聽聽httpQueue.async(doneCallback)
聽聽}
}

If you get the point.

yes, agreed.

I pushed this:

https://github.com/swift-server/http/pull/86

Take this as a point for discussion. I chose to pass in an optional DispatchQueue (sync servers would pass in nil). It could also be a non-optional protocol which DispatchQueue implements.

I also attached specific semantics:
- the handler is called on the queue which is passed in
- hence callbacks don鈥檛 need synchronisation if they stay on the queue

Maybe the protocol variant is better, but I wanted to avoid an additional HTTPSynchronizationContext protocol if possible. And my proposal works OK for both sync and async (but not for other async options like uv).

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

I鈥檓 not entirely sure I can follow you here. IMO the `HTTPRequestHandling` protocol doesn鈥檛 belong here. It can be useful for OO oriented APIs, but it provided zero value to this framework.

So the primary handler type would be a closure:

聽聽(HTTPRequest, HTTPResponseWriter, SyncContext)

And I can鈥檛 make the SyncContext generic in this, right?

hh

路路路

On 6. Nov 2017, at 19:20, Johannes Wei脽 <johannesweiss@apple.com> wrote:

Hi Helge,

Why not a

protocol HTTPRequestHandling {
聽聽聽associatedtype Context
}

then an implementation that uses DispatchQueues can make `typealias Context = DispatchQueue` or whatever it feels like?

Thinking about this, we could make the whole server class generic and do something like Apache MPM to select synchronous/async MPMs.

Like so:

聽聽let server = HTTPServer<SyncSocketMPM>(port: 0xF00)

and

聽聽let server = HTTPServer<DispatchMPM>(port: 1337)

where we have

聽聽protocol HTTPServerProcessingModule {
聽聽聽聽associatedtype SyncContext
聽聽聽聽associatedtype RequestHead
聽聽聽聽associatedtype ResponseWriter
聽聽聽聽associatedtype Configuration // like Apache Server MPM config
聽聽聽聽typealias Handler = ( RequestHead, ResponseWriter, SyncContext )
聽聽}

聽聽class HTTPServer<MPM: HTTPProcessingModule> {

聽聽聽聽init(port : Int? = nil,
聽聽聽聽聽聽聽聽聽handler : MPM.Handler,
聽聽聽聽聽聽聽聽聽configuration : MPM.Configuration? = nil)
聽聽聽聽{ ..}
聽聽}

Maybe that is a overkill for this effort, but it may be a sweet way to switch implementations w/o dynamic overhead.

What do you think?

hh

路路路

On 6. Nov 2017, at 19:52, Helge He脽 via swift-server-dev <swift-server-dev@swift.org> wrote:

On 6. Nov 2017, at 19:20, Johannes Wei脽 <johannesweiss@apple.com> wrote:

Hi Helge,

Why not a

protocol HTTPRequestHandling {
聽聽associatedtype Context
}

then an implementation that uses DispatchQueues can make `typealias Context = DispatchQueue` or whatever it feels like?

I鈥檓 not entirely sure I can follow you here. IMO the `HTTPRequestHandling` protocol doesn鈥檛 belong here. It can be useful for OO oriented APIs, but it provided zero value to this framework.

So the primary handler type would be a closure:

(HTTPRequest, HTTPResponseWriter, SyncContext)

And I can鈥檛 make the SyncContext generic in this, right?

Terms of Service

Privacy Policy

Cookie Policy