Swift-server-dev Digest, Vol 2, Issue 1

Server Development
21

Tanner Nelson
Vapor
+1 (435) 773-2831

There has been some discussions about this in Kitura already (Issues · Kitura/Kitura-net · GitHub) and I think we can use what they found out at the moment.
In my opinion, if we are not able to give the same (or better) performance we should go for a wrapper of a more performant lib, such as http_parser.

They claim that their port is just ~1/3 slower, that seems perfectly reasonable to me. I’m all for using it.

As far as I can tell, the package in question here is a carbon copy of the C parser (UnsafePointers, asserts, global funcs). The only difference is it's 1/3 slower.

A big performance hit like that is not worth it unless we're getting improved readability or safety out of the code.

···

On Nov 3, 2016, at 4:34 PM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:
On 3 Nov 2016, at 19:41, Alfredo Delli Bovi <alfredo.dellibovi@gmail.com> wrote:

I also agree with the overall sentiment that a ‘pure’ Swift solution should be preferred.

Of course it’s also matter of the workload that have, we could always start with a wrapper, so we are able to define the APIs layer and ship a version with it and later on changing the implementation with a pure Swift.

Yes, the API should sit on top of this and be essentially parser agnostic.

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

22

I think wrapping a battle-tested C library is the way to go. Using a C library is not the same as writing one. In our case we’ll have to deal with C APIs anyway (POSIX), so the experience won’t be something we’re not used to already. Pragmatically here are the pros and cons:

Pros
- Faster to wrap a C lib than to write everything in Swift
- Safer to use a battle-tested C lib
- Better performance when handling strings (major current bottleneck for Swift)

Cons
- We need the OK from Chris Lattner
- If a bug is found we have to either:
  - fix it ourselves and push back upstream
  - if it’s something we don’t know how to fix we report and wait for the fix

I’ve been using http_parser on Zewo for over an year and I haven’t found a single bug yet. That’s *very* unlikely to happen if we write a Swift parser ourselves.

About the binary data discussion. I think something so important like that should reside in the standard library. I don’t think a protocol is a good idea because it promotes the creation of even more binary abstractions which is no good. I know the proposal of a new “data” type on the standard library falls on what I just said, *but* in a perfect world Foundation would lose NSData in favor of the standard library Data pretty much like it did for NSString/String. Eventually all other frameworks would be using standard library’s Data too.

The feeling I got from Ted Kremenek is that we should picture the best scenario and then see where our work might land. This is the perfect case for that.

···

On Nov 3, 2016, at 6:49 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

Following up with what Tanner and Chris said. I think HTTP 2 is particularly difficult, and you're right chris, bringing in a c library, even if just temporarily will help expedite the process in an efficient way.

I am however pretty ardently against porting C code to Swift as opposed to rethinking things for a Swift paradigm. If that's the case, I'd prefer to either link to the c code, or bring it into the Swift compiler. The end result would appear to be the same.

- Logan

On Thu, Nov 3, 2016 at 4:46 PM Tanner Nelson via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:

Tanner Nelson
Vapor
+1 (435) 773-2831 <tel:(435)%20773-2831>

On Nov 3, 2016, at 4:34 PM, Helge Heß via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:

On 3 Nov 2016, at 19:41, Alfredo Delli Bovi <alfredo.dellibovi@gmail.com <mailto:alfredo.dellibovi@gmail.com>> wrote:

There has been some discussions about this in Kitura already (Issues · Kitura/Kitura-net · GitHub) and I think we can use what they found out at the moment.
In my opinion, if we are not able to give the same (or better) performance we should go for a wrapper of a more performant lib, such as http_parser.

They claim that their port is just ~1/3 slower, that seems perfectly reasonable to me. I’m all for using it.

As far as I can tell, the package in question here is a carbon copy of the C parser (UnsafePointers, asserts, global funcs). The only difference is it's 1/3 slower.

A big performance hit like that is not worth it unless we're getting improved readability or safety out of the code.

I also agree with the overall sentiment that a ‘pure’ Swift solution should be preferred.

Of course it’s also matter of the workload that have, we could always start with a wrapper, so we are able to define the APIs layer and ship a version with it and later on changing the implementation with a pure Swift.

Yes, the API should sit on top of this and be essentially parser agnostic.

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

23

Following up with what Tanner and Chris said. I think HTTP 2 is
particularly difficult, and you're right chris, bringing in a c library,
even if just temporarily will help expedite the process in an efficient way.

I am however pretty ardently against porting C code to Swift as opposed to
rethinking things for a Swift paradigm. If that's the case, I'd prefer to
either link to the c code, or bring it into the Swift compiler. The end
result would appear to be the same.

- Logan

···

On Thu, Nov 3, 2016 at 4:46 PM Tanner Nelson via swift-server-dev < swift-server-dev@swift.org> wrote:

Tanner Nelson
Vapor
+1 (435) 773-2831 <(435)%20773-2831>

On Nov 3, 2016, at 4:34 PM, Helge Heß via swift-server-dev < > swift-server-dev@swift.org> wrote:

On 3 Nov 2016, at 19:41, Alfredo Delli Bovi <alfredo.dellibovi@gmail.com> > wrote:

There has been some discussions about this in Kitura already (
Issues · Kitura/Kitura-net · GitHub) and I think we can use
what they found out at the moment.
In my opinion, if we are not able to give the same (or better) performance
we should go for a wrapper of a more performant lib, such as http_parser.

They claim that their port is just ~1/3 slower, that seems perfectly
reasonable to me. I’m all for using it.

As far as I can tell, the package in question here is a carbon copy of the
C parser (UnsafePointers, asserts, global funcs). The only difference is
it's 1/3 slower.

A big performance hit like that is not worth it unless we're getting
improved readability or safety out of the code.

I also agree with the overall sentiment that a ‘pure’ Swift solution
should be preferred.

Of course it’s also matter of the workload that have, we could always
start with a wrapper, so we are able to define the APIs layer and ship a
version with it and later on changing the implementation with a pure Swift.

Yes, the API should sit on top of this and be essentially parser agnostic.

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

24

HTTP/2 protocol support in itself doesn’t seem very complex to me (the protocol). The thing which makes it complex is reasonable scheduling of request/response handlers (the multiplexed streams on the single socket). And that is very much tied to the scheduling library people use, hence I wonder what can be done here in a generic way.

hh

···

On 3 Nov 2016, at 21:27, Chris Bailey via swift-server-dev <swift-server-dev@swift.org> wrote:

I think one thing that needs to be considered is HTTP/2 support - implementing HTTP/2 is significantly more complex that HTTP 1.x, so if we want to support HTTP/2 it may make more sense to look at what we can reuse (the nghttp2 library?)

25

I agree with Paulo. One of Swift's strong points is its ability to bind
with all sorts of existing "system" C based APIs. Especially C APIs that
are extremely well tested and maintained. We should all remember that
while many of us may have taken the http_parser from Node, Node itself
took it from Nginx. This code comes with great pedigree.

Why should we spend time developing an HTTP parser in Swift when we should
be spending our collective time higher up the Web Server Framework
toolchain? Alfredo Delli Bovi quoted the work that was done Dave Sperling
to port the HTTP Parser Kitura was using to Swift. While he ported the
code, he did not write the complete Unit Test Suite needed for that code.
His own estimate was that writing the tests would be approximately two to
three times the effort of the original port. I also seem to remember that
the performance hit for that particular code was more than 33%.

Another point, which is especially true on current Swift drivers, is that
any code that is going to parse something like HTTP requests and responses
is probably going to have to work at the byte level and not that String
level. This is due to performance issues. At the byte level its actually a
lot easier to write C code, than Swift code. This may change in the
future, but lets wait until this kind of thing gets more performant in
Swift.

Some have said that readability is more important than performance. I for
one, who have read the http_parser code from time to time, find rather
readable for a tightly written finite state machine. Here I also beg to
disagree that performance isn't important. In a server one wants to spend
cycles doing real work for the requestor, not figuring out what he wants
you to do. To scale one needs to be performant.

Shmuel Kallner
STSM Smart Client Platforms group
Tel: +972-4829-6430
e-mail: kallner@il.ibm.com

···

From: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
To: Logan Wright <logan@qutheory.io>
Cc: Swift Server Dev <swift-server-dev@swift.org>
Date: 04/11/2016 00:15
Subject: Re: [swift-server-dev] HTTP Parser
Sent by: swift-server-dev-bounces@swift.org

I think wrapping a battle-tested C library is the way to go. Using a C
library is not the same as writing one. In our case we?ll have to deal
with C APIs anyway (POSIX), so the experience won?t be something we?re not
used to already. Pragmatically here are the pros and cons:

Pros
- Faster to wrap a C lib than to write everything in Swift
- Safer to use a battle-tested C lib
- Better performance when handling strings (major current bottleneck for
Swift)

Cons
- We need the OK from Chris Lattner
- If a bug is found we have to either:
  - fix it ourselves and push back upstream
  - if it?s something we don?t know how to fix we report and wait for the
fix

I?ve been using http_parser on Zewo for over an year and I haven?t found a
single bug yet. That?s *very* unlikely to happen if we write a Swift
parser ourselves.

About the binary data discussion. I think something so important like that
should reside in the standard library. I don?t think a protocol is a good
idea because it promotes the creation of even more binary abstractions
which is no good. I know the proposal of a new ?data? type on the standard
library falls on what I just said, *but* in a perfect world Foundation
would lose NSData in favor of the standard library Data pretty much like
it did for NSString/String. Eventually all other frameworks would be using
standard library?s Data too.

The feeling I got from Ted Kremenek is that we should picture the best
scenario and then see where our work might land. This is the perfect case
for that.

On Nov 3, 2016, at 6:49 PM, Logan Wright via swift-server-dev < swift-server-dev@swift.org> wrote:

Following up with what Tanner and Chris said. I think HTTP 2 is
particularly difficult, and you're right chris, bringing in a c library,
even if just temporarily will help expedite the process in an efficient
way.

I am however pretty ardently against porting C code to Swift as opposed to
rethinking things for a Swift paradigm. If that's the case, I'd prefer to
either link to the c code, or bring it into the Swift compiler. The end
result would appear to be the same.

- Logan

On Thu, Nov 3, 2016 at 4:46 PM Tanner Nelson via swift-server-dev < swift-server-dev@swift.org> wrote:

Tanner Nelson
Vapor
+1 (435) 773-2831

On Nov 3, 2016, at 4:34 PM, Helge Heß via swift-server-dev < swift-server-dev@swift.org> wrote:

On 3 Nov 2016, at 19:41, Alfredo Delli Bovi <alfredo.dellibovi@gmail.com> wrote:
There has been some discussions about this in Kitura already (
Issues · Kitura/Kitura-net · GitHub) and I think we can use
what they found out at the moment.
In my opinion, if we are not able to give the same (or better) performance
we should go for a wrapper of a more performant lib, such as http_parser.

They claim that their port is just ~1/3 slower, that seems perfectly
reasonable to me. I?m all for using it.

As far as I can tell, the package in question here is a carbon copy of the
C parser (UnsafePointers, asserts, global funcs). The only difference is
it's 1/3 slower.

A big performance hit like that is not worth it unless we're getting
improved readability or safety out of the code.

I also agree with the overall sentiment that a ?pure? Swift solution
should be preferred.

Of course it?s also matter of the workload that have, we could always
start with a wrapper, so we are able to define the APIs layer and ship a
version with it and later on changing the implementation with a pure
Swift.

Yes, the API should sit on top of this and be essentially parser agnostic.

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

26

Completely agree.
Also, a poor performance at that stage of the request could lead to security issues too: i.e. DDoS attacks would be easier to execute.

···

—
Alfredo Delli Bovi

On 4 Nov 2016, at 09:50, Samuel Kallner via swift-server-dev <swift-server-dev@swift.org> wrote:

I agree with Paulo. One of Swift's strong points is its ability to bind with all sorts of existing "system" C based APIs. Especially C APIs that are extremely well tested and maintained. We should all remember that while many of us may have taken the http_parser from Node, Node itself took it from Nginx. This code comes with great pedigree.

Why should we spend time developing an HTTP parser in Swift when we should be spending our collective time higher up the Web Server Framework toolchain? Alfredo Delli Bovi quoted the work that was done Dave Sperling to port the HTTP Parser Kitura was using to Swift. While he ported the code, he did not write the complete Unit Test Suite needed for that code. His own estimate was that writing the tests would be approximately two to three times the effort of the original port. I also seem to remember that the performance hit for that particular code was more than 33%.

Another point, which is especially true on current Swift drivers, is that any code that is going to parse something like HTTP requests and responses is probably going to have to work at the byte level and not that String level. This is due to performance issues. At the byte level its actually a lot easier to write C code, than Swift code. This may change in the future, but lets wait until this kind of thing gets more performant in Swift.

Some have said that readability is more important than performance. I for one, who have read the http_parser code from time to time, find rather readable for a tightly written finite state machine. Here I also beg to disagree that performance isn't important. In a server one wants to spend cycles doing real work for the requestor, not figuring out what he wants you to do. To scale one needs to be performant.

Shmuel Kallner
STSM Smart Client Platforms group
Tel: +972-4829-6430
e-mail: kallner@il.ibm.com

From: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
To: Logan Wright <logan@qutheory.io>
Cc: Swift Server Dev <swift-server-dev@swift.org>
Date: 04/11/2016 00:15
Subject: Re: [swift-server-dev] HTTP Parser
Sent by: swift-server-dev-bounces@swift.org

I think wrapping a battle-tested C library is the way to go. Using a C library is not the same as writing one. In our case we’ll have to deal with C APIs anyway (POSIX), so the experience won’t be something we’re not used to already. Pragmatically here are the pros and cons:

Pros
- Faster to wrap a C lib than to write everything in Swift
- Safer to use a battle-tested C lib
- Better performance when handling strings (major current bottleneck for Swift)

Cons
- We need the OK from Chris Lattner
- If a bug is found we have to either:
  - fix it ourselves and push back upstream
  - if it’s something we don’t know how to fix we report and wait for the fix

I’ve been using http_parser on Zewo for over an year and I haven’t found a single bug yet. That’s *very* unlikely to happen if we write a Swift parser ourselves.

About the binary data discussion. I think something so important like that should reside in the standard library. I don’t think a protocol is a good idea because it promotes the creation of even more binary abstractions which is no good. I know the proposal of a new “data” type on the standard library falls on what I just said, *but* in a perfect world Foundation would lose NSData in favor of the standard library Data pretty much like it did for NSString/String. Eventually all other frameworks would be using standard library’s Data too.

The feeling I got from Ted Kremenek is that we should picture the best scenario and then see where our work might land. This is the perfect case for that.

On Nov 3, 2016, at 6:49 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:

Following up with what Tanner and Chris said. I think HTTP 2 is particularly difficult, and you're right chris, bringing in a c library, even if just temporarily will help expedite the process in an efficient way.

I am however pretty ardently against porting C code to Swift as opposed to rethinking things for a Swift paradigm. If that's the case, I'd prefer to either link to the c code, or bring it into the Swift compiler. The end result would appear to be the same.

- Logan

On Thu, Nov 3, 2016 at 4:46 PM Tanner Nelson via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:

Tanner Nelson
Vapor
+1 (435) 773-2831 <tel:(435)%20773-2831>

On Nov 3, 2016, at 4:34 PM, Helge Heß via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:

On 3 Nov 2016, at 19:41, Alfredo Delli Bovi <alfredo.dellibovi@gmail.com <mailto:alfredo.dellibovi@gmail.com>> wrote:
There has been some discussions about this in Kitura already (Issues · Kitura/Kitura-net · GitHub) and I think we can use what they found out at the moment.
In my opinion, if we are not able to give the same (or better) performance we should go for a wrapper of a more performant lib, such as http_parser.

They claim that their port is just ~1/3 slower, that seems perfectly reasonable to me. I’m all for using it.

As far as I can tell, the package in question here is a carbon copy of the C parser (UnsafePointers, asserts, global funcs). The only difference is it's 1/3 slower.

A big performance hit like that is not worth it unless we're getting improved readability or safety out of the code.

I also agree with the overall sentiment that a ‘pure’ Swift solution should be preferred.

Of course it’s also matter of the workload that have, we could always start with a wrapper, so we are able to define the APIs layer and ship a version with it and later on changing the implementation with a pure Swift.

Yes, the API should sit on top of this and be essentially parser agnostic.

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

27

Just to clarify, when I talk about Swift, I am absolutely not talking about
1:1 port. Rewriting C in Swift is pointless if we're not going to rethink
it for the language. If we collectively agree to use underlying C
libraries, then compiling them directly is my preference.

That said, even if we do end up doing that short term, I'd like to include
plans or at least intentions to do things in Swift. I (and I'm sure many of
us) generally prefer working in Swift because it's safe, succinct, and
clean. We also have several instances of existing HTTP parsers we could
pull from, it's not like we're starting from 0 as a group.

For HTTP2, I think everyone is in agreement that a c library is the way to
go. I'd like to give HTTP a bit more time for opinions to come in.

- Logan

···

On Fri, Nov 4, 2016 at 5:25 AM Alfredo Delli Bovi via swift-server-dev < swift-server-dev@swift.org> wrote:

Completely agree.
Also, a poor performance at that stage of the request could lead to
security issues too: i.e. DDoS attacks would be easier to execute.
—
Alfredo Delli Bovi

On 4 Nov 2016, at 09:50, Samuel Kallner via swift-server-dev < > swift-server-dev@swift.org> wrote:

I agree with Paulo. One of Swift's strong points is its ability to bind
with all sorts of existing "system" C based APIs. Especially C APIs that
are extremely well tested and maintained. We should all remember that while
many of us may have taken the http_parser from Node, Node itself took it
from Nginx. This code comes with great pedigree.

Why should we spend time developing an HTTP parser in Swift when we should
be spending our collective time higher up the Web Server Framework
toolchain? Alfredo Delli Bovi quoted the work that was done Dave Sperling
to port the HTTP Parser Kitura was using to Swift. While he ported the
code, he did not write the complete Unit Test Suite needed for that code.
His own estimate was that writing the tests would be approximately two to
three times the effort of the original port. I also seem to remember that
the performance hit for that particular code was more than 33%.

Another point, which is especially true on current Swift drivers, is that
any code that is going to parse something like HTTP requests and responses
is probably going to have to work at the byte level and not that String
level. This is due to performance issues. At the byte level its actually a
lot easier to write C code, than Swift code. This may change in the future,
but lets wait until this kind of thing gets more performant in Swift.

Some have said that readability is more important than performance. I for
one, who have read the http_parser code from time to time, find rather
readable for a tightly written finite state machine. Here I also beg to
disagree that performance isn't important. In a server one wants to spend
cycles doing real work for the requestor, not figuring out what he wants
you to do. To scale one needs to be performant.

Shmuel Kallner
STSM Smart Client Platforms group
Tel: +972-4829-6430
e-mail: kallner@il.ibm.com

From: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
To: Logan Wright <logan@qutheory.io>
Cc: Swift Server Dev <swift-server-dev@swift.org>
Date: 04/11/2016 00:15
Subject: Re: [swift-server-dev] HTTP Parser
Sent by: swift-server-dev-bounces@swift.org
------------------------------

I think wrapping a battle-tested C library is the way to go. Using a C
library is not the same as writing one. In our case we’ll have to deal with
C APIs anyway (POSIX), so the experience won’t be something we’re not used
to already. Pragmatically here are the pros and cons:

Pros
- Faster to wrap a C lib than to write everything in Swift
- Safer to use a battle-tested C lib
- Better performance when handling strings (major current bottleneck for
Swift)

Cons
- We need the OK from Chris Lattner
- If a bug is found we have to either:
  - fix it ourselves and push back upstream
  - if it’s something we don’t know how to fix we report and wait for the
fix

I’ve been using http_parser on Zewo for over an year and I haven’t found a
single bug yet. That’s *very* unlikely to happen if we write a Swift parser
ourselves.

About the binary data discussion. I think something so important like that
should reside in the standard library. I don’t think a protocol is a good
idea because it promotes the creation of even more binary abstractions
which is no good. I know the proposal of a new “data” type on the standard
library falls on what I just said, *but* in a perfect world Foundation
would lose NSData in favor of the standard library Data pretty much like it
did for NSString/String. Eventually all other frameworks would be using
standard library’s Data too.

The feeling I got from Ted Kremenek is that we should picture the best
scenario and then see where our work might land. This is the perfect case
for that.

On Nov 3, 2016, at 6:49 PM, Logan Wright via swift-server-dev < > *swift-server-dev@swift.org* <swift-server-dev@swift.org>> wrote:

Following up with what Tanner and Chris said. I think HTTP 2 is
particularly difficult, and you're right chris, bringing in a c library,
even if just temporarily will help expedite the process in an efficient way.

I am however pretty ardently against porting C code to Swift as opposed to
rethinking things for a Swift paradigm. If that's the case, I'd prefer to
either link to the c code, or bring it into the Swift compiler. The end
result would appear to be the same.

- Logan

On Thu, Nov 3, 2016 at 4:46 PM Tanner Nelson via swift-server-dev < > *swift-server-dev@swift.org* <swift-server-dev@swift.org>> wrote:

Tanner Nelson
Vapor
*+1 (435) 773-2831* <(435)%20773-2831>

On Nov 3, 2016, at 4:34 PM, Helge Heß via swift-server-dev < > *swift-server-dev@swift.org* <swift-server-dev@swift.org>> wrote:

On 3 Nov 2016, at 19:41, Alfredo Delli Bovi <*alfredo.dellibovi@gmail.com* > <alfredo.dellibovi@gmail.com>> wrote:
There has been some discussions about this in Kitura already (
*Swap HTTPParser Swift package for http-parser · Issue #52 · Kitura/Kitura-net · GitHub*
<Issues · Kitura/Kitura-net · GitHub) and I think we can
use what they found out at the moment.
In my opinion, if we are not able to give the same (or better) performance
we should go for a wrapper of a more performant lib, such as http_parser.

They claim that their port is just ~1/3 slower, that seems perfectly
reasonable to me. I’m all for using it.

As far as I can tell, the package in question here is a carbon copy of the
C parser (UnsafePointers, asserts, global funcs). The only difference is
it's 1/3 slower.

A big performance hit like that is not worth it unless we're getting
improved readability or safety out of the code.

I also agree with the overall sentiment that a ‘pure’ Swift solution
should be preferred.

Of course it’s also matter of the workload that have, we could always
start with a wrapper, so we are able to define the APIs layer and ship a
version with it and later on changing the implementation with a pure Swift.

Yes, the API should sit on top of this and be essentially parser agnostic.

hh

_______________________________________________
swift-server-dev mailing list
*swift-server-dev@swift.org* <swift-server-dev@swift.org>
*https://lists.swift.org/mailman/listinfo/swift-server-dev*
<https://lists.swift.org/mailman/listinfo/swift-server-dev&gt;

_______________________________________________
swift-server-dev mailing list
*swift-server-dev@swift.org* <swift-server-dev@swift.org>
*https://lists.swift.org/mailman/listinfo/swift-server-dev*
<https://lists.swift.org/mailman/listinfo/swift-server-dev&gt;
_______________________________________________
swift-server-dev mailing list
*swift-server-dev@swift.org* <swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

28

I’m not entirely sure why you have double standards here. Why would you come up with a safe, succinct and clean HTTP/1.x parser but not do the same for HTTP/2?

hh

···

On 4 Nov 2016, at 20:11, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

That said, even if we do end up doing that short term, I'd like to include plans or at least intentions to do things in Swift. I (and I'm sure many of us) generally prefer working in Swift because it's safe, succinct, and clean. We also have several instances of existing HTTP parsers we could pull from, it's not like we're starting from 0 as a group.

For HTTP2, I think everyone is in agreement that a c library is the way to go. I'd like to give HTTP a bit more time for opinions to come in.

29

I think the right solution in general is to use C quite aggressively at first, but wall it off behind Swift facades. Later, we can revisit our C dependencies and replace them with Swift as we feel it's appropriate. Ideally, we'd use protocols here so the replacement is as simple as possible.

Here's my question: Can we have our HTTP parser use Foundation's existing `URLRequest` and `HTTPURLResponse` types as Swift-facing outputs and inputs? Or are these types too-tightly designed for an HTTP client's needs, and we need different types for an HTTP server? Or are there minor modifications the Foundation team is willing to make in Apple Foundation to extend them for our use case? If we *can* use these types on both client and server, I think that would be ideal—it would allow people to reuse their knowledge and potentially a little bit of code.

···

On Nov 4, 2016, at 12:18 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

Helge, if I had my way, we wouldn't touch C in any way whatsoever for any of the server side libraries except as an absolute last resort, but this life is full of compromise, and I'm trying to be amicable to come up with solutions that we can agree on.

For HTTP2 as well, I'd like plans to eventually move to pure swift native implementations rethought for the language.

--
Brent Royal-Gordon
Architechies

30

I think it is pretty reasonable for us to modify URLRequest or HTTPURLResponse. That being said it would need to be well encapsulated, most likely limited to primarily swift, it couldn’t break existing clients on iOS or macOS etc. Extensions might be a great way to add that functionality. For a full addition we would need to of course go through full API proposals and such and most likely have some rough draft of the implementation before the proposal stage as well.

···

On Nov 4, 2016, at 1:34 PM, Brent Royal-Gordon via swift-server-dev <swift-server-dev@swift.org> wrote:

On Nov 4, 2016, at 12:18 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

Helge, if I had my way, we wouldn't touch C in any way whatsoever for any of the server side libraries except as an absolute last resort, but this life is full of compromise, and I'm trying to be amicable to come up with solutions that we can agree on.

For HTTP2 as well, I'd like plans to eventually move to pure swift native implementations rethought for the language.

I think the right solution in general is to use C quite aggressively at first, but wall it off behind Swift facades. Later, we can revisit our C dependencies and replace them with Swift as we feel it's appropriate. Ideally, we'd use protocols here so the replacement is as simple as possible.

Here's my question: Can we have our HTTP parser use Foundation's existing `URLRequest` and `HTTPURLResponse` types as Swift-facing outputs and inputs? Or are these types too-tightly designed for an HTTP client's needs, and we need different types for an HTTP server? Or are there minor modifications the Foundation team is willing to make in Apple Foundation to extend them for our use case? If we *can* use these types on both client and server, I think that would be ideal—it would allow people to reuse their knowledge and potentially a little bit of code.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

31

Hi Paulo,

The design of Foundation’s HTTP request/response is very tied to Foundation. I’d prefer we come up with our own design. Even though the name of the work group is Swift Server APIs, we’re still developing solutions for clients that could be used on the server side. An HTTP client for example. Because we have different frameworks with different concurrency models we need a design that’s flexible enough for that. Trying to adapt `URLRequest` and `HTTPURLResponse` would first, impact existing applications that use it on iOS for example, and second, we might not have enough freedom with the API design to accomplish that. Even if the Foundation team is in favor of changing the design, there’s only a small level of change in the design we can make so the API doesn’t feel alien to the rest of Foundation.

If you’re thinking about iOS specifically. I think some of the libraries we build here could certainly be used there as well. I can see an HTTP client built from this effort running on iOS. This way we don’t mess around with existing software. We all know people don’t like when their code breaks.

I think if the API design feels “alien” to Foundation, then we either need to update the Foundation API to feel like it fits in with Swift or design the server API to fit in with Foundation.

I’ll keep pushing for this throughout the process. It’s really important that we have a coherent stack of software from top to bottom. Ending up with different model objects for URLRequest used in the same app via two frameworks would be really unfortunate.

This comes at a cost, of course, but consistency is not something we should dismiss so easily right out of the gate.

- Tony

···

On Nov 4, 2016, at 1:58 PM, Paulo Faria via swift-server-dev <swift-server-dev@swift.org> wrote:

On Nov 4, 2016, at 6:34 PM, Brent Royal-Gordon via swift-server-dev <swift-server-dev@swift.org> wrote:

On Nov 4, 2016, at 12:18 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

Helge, if I had my way, we wouldn't touch C in any way whatsoever for any of the server side libraries except as an absolute last resort, but this life is full of compromise, and I'm trying to be amicable to come up with solutions that we can agree on.

For HTTP2 as well, I'd like plans to eventually move to pure swift native implementations rethought for the language.

I think the right solution in general is to use C quite aggressively at first, but wall it off behind Swift facades. Later, we can revisit our C dependencies and replace them with Swift as we feel it's appropriate. Ideally, we'd use protocols here so the replacement is as simple as possible.

Here's my question: Can we have our HTTP parser use Foundation's existing `URLRequest` and `HTTPURLResponse` types as Swift-facing outputs and inputs? Or are these types too-tightly designed for an HTTP client's needs, and we need different types for an HTTP server? Or are there minor modifications the Foundation team is willing to make in Apple Foundation to extend them for our use case? If we *can* use these types on both client and server, I think that would be ideal—it would allow people to reuse their knowledge and potentially a little bit of code.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

32

I don’t see why one would want to avoid Foundation, since it should be available anywhere Swift is via Swift Foundation.

  -- Chris

···

On Nov 4, 2016, at 1:58 PM, Paulo Faria via swift-server-dev <swift-server-dev@swift.org> wrote:

The design of Foundation’s HTTP request/response is very tied to Foundation. I’d prefer we come up with our own design. Even though the name of the work group is Swift Server APIs, we’re still developing solutions for clients that could be used on the server side.

33

Logan, would you be kind to explain why you think we shouldn’t touch C? My argument is that using a C library that we all know works well gives us time to work on more important things, like the user-facing API. You mentioned that we all like Swift because it’s safe, succinct and clean. I think we can all agree with that, but that statement doesn’t correlate to using a C lib. We wouldn’t be _implementing_ the parser in c, we would be _using_ a existing c parser. So the work would fall into what we’ll already have to do when dealing with C POSIX APIs, for example. Using a C lib would be one less thing to maintain. So again we can focus on creating safe, succinct and clean code where it really matters, the API level.

···

On Nov 4, 2016, at 5:18 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

Helge, if I had my way, we wouldn't touch C in any way whatsoever for any of the server side libraries except as an absolute last resort, but this life is full of compromise, and I'm trying to be amicable to come up with solutions that we can agree on.

For HTTP2 as well, I'd like plans to eventually move to pure swift native implementations rethought for the language.

On Fri, Nov 4, 2016 at 3:17 PM Helge Heß via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:
On 4 Nov 2016, at 20:11, Logan Wright via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:
> That said, even if we do end up doing that short term, I'd like to include plans or at least intentions to do things in Swift. I (and I'm sure many of us) generally prefer working in Swift because it's safe, succinct, and clean. We also have several instances of existing HTTP parsers we could pull from, it's not like we're starting from 0 as a group.
>
> For HTTP2, I think everyone is in agreement that a c library is the way to go. I'd like to give HTTP a bit more time for opinions to come in.

I’m not entirely sure why you have double standards here. Why would you come up with a safe, succinct and clean HTTP/1.x parser but not do the same for HTTP/2?

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

34

The design of Foundation’s HTTP request/response is very tied to Foundation. I’d prefer we come up with our own design. Even though the name of the work group is Swift Server APIs, we’re still developing solutions for clients that could be used on the server side. An HTTP client for example. Because we have different frameworks with different concurrency models we need a design that’s flexible enough for that. Trying to adapt `URLRequest` and `HTTPURLResponse` would first, impact existing applications that use it on iOS for example, and second, we might not have enough freedom with the API design to accomplish that. Even if the Foundation team is in favor of changing the design, there’s only a small level of change in the design we can make so the API doesn’t feel alien to the rest of Foundation.

If you’re thinking about iOS specifically. I think some of the libraries we build here could certainly be used there as well. I can see an HTTP client built from this effort running on iOS. This way we don’t mess around with existing software. We all know people don’t like when their code breaks.

···

On Nov 4, 2016, at 6:34 PM, Brent Royal-Gordon via swift-server-dev <swift-server-dev@swift.org> wrote:

On Nov 4, 2016, at 12:18 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

Helge, if I had my way, we wouldn't touch C in any way whatsoever for any of the server side libraries except as an absolute last resort, but this life is full of compromise, and I'm trying to be amicable to come up with solutions that we can agree on.

For HTTP2 as well, I'd like plans to eventually move to pure swift native implementations rethought for the language.

I think the right solution in general is to use C quite aggressively at first, but wall it off behind Swift facades. Later, we can revisit our C dependencies and replace them with Swift as we feel it's appropriate. Ideally, we'd use protocols here so the replacement is as simple as possible.

Here's my question: Can we have our HTTP parser use Foundation's existing `URLRequest` and `HTTPURLResponse` types as Swift-facing outputs and inputs? Or are these types too-tightly designed for an HTTP client's needs, and we need different types for an HTTP server? Or are there minor modifications the Foundation team is willing to make in Apple Foundation to extend them for our use case? If we *can* use these types on both client and server, I think that would be ideal—it would allow people to reuse their knowledge and potentially a little bit of code.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

35

Helge, if I had my way, we wouldn't touch C in any way whatsoever for any
of the server side libraries except as an absolute last resort, but this
life is full of compromise, and I'm trying to be amicable to come up with
solutions that we can agree on.

For HTTP2 as well, I'd like plans to eventually move to pure swift native
implementations rethought for the language.

···

On Fri, Nov 4, 2016 at 3:17 PM Helge Heß via swift-server-dev < swift-server-dev@swift.org> wrote:

On 4 Nov 2016, at 20:11, Logan Wright via swift-server-dev < > swift-server-dev@swift.org> wrote:
> That said, even if we do end up doing that short term, I'd like to
include plans or at least intentions to do things in Swift. I (and I'm sure
many of us) generally prefer working in Swift because it's safe, succinct,
and clean. We also have several instances of existing HTTP parsers we could
pull from, it's not like we're starting from 0 as a group.
>
> For HTTP2, I think everyone is in agreement that a c library is the way
to go. I'd like to give HTTP a bit more time for opinions to come in.

I’m not entirely sure why you have double standards here. Why would you
come up with a safe, succinct and clean HTTP/1.x parser but not do the same
for HTTP/2?

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

36

But as part of this project you need to form a group opinion and telling someone else his idea about this is wasted time may not be the best approach

I didn’t say the idea is a waste of time. I said implementing and maintaining everything in Swift would be very time-consuming. And that’s not an attack on this idea at all. I’m really sorry and I apologize if it sounded that way. I was just pragmatically pointing that it would take longer for us to achieve our goal that way.

···

On Nov 4, 2016, at 7:07 PM, Tony Parker <anthony.parker@apple.com> wrote:

Hi Paulo,

On Nov 4, 2016, at 1:58 PM, Paulo Faria via swift-server-dev <swift-server-dev@swift.org> wrote:

The design of Foundation’s HTTP request/response is very tied to Foundation. I’d prefer we come up with our own design. Even though the name of the work group is Swift Server APIs, we’re still developing solutions for clients that could be used on the server side. An HTTP client for example. Because we have different frameworks with different concurrency models we need a design that’s flexible enough for that. Trying to adapt `URLRequest` and `HTTPURLResponse` would first, impact existing applications that use it on iOS for example, and second, we might not have enough freedom with the API design to accomplish that. Even if the Foundation team is in favor of changing the design, there’s only a small level of change in the design we can make so the API doesn’t feel alien to the rest of Foundation.

If you’re thinking about iOS specifically. I think some of the libraries we build here could certainly be used there as well. I can see an HTTP client built from this effort running on iOS. This way we don’t mess around with existing software. We all know people don’t like when their code breaks.

I think if the API design feels “alien” to Foundation, then we either need to update the Foundation API to feel like it fits in with Swift or design the server API to fit in with Foundation.

I’ll keep pushing for this throughout the process. It’s really important that we have a coherent stack of software from top to bottom. Ending up with different model objects for URLRequest used in the same app via two frameworks would be really unfortunate.

This comes at a cost, of course, but consistency is not something we should dismiss so easily right out of the gate.

- Tony

On Nov 4, 2016, at 6:34 PM, Brent Royal-Gordon via swift-server-dev <swift-server-dev@swift.org> wrote:

On Nov 4, 2016, at 12:18 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

Helge, if I had my way, we wouldn't touch C in any way whatsoever for any of the server side libraries except as an absolute last resort, but this life is full of compromise, and I'm trying to be amicable to come up with solutions that we can agree on.

For HTTP2 as well, I'd like plans to eventually move to pure swift native implementations rethought for the language.

I think the right solution in general is to use C quite aggressively at first, but wall it off behind Swift facades. Later, we can revisit our C dependencies and replace them with Swift as we feel it's appropriate. Ideally, we'd use protocols here so the replacement is as simple as possible.

Here's my question: Can we have our HTTP parser use Foundation's existing `URLRequest` and `HTTPURLResponse` types as Swift-facing outputs and inputs? Or are these types too-tightly designed for an HTTP client's needs, and we need different types for an HTTP server? Or are there minor modifications the Foundation team is willing to make in Apple Foundation to extend them for our use case? If we *can* use these types on both client and server, I think that would be ideal—it would allow people to reuse their knowledge and potentially a little bit of code.

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

37

With respect to adopting other core Swift ideas like value types: we turned 20 classes into value types last year. This included a ton of work to adopt standard library protocols and improve type safety. It is an absolutely huge surface amount of Foundation’s total API surface area. It’s a large statement about how much we value making Foundation’s API consistent for Swift.

Yeah! I really love the effort you’re putting into making Foundation more “Swifty”. That doesn’t go unnoticed.

···

On Nov 4, 2016, at 8:49 PM, Tony Parker <anthony.parker@apple.com> wrote:

However, I do not believe there is such a fundamental conceptual mismatch here that we cannot preserve one of the most useful aspects of developing with the iOS, macOS SDKs - consistent types and low impedance mismatch between API at all levels of the stack.

I definitely value "consistent types and low impedance mismatch between API at all levels of the stack”. I just hope the APIs are designed with the best of Swift in mind. Then after we get a good design, we can see if it fits Foundation, and then make the adaptations there. I think this approach would make Foundation follow along and become even more “Swiftier” with time. Of course this is easier said than done, but I believe that’s what would make the ecosystem as a whole become better for Swift.

38

Logan, would you be kind to explain why you think we shouldn’t touch C?

Well, I think the rational is pretty clear. Staying within a single language has obvious benefits. Add to that safety features inherent to Swift designed solutions (buffer overruns and such) as well as the relative beauty of a modern language.

http_parser is a very C thing, it makes extensive use of pointers, goto’s and other C ‘tricks' to accomplish the performance it has. It is reasonably small and focused code but certainly not code which is straight forward to read&understand.

At the same time http_parser is a nice demo on how a high performance C core implementation is used in a high-level language environment (Node.js). The user of Node.js doesn’t have to deal with the C stuff at all.

Anyways I stick to my original opinion:
—snip---
Personally I’d suggest a small wrapper around this one: GitHub - nodejs/http-parser: http request/response parser for c
—snap—

But if someone has a great Swift parser providing the key features I’m interested in (push based, as zero copy as possible, reasonably fast), I’m quite interested too :->

hh

···

On 4 Nov 2016, at 21:20, Paulo Faria <paulo@zewo.io> wrote:

My argument is that using a C library that we all know works well gives us time to work on more important things, like the user-facing API. You mentioned that we all like Swift because it’s safe, succinct and clean. I think we can all agree with that, but that statement doesn’t correlate to using a C lib. We wouldn’t be _implementing_ the parser in c, we would be _using_ a existing c parser. So the work would fall into what we’ll already have to do when dealing with C POSIX APIs, for example. Using a C lib would be one less thing to maintain. So again we can focus on creating safe, succinct and clean code where it really matters, the API level.

On Nov 4, 2016, at 5:18 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

Helge, if I had my way, we wouldn't touch C in any way whatsoever for any of the server side libraries except as an absolute last resort, but this life is full of compromise, and I'm trying to be amicable to come up with solutions that we can agree on.

For HTTP2 as well, I'd like plans to eventually move to pure swift native implementations rethought for the language.

On Fri, Nov 4, 2016 at 3:17 PM Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:
On 4 Nov 2016, at 20:11, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:
> That said, even if we do end up doing that short term, I'd like to include plans or at least intentions to do things in Swift. I (and I'm sure many of us) generally prefer working in Swift because it's safe, succinct, and clean. We also have several instances of existing HTTP parsers we could pull from, it's not like we're starting from 0 as a group.
>
> For HTTP2, I think everyone is in agreement that a c library is the way to go. I'd like to give HTTP a bit more time for opinions to come in.

I’m not entirely sure why you have double standards here. Why would you come up with a safe, succinct and clean HTTP/1.x parser but not do the same for HTTP/2?

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

39

I did that in a Swift 2 GCD (and http-parser) based iOS HTTP proxy for a customer project. That is, parse stuff into NSURLRequest and NSHTTPURLResponse. I think in general this is the right approach to do this, but I also ran into a few issues. E.g. one was that NSURLRequest takes an NSURL, hence it can’t be used as-is for a `CONNECT host:port …` request.

The other thing I dislike about the two is that the API isn’t symmetric. E.g. in NSURLRequest you grab the headers via

  var allHTTPHeaderFields: [String : String]?

whereas in NSHTTPURLResponse you do:

  var allHeaderFields: [AnyHashable : Any]

NSURLRequest has

  func value(forHTTPHeaderField: String)

which NSHTTPURLResponse doesn’t have, etc.

But I suppose stuff like that should be fixed in Foundation too. And it isn’t a blocker, I ended up writing extensions for such.
And maybe that is the right solution for Swift-Server: Have protocols for that and let the two be just one implementation of them. E.g. a libcurl C request struct could be another.

Having said that: there is one detail which may be hard for the API as it may tie into the way the I/O is done: the body streams. In my project I just used the two classes for the headers only.

hh

···

On 4 Nov 2016, at 21:34, Brent Royal-Gordon <brent@architechies.com> wrote:

Here's my question: Can we have our HTTP parser use Foundation's existing `URLRequest` and `HTTPURLResponse` types as Swift-facing outputs and inputs?

40

It is a matter of priorities.
For some (not me) being pure Swift is a very high priority and I can see the thinking behind that.
For others pure Swift doesn’t matter so much for various different reasons, some may prefer Swift but chose C for pragmatic reasons, some are fine with C in general and don’t care, etc.
For yet another group C is as much a 1st-level language feature as C is a 1st-level language to Objective-C/C++.

And let me now do the Java 8 Update 111 :->

hh

···

On 4 Nov 2016, at 23:08, Paulo Faria via swift-server-dev <swift-server-dev@swift.org> wrote:

But as part of this project you need to form a group opinion and telling someone else his idea about this is wasted time may not be the best approach

I didn’t say the idea is a waste of time. I said implementing and maintaining everything in Swift would be very time-consuming. And that’s not an attack on this idea at all. I’m really sorry and I apologize if it sounded that way. I was just pragmatically pointing that it would take longer for us to achieve our goal that way.