Swift-server-dev Digest, Vol 2, Issue 1

[steve_algernon]

NSHTTPURLRequest and NHTTPURLResponse should be written in terms of a richer HTTP message object. I would suggest NSHTTPMessage => { NSHTTPRequestMessage, NSHTTPResponseMessage } with a class method to parse and construct.

(Currently, NSURLRequest carries a reference to any payload, while NSURLSession expects the payload to be separate from the request header. NSURLResponse doesn't provide the response as a data, and I don't like having the two not be reflexive.)

There's two reasons for this: (1) The current API doesn't allow for correct behavior for multiple HTTP Headers. If you Set-Cookie: a=x and Set-Cookie: b=x, asking for the Set-Cookie header will give you SetCookie value of "a=x; b=x", which is not a valid cookie. There may be other headers for which this combining is incorrect.

(2) CFHTTPMessageRef in CFNetwork implements the underlying behavior here, but that object is not exposed in Foundation. I don't know if CFHTTPMessage ever made it out into the CFNetwork open source drop of, what 2003? but it could do with a refresh.

That being said, the header parsing is pretty trivial. (Except where it isn't, eg multipart-mixed-replace and chunked-encoding Trailers)

--sma

···

On Nov 4, 2016, at 1:34 PM, Brent Royal-Gordon via swift-server-dev <swift-server-dev@swift.org> wrote:

Here's my question: Can we have our HTTP parser use Foundation's existing `URLRequest` and `HTTPURLResponse` types as Swift-facing outputs and inputs? Or are these types too-tightly designed for an HTTP client's needs, and we need different types for an HTTP server? Or are there minor modifications the Foundation team is willing to make in Apple Foundation to extend them for our use case? If we *can* use these types on both client and server, I think that would be ideal—it would allow people to reuse their knowledge and potentially a little bit of code.

[Tony_Parker]

Hi Tony!

I’m glad that you touched this subject. That’s something I think we should sort out early. How is the relationship between the Swift Server APIs and Foundation going to be. I think we can all agree that we don’t want duplicate APIs, and I love that you’re willing to do what it takes for it. I just worry about this specific line you said:

or design the server API to fit in with Foundation.

The reason I don’t think this is the best solution is because Foundation wasn’t designed for Swift. It uses concepts which are very well aligned with Objective-C, and it works beautifully there. I can see the effort on making Foundation be more Swifty, but I’m not sure about the level of changes you’re willing to go to make this happen. Specially because the biggest difference is on the programming paradigm used. Foundation uses OOP favoring reference types and inheritance as the main paradigm. We all know that Swift on the other hand proposes a whole new paradigm POP (Protocol oriented programming) favoring value types and protocols. Is it realistic to expect that Foundation on Swift would be changed to use this paradigm while maintaining Objective-C’s foundation with it’s original paradigm? I feel the weight of legacy will have a big impact on the design if we decide to make the server APIs work with Foundation instead of the other way around. I see the Swift Server APIs as a chance to design libraries that are 100% design with the beauty of Swift in mind. If we adapt to Foundation I feel a lot of that will be lost. I’m not saying we should just ditch Foundation. What I would love to see is a true Swift Foundation and the Server APIs sharing code with it.

Without getting too much into this topic, which can quickly turn a civil discussion into something else, Foundation has been "protocol oriented" all along. This is the reason that Foundation has class clusters. NSData, NSString, NSArray, NSDictionary, NSSet, NSAttributedString, NSCalendar, NSLocale, and lots more — those are all abstract base classes, which provide a default implementation for some methods and expect subclasses to provide the implementation of a few core methods.

Now it is clearly true that Swift allows us to expand upon this by adopting multiple protocols. However, I do not believe there is such a fundamental conceptual mismatch here that we cannot preserve one of the most useful aspects of developing with the iOS, macOS SDKs - consistent types and low impedance mismatch between API at all levels of the stack.

With respect to adopting other core Swift ideas like value types: we turned 20 classes into value types last year. This included a ton of work to adopt standard library protocols and improve type safety. It is an absolutely huge surface amount of Foundation’s total API surface area. It’s a large statement about how much we value making Foundation’s API consistent for Swift.

- Tony

···

On Nov 4, 2016, at 2:55 PM, Paulo Faria via swift-server-dev <swift-server-dev@swift.org> wrote:

On Nov 4, 2016, at 7:38 PM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

On 4 Nov 2016, at 21:34, Brent Royal-Gordon <brent@architechies.com> wrote:

Here's my question: Can we have our HTTP parser use Foundation's existing `URLRequest` and `HTTPURLResponse` types as Swift-facing outputs and inputs?

I did that in a Swift 2 GCD (and http-parser) based iOS HTTP proxy for a customer project. That is, parse stuff into NSURLRequest and NSHTTPURLResponse. I think in general this is the right approach to do this, but I also ran into a few issues. E.g. one was that NSURLRequest takes an NSURL, hence it can’t be used as-is for a `CONNECT host:port …` request.

The other thing I dislike about the two is that the API isn’t symmetric. E.g. in NSURLRequest you grab the headers via

var allHTTPHeaderFields: [String : String]?

whereas in NSHTTPURLResponse you do:

var allHeaderFields: [AnyHashable : Any]

NSURLRequest has

func value(forHTTPHeaderField: String)

which NSHTTPURLResponse doesn’t have, etc.

But I suppose stuff like that should be fixed in Foundation too. And it isn’t a blocker, I ended up writing extensions for such.
And maybe that is the right solution for Swift-Server: Have protocols for that and let the two be just one implementation of them. E.g. a libcurl C request struct could be another.

Having said that: there is one detail which may be hard for the API as it may tie into the way the I/O is done: the body streams. In my project I just used the two classes for the headers only.

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

[Paulo_Faria]

Hey Helge.

Just as I’m not in favor of _implementing_ everything in Swift for the sake of implementing it in Swift. I’m also not in favor of _using_ an existing C lib for the sake of using an existing C lib. Like you said, if someone comes up with a Swift implementation that it’s thoroughly tested, push-based, as zero copy as possible and as fast or faster than `http_parser`, then I think it makes sense for us to use it. The thing is, that’s unlikely to happen. And if it does it would take quite a while for it to reach the stability of http_parser. So pragmatically I see no reason for us to use our precious time reinventing the wheel. We have to remember that using a lib is very different than writing it. If we write in Swift we’ll have more code to maintain. Using a C lib that works, means we don’t even need to know how it works. We just need to know it works.

···

On Nov 4, 2016, at 7:18 PM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

On 4 Nov 2016, at 21:20, Paulo Faria <paulo@zewo.io <mailto:paulo@zewo.io>> wrote:

Logan, would you be kind to explain why you think we shouldn’t touch C?

Well, I think the rational is pretty clear. Staying within a single language has obvious benefits. Add to that safety features inherent to Swift designed solutions (buffer overruns and such) as well as the relative beauty of a modern language.

http_parser is a very C thing, it makes extensive use of pointers, goto’s and other C ‘tricks' to accomplish the performance it has. It is reasonably small and focused code but certainly not code which is straight forward to read&understand.

At the same time http_parser is a nice demo on how a high performance C core implementation is used in a high-level language environment (Node.js). The user of Node.js doesn’t have to deal with the C stuff at all.

Anyways I stick to my original opinion:
—snip---
Personally I’d suggest a small wrapper around this one: GitHub - nodejs/http-parser: http request/response parser for c
—snap—

But if someone has a great Swift parser providing the key features I’m interested in (push based, as zero copy as possible, reasonably fast), I’m quite interested too :->

hh

My argument is that using a C library that we all know works well gives us time to work on more important things, like the user-facing API. You mentioned that we all like Swift because it’s safe, succinct and clean. I think we can all agree with that, but that statement doesn’t correlate to using a C lib. We wouldn’t be _implementing_ the parser in c, we would be _using_ a existing c parser. So the work would fall into what we’ll already have to do when dealing with C POSIX APIs, for example. Using a C lib would be one less thing to maintain. So again we can focus on creating safe, succinct and clean code where it really matters, the API level.

On Nov 4, 2016, at 5:18 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

Helge, if I had my way, we wouldn't touch C in any way whatsoever for any of the server side libraries except as an absolute last resort, but this life is full of compromise, and I'm trying to be amicable to come up with solutions that we can agree on.

For HTTP2 as well, I'd like plans to eventually move to pure swift native implementations rethought for the language.

On Fri, Nov 4, 2016 at 3:17 PM Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:
On 4 Nov 2016, at 20:11, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

That said, even if we do end up doing that short term, I'd like to include plans or at least intentions to do things in Swift. I (and I'm sure many of us) generally prefer working in Swift because it's safe, succinct, and clean. We also have several instances of existing HTTP parsers we could pull from, it's not like we're starting from 0 as a group.

For HTTP2, I think everyone is in agreement that a c library is the way to go. I'd like to give HTTP a bit more time for opinions to come in.

I’m not entirely sure why you have double standards here. Why would you come up with a safe, succinct and clean HTTP/1.x parser but not do the same for HTTP/2?

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev

[Paulo_Faria]

Hi Tony!

I’m glad that you touched this subject. That’s something I think we should sort out early. How is the relationship between the Swift Server APIs and Foundation going to be. I think we can all agree that we don’t want duplicate APIs, and I love that you’re willing to do what it takes for it. I just worry about this specific line you said:

or design the server API to fit in with Foundation.

The reason I don’t think this is the best solution is because Foundation wasn’t designed for Swift. It uses concepts which are very well aligned with Objective-C, and it works beautifully there. I can see the effort on making Foundation be more Swifty, but I’m not sure about the level of changes you’re willing to go to make this happen. Specially because the biggest difference is on the programming paradigm used. Foundation uses OOP favoring reference types and inheritance as the main paradigm. We all know that Swift on the other hand proposes a whole new paradigm POP (Protocol oriented programming) favoring value types and protocols. Is it realistic to expect that Foundation on Swift would be changed to use this paradigm while maintaining Objective-C’s foundation with it’s original paradigm? I feel the weight of legacy will have a big impact on the design if we decide to make the server APIs work with Foundation instead of the other way around. I see the Swift Server APIs as a chance to design libraries that are 100% design with the beauty of Swift in mind. If we adapt to Foundation I feel a lot of that will be lost. I’m not saying we should just ditch Foundation. What I would love to see is a true Swift Foundation and the Server APIs sharing code with it.

···

On Nov 4, 2016, at 7:38 PM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

On 4 Nov 2016, at 21:34, Brent Royal-Gordon <brent@architechies.com> wrote:

Here's my question: Can we have our HTTP parser use Foundation's existing `URLRequest` and `HTTPURLResponse` types as Swift-facing outputs and inputs?

I did that in a Swift 2 GCD (and http-parser) based iOS HTTP proxy for a customer project. That is, parse stuff into NSURLRequest and NSHTTPURLResponse. I think in general this is the right approach to do this, but I also ran into a few issues. E.g. one was that NSURLRequest takes an NSURL, hence it can’t be used as-is for a `CONNECT host:port …` request.

The other thing I dislike about the two is that the API isn’t symmetric. E.g. in NSURLRequest you grab the headers via

var allHTTPHeaderFields: [String : String]?

whereas in NSHTTPURLResponse you do:

var allHeaderFields: [AnyHashable : Any]

NSURLRequest has

func value(forHTTPHeaderField: String)

which NSHTTPURLResponse doesn’t have, etc.

But I suppose stuff like that should be fixed in Foundation too. And it isn’t a blocker, I ended up writing extensions for such.
And maybe that is the right solution for Swift-Server: Have protocols for that and let the two be just one implementation of them. E.g. a libcurl C request struct could be another.

Having said that: there is one detail which may be hard for the API as it may tie into the way the I/O is done: the body streams. In my project I just used the two classes for the headers only.

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

[Brent_Royal-Gordon]

I did that in a Swift 2 GCD (and http-parser) based iOS HTTP proxy for a customer project. That is, parse stuff into NSURLRequest and NSHTTPURLResponse. I think in general this is the right approach to do this, but I also ran into a few issues. E.g. one was that NSURLRequest takes an NSURL, hence it can’t be used as-is for a `CONNECT host:port …` request.

This may be the kind of thing that needs extension. (As an additional benefit, it would help on the client side, too—client-side URLRequests are equally unable to represent CONNECT method use.)

And maybe that is the right solution for Swift-Server: Have protocols for that and let the two be just one implementation of them. E.g. a libcurl C request struct could be another.

Personally, I imagine that libcurl would be handled by translating from `URLRequest` to its preferred structures, but there are different ways to handle that.

Having said that: there is one detail which may be hard for the API as it may tie into the way the I/O is done: the body streams. In my project I just used the two classes for the headers only.

`bodyStream` is an `InputStream`, so is this an issue if you subclass `InputStream` to represent a socket (or, I suppose, the next Content-Length bytes of a socket)? Or is there some other issue here I'm not thinking of?

···

On Nov 4, 2016, at 2:38 PM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

--
Brent Royal-Gordon
Architechies

[Chris_Bailey]

I think the right solution in general is to use C quite aggressively at

first, but wall it off behind Swift facades. Later, we can revisit our C
dependencies and replace them with Swift as we feel it's appropriate.
Ideally, we'd use protocols here so the replacement is as simple as
possible.

+1

Chris

<swift-server-dev@swift.org>
swift-server-dev <swift-server-dev@swift.org>
Sent by: swift-server-dev-bounces@swift.org

Helge, if I had my way, we wouldn't touch C in any way whatsoever for

any of the server side libraries except as an absolute last resort, but
this life is full of compromise, and I'm trying to be amicable to come up
with solutions that we can agree on.

For HTTP2 as well, I'd like plans to eventually move to pure swift

native implementations rethought for the language.

I think the right solution in general is to use C quite aggressively at
first, but wall it off behind Swift facades. Later, we can revisit our C
dependencies and replace them with Swift as we feel it's appropriate.
Ideally, we'd use protocols here so the replacement is as simple as
possible.

Here's my question: Can we have our HTTP parser use Foundation's existing
`URLRequest` and `HTTPURLResponse` types as Swift-facing outputs and
inputs? Or are these types too-tightly designed for an HTTP client's
needs, and we need different types for an HTTP server? Or are there minor
modifications the Foundation team is willing to make in Apple Foundation
to extend them for our use case? If we *can* use these types on both
client and server, I think that would be ideal—it would allow people to
reuse their knowledge and potentially a little bit of code.

···

From: Brent Royal-Gordon via swift-server-dev
To: Logan Wright <logan@qutheory.io>, Logan Wright via
Date: 04/11/2016 20:34
Subject: Re: [swift-server-dev] HTTP Parser

On Nov 4, 2016, at 12:18 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

--
Brent Royal-Gordon
Architechies

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

[Chris_Bailey]

Whilst it might not feel like it, it seems to be that we're all pulling in
the same direction.

Tony said:

I think if the API design feels “alien” to Foundation, then we either

need to update the Foundation API to feel like it fits in with Swift or
design the server API to fit in with Foundation.

I’ll keep pushing for this throughout the process. It’s really

important that we have a coherent stack of software from top to bottom.
Ending up with different model objects for URLRequest used in the same app
via two frameworks would be really unfortunate.

I think this completely aligns with what everyone is saying - we want/need
a coherent API stack with seamless interop between any new APIs and the
existing ones. The only thing we need to work on is the approach to
achieving that.

For me, and I think that this is the approach that Paulo is pushing for,
is that we should first look at "what do we think is the right approach",
without any constraints. Once we understand that, and any gap there is
between that and the Foundation approach, we can determine how we close
that gap - whether we can do that by evolving APIs, or whether it requires
compromises for the sake of developer experience and application code
portability, etc.

Chris

···

From: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
To: Tony Parker <anthony.parker@apple.com>
Cc: Swift Server Dev <swift-server-dev@swift.org>
Date: 04/11/2016 23:08
Subject: Re: [swift-server-dev] HTTP Parser
Sent by: swift-server-dev-bounces@swift.org

With respect to adopting other core Swift ideas like value types: we

turned 20 classes into value types last year. This included a ton of work
to adopt standard library protocols and improve type safety. It is an
absolutely huge surface amount of Foundation’s total API surface area.
It’s a large statement about how much we value making Foundation’s API
consistent for Swift.

Yeah! I really love the effort you’re putting into making Foundation more
“Swifty”. That doesn’t go unnoticed.

On Nov 4, 2016, at 8:49 PM, Tony Parker <anthony.parker@apple.com> wrote:

However, I do not believe there is such a fundamental conceptual mismatch
here that we cannot preserve one of the most useful aspects of developing
with the iOS, macOS SDKs - consistent types and low impedance mismatch
between API at all levels of the stack.

I definitely value "consistent types and low impedance mismatch between
API at all levels of the stack”. I just hope the APIs are designed with
the best of Swift in mind. Then after we get a good design, we can see if
it fits Foundation, and then make the adaptations there. I think this
approach would make Foundation follow along and become even more
“Swiftier” with time. Of course this is easier said than done, but I
believe that’s what would make the ecosystem as a whole become better for
Swift._______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

[cloutiertyler (Tyler Cloutier)]

I would agree that wrapping the nodejs http_parser is probably the way to go for now. There is a lot of places we could spend our time, but given that that is an extremely well tested program at this point, I don’t see the need to put any wood behind that arrow. Writing a Swift wrapper around it is not difficult and frees us up to work on other more pressing things. In fact many of us, have already done it <https://github.com/SwiftOnEdge/Edge/blob/master/Sources/HTTP/Parser.swift&gt;\.

Tyler

···

On Nov 4, 2016, at 2:18 PM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

On 4 Nov 2016, at 21:20, Paulo Faria <paulo@zewo.io> wrote:

Logan, would you be kind to explain why you think we shouldn’t touch C?

Well, I think the rational is pretty clear. Staying within a single language has obvious benefits. Add to that safety features inherent to Swift designed solutions (buffer overruns and such) as well as the relative beauty of a modern language.

http_parser is a very C thing, it makes extensive use of pointers, goto’s and other C ‘tricks' to accomplish the performance it has. It is reasonably small and focused code but certainly not code which is straight forward to read&understand.

At the same time http_parser is a nice demo on how a high performance C core implementation is used in a high-level language environment (Node.js). The user of Node.js doesn’t have to deal with the C stuff at all.

Anyways I stick to my original opinion:
—snip---
Personally I’d suggest a small wrapper around this one: GitHub - nodejs/http-parser: http request/response parser for c
—snap—

But if someone has a great Swift parser providing the key features I’m interested in (push based, as zero copy as possible, reasonably fast), I’m quite interested too :->

hh

My argument is that using a C library that we all know works well gives us time to work on more important things, like the user-facing API. You mentioned that we all like Swift because it’s safe, succinct and clean. I think we can all agree with that, but that statement doesn’t correlate to using a C lib. We wouldn’t be _implementing_ the parser in c, we would be _using_ a existing c parser. So the work would fall into what we’ll already have to do when dealing with C POSIX APIs, for example. Using a C lib would be one less thing to maintain. So again we can focus on creating safe, succinct and clean code where it really matters, the API level.

On Nov 4, 2016, at 5:18 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

Helge, if I had my way, we wouldn't touch C in any way whatsoever for any of the server side libraries except as an absolute last resort, but this life is full of compromise, and I'm trying to be amicable to come up with solutions that we can agree on.

For HTTP2 as well, I'd like plans to eventually move to pure swift native implementations rethought for the language.

On Fri, Nov 4, 2016 at 3:17 PM Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:
On 4 Nov 2016, at 20:11, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

That said, even if we do end up doing that short term, I'd like to include plans or at least intentions to do things in Swift. I (and I'm sure many of us) generally prefer working in Swift because it's safe, succinct, and clean. We also have several instances of existing HTTP parsers we could pull from, it's not like we're starting from 0 as a group.

For HTTP2, I think everyone is in agreement that a c library is the way to go. I'd like to give HTTP a bit more time for opinions to come in.

I’m not entirely sure why you have double standards here. Why would you come up with a safe, succinct and clean HTTP/1.x parser but not do the same for HTTP/2?

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

[Joannis_Orlandos]

I think that if you wrap C libraries you'll get stuck in the wrong mentality. Working with C libraries encourages thinking without protocols. Protocols like `ExpressibleByDictionaryLiteral` and `Sequence` are really easy to implement and make usage of conforming types really beautiful. I think that a lot of APIs will be designed with the wrong mentality, at least initially, because they're alien to Swift. And that'll create APIs for these libraries that are not at the level where they could be in a language like Swift.

I don't think we need to rush ourselves with trying to create the libraries rapidly. If you want to have a common set of libraries in which the entire community will cooperate I think Swift is the only way to go. Many people have already written big libraries and frameworks in Swift. And I myself would never remove my own Swift code in favour of C-reliant code. And I doubt it's just me. I think that people whom prefer pure Swift will stick with (their own) existing functionality and rather build on top of a C reliant codebase.

Besides that all I feel like there's no need to rush it. A good set of libraries needs time. If you have a million people working on something for a week it will never be as good as a hundred people for a year or two. A good API/library needs time to be thought out and will need more than a few nights of rest before I think the idea alone should be considered complete. I think this counts for C-based and pure Swift libraries.

And if we'd just take the time to develop something great it will have a much greater impact on the Server Side Swift world than it would otherwise.

Joannis

···

On 6 Nov 2016, at 10:44, Chris Bailey via swift-server-dev <swift-server-dev@swift.org> wrote:

Whilst it might not feel like it, it seems to be that we're all pulling in the same direction.

Tony said:
>> I think if the API design feels “alien” to Foundation, then we either need to update the Foundation API to feel like it fits in with Swift or design the server API to fit in with Foundation.

>> I’ll keep pushing for this throughout the process. It’s really important that we have a coherent stack of software from top to bottom. Ending up with different model objects for URLRequest used in the same app via two frameworks would be really unfortunate.

I think this completely aligns with what everyone is saying - we want/need a coherent API stack with seamless interop between any new APIs and the existing ones. The only thing we need to work on is the approach to achieving that.

For me, and I think that this is the approach that Paulo is pushing for, is that we should first look at "what do we think is the right approach", without any constraints. Once we understand that, and any gap there is between that and the Foundation approach, we can determine how we close that gap - whether we can do that by evolving APIs, or whether it requires compromises for the sake of developer experience and application code portability, etc.

Chris

From: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
To: Tony Parker <anthony.parker@apple.com>
Cc: Swift Server Dev <swift-server-dev@swift.org>
Date: 04/11/2016 23:08
Subject: Re: [swift-server-dev] HTTP Parser
Sent by: swift-server-dev-bounces@swift.org

> With respect to adopting other core Swift ideas like value types: we turned 20 classes into value types last year. This included a ton of work to adopt standard library protocols and improve type safety. It is an absolutely huge surface amount of Foundation’s total API surface area. It’s a large statement about how much we value making Foundation’s API consistent for Swift.

Yeah! I really love the effort you’re putting into making Foundation more “Swifty”. That doesn’t go unnoticed.

On Nov 4, 2016, at 8:49 PM, Tony Parker <anthony.parker@apple.com <mailto:anthony.parker@apple.com>> wrote:

However, I do not believe there is such a fundamental conceptual mismatch here that we cannot preserve one of the most useful aspects of developing with the iOS, macOS SDKs - consistent types and low impedance mismatch between API at all levels of the stack.

I definitely value "consistent types and low impedance mismatch between API at all levels of the stack”. I just hope the APIs are designed with the best of Swift in mind. Then after we get a good design, we can see if it fits Foundation, and then make the adaptations there. I think this approach would make Foundation follow along and become even more “Swiftier” with time. Of course this is easier said than done, but I believe that’s what would make the ecosystem as a whole become better for Swift._______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

[Chris_Bailey]

To add some data into the debate...

The C based http_parser used by Node.js (and original from NGINX) is
approx 3K lines of code. If we think strictly in terms of effort to
implement (and ignore maintenance costs or what's required to reach the
same quality levels etc) then it shouldn't be an unreasonable amount of
effort to do a Swift implementation - and in fact we could look at both
approaches and see which is preferable (as both approaches have been taken
by Swift frameworks already).

The nghttp2 library however is approx 64K lines of code (looking only at
the src directory). As such, its a much bigger task to implement, and to
do so in a bug-free manner - note that the tests are another 17K lines of
code.

As an aside, wrapping the nghttp2 library is the approach being taken by
Node.js for their HTTP/2 support.

Chris

···

From: Tyler Cloutier via swift-server-dev <swift-server-dev@swift.org>
To: Helge Heß <me@helgehess.eu>
Cc: Swift Server Dev <swift-server-dev@swift.org>
Date: 06/11/2016 19:35
Subject: Re: [swift-server-dev] HTTP Parser
Sent by: swift-server-dev-bounces@swift.org

I would agree that wrapping the nodejs http_parser is probably the way to
go for now. There is a lot of places we could spend our time, but given
that that is an extremely well tested program at this point, I don’t see
the need to put any wood behind that arrow. Writing a Swift wrapper around
it is not difficult and frees us up to work on other more pressing things.
In fact many of us, have already done it.

Tyler

On Nov 4, 2016, at 2:18 PM, Helge Heß via swift-server-dev < swift-server-dev@swift.org> wrote:

On 4 Nov 2016, at 21:20, Paulo Faria <paulo@zewo.io> wrote:
Logan, would you be kind to explain why you think we shouldn’t touch C?

Well, I think the rational is pretty clear. Staying within a single
language has obvious benefits. Add to that safety features inherent to
Swift designed solutions (buffer overruns and such) as well as the
relative beauty of a modern language.

http_parser is a very C thing, it makes extensive use of pointers, goto’s
and other C ‘tricks' to accomplish the performance it has. It is
reasonably small and focused code but certainly not code which is straight
forward to read&understand.

At the same time http_parser is a nice demo on how a high performance C
core implementation is used in a high-level language environment
(Node.js). The user of Node.js doesn’t have to deal with the C stuff at
all.

Anyways I stick to my original opinion:
—snip---
Personally I’d suggest a small wrapper around this one:

GitHub - nodejs/http-parser: http request/response parser for c

—snap—

But if someone has a great Swift parser providing the key features I’m
interested in (push based, as zero copy as possible, reasonably fast), I’m
quite interested too :->

hh

My argument is that using a C library that we all know works well gives us
time to work on more important things, like the user-facing API. You
mentioned that we all like Swift because it’s safe, succinct and clean. I
think we can all agree with that, but that statement doesn’t correlate to
using a C lib. We wouldn’t be _implementing_ the parser in c, we would be
_using_ a existing c parser. So the work would fall into what we’ll
already have to do when dealing with C POSIX APIs, for example. Using a C
lib would be one less thing to maintain. So again we can focus on creating
safe, succinct and clean code where it really matters, the API level.

On Nov 4, 2016, at 5:18 PM, Logan Wright via swift-server-dev < swift-server-dev@swift.org> wrote:

Helge, if I had my way, we wouldn't touch C in any way whatsoever for any
of the server side libraries except as an absolute last resort, but this
life is full of compromise, and I'm trying to be amicable to come up with
solutions that we can agree on.

For HTTP2 as well, I'd like plans to eventually move to pure swift native
implementations rethought for the language.

On Fri, Nov 4, 2016 at 3:17 PM Helge Heß via swift-server-dev < swift-server-dev@swift.org> wrote:
On 4 Nov 2016, at 20:11, Logan Wright via swift-server-dev < swift-server-dev@swift.org> wrote:
That said, even if we do end up doing that short term, I'd like to include
plans or at least intentions to do things in Swift. I (and I'm sure many
of us) generally prefer working in Swift because it's safe, succinct, and
clean. We also have several instances of existing HTTP parsers we could
pull from, it's not like we're starting from 0 as a group.

For HTTP2, I think everyone is in agreement that a c library is the way to
go. I'd like to give HTTP a bit more time for opinions to come in.

I’m not entirely sure why you have double standards here. Why would you
come up with a safe, succinct and clean HTTP/1.x parser but not do the
same for HTTP/2?

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

[Paulo_Faria]

Hey Joannis! Nice to see you here.

I agree with you 100℅ we shouldn't rush the design of the APIs. I didn't mean that at all. I meant exactly the opposite. We should spend as much time as possible on the API design, and wrapping a C library would allows to distribute our total time of development focusing a lot more on the API design than on the development and maintenance of the internals. We should definitely design the API with the best of Swift using protocols and everything else you mentioned, and wraping C doesn't restrain us in any way to do that. There's nothing we can't do on the API level because we're wrapping C. What Chris Bailey mentioned about my mentality before is totally right. I believe we should spend as much time as possible designing the APIs without any constraint from, for example, Foundation or any C library, and in the Foundation case if really needed we should compromise. I just know from experience that wrapping C libs doesn't constraint at all the "Swiftyness" of the user-facing API. This is pure conjecture as I didn't really dig in, but I believe Swift's standard library itself uses libicu for String's internal on Linux. If this is wrong someone please correct me. I also believe Foundation wraps libcurl for the HTTP stuff.

I think that if you wrap C libraries you'll get stuck in the wrong mentality. Working with C libraries encourages thinking without protocols. Protocols like `ExpressibleByDictionaryLiteral` and `Sequence` are really easy to implement and make usage of conforming types really beautiful. I think that a lot of APIs will be designed with the wrong mentality, at least initially, because they're alien to Swift. And that'll create APIs for these libraries that are not at the level where they could be in a language like Swift.

I don't think we need to rush ourselves with trying to create the libraries rapidly. If you want to have a common set of libraries in which the entire community will cooperate I think Swift is the only way to go. Many people have already written big libraries and frameworks in Swift. And I myself would never remove my own Swift code in favour of C-reliant code. And I doubt it's just me. I think that people whom prefer pure Swift will stick with (their own) existing functionality and rather build on top of a C reliant codebase.

Besides that all I feel like there's no need to rush it. A good set of libraries needs time. If you have a million people working on something for a week it will never be as good as a hundred people for a year or two. A good API/library needs time to be thought out and will need more than a few nights of rest before I think the idea alone should be considered complete. I think this counts for C-based and pure Swift libraries.

And if we'd just take the time to develop something great it will have a much greater impact on the Server Side Swift world than it would otherwise.

Joannis

Whilst it might not feel like it, it seems to be that we're all pulling in the same direction.

Tony said:

I think if the API design feels “alien” to Foundation, then we either need to update the Foundation API to feel like it fits in with Swift or design the server API to fit in with Foundation.

I’ll keep pushing for this throughout the process. It’s really important that we have a coherent stack of software from top to bottom. Ending up with different model objects for URLRequest used in the same app via two frameworks would be really unfortunate.

I think this completely aligns with what everyone is saying - we want/need a coherent API stack with seamless interop between any new APIs and the existing ones. The only thing we need to work on is the approach to achieving that.

For me, and I think that this is the approach that Paulo is pushing for, is that we should first look at "what do we think is the right approach", without any constraints. Once we understand that, and any gap there is between that and the Foundation approach, we can determine how we close that gap - whether we can do that by evolving APIs, or whether it requires compromises for the sake of developer experience and application code portability, etc.

Chris

···

---- On Sun, 06 Nov 2016 07:54:15 -0200 joannis@orlandos.nl wrote ----
On 6 Nov 2016, at 10:44, Chris Bailey via swift-server-dev <swift-server-dev@swift.org> wrote:

From: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
To: Tony Parker <anthony.parker@apple.com>
Cc: Swift Server Dev <swift-server-dev@swift.org>
Date: 04/11/2016 23:08
Subject: Re: [swift-server-dev] HTTP Parser
Sent by: swift-server-dev-bounces@swift.org

With respect to adopting other core Swift ideas like value types: we turned 20 classes into value types last year. This included a ton of work to adopt standard library protocols and improve type safety. It is an absolutely huge surface amount of Foundation’s total API surface area. It’s a large statement about how much we value making Foundation’s API consistent for Swift.

Yeah! I really love the effort you’re putting into making Foundation more “Swifty”. That doesn’t go unnoticed.

On Nov 4, 2016, at 8:49 PM, Tony Parker <anthony.parker@apple.com> wrote:

However, I do not believe there is such a fundamental conceptual mismatch here that we cannot preserve one of the most useful aspects of developing with the iOS, macOS SDKs - consistent types and low impedance mismatch between API at all levels of the stack.

I definitely value "consistent types and low impedance mismatch between API at all levels of the stack”. I just hope the APIs are designed with the best of Swift in mind. Then after we get a good design, we can see if it fits Foundation, and then make the adaptations there. I think this approach would make Foundation follow along and become even more “Swiftier” with time. Of course this is easier said than done, but I believe that’s what would make the ecosystem as a whole become better for Swift._______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

[Logan_Wright1 (Logan Wright)]

Here are some of the most basic reasons why I would prefer to use swift
implementations.

- Easier for Swift contributors to enter and participate
- More readable code
- Faster in long run
- More concise code
- Fully embraces Swift language
- Easier to maintain with control over the library

Again, I am not saying this all needs to happen immediately, but I think
assuming that we'll only ever use c libraries in perpetuity is an extremely
short sighted vision of how the server apis can grow. We should also keep
in mind that when talking about HTTP, this isn't a from scratch rewrite as
seems to be suggested several times. There are a few (likely more) parsers
written in pure swift that we could mix and match to get something solid
into the library. The fact that these libraries exist makes wrapping c seem
like a step backwards in the progression of Swift which I am against. A
perfectly reasonable compromise (as suggested before) seems to be initially
wrapping some c code while leaving room in the design for Swift
implementations long term if necessary.

I'd still like to get more voices, we seem to have heard from about 7
people or so, and from what I can tell, the count is about 4/3. Anyone
following along care to weigh in a little bit so the same few people aren't
rehashing the same talking points.

- Logan

···

On Sun, Nov 6, 2016 at 7:37 AM Paulo Faria via swift-server-dev < swift-server-dev@swift.org> wrote:

Hey Joannis! Nice to see you here.

I agree with you 100℅ we shouldn't rush the design of the APIs. I didn't
mean that at all. I meant exactly the opposite. We should spend as much
time as possible on the API design, and wrapping a C library would allows
to distribute our total time of development focusing a lot more on the API
design than on the development and maintenance of the internals. We should
definitely design the API with the best of Swift using protocols and
everything else you mentioned, and wraping C doesn't restrain us in any way
to do that. There's nothing we can't do on the API level because we're
wrapping C. What Chris Bailey mentioned about my mentality before is
totally right. I believe we should spend as much time as possible designing
the APIs without any constraint from, for example, Foundation or any C
library, and in the Foundation case if really needed we should compromise.
I just know from experience that wrapping C libs doesn't constraint at all
the "Swiftyness" of the user-facing API. This is pure conjecture as I
didn't really dig in, but I believe Swift's standard library itself uses
libicu for String's internal on Linux. If this is wrong someone please
correct me. I also believe Foundation wraps libcurl for the HTTP stuff.

---- On Sun, 06 Nov 2016 07:54:15 -0200 * joannis@orlandos.nl > <joannis@orlandos.nl> * wrote ----

I think that if you wrap C libraries you'll get stuck in the wrong
mentality. Working with C libraries encourages thinking without protocols.
Protocols like `ExpressibleByDictionaryLiteral` and `Sequence` are really
easy to implement and make usage of conforming types really beautiful. I
think that a lot of APIs will be designed with the wrong mentality, at
least initially, because they're alien to Swift. And that'll create APIs
for these libraries that are not at the level where they could be in a
language like Swift.

I don't think we need to rush ourselves with trying to create the
libraries rapidly. If you want to have a common set of libraries in which
the entire community will cooperate I think Swift is the only way to go.
Many people have already written big libraries and frameworks in Swift. And
I myself would never remove my own Swift code in favour of C-reliant code.
And I doubt it's just me. I think that people whom prefer pure Swift will
stick with (their own) existing functionality and rather build on top of a
C reliant codebase.

Besides that all I feel like there's no need to rush it. A good set of
libraries needs time. If you have a million people working on something for
a week it will never be as good as a hundred people for a year or two. A
good API/library needs time to be thought out and will need more than a few
nights of rest before I think the idea alone should be considered complete.
I think this counts for C-based and pure Swift libraries.

And if we'd just take the time to develop something great it will have a
much greater impact on the Server Side Swift world than it would otherwise.

Joannis

On 6 Nov 2016, at 10:44, Chris Bailey via swift-server-dev < > swift-server-dev@swift.org> wrote:

Whilst it might not feel like it, it seems to be that we're all pulling in
the same direction.

Tony said:
>> I think if the API design feels “alien” to Foundation, then we either
need to update the Foundation API to feel like it fits in with Swift or
design the server API to fit in with Foundation.

>> I’ll keep pushing for this throughout the process. It’s really
important that we have a coherent stack of software from top to bottom.
Ending up with different model objects for URLRequest used in the same app
via two frameworks would be really unfortunate.

I think this completely aligns with what everyone is saying - we want/need
a coherent API stack with seamless interop between any new APIs and the
existing ones. The only thing we need to work on is the approach to
achieving that.

For me, and I think that this is the approach that Paulo is pushing for,
is that we should first look at "what do we think is the right approach",
without any constraints. Once we understand that, and any gap there is
between that and the Foundation approach, we can determine how we close
that gap - whether we can do that by evolving APIs, or whether it requires
compromises for the sake of developer experience and application code
portability, etc.

Chris

From: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
To: Tony Parker <anthony.parker@apple.com>
Cc: Swift Server Dev <swift-server-dev@swift.org>
Date: 04/11/2016 23:08
Subject: Re: [swift-server-dev] HTTP Parser
Sent by: swift-server-dev-bounces@swift.org
------------------------------

> With respect to adopting other core Swift ideas like value types: we
turned 20 classes into value types last year. This included a ton of work
to adopt standard library protocols and improve type safety. It is an
absolutely huge surface amount of Foundation’s total API surface area. It’s
a large statement about how much we value making Foundation’s API
consistent for Swift.

Yeah! I really love the effort you’re putting into making Foundation more
“Swifty”. That doesn’t go unnoticed.

On Nov 4, 2016, at 8:49 PM, Tony Parker < <anthony.parker@apple.com>*anthony.parker@apple.com > <anthony.parker@apple.com>*> wrote:

However, I do not believe there is such a fundamental conceptual mismatch
here that we cannot preserve one of the most useful aspects of developing
with the iOS, macOS SDKs - consistent types and low impedance mismatch
between API at all levels of the stack.

I definitely value "consistent types and low impedance mismatch between
API at all levels of the stack”. I just hope the APIs are designed with the
best of Swift in mind. Then after we get a good design, we can see if it
fits Foundation, and then make the adaptations there. I think this approach
would make Foundation follow along and become even more “Swiftier” with
time. Of course this is easier said than done, but I believe that’s what
would make the ecosystem as a whole become better for Swift.
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
<https://lists.swift.org/mailman/listinfo/swift-server-dev&gt;
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

[Catalin_Stan (Cătălin Stan)]

Hi everyone,

Just wanted to add a couple of thoughts to the conversation.

We have set this ambitions deadline in mind for the first version of the API’s and, with that in sight, it is highly unlikely that we’ll be able to write a parser in Swift. Also it is a monumental task to rewrite the whole thing - with all it’s specs, edge cases, support for v1.1, v2, byte-ranges, pipelining, multipart requests, a.s.o. - I know. I’ve done it. It sucks.

That being said, I cannot ignore the opportunity that we have at this point. Developing a parser from scratch, while working along side people that develop Foundation would allow us to also improve the language at the same time. Is there need for something similar to pointer arithmetic? Let’s make it. Does Foundation make too many data copy ops? Let’s make it so that it doesn’t. Is dispatch missing a kernel space component so that it can come close to Darwin in terms of speed? Let’s get that done as well.

All in all, I think that it’s worth looking more towards the long term benefits, rather the somewhat immediate result of having it “now”. After all it’s not like we have to *convince* the swift community to adopt it. The need is already there. Also, since everyone has already, to a larger or lesser extent, wrapped node’s parser around swift API’s, what’s the point of doing that again? Just to be able to say that this implementation “comes with swift”? I don’t believe it’s worth it. Again, this is just personal preference; I guess, I believe in doing things once, and not coming back to them over and over again. I digress.

As as I said in the begging, I do understand the time constraints and the inherent value that comes with using a tried and tested solution thus allowing us to focus on the design of the developer facing APIs. All in all, I agree with Chris that the first thing should be to spec out what an “ideal world” version would contain, without all the constraints we have.

Apologies for the rant,

Catalin

···

On 7 Nov 2016, at 02.59, Chris Bailey via swift-server-dev <swift-server-dev@swift.org> wrote:

To add some data into the debate...

The C based http_parser used by Node.js (and original from NGINX) is approx 3K lines of code. If we think strictly in terms of effort to implement (and ignore maintenance costs or what's required to reach the same quality levels etc) then it shouldn't be an unreasonable amount of effort to do a Swift implementation - and in fact we could look at both approaches and see which is preferable (as both approaches have been taken by Swift frameworks already).

The nghttp2 library however is approx 64K lines of code (looking only at the src directory). As such, its a much bigger task to implement, and to do so in a bug-free manner - note that the tests are another 17K lines of code.

As an aside, wrapping the nghttp2 library is the approach being taken by Node.js for their HTTP/2 support.

Chris

From: Tyler Cloutier via swift-server-dev <swift-server-dev@swift.org>
To: Helge Heß <me@helgehess.eu>
Cc: Swift Server Dev <swift-server-dev@swift.org>
Date: 06/11/2016 19:35
Subject: Re: [swift-server-dev] HTTP Parser
Sent by: swift-server-dev-bounces@swift.org

I would agree that wrapping the nodejs http_parser is probably the way to go for now. There is a lot of places we could spend our time, but given that that is an extremely well tested program at this point, I don’t see the need to put any wood behind that arrow. Writing a Swift wrapper around it is not difficult and frees us up to work on other more pressing things. In fact many of us, have already done it <https://github.com/SwiftOnEdge/Edge/blob/master/Sources/HTTP/Parser.swift&gt;\.

Tyler

On Nov 4, 2016, at 2:18 PM, Helge Heß via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:

On 4 Nov 2016, at 21:20, Paulo Faria <paulo@zewo.io <mailto:paulo@zewo.io>> wrote:
Logan, would you be kind to explain why you think we shouldn’t touch C?

Well, I think the rational is pretty clear. Staying within a single language has obvious benefits. Add to that safety features inherent to Swift designed solutions (buffer overruns and such) as well as the relative beauty of a modern language.

http_parser is a very C thing, it makes extensive use of pointers, goto’s and other C ‘tricks' to accomplish the performance it has. It is reasonably small and focused code but certainly not code which is straight forward to read&understand.

At the same time http_parser is a nice demo on how a high performance C core implementation is used in a high-level language environment (Node.js). The user of Node.js doesn’t have to deal with the C stuff at all.

Anyways I stick to my original opinion:
—snip---
Personally I’d suggest a small wrapper around this one: GitHub - nodejs/http-parser: http request/response parser for c
—snap—

But if someone has a great Swift parser providing the key features I’m interested in (push based, as zero copy as possible, reasonably fast), I’m quite interested too :->

hh

My argument is that using a C library that we all know works well gives us time to work on more important things, like the user-facing API. You mentioned that we all like Swift because it’s safe, succinct and clean. I think we can all agree with that, but that statement doesn’t correlate to using a C lib. We wouldn’t be _implementing_ the parser in c, we would be _using_ a existing c parser. So the work would fall into what we’ll already have to do when dealing with C POSIX APIs, for example. Using a C lib would be one less thing to maintain. So again we can focus on creating safe, succinct and clean code where it really matters, the API level.

On Nov 4, 2016, at 5:18 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:

Helge, if I had my way, we wouldn't touch C in any way whatsoever for any of the server side libraries except as an absolute last resort, but this life is full of compromise, and I'm trying to be amicable to come up with solutions that we can agree on.

For HTTP2 as well, I'd like plans to eventually move to pure swift native implementations rethought for the language.

On Fri, Nov 4, 2016 at 3:17 PM Helge Heß via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:
On 4 Nov 2016, at 20:11, Logan Wright via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:
That said, even if we do end up doing that short term, I'd like to include plans or at least intentions to do things in Swift. I (and I'm sure many of us) generally prefer working in Swift because it's safe, succinct, and clean. We also have several instances of existing HTTP parsers we could pull from, it's not like we're starting from 0 as a group.

For HTTP2, I think everyone is in agreement that a c library is the way to go. I'd like to give HTTP a bit more time for opinions to come in.

I’m not entirely sure why you have double standards here. Why would you come up with a safe, succinct and clean HTTP/1.x parser but not do the same for HTTP/2?

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

[Helge_Hess1 (Helge Heß)]

Ahh common! :-) The http_parser is just the parser. nghttp2 is a full HTTP networking library, pretty much a C++ version of what this groups wants to come up with! Included in your LOC is stuff like a `memcached_dispatcher.h/.cc` and `ruby_module.h/.cc` …

Do you really want to use nghttp2? Unlike the name suggests this isn’t a plain-C thing, it is actually a C++ code base. Scanning the code it looks like that would bring in the C++ stdlib, Boost!, libevent and who knows what. This is fine for Node which is C++ in the first place, but I really don’t find it desirable to pull in such a big thing as part of this effort.

hh

P.S.: As a sidenote, can’t someone (hm hm, who would come to mind?) sponsor the CryptoSwift guy to do a java.security like Security standard framework (which includes TLS)? That would be pretty cool :-)

···

On 07 Nov 2016, at 02:59, Chris Bailey <BAILEYC@uk.ibm.com> wrote:

To add some data into the debate...

The C based http_parser used by Node.js (and original from NGINX) is approx 3K lines of code. If we think strictly in terms of effort to implement (and ignore maintenance costs or what's required to reach the same quality levels etc) then it shouldn't be an unreasonable amount of effort to do a Swift implementation - and in fact we could look at both approaches and see which is preferable (as both approaches have been taken by Swift frameworks already).

The nghttp2 library however is approx 64K lines of code (looking only at the src directory).

[Benoit_Person]

I am new here and mostly following along (so take all I say with a
"newbie" pov ;) ), but is there actual value in discussing whether or
not the implementation should be Swift-based or a wrapper around C
right now?

Let's focus on designing a Swift-friendly API, unconstrained by any
"implementation detail" (C-wrapper or pure Swift), and when we have
that, make an educated decision on whether or not it can be built by
wrapping a C library to speed up delivery and iterations on the API?

Ben

···

On 6 November 2016 at 18:09, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

Here are some of the most basic reasons why I would prefer to use swift implementations.

- Easier for Swift contributors to enter and participate
- More readable code
- Faster in long run
- More concise code
- Fully embraces Swift language
- Easier to maintain with control over the library

Again, I am not saying this all needs to happen immediately, but I think assuming that we'll only ever use c libraries in perpetuity is an extremely short sighted vision of how the server apis can grow. We should also keep in mind that when talking about HTTP, this isn't a from scratch rewrite as seems to be suggested several times. There are a few (likely more) parsers written in pure swift that we could mix and match to get something solid into the library. The fact that these libraries exist makes wrapping c seem like a step backwards in the progression of Swift which I am against. A perfectly reasonable compromise (as suggested before) seems to be initially wrapping some c code while leaving room in the design for Swift implementations long term if necessary.

I'd still like to get more voices, we seem to have heard from about 7 people or so, and from what I can tell, the count is about 4/3. Anyone following along care to weigh in a little bit so the same few people aren't rehashing the same talking points.

- Logan

On Sun, Nov 6, 2016 at 7:37 AM Paulo Faria via swift-server-dev <swift-server-dev@swift.org> wrote:

Hey Joannis! Nice to see you here.

I agree with you 100℅ we shouldn't rush the design of the APIs. I didn't mean that at all. I meant exactly the opposite. We should spend as much time as possible on the API design, and wrapping a C library would allows to distribute our total time of development focusing a lot more on the API design than on the development and maintenance of the internals. We should definitely design the API with the best of Swift using protocols and everything else you mentioned, and wraping C doesn't restrain us in any way to do that. There's nothing we can't do on the API level because we're wrapping C. What Chris Bailey mentioned about my mentality before is totally right. I believe we should spend as much time as possible designing the APIs without any constraint from, for example, Foundation or any C library, and in the Foundation case if really needed we should compromise. I just know from experience that wrapping C libs doesn't constraint at all the "Swiftyness" of the user-facing API. This is pure conjecture as I didn't really dig in, but I believe Swift's standard library itself uses libicu for String's internal on Linux. If this is wrong someone please correct me. I also believe Foundation wraps libcurl for the HTTP stuff.

---- On Sun, 06 Nov 2016 07:54:15 -0200 joannis@orlandos.nl wrote ----

I think that if you wrap C libraries you'll get stuck in the wrong mentality. Working with C libraries encourages thinking without protocols. Protocols like `ExpressibleByDictionaryLiteral` and `Sequence` are really easy to implement and make usage of conforming types really beautiful. I think that a lot of APIs will be designed with the wrong mentality, at least initially, because they're alien to Swift. And that'll create APIs for these libraries that are not at the level where they could be in a language like Swift.

I don't think we need to rush ourselves with trying to create the libraries rapidly. If you want to have a common set of libraries in which the entire community will cooperate I think Swift is the only way to go. Many people have already written big libraries and frameworks in Swift. And I myself would never remove my own Swift code in favour of C-reliant code. And I doubt it's just me. I think that people whom prefer pure Swift will stick with (their own) existing functionality and rather build on top of a C reliant codebase.

Besides that all I feel like there's no need to rush it. A good set of libraries needs time. If you have a million people working on something for a week it will never be as good as a hundred people for a year or two. A good API/library needs time to be thought out and will need more than a few nights of rest before I think the idea alone should be considered complete. I think this counts for C-based and pure Swift libraries.

And if we'd just take the time to develop something great it will have a much greater impact on the Server Side Swift world than it would otherwise.

Joannis

On 6 Nov 2016, at 10:44, Chris Bailey via swift-server-dev <swift-server-dev@swift.org> wrote:

Whilst it might not feel like it, it seems to be that we're all pulling in the same direction.

Tony said:
>> I think if the API design feels “alien” to Foundation, then we either need to update the Foundation API to feel like it fits in with Swift or design the server API to fit in with Foundation.

>> I’ll keep pushing for this throughout the process. It’s really important that we have a coherent stack of software from top to bottom. Ending up with different model objects for URLRequest used in the same app via two frameworks would be really unfortunate.

I think this completely aligns with what everyone is saying - we want/need a coherent API stack with seamless interop between any new APIs and the existing ones. The only thing we need to work on is the approach to achieving that.

For me, and I think that this is the approach that Paulo is pushing for, is that we should first look at "what do we think is the right approach", without any constraints. Once we understand that, and any gap there is between that and the Foundation approach, we can determine how we close that gap - whether we can do that by evolving APIs, or whether it requires compromises for the sake of developer experience and application code portability, etc.

Chris

From: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
To: Tony Parker <anthony.parker@apple.com>
Cc: Swift Server Dev <swift-server-dev@swift.org>
Date: 04/11/2016 23:08
Subject: Re: [swift-server-dev] HTTP Parser
Sent by: swift-server-dev-bounces@swift.org
________________________________

> With respect to adopting other core Swift ideas like value types: we turned 20 classes into value types last year. This included a ton of work to adopt standard library protocols and improve type safety. It is an absolutely huge surface amount of Foundation’s total API surface area. It’s a large statement about how much we value making Foundation’s API consistent for Swift.

Yeah! I really love the effort you’re putting into making Foundation more “Swifty”. That doesn’t go unnoticed.

On Nov 4, 2016, at 8:49 PM, Tony Parker <anthony.parker@apple.com> wrote:

However, I do not believe there is such a fundamental conceptual mismatch here that we cannot preserve one of the most useful aspects of developing with the iOS, macOS SDKs - consistent types and low impedance mismatch between API at all levels of the stack.

I definitely value "consistent types and low impedance mismatch between API at all levels of the stack”. I just hope the APIs are designed with the best of Swift in mind. Then after we get a good design, we can see if it fits Foundation, and then make the adaptations there. I think this approach would make Foundation follow along and become even more “Swiftier” with time. Of course this is easier said than done, but I believe that’s what would make the ecosystem as a whole become better for Swift._______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

[Zach_Drayer]

<forking "HTTP Parser", thanks for reading and/or sorry in advance for any for threading issues that crop up>

P.S.: As a sidenote, can’t someone (hm hm, who would come to mind?) sponsor the CryptoSwift guy to do a java.security like Security standard framework (which includes TLS)? That would be pretty cool :-)

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Assuming we want a pure-Swift crypto framework _right now_:

Before any crypto code can safely be written in Swift, there needs to be compiler support to prevent it from optimizing things incorrectly (for example, a memset() designed to clear a buffer of secret data).

I may have missed discussions about this on swift-evolution (and maybe swift-server-dev is a place to flesh out a SE proposal to send their way?), but, i'm not sure if all that work's been scoped out anywhere, or if it has, what the timeframe is. ABI stability might even be a soft requirement here before anyone can reasonably get started on this.

So, I don't think we really want a pure Swift crypto library yet.

Instead: while in C, Security.framework, and CommonCrypto.framework from  are both fully open source, and battle-hardened. This is one of (rare) the cases where practicality and strongly deferring to the thing that already exists seem to be the safe option (and not only in opinion).

Anyway, (to me) the more interesting questions that we can talk about are over API:
- what constructs are needed (bignum? secure random? hmac? checksumming? key derivation? wrapping a socket in a big TLS hug?)
- what umbrella(s) do these constructs live in? (maybe bignum is in separate framework, or hugging sockets with TLS is an i/o construct rather than a crypto construct, or.. etc)
- is there anything to explicitly not support? securing a socket with sslv2 and sslv3 seem like obvious candidates here, but maybe the API is even more opinionated and doesn't let you create weak RSA keys, etc.
- ??? other exciting things to consider here

-z

···

On Nov 7, 2016, at 11:20 AM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

[Paulo_Faria]

I agree with you Ben. (:

Drawing from what Helge said and my own experience with parsers, I think the best design for parsers is the streaming (or incremental) parser. Streaming parsers work by the following mechanism. You send chunks of data and when the parser has gathered enough data to construct the parsed value it gives you the request. The pseudo-code would be something like this:

let parser = RequestParser()

while !stream.closed {
  let chunk = stream.read(bufferSize)

  try parser.parse(chunk) { request in
      // do whatever with the request
  }

  ...
}

So we read a chunk from the tcp stream, for example, and then pass that chunk to the parser. If the parser doesn’t have enough information to create a request, the call to `parse` returns and then the loop continues. We read from the stream again and send it to the parser. Let’s say we have enough information to construct a request now. When that happens the `parse` function calls the closure sending the constructed request. The closure is non escaping, which means that `parse` will only return after the closure returns. This means the parser acts basically like an “if” statement activating the closure when there’s a request or just moving on if it doesn’t.

The `parse` function could return an optional Request instead of taking a closure. But this would not be ideal if the data that is sent to the parser contains enough information to produce more than one request. With the closure design, if the parser has enough information to construct two or more request, all it has to do is send each request to the closure. If we used the optional approach we would have return the first parsed request, then store the second one and we would only be able to return that second request if the `parse` function is called again, and we have no guarantees about that. So we could effectively lose requests this way.

So instead of the optional we could still return an array of requests in case we have parsed more than one request. The pseudo-code would be something like this:

let parser = RequestParser()

while !stream.closed {
  let chunk = stream.read(bufferSize)

  for request in try parser.parse(chunk) {
      // do whatever with the request
  }

  ...
}

This would have the same effect of the closure-based API, but it wouldn’t be as optimal because it would require the parser to store all parsed requests in an array before it returns it. With the closure-based API we don’t need to store the parsed requests. We just send a request to the closure as soon as we construct one.

The streaming parser design is pretty good because it allows you to provide convenience APIs which you can use for one-shot parsing. Where you know (or expect) to have binary data that fully represents a request. Then you can provide this API on top of the streaming parser.

let data = getBinaryDataRepresentingAFullRequestFromSomewhere()
let request = try RequestParser.parse(data)

This scenario is very unlikely when it comes to HTTP parsing in real applications, but if it’s needed for say unit testing purposes, we have a nice way to do that.

This design applies to HTTP request and HTTP responses and it works the same way for both. I only used requests here because that’s what we would be doing on the HTTP server side. On the HTTP client side it would be the same mechanism, but it would parse responses instead of requests.

···

On Nov 6, 2016, at 4:09 PM, Benoit Person <benoit.person@gmail.com> wrote:

I am new here and mostly following along (so take all I say with a
"newbie" pov ;) ), but is there actual value in discussing whether or
not the implementation should be Swift-based or a wrapper around C
right now?

Let's focus on designing a Swift-friendly API, unconstrained by any
"implementation detail" (C-wrapper or pure Swift), and when we have
that, make an educated decision on whether or not it can be built by
wrapping a C library to speed up delivery and iterations on the API?

Ben

On 6 November 2016 at 18:09, Logan Wright via swift-server-dev > <swift-server-dev@swift.org> wrote:

Here are some of the most basic reasons why I would prefer to use swift implementations.

- Easier for Swift contributors to enter and participate
- More readable code
- Faster in long run
- More concise code
- Fully embraces Swift language
- Easier to maintain with control over the library

Again, I am not saying this all needs to happen immediately, but I think assuming that we'll only ever use c libraries in perpetuity is an extremely short sighted vision of how the server apis can grow. We should also keep in mind that when talking about HTTP, this isn't a from scratch rewrite as seems to be suggested several times. There are a few (likely more) parsers written in pure swift that we could mix and match to get something solid into the library. The fact that these libraries exist makes wrapping c seem like a step backwards in the progression of Swift which I am against. A perfectly reasonable compromise (as suggested before) seems to be initially wrapping some c code while leaving room in the design for Swift implementations long term if necessary.

I'd still like to get more voices, we seem to have heard from about 7 people or so, and from what I can tell, the count is about 4/3. Anyone following along care to weigh in a little bit so the same few people aren't rehashing the same talking points.

- Logan

On Sun, Nov 6, 2016 at 7:37 AM Paulo Faria via swift-server-dev <swift-server-dev@swift.org> wrote:

Hey Joannis! Nice to see you here.

I agree with you 100℅ we shouldn't rush the design of the APIs. I didn't mean that at all. I meant exactly the opposite. We should spend as much time as possible on the API design, and wrapping a C library would allows to distribute our total time of development focusing a lot more on the API design than on the development and maintenance of the internals. We should definitely design the API with the best of Swift using protocols and everything else you mentioned, and wraping C doesn't restrain us in any way to do that. There's nothing we can't do on the API level because we're wrapping C. What Chris Bailey mentioned about my mentality before is totally right. I believe we should spend as much time as possible designing the APIs without any constraint from, for example, Foundation or any C library, and in the Foundation case if really needed we should compromise. I just know from experience that wrapping C libs doesn't constraint at all the "Swiftyness" of the user-facing API. This is pure conjecture as I didn't really dig in, but I believe Swift's standard library itself uses libicu for String's internal on Linux. If this is wrong someone please correct me. I also believe Foundation wraps libcurl for the HTTP stuff.

---- On Sun, 06 Nov 2016 07:54:15 -0200 joannis@orlandos.nl wrote ----

I think that if you wrap C libraries you'll get stuck in the wrong mentality. Working with C libraries encourages thinking without protocols. Protocols like `ExpressibleByDictionaryLiteral` and `Sequence` are really easy to implement and make usage of conforming types really beautiful. I think that a lot of APIs will be designed with the wrong mentality, at least initially, because they're alien to Swift. And that'll create APIs for these libraries that are not at the level where they could be in a language like Swift.

I don't think we need to rush ourselves with trying to create the libraries rapidly. If you want to have a common set of libraries in which the entire community will cooperate I think Swift is the only way to go. Many people have already written big libraries and frameworks in Swift. And I myself would never remove my own Swift code in favour of C-reliant code. And I doubt it's just me. I think that people whom prefer pure Swift will stick with (their own) existing functionality and rather build on top of a C reliant codebase.

Besides that all I feel like there's no need to rush it. A good set of libraries needs time. If you have a million people working on something for a week it will never be as good as a hundred people for a year or two. A good API/library needs time to be thought out and will need more than a few nights of rest before I think the idea alone should be considered complete. I think this counts for C-based and pure Swift libraries.

And if we'd just take the time to develop something great it will have a much greater impact on the Server Side Swift world than it would otherwise.

Joannis

On 6 Nov 2016, at 10:44, Chris Bailey via swift-server-dev <swift-server-dev@swift.org> wrote:

Whilst it might not feel like it, it seems to be that we're all pulling in the same direction.

Tony said:

I think if the API design feels “alien” to Foundation, then we either need to update the Foundation API to feel like it fits in with Swift or design the server API to fit in with Foundation.

I’ll keep pushing for this throughout the process. It’s really important that we have a coherent stack of software from top to bottom. Ending up with different model objects for URLRequest used in the same app via two frameworks would be really unfortunate.

I think this completely aligns with what everyone is saying - we want/need a coherent API stack with seamless interop between any new APIs and the existing ones. The only thing we need to work on is the approach to achieving that.

For me, and I think that this is the approach that Paulo is pushing for, is that we should first look at "what do we think is the right approach", without any constraints. Once we understand that, and any gap there is between that and the Foundation approach, we can determine how we close that gap - whether we can do that by evolving APIs, or whether it requires compromises for the sake of developer experience and application code portability, etc.

Chris

From: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
To: Tony Parker <anthony.parker@apple.com>
Cc: Swift Server Dev <swift-server-dev@swift.org>
Date: 04/11/2016 23:08
Subject: Re: [swift-server-dev] HTTP Parser
Sent by: swift-server-dev-bounces@swift.org
________________________________

With respect to adopting other core Swift ideas like value types: we turned 20 classes into value types last year. This included a ton of work to adopt standard library protocols and improve type safety. It is an absolutely huge surface amount of Foundation’s total API surface area. It’s a large statement about how much we value making Foundation’s API consistent for Swift.

Yeah! I really love the effort you’re putting into making Foundation more “Swifty”. That doesn’t go unnoticed.

On Nov 4, 2016, at 8:49 PM, Tony Parker <anthony.parker@apple.com> wrote:

However, I do not believe there is such a fundamental conceptual mismatch here that we cannot preserve one of the most useful aspects of developing with the iOS, macOS SDKs - consistent types and low impedance mismatch between API at all levels of the stack.

I definitely value "consistent types and low impedance mismatch between API at all levels of the stack”. I just hope the APIs are designed with the best of Swift in mind. Then after we get a good design, we can see if it fits Foundation, and then make the adaptations there. I think this approach would make Foundation follow along and become even more “Swiftier” with time. Of course this is easier said than done, but I believe that’s what would make the ecosystem as a whole become better for Swift._______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

[jgiola (Jacopo Andrea Giola)]

I'm new here too, but count me in the camp to first draft a Swift friendly API that promote the use of the latest standard and security measures by default before and decide where to implement them in pure Swift or through wrapping C libraries.

···

Sent from my iPad

On 6 Nov 2016, at 19:09, Benoit Person via swift-server-dev <swift-server-dev@swift.org> wrote:

I am new here and mostly following along (so take all I say with a
"newbie" pov ;) ), but is there actual value in discussing whether or
not the implementation should be Swift-based or a wrapper around C
right now?

Let's focus on designing a Swift-friendly API, unconstrained by any
"implementation detail" (C-wrapper or pure Swift), and when we have
that, make an educated decision on whether or not it can be built by
wrapping a C library to speed up delivery and iterations on the API?

Ben

On 6 November 2016 at 18:09, Logan Wright via swift-server-dev > <swift-server-dev@swift.org> wrote:

Here are some of the most basic reasons why I would prefer to use swift implementations.

- Easier for Swift contributors to enter and participate
- More readable code
- Faster in long run
- More concise code
- Fully embraces Swift language
- Easier to maintain with control over the library

Again, I am not saying this all needs to happen immediately, but I think assuming that we'll only ever use c libraries in perpetuity is an extremely short sighted vision of how the server apis can grow. We should also keep in mind that when talking about HTTP, this isn't a from scratch rewrite as seems to be suggested several times. There are a few (likely more) parsers written in pure swift that we could mix and match to get something solid into the library. The fact that these libraries exist makes wrapping c seem like a step backwards in the progression of Swift which I am against. A perfectly reasonable compromise (as suggested before) seems to be initially wrapping some c code while leaving room in the design for Swift implementations long term if necessary.

I'd still like to get more voices, we seem to have heard from about 7 people or so, and from what I can tell, the count is about 4/3. Anyone following along care to weigh in a little bit so the same few people aren't rehashing the same talking points.

- Logan

On Sun, Nov 6, 2016 at 7:37 AM Paulo Faria via swift-server-dev <swift-server-dev@swift.org> wrote:

Hey Joannis! Nice to see you here.

I agree with you 100℅ we shouldn't rush the design of the APIs. I didn't mean that at all. I meant exactly the opposite. We should spend as much time as possible on the API design, and wrapping a C library would allows to distribute our total time of development focusing a lot more on the API design than on the development and maintenance of the internals. We should definitely design the API with the best of Swift using protocols and everything else you mentioned, and wraping C doesn't restrain us in any way to do that. There's nothing we can't do on the API level because we're wrapping C. What Chris Bailey mentioned about my mentality before is totally right. I believe we should spend as much time as possible designing the APIs without any constraint from, for example, Foundation or any C library, and in the Foundation case if really needed we should compromise. I just know from experience that wrapping C libs doesn't constraint at all the "Swiftyness" of the user-facing API. This is pure conjecture as I didn't really dig in, but I believe Swift's standard library itself uses libicu for String's internal on Linux. If this is wrong someone please correct me. I also believe Foundation wraps libcurl for the HTTP stuff.

---- On Sun, 06 Nov 2016 07:54:15 -0200 joannis@orlandos.nl wrote ----

I think that if you wrap C libraries you'll get stuck in the wrong mentality. Working with C libraries encourages thinking without protocols. Protocols like `ExpressibleByDictionaryLiteral` and `Sequence` are really easy to implement and make usage of conforming types really beautiful. I think that a lot of APIs will be designed with the wrong mentality, at least initially, because they're alien to Swift. And that'll create APIs for these libraries that are not at the level where they could be in a language like Swift.

I don't think we need to rush ourselves with trying to create the libraries rapidly. If you want to have a common set of libraries in which the entire community will cooperate I think Swift is the only way to go. Many people have already written big libraries and frameworks in Swift. And I myself would never remove my own Swift code in favour of C-reliant code. And I doubt it's just me. I think that people whom prefer pure Swift will stick with (their own) existing functionality and rather build on top of a C reliant codebase.

Besides that all I feel like there's no need to rush it. A good set of libraries needs time. If you have a million people working on something for a week it will never be as good as a hundred people for a year or two. A good API/library needs time to be thought out and will need more than a few nights of rest before I think the idea alone should be considered complete. I think this counts for C-based and pure Swift libraries.

And if we'd just take the time to develop something great it will have a much greater impact on the Server Side Swift world than it would otherwise.

Joannis

On 6 Nov 2016, at 10:44, Chris Bailey via swift-server-dev <swift-server-dev@swift.org> wrote:

Whilst it might not feel like it, it seems to be that we're all pulling in the same direction.

Tony said:

I think if the API design feels “alien” to Foundation, then we either need to update the Foundation API to feel like it fits in with Swift or design the server API to fit in with Foundation.

I’ll keep pushing for this throughout the process. It’s really important that we have a coherent stack of software from top to bottom. Ending up with different model objects for URLRequest used in the same app via two frameworks would be really unfortunate.

I think this completely aligns with what everyone is saying - we want/need a coherent API stack with seamless interop between any new APIs and the existing ones. The only thing we need to work on is the approach to achieving that.

For me, and I think that this is the approach that Paulo is pushing for, is that we should first look at "what do we think is the right approach", without any constraints. Once we understand that, and any gap there is between that and the Foundation approach, we can determine how we close that gap - whether we can do that by evolving APIs, or whether it requires compromises for the sake of developer experience and application code portability, etc.

Chris

From: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
To: Tony Parker <anthony.parker@apple.com>
Cc: Swift Server Dev <swift-server-dev@swift.org>
Date: 04/11/2016 23:08
Subject: Re: [swift-server-dev] HTTP Parser
Sent by: swift-server-dev-bounces@swift.org
________________________________

With respect to adopting other core Swift ideas like value types: we turned 20 classes into value types last year. This included a ton of work to adopt standard library protocols and improve type safety. It is an absolutely huge surface amount of Foundation’s total API surface area. It’s a large statement about how much we value making Foundation’s API consistent for Swift.

Yeah! I really love the effort you’re putting into making Foundation more “Swifty”. That doesn’t go unnoticed.

On Nov 4, 2016, at 8:49 PM, Tony Parker <anthony.parker@apple.com> wrote:

However, I do not believe there is such a fundamental conceptual mismatch here that we cannot preserve one of the most useful aspects of developing with the iOS, macOS SDKs - consistent types and low impedance mismatch between API at all levels of the stack.

I definitely value "consistent types and low impedance mismatch between API at all levels of the stack”. I just hope the APIs are designed with the best of Swift in mind. Then after we get a good design, we can see if it fits Foundation, and then make the adaptations there. I think this approach would make Foundation follow along and become even more “Swiftier” with time. Of course this is easier said than done, but I believe that’s what would make the ecosystem as a whole become better for Swift._______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

[Helge_Hess1 (Helge Heß)]

P.S.: As a sidenote, can’t someone (hm hm, who would come to mind?) sponsor the CryptoSwift guy to do a java.security like Security standard framework (which includes TLS)? That would be pretty cool :-)

Assuming we want a pure-Swift crypto framework _right now_:

Before any crypto code can safely be written in Swift, there needs to be compiler support to prevent it from optimizing things incorrectly (for example, a memset() designed to clear a buffer of secret data).

OK.

So, I don't think we really want a pure Swift crypto library yet.

OK.

Instead: while in C, Security.framework, and CommonCrypto.framework from  are both fully open source, and battle-hardened.

Do they provide a clean Linux story too? (.framework in the name kinda suggests they don’t :-)

Do they work with non-blocking I/O? The API read/write/handshake API suggests that they may not?

Looks like the approach people currently use is either Security.framework on Darwin and Open/LibreSSL on other platforms, or using OpenSSL on both. Some even forking OpenSSL.

This is one of (rare) the cases where practicality and strongly deferring to the thing that already exists seem to be the safe option (and not only in opinion).

This is all true, but then someone also thought it makes sense to have Security.framework despite OpenSSL being available. Or java.security. Or Windows SChannel/security. Just saying …

The thing which makes a pure-Swift thing attractive is that it would be cross platform and readily available. The C libs are really diverse between the platforms and forking OpenSSL into an embedded module is not exactly 1337 either.

Anyway, (to me) the more interesting questions that we can talk about are over API:
- what constructs are needed (bignum? secure random? hmac? checksumming? key derivation? wrapping a socket in a big TLS hug?)

The thing I need and which is a little awkward to get is async TLS, both client and server. I started working on an OpenSSL BIO and see whether I can get that running on top of GCD, looks like it should be possible.

For the hashes that I needed so far (just MD5 and SHA1) I use CryptoSwift.

hh

···

On 07 Nov 2016, at 13:48, Zach Drayer <zachary.drayer@gmail.com> wrote:

On Nov 7, 2016, at 11:20 AM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

[mattd]

Keep in mind that some consumers of a crypto library need compliance with
FIPS 140-2 Level 1. Writing a crypto library and obtaining the necessary
FIPS validation certificate can be an expensive and time-consuming process.
A pure-Swift library would be no different in this regard if there is a
need for compliance with FIPS 140-2 Level 1.

Since Security.framework and CommonCrypto.framework are already FIPS 140-2
Level 1 certified, it would seem that those would be reasonable targets for
wrapping on Darwin. However, other platforms (with the exception of Windows
which has it's own FIPS certified crypto library) would likely need to be
based on Open/LibreSSL. Some commercially available Linux distros provide a
certified OpenSSL FIPS object module. In addition, for those that can get
by with a private label validation, compiling your own FIPS object module
is an alternative. Keep in mind that LibreSSL completely removed it's
support for the FIPS object module.

For those curious about OpenSSL and FIPS, see the following:

https://www.openssl.org/docs/fipsnotes.html

Ultimately, I think we'll want to stick with the known crypto libraries and
frameworks that are compliant with FIPS 140-2 Level 1 until a decision can
be made around the need/desire for a pure-Swift implementation and
sponsor(s) for obtaining the validation. There are going to come cases
where some software or perhaps embedded hardware that use Swift for crypto
purposes will require FIPS 140-2 Level 1 validation.

Then again, perhaps FIPS isn't a concern for people at all. If so, please
disregard :)

Sorry if this was already addressed in the Security meeting.

Cheers,
Matt

···

On Mon, Nov 7, 2016 at 9:45 AM, Helge Heß via swift-server-dev < swift-server-dev@swift.org> wrote:

On 07 Nov 2016, at 13:48, Zach Drayer <zachary.drayer@gmail.com> wrote:
> On Nov 7, 2016, at 11:20 AM, Helge Heß via swift-server-dev < > swift-server-dev@swift.org> wrote:
>> P.S.: As a sidenote, can’t someone (hm hm, who would come to mind?)
sponsor the CryptoSwift guy to do a java.security like Security standard
framework (which includes TLS)? That would be pretty cool :-)
>
> Assuming we want a pure-Swift crypto framework _right now_:
>
> Before any crypto code can safely be written in Swift, there needs to be
compiler support to prevent it from optimizing things incorrectly (for
example, a memset() designed to clear a buffer of secret data).

OK.

> So, I don't think we really want a pure Swift crypto library yet.

OK.

> Instead: while in C, Security.framework, and CommonCrypto.framework from
 are both fully open source, and battle-hardened.

Do they provide a clean Linux story too? (.framework in the name kinda
suggests they don’t :-)

Do they work with non-blocking I/O? The API read/write/handshake API
suggests that they may not?

Looks like the approach people currently use is either Security.framework
on Darwin and Open/LibreSSL on other platforms, or using OpenSSL on both.
Some even forking OpenSSL.

> This is one of (rare) the cases where practicality and strongly
deferring to the thing that already exists seem to be the safe option (and
not only in opinion).

This is all true, but then someone also thought it makes sense to have
Security.framework despite OpenSSL being available. Or java.security. Or
Windows SChannel/security. Just saying …

The thing which makes a pure-Swift thing attractive is that it would be
cross platform and readily available. The C libs are really diverse between
the platforms and forking OpenSSL into an embedded module is not exactly
1337 either.

> Anyway, (to me) the more interesting questions that we can talk about
are over API:
> - what constructs are needed (bignum? secure random? hmac? checksumming?
key derivation? wrapping a socket in a big TLS hug?)

The thing I need and which is a little awkward to get is async TLS, both
client and server. I started working on an OpenSSL BIO and see whether I
can get that running on top of GCD, looks like it should be possible.

For the hashes that I needed so far (just MD5 and SHA1) I use CryptoSwift.

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev