Prototype of the discussed HTTP API Spec

Server Development
1

There's been a lot of discussion around the HTTP API spec originally
created by Johannes Weiss with input of a number of other people
(particularly Helga Hess):
        
https://lists.swift.org/pipermail/swift-server-dev/Week-of-Mon-20170403/000422.html

Since then some work has been done by Carl Brown and others on a prototype
of a clean (not derived from existing code) implementation of that API.
You can find that prototype here:
https://github.com/carlbrown/HTTPSketch

The API itself is essentially implemented in three files:
HTTPRequest:
https://github.com/carlbrown/HTTPSketch/blob/master/Sources/HTTPSketch/HTTPRequest.swift
HTTPResponse:
https://github.com/carlbrown/HTTPSketch/blob/master/Sources/HTTPSketch/HTTPResponse.swift

HTTPCommon:
https://github.com/carlbrown/HTTPSketch/blob/master/Sources/HTTPSketch/HTTPCommon.swift

The prototype has some dependencies on the CHTTParser and BlueSocket
libraries from IBM-Swift in order to have working implementation, but
we've attempted to abstract those away so that they're easily replaced
with another implementation - eventually the aim is to move to standard
implementations from the Server API project and to utilise the secure
transport implementation being worked on under the Security stream.

We're proposing to use this as the starting point and move it into the
swift-server org on GitHub. From there we can then collaborate and iterate
on improving/evolving the API surface and the implementation through
discussion on the mailing list and PRs.

The hope is that having an independent concrete implementation should make
it much easier for us to iterate on and experiment with changes to the API
and implementation as can all see how it affects a real implementation,
including being able to assess performance and memory footprint costs of
alternative approaches, etc.

Please take a look. If people are happy to use this as an initial
implementation of HTTP API spec that's been layed out so far, we'll move
it over to the swift-server org on GitHub. This is by no means the final
API or implementation - there's definitely lots that still needs to be
discussed. Once its on the swift-server org we'll start looking at the API
and implementation choices in-depth via the mailing list and PRs.

Chris
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2

I'm glad we can start talking in source code now. Chris sent me the link
before, so I had time to create a counterproposal concerning the 3 files
mentioned. In the readme I have the rationale for the design decisions. I
also used jazzy to create an API reference. Here are the links:

https://github.com/paulofaria/http-api-proposal
https://paulofaria.github.io/http-api-proposal/

···

On 26 May 2017 at 13:00, Chris Bailey via swift-server-dev < swift-server-dev@swift.org> wrote:

There's been a lot of discussion around the HTTP API spec originally
created by Johannes Weiss with input of a number of other people
(particularly Helga Hess):
        The swift-server-dev Archives
Week-of-Mon-20170403/000422.html

Since then some work has been done by Carl Brown and others on a prototype
of a clean (not derived from existing code) implementation of that API. You
can find that prototype here: carlbrown (Carl Brown) · GitHub
HTTPSketch

The API itself is essentially implemented in three files:
HTTPRequest: https://github.com/carlbrown/HTTPSketch/blob/master/
Sources/HTTPSketch/HTTPRequest.swift
HTTPResponse: https://github.com/carlbrown/HTTPSketch/blob/master/
Sources/HTTPSketch/HTTPResponse.swift

HTTPCommon: https://github.com/carlbrown/HTTPSketch/blob/master/
Sources/HTTPSketch/HTTPCommon.swift

The prototype has some dependencies on the CHTTParser and BlueSocket
libraries from IBM-Swift in order to have working implementation, but we've
attempted to abstract those away so that they're easily replaced with
another implementation - eventually the aim is to move to standard
implementations from the Server API project and to utilise the secure
transport implementation being worked on under the Security stream.

We're proposing to use this as the starting point and move it into the
swift-server org on GitHub. From there we can then collaborate and iterate
on improving/evolving the API surface and the implementation through
discussion on the mailing list and PRs.

The hope is that having an independent concrete implementation should make
it much easier for us to iterate on and experiment with changes to the API
and implementation as can all see how it affects a real implementation,
including being able to assess performance and memory footprint costs of
alternative approaches, etc.

Please take a look. If people are happy to use this as an initial
implementation of HTTP API spec that's been layed out so far, we'll move it
over to the swift-server org on GitHub. This is by no means the final API
or implementation - there's definitely lots that still needs to be
discussed. Once its on the swift-server org we'll start looking at the API
and implementation choices in-depth via the mailing list and PRs.

Chris
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

3

From what I've seen. My proposal is very similar to the one Chris posted.

The main difference is concerning the HTTP body. My proposal is far simpler
and easier to integrate with existing frameworks.

Here's a link to the synchronous version:

https://github.com/paulofaria/http-api-proposal/blob/master/Sources/HTTP.swift#L993-L1001

And here's a link to the asynchronous version:

https://github.com/paulofaria/http-api-proposal/blob/master/Sources/HTTP.swift#L1126-L1134

The only issue with this approach is the trailing headers. But I'm not sure
support for it is implemented in the code Chris linked. I'll add support
for it this weekend.

4

Hi, Paulo,

  Like Helge said, Johannes was already talking in source code (see The swift-server-dev The Week Of Monday 3 April 2017 Archive by thread and The swift-server-dev The Week Of Monday 10 April 2017 Archive by thread for the week of threads), and I tried to follow along with that API which had already talked about on the list.

  I was hoping at this point (or some point soon) that we could start talking about an implementation instead of just arguing about the API some more in the abstract.

  I certainly expect we will make changes to the API as we go, I just hope that those changes will have an implementation behind them so that we'll be able to discuss those changes with an understanding of what the performance implications are and examples of how the new API is intended to be used.

  But like I said to Helge, the question of the moment (for you and for the group) is - do you think this is a good enough starting point for us to move it to the Swift on Server · GitHub organization and start iterating via GitHub issues & Pull Requests, or do you think we're still at the stage where we need to have more email discussions about it before we're ready to take that step?

-Carl

···

--
Carl Brown, Swift@IBM
Carl.Brown1@IBM.com <mailto:Carl.Brown1@IBM.com> (Work)
Austin, TX

On May 26, 2017, at 3:32 PM, Paulo Faria via swift-server-dev <swift-server-dev@swift.org> wrote:

I'm glad we can start talking in source code now. Chris sent me the link before, so I had time to create a counterproposal concerning the 3 files mentioned. In the readme I have the rationale for the design decisions. I also used jazzy to create an API reference. Here are the links:

https://github.com/paulofaria/http-api-proposal
https://paulofaria.github.io/http-api-proposal/

On 26 May 2017 at 13:00, Chris Bailey via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:
There's been a lot of discussion around the HTTP API spec originally created by Johannes Weiss with input of a number of other people (particularly Helga Hess):
        [swift-server-dev] HTTP API Sketch v2

Since then some work has been done by Carl Brown and others on a prototype of a clean (not derived from existing code) implementation of that API. You can find that prototype here: https://github.com/carlbrown/HTTPSketch

The API itself is essentially implemented in three files:
HTTPRequest: https://github.com/carlbrown/HTTPSketch/blob/master/Sources/HTTPSketch/HTTPRequest.swift
HTTPResponse: https://github.com/carlbrown/HTTPSketch/blob/master/Sources/HTTPSketch/HTTPResponse.swift

HTTPCommon: https://github.com/carlbrown/HTTPSketch/blob/master/Sources/HTTPSketch/HTTPCommon.swift

The prototype has some dependencies on the CHTTParser and BlueSocket libraries from IBM-Swift in order to have working implementation, but we've attempted to abstract those away so that they're easily replaced with another implementation - eventually the aim is to move to standard implementations from the Server API project and to utilise the secure transport implementation being worked on under the Security stream.

We're proposing to use this as the starting point and move it into the swift-server org on GitHub. From there we can then collaborate and iterate on improving/evolving the API surface and the implementation through discussion on the mailing list and PRs.

The hope is that having an independent concrete implementation should make it much easier for us to iterate on and experiment with changes to the API and implementation as can all see how it affects a real implementation, including being able to assess performance and memory footprint costs of alternative approaches, etc.

Please take a look. If people are happy to use this as an initial implementation of HTTP API spec that's been layed out so far, we'll move it over to the swift-server org on GitHub. This is by no means the final API or implementation - there's definitely lots that still needs to be discussed. Once its on the swift-server org we'll start looking at the API and implementation choices in-depth via the mailing list and PRs.

Chris
Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

5

I'm glad we can start talking in source code now.

Johannes’ stuff was source code too.

Chris sent me the link before, so I had time to create a counterproposal concerning the 3 files mentioned. In the readme I have the rationale for the design decisions. I also used jazzy to create an API reference. Here are the links:

No HTTP prefix

There's no need to prefix the types with HTTP because
they are already defined inside the HTTP module

Swift modules are not really namespaces but just modules.
If you’d want a namespace, it should be sth like

  enum HTTP {
    class Request {}
  }

that would be kinda OK. In general I prefer just calling
things what they are, HTTPRequest in this case.

The think you run into when using generic names is that:

  import HTTP
  import SOAP

let request = Request() // …

‘Request' is just too generic when looking at the source. It
makes some sense _within_ the implementing module, but
less in clients consuming the API.

Nesting Method/Status

I guess that makes sense. I still find it weird to use
closed API (enums) for an open protocol. I’d prefer a
solution using constants.

Lowercase Request.Method cases

That likely makes sense for consistency.

Most common APIs are synchronous. Asynchronous APIs are
the exception in general

This is highly controversial :-) Both variants exist and
have their pros and cons, but that "Asynchronous APIs are
the exception” sounds super weird. E.g. Node.js is
all asynchronous and the synchronous APIs are explicitly
marked via ‘sync’ (like readFileSync instead of the proper
readFile).

As I demonstrated before the API doesn't necessarily have to
be different. (my demo using Johannes' completely async API
on top of Apache, doing it completely synchronously).

In Swift async-vs-sync is clearly expressed by `@escaping`
in the signature. No specific naming needed in the first
place?

Use UnsafeMutableRawBufferPointer and UnsafeRawBufferPointer
instead of Foundation.Data

In general, YES. But the proposal uses *DispatchData* and not
Data.
(Though I find it a little weird that Data is so specific now in the
first place.)

As mentioned before, if you are performance aware, vectors
are kinda like a must. (which is what DispatchData can do
internally, though I think the API is AFAIK not exposed to
Swift)

hh

···

On 26. May 2017, at 22:32, Paulo Faria via swift-server-dev <swift-server-dev@swift.org> wrote:

6

HTTPRequest: https://github.com/carlbrown/HTTPSketch/blob/master/Sources/HTTPSketch/HTTPRequest.swift

  /// HTTP Methods handled by http_parser.[ch] supports
  public enum HTTPMethod: String {
    // case custom(method: String)
    case UNKNOWN

Don’t restrict the API to a specific HTTP parser implementation. The number of methods supported by http_parser changed multiple times in the past (I myself added one to the official tree).

I still highly recommend using constants instead of an enum here. The methods *will* change in the future. (At least BATCH is going to be added)
At the minimum do the custom(String) variant. UNKNOWN seems unacceptable to me.

In the same run, I don’t understand why an enum is used for methods but not for headers. Extra weird because the method is just a header in HTTP/2 …
I’d prefer constants for both.

func writeBody(data: DispatchData, completion: @escaping (Result<POSIXError, ()>) -> Void)

Do we really want to expose POSIXError’s? I’d say have an own error type for HTTP level errors and maybe something like this:

  enum HTTPError {
    case connectionClosed(cause: POSIXError?)
    ...
  }

Also: DispatchData vs Data vs Ptr.

public enum HTTPResponseStatus: UInt, RawRepresentable {
   /* The original spec used custom if you want to use a non-standard response code or
    have it available in a (UInt, String) pair from a higher-level web framework.
    
    Swift 3 doesn't like it - will revisit in Swift 4*/
   //case custom(code: UInt, reasonPhrase: String)

Swift 3 does like it, you just can’t combine raw w/ extra values. That just doesn’t make sense. I think the only thing you could hope for is that this works on a raw:

  case custom(code: UInt)

Also, why a `UInt`? Do you foresee 64-bit status codes? Should be UInt16.

Again: I’d prefer constants because status codes are defined as part of RFCs and
will *definitely* change in the future. Something like

  struct HTTPStatus {
    let code : UInt16

    static let `continue` : UInt16 = 100
    etc.
  }

That is future and source-compat proof.

public struct HTTPHeaders {
    var storage: [String:[String]] /* lower cased keys */
    var original: [(String, String)] /* original casing */

I don’t think the storage semantics should be part of the API. I’d prefer
a protocol here providing indexed access.

Finally, I still propose calling it HTTPRequestHead, HTTPResponseHead. The body belongs to the request/response. Don’t care too much, but would be better.

hh

···

On 26. May 2017, at 22:00, Chris Bailey via swift-server-dev <swift-server-dev@swift.org> wrote:

7

Hi, Helge,

For purposes of building this prototype, I tried to stay with as many of Johannes' original type signatures as I could. Therefore, I used an enum for HTTPMethod because that's what Johannes had in his email from [swift-server-dev] HTTP API Sketch v2

I'm happy to get into that and other specifics as we go (and yes I added the `RawRepresentable` to the enum because I'm more used to thinking in "200" and "404" than .ok and .notFound, and you are right, a `UInt16` would have been better for status (and maybe I shouldn't have done it at all) - my bad).

But the question of the moment (for you and for the group) is - do you think this is a good enough starting point for us to move it to the Swift on Server · GitHub organization and start iterating via GitHub issues & Pull Requests, or do you think we're still at the stage where we need to have more email discussions about it before we're ready to take that step?

-Carl

···

--
Carl Brown, Swift@IBM
Carl.Brown1@IBM.com <mailto:Carl.Brown1@IBM.com> (Work)
Austin, TX

On May 26, 2017, at 4:22 PM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

On 26. May 2017, at 22:00, Chris Bailey via swift-server-dev <swift-server-dev@swift.org> wrote:

HTTPRequest: https://github.com/carlbrown/HTTPSketch/blob/master/Sources/HTTPSketch/HTTPRequest.swift

/// HTTP Methods handled by http_parser.[ch] supports
public enum HTTPMethod: String {
   // case custom(method: String)
   case UNKNOWN

Don’t restrict the API to a specific HTTP parser implementation. The number of methods supported by http_parser changed multiple times in the past (I myself added one to the official tree).

I still highly recommend using constants instead of an enum here. The methods *will* change in the future. (At least BATCH is going to be added)
At the minimum do the custom(String) variant. UNKNOWN seems unacceptable to me.

In the same run, I don’t understand why an enum is used for methods but not for headers. Extra weird because the method is just a header in HTTP/2 …
I’d prefer constants for both.

func writeBody(data: DispatchData, completion: @escaping (Result<POSIXError, ()>) -> Void)

Do we really want to expose POSIXError’s? I’d say have an own error type for HTTP level errors and maybe something like this:

enum HTTPError {
   case connectionClosed(cause: POSIXError?)
   ...
}

Also: DispatchData vs Data vs Ptr.

public enum HTTPResponseStatus: UInt, RawRepresentable {
  /* The original spec used custom if you want to use a non-standard response code or
   have it available in a (UInt, String) pair from a higher-level web framework.

   Swift 3 doesn't like it - will revisit in Swift 4*/
  //case custom(code: UInt, reasonPhrase: String)

Swift 3 does like it, you just can’t combine raw w/ extra values. That just doesn’t make sense. I think the only thing you could hope for is that this works on a raw:

case custom(code: UInt)

Also, why a `UInt`? Do you foresee 64-bit status codes? Should be UInt16.

Again: I’d prefer constants because status codes are defined as part of RFCs and
will *definitely* change in the future. Something like

struct HTTPStatus {
   let code : UInt16

   static let `continue` : UInt16 = 100
   etc.
}

That is future and source-compat proof.

public struct HTTPHeaders {
   var storage: [String:[String]] /* lower cased keys */
   var original: [(String, String)] /* original casing */

I don’t think the storage semantics should be part of the API. I’d prefer
a protocol here providing indexed access.

Finally, I still propose calling it HTTPRequestHead, HTTPResponseHead. The body belongs to the request/response. Don’t care too much, but would be better.

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

8

Helge! I agree with most of your suggestions and I'll update my proposal
with them. There's only one I'm not with you and it's about the HTTP
prefix. The example you mentioned could be easily solved by adding `HTTP.`
when referring to the specific type.

import HTTP
import SOAP

let request = HTTP.Request()

We have used Request and Response without prefixes for about two years and
not once anyone complained about that. I imagine others who use the same
naming scheme can testify to that.

About Sync/Async prefix. I was talking about type prefixes not method
prefixes. We definitely shouldn't prefix function APIs with sync or async.
I honestly don't care too much about which one to use as a default. I just
think it makes more sense to prefix the Async ones.

About raw buffers we could provide APIs that take an array of
Unsafe[Mutable]RawBufferPointer
to pass them to readv, writev and company. Those could also be easily
converted to DispatchData in the frameworks that prefer to use it.

···

On 26 May 2017 at 15:27, Carl Brown via swift-server-dev < swift-server-dev@swift.org> wrote:

Hi, Helge,

For purposes of building this prototype, I tried to stay with as many of
Johannes' original type signatures as I could. Therefore, I used an enum
for HTTPMethod because that's what Johannes had in his email from
The swift-server-dev Archives
Week-of-Mon-20170403/000422.html

I'm happy to get into that and other specifics as we go (and yes I added
the `RawRepresentable` to the enum because I'm more used to thinking in
"200" and "404" than .ok and .notFound, and you are right, a `UInt16` would
have been better for status (and maybe I shouldn't have done it at all) -
my bad).

But the question of the moment (for you and for the group) is - do you
think this is a good enough starting point for us to move it to the
Swift on Server · GitHub organization and start iterating via
GitHub issues & Pull Requests, or do you think we're still at the stage
where we need to have more email discussions about it before we're ready to
take that step?

-Carl

--
Carl Brown, Swift@IBM
Carl.Brown1@IBM.com (Work)
Austin, TX

On May 26, 2017, at 4:22 PM, Helge Heß via swift-server-dev < > swift-server-dev@swift.org> wrote:

On 26. May 2017, at 22:00, Chris Bailey via swift-server-dev < > swift-server-dev@swift.org> wrote:

HTTPRequest: https://github.com/carlbrown/HTTPSketch/blob/
master/Sources/HTTPSketch/HTTPRequest.swift

/// HTTP Methods handled by http_parser.[ch] supports
public enum HTTPMethod: String {
   // case custom(method: String)
   case UNKNOWN

Don’t restrict the API to a specific HTTP parser implementation. The
number of methods supported by http_parser changed multiple times in the
past (I myself added one to the official tree).

I still highly recommend using constants instead of an enum here. The
methods *will* change in the future. (At least BATCH is going to be added)
At the minimum do the custom(String) variant. UNKNOWN seems unacceptable
to me.

In the same run, I don’t understand why an enum is used for methods but
not for headers. Extra weird because the method is just a header in HTTP/2 …
I’d prefer constants for both.

func writeBody(data: DispatchData, completion: @escaping
(Result<POSIXError, ()>) -> Void)

Do we really want to expose POSIXError’s? I’d say have an own error type
for HTTP level errors and maybe something like this:

enum HTTPError {
   case connectionClosed(cause: POSIXError?)
   ...
}

Also: DispatchData vs Data vs Ptr.

public enum HTTPResponseStatus: UInt, RawRepresentable {
  /* The original spec used custom if you want to use a non-standard
response code or
   have it available in a (UInt, String) pair from a higher-level web
framework.

   Swift 3 doesn't like it - will revisit in Swift 4*/
  //case custom(code: UInt, reasonPhrase: String)

Swift 3 does like it, you just can’t combine raw w/ extra values. That
just doesn’t make sense. I think the only thing you could hope for is that
this works on a raw:

case custom(code: UInt)

Also, why a `UInt`? Do you foresee 64-bit status codes? Should be UInt16.

Again: I’d prefer constants because status codes are defined as part of
RFCs and
will *definitely* change in the future. Something like

struct HTTPStatus {
   let code : UInt16

   static let `continue` : UInt16 = 100
   etc.
}

That is future and source-compat proof.

public struct HTTPHeaders {
   var storage: [String:[String]] /* lower cased keys */
   var original: [(String, String)] /* original casing */

I don’t think the storage semantics should be part of the API. I’d prefer
a protocol here providing indexed access.

Finally, I still propose calling it HTTPRequestHead, HTTPResponseHead. The
body belongs to the request/response. Don’t care too much, but would be
better.

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

9

Hi Carl,

But the question of the moment (for you and for the group) is - do you think this is a good enough starting point for us to move it to the Swift on Server · GitHub organization and start iterating via GitHub issues & Pull Requests, or do you think we're still at the stage where we need to have more email discussions about it before we're ready to take that step?

that seems perfectly reasonable to me.

What I personally would like best is submitting Johannes’ proposal first, and on top of that submit your changes (w/ explanations what you changed and why in the commits).

I have Johannes’ original proposal in code over here, in case you need it as a basis:

  https://github.com/ApacheExpress/ApacheExpress/tree/s3wg-api-proposal-1/S3WGAPIProposal1

hh

10

Helge! I agree with most of your suggestions and I'll update my proposal with them. There's only one I'm not with you and it's about the HTTP prefix. The example you mentioned could be easily solved by adding `HTTP.` when referring to the specific type.

import HTTP
import SOAP

let request = HTTP.Request()

We have used Request and Response without prefixes for about two years and not once anyone complained about that. I imagine others who use the same naming scheme can testify to that.

Having seen such overloading in Java frameworks before I simply disagree with that. The object in question is plainly a HTTPRequest(Head) not just some arbitrary request. The ‘HTTP’ really is part of the thing. In fact the ‘arbitrary’ request could be an NSOperation, providing functionalities such as pause, cancel, etc.

In a way it is like

  import Cat

  let pet = Animal() // resolving to Cat.Animal()

But well, if people really like that … ;-) I vote against it.

About Sync/Async prefix. I was talking about type prefixes not method prefixes. We definitely shouldn't prefix function APIs with sync or async. I honestly don't care too much about which one to use as a default. I just think it makes more sense to prefix the Async ones.

Your particular setup is sync, right? ;->

About raw buffers we could provide APIs that take an array of Unsafe[Mutable]RawBufferPointer to pass them to readv, writev and company. Those could also be easily converted to DispatchData in the frameworks that prefer to use it.

This is a little related to sync vs async. For async you usually need some way to hold on to a buffer, and that is DispatchData. For sync you usually don’t have to, and pointers to buffers can be faster.

Though I do think that sync is more appropriate for many server builders, I think that the goal for Swift-Server would be more oriented towards async. I may be wrong.

hh

···

On May 27, 2017, at 12:55 AM, Paulo Faria <paulo@zewo.io> wrote:

11

Also, POSIXError is not an actual thing a user can create, right? It is just a namespace. Should be POSIXErrorCode?

hh

···

On 26 May 2017, at 23:22, Helge Heß <me@helgehess.eu> wrote:

func writeBody(data: DispatchData, completion: @escaping (Result<POSIXError, ()>) -> Void)

Do we really want to expose POSIXError’s? I’d say have an own error type for HTTP level errors and maybe something like this:

enum HTTPError {
   case connectionClosed(cause: POSIXError?)
   ...
}

12

When importing HTTP you know you'll deal with HTTP requests and not anything else. And if you do import another module which also has Request (never happened to me) you have to make it clear which one you're referring to. So you'll have to do HTTP.Request, SOAP.Request. So in both scenarios, there won't be any confusion. On the other hand prefixing everything ends up adding noise to the API.

I don’t think adding a prefix will add too much noise to the API, and if it does it cost less than resolving namespace collision.
It is true that user can use HTTP.Request and SOAP.Request to make it clean which one ppl is referring to, but it also make code hard to debug once anyone has accidentally forgot to add the “HTTP.” preflx, and this is kind of error that the compiler cannot catch sometimes, especially when they have members and methods used in the function body with the same name and type (which for a req/res this should be pretty common).

Michael

13

Swift is still minority in server side and early user are mostly from iOS and no real complex project AFAIK had built on any of the swift web framework available yet. And even people encounter this problem it’s not likely anyone will report it, since it’s not a bug but what the framework authors choose to do.

On the other hand, a mature example, Netty, written in Java which has a similar namespace system as swift (using . ), includes HTTP as the prefix,
https://netty.io/4.0/api/io/netty/handler/codec/http/HttpRequest.html

Request and Response sounds like name of a protocol for me instead of a real type.

Michael

···

On May 26, 2017, at 5:02 PM, Paulo Faria <paulo@zewo.io> wrote:

> I don’t think adding a prefix will add too much noise to the API, and if it does it cost less than resolving namespace collision.
> It is true that user can use HTTP.Request and SOAP.Request to make it clean which one ppl is referring to, but it also make code hard to debug once anyone has accidentally forgot to add the “HTTP.” preflx, and this is kind of error that the compiler cannot catch sometimes, especially when they have members and methods used in the function body with the same name and type (which for a req/res this should be pretty common).

I honestly never seen it happen. Vapor is one of the most popular frameworks and they also use Request and Response. I'd love to see real-life reports of issues regarding this matter.

14

In general I agree this is how we should process, components by components, arguing around implementation detail won’t help much here.

Tho we do need to agree an architecture to start with before we know what are the components we need.

I do agree that version, req method and res status is something quite orthogonal to the actual architecture hence a good place to discuss ahead.

Michael

···

On May 26, 2017, at 4:57 PM, Paulo Faria via swift-server-dev <swift-server-dev@swift.org> wrote:

> What do you mean by "the framework" here? I can think of several things that could be referred to by that phrase, and I'm not sure which one you mean.

Sorry, it was really confusing. I meant higher level web frameworks like Kitura, for example. It looks like the code you have go all the way up there.. The WebApp type says it all. I don't think we should have this at all. At least not for now.. I believe we should only go up to the HTTP server/client. I think the types we should implement are in order (naming aside, adding prefixes or not, Sync, Async etc)

Week 1

Version
Headers
Message
Request
Response

Week 2

SyncBody
AsyncBody

Week 3

SyncRequestParser
AsyncRequestParser
SyncResponseParser
AsyncResponseParser

Week 4

SyncRequestSerializer
AsyncRequestSerializer
SyncResponseSerializer
AsyncResponseSerializer

Week 5

SyncServer
AsyncServer

Week 6

SyncClient
AsyncClient

We would have one week to discuss each the API of the types in each set. After we decide on the API we implement in the following week while discussing the next set of APIs. Once we settle on the API for each type implementing shouldn't take more than 2 days based on our experience with the subject.

Using this scheme we would have the API ready in mid-july. What do you guys think? I really think this is a good way to follow on the discussion/implementation.

On 26 May 2017 at 20:40, Carl Brown <carl.brown.swift@linuxswift.com <mailto:carl.brown.swift@linuxswift.com>> wrote:

On May 26, 2017, at 6:07 PM, Paulo Faria via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:

Hi, Carl!

I honestly think the code is too overwhelming.

I can see that. It was important to us to get something that actually worked end-to-end, could be profiled and had test coverage. We couldn't get it much smaller.

I'm also a bit confused about the IBM dependencies. Is this supposed to stay in the swift server repos or is it just momentary so we have working code?

It's a stand-in until the Transport part of the working group has a reference implementation of a socket/stream API and the Security part of the working group has a reference implementation of SSL/TLS. The HTTP part of the group is just ahead of everyone else.

I'm not very fond of the first option if it is the case, and about the second option... If it's just to see working code.. Then we should have in swift server org only the HTTP module and then have the actual implementation of the framework reside elsewhere importing HTTP module as a dependency. This way we have an view of how people will import and use the framework. It also gives the other frameworks a chance to implement their solutions on top of the common HTTP module.

What do you mean by "the framework" here? I can think of several things that could be referred to by that phrase, and I'm not sure which one you mean.

I think we should start moving small bits of code, for example, Version, Headers, Message, Request and Response. After we have that code in the repo working with all the frameworks which are validating the API, then we move on to the body (which is probably going to be the hardest part to settle for everyone). After that we go to the parser/serializer.

Again, I'm not sure what you mean by "the frameworks" here, but putting that aside, there's the issue of timing. I can see value in asking the group for comments on a small piece of code and not moving on to the next small piece until after everyone has had a chance to comment, but given the pace of progress that's been made by this group so far, don't you think that would take a very, very long time?

-Carl

On 26 May 2017 at 19:51, Helge Heß via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:
Hi Carl,

> But the question of the moment (for you and for the group) is - do you think this is a good enough starting point for us to move it to the Swift on Server · GitHub organization and start iterating via GitHub issues & Pull Requests, or do you think we're still at the stage where we need to have more email discussions about it before we're ready to take that step?

that seems perfectly reasonable to me.

What I personally would like best is submitting Johannes’ proposal first, and on top of that submit your changes (w/ explanations what you changed and why in the commits).

I have Johannes’ original proposal in code over here, in case you need it as a basis:

  https://github.com/ApacheExpress/ApacheExpress/tree/s3wg-api-proposal-1/S3WGAPIProposal1

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

15

Hi Carl,

But the question of the moment (for you and for the group) is - do you think this is a good enough starting point for us to move it to the Swift on Server · GitHub organization and start iterating via GitHub issues & Pull Requests, or do you think we're still at the stage where we need to have more email discussions about it before we're ready to take that step?

that seems perfectly reasonable to me.

What I personally would like best is submitting Johannes’ proposal first, and on top of that submit your changes (w/ explanations what you changed and why in the commits).

I have Johannes’ original proposal in code over here, in case you need it as a basis:

https://github.com/ApacheExpress/ApacheExpress/tree/s3wg-api-proposal-1/S3WGAPIProposal1

This looks like an earlier version of Johannes' code (probably from https://lists.swift.org/pipermail/swift-server-dev/Week-of-Mon-20170320/000329.html\). I was working off the April 5th version. (The one with "HTTP API Sketch v2" in the subject from https://lists.swift.org/pipermail/swift-server-dev/Week-of-Mon-20170403/000422.html\).

Do we need to start with the March version?

-Carl

···

On May 26, 2017, at 5:51 PM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

16

Hi, Carl!

I honestly think the code is too overwhelming.

I can see that. It was important to us to get something that actually worked end-to-end, could be profiled and had test coverage. We couldn't get it much smaller.

I'm also a bit confused about the IBM dependencies. Is this supposed to stay in the swift server repos or is it just momentary so we have working code?

It's a stand-in until the Transport part of the working group has a reference implementation of a socket/stream API and the Security part of the working group has a reference implementation of SSL/TLS. The HTTP part of the group is just ahead of everyone else.

I'm not very fond of the first option if it is the case, and about the second option... If it's just to see working code.. Then we should have in swift server org only the HTTP module and then have the actual implementation of the framework reside elsewhere importing HTTP module as a dependency. This way we have an view of how people will import and use the framework. It also gives the other frameworks a chance to implement their solutions on top of the common HTTP module.

What do you mean by "the framework" here? I can think of several things that could be referred to by that phrase, and I'm not sure which one you mean.

I think we should start moving small bits of code, for example, Version, Headers, Message, Request and Response. After we have that code in the repo working with all the frameworks which are validating the API, then we move on to the body (which is probably going to be the hardest part to settle for everyone). After that we go to the parser/serializer.

Again, I'm not sure what you mean by "the frameworks" here, but putting that aside, there's the issue of timing. I can see value in asking the group for comments on a small piece of code and not moving on to the next small piece until after everyone has had a chance to comment, but given the pace of progress that's been made by this group so far, don't you think that would take a very, very long time?

-Carl

···

On May 26, 2017, at 6:07 PM, Paulo Faria via swift-server-dev <swift-server-dev@swift.org> wrote:

On 26 May 2017 at 19:51, Helge Heß via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:
Hi Carl,

> But the question of the moment (for you and for the group) is - do you think this is a good enough starting point for us to move it to the Swift on Server · GitHub organization and start iterating via GitHub issues & Pull Requests, or do you think we're still at the stage where we need to have more email discussions about it before we're ready to take that step?

that seems perfectly reasonable to me.

What I personally would like best is submitting Johannes’ proposal first, and on top of that submit your changes (w/ explanations what you changed and why in the commits).

I have Johannes’ original proposal in code over here, in case you need it as a basis:

  https://github.com/ApacheExpress/ApacheExpress/tree/s3wg-api-proposal-1/S3WGAPIProposal1

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

17

Hi, Carl!

I honestly think the code is too overwhelming. I'm also a bit confused
about the IBM dependencies. Is this supposed to stay in the swift server
repos or is it just momentary so we have working code? I'm not very fond of
the first option if it is the case, and about the second option... If it's
just to see working code.. Then we should have in swift server org only the
HTTP module and then have the actual implementation of the framework reside
elsewhere importing HTTP module as a dependency. This way we have an view
of how people will import and use the framework. It also gives the other
frameworks a chance to implement their solutions on top of the common HTTP
module.

I think we should start moving small bits of code, for example, Version,
Headers, Message, Request and Response. After we have that code in the repo
working with all the frameworks which are validating the API, then we move
on to the body (which is probably going to be the hardest part to settle
for everyone). After that we go to the parser/serializer.

···

On 26 May 2017 at 19:51, Helge Heß via swift-server-dev < swift-server-dev@swift.org> wrote:

Hi Carl,

> But the question of the moment (for you and for the group) is - do you
think this is a good enough starting point for us to move it to the
Swift on Server · GitHub organization and start iterating via
GitHub issues & Pull Requests, or do you think we're still at the stage
where we need to have more email discussions about it before we're ready to
take that step?

that seems perfectly reasonable to me.

What I personally would like best is submitting Johannes’ proposal first,
and on top of that submit your changes (w/ explanations what you changed
and why in the commits).

I have Johannes’ original proposal in code over here, in case you need it
as a basis:

  GitHub - ApacheExpress/ApacheExpress at s3wg-api-proposal-1
S3WGAPIProposal1

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

18

I'm not sure I follow your example. Cat.Animal doesn't make much sense to
me while Animal.Cat makes perfect sense.

import Animal

let pet = Cat()

When importing HTTP you know you'll deal with HTTP requests and not
anything else. And if you do import another module which also has Request
(never happened to me) you have to make it clear which one you're referring
to. So you'll have to do HTTP.Request, SOAP.Request. So in both scenarios,
there won't be any confusion. On the other hand prefixing everything ends
up adding noise to the API.

Your particular setup is sync, right? ;->

It is, but I really don't care about prefixing with Sync, if needed. I
honestly find it weird to mark an API Sync since really, most of the APIs
are synchronous (in general of course). Is there any async API in Swift's
standard library? But again, non-issue for me. I'd be totally ok with Sync.

For async you usually need some way to hold on to a buffer, and that is

DispatchData

Sorry can you elucidate that? If you have the pointer, you're holding the
buffer aren't you? I'm not sure I follow the issue.

···

On 26 May 2017 at 20:09, Helge Heß via swift-server-dev < swift-server-dev@swift.org> wrote:

On May 27, 2017, at 12:55 AM, Paulo Faria <paulo@zewo.io> wrote:
> Helge! I agree with most of your suggestions and I'll update my proposal
with them. There's only one I'm not with you and it's about the HTTP
prefix. The example you mentioned could be easily solved by adding `HTTP.`
when referring to the specific type.
>
> import HTTP
> import SOAP
>
> let request = HTTP.Request()
>
> We have used Request and Response without prefixes for about two years
and not once anyone complained about that. I imagine others who use the
same naming scheme can testify to that.

Having seen such overloading in Java frameworks before I simply disagree
with that. The object in question is plainly a HTTPRequest(Head) not just
some arbitrary request. The ‘HTTP’ really is part of the thing. In fact the
‘arbitrary’ request could be an NSOperation, providing functionalities such
as pause, cancel, etc.

In a way it is like

  import Cat

  let pet = Animal() // resolving to Cat.Animal()

But well, if people really like that … ;-) I vote against it.

> About Sync/Async prefix. I was talking about type prefixes not method
prefixes. We definitely shouldn't prefix function APIs with sync or async.
I honestly don't care too much about which one to use as a default. I just
think it makes more sense to prefix the Async ones.

Your particular setup is sync, right? ;->

> About raw buffers we could provide APIs that take an array of
Unsafe[Mutable]RawBufferPointer to pass them to readv, writev and
company. Those could also be easily converted to DispatchData in the
frameworks that prefer to use it.

This is a little related to sync vs async. For async you usually need some
way to hold on to a buffer, and that is DispatchData. For sync you usually
don’t have to, and pointers to buffers can be faster.

Though I do think that sync is more appropriate for many server builders,
I think that the goal for Swift-Server would be more oriented towards
async. I may be wrong.

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

19

I really think we should start with Version, Headers, Message, Request and
Response and iterate.

···

On 26 May 2017 at 20:21, Carl Brown via swift-server-dev < swift-server-dev@swift.org> wrote:

On May 26, 2017, at 5:51 PM, Helge Heß via swift-server-dev < > swift-server-dev@swift.org> wrote:

Hi Carl,

But the question of the moment (for you and for the group) is - do you
think this is a good enough starting point for us to move it to the
Swift on Server · GitHub organization and start iterating via
GitHub issues & Pull Requests, or do you think we're still at the stage
where we need to have more email discussions about it before we're ready to
take that step?

that seems perfectly reasonable to me.

What I personally would like best is submitting Johannes’ proposal first,
and on top of that submit your changes (w/ explanations what you changed
and why in the commits).

I have Johannes’ original proposal in code over here, in case you need it
as a basis:

GitHub - ApacheExpress/ApacheExpress at s3wg-api-proposal-1
S3WGAPIProposal1

This looks like an earlier version of Johannes' code (probably from
The swift-server-dev Archives
Week-of-Mon-20170320/000329.html). I was working off the April 5th
version. (The one with "HTTP API Sketch v2" in the subject from
The swift-server-dev Archives
Week-of-Mon-20170403/000422.html).

Do we need to start with the March version?

-Carl

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

20

What do you mean by "the framework" here? I can think of several things

that could be referred to by that phrase, and I'm not sure which one you
mean.

Sorry, it was really confusing. I meant higher level web frameworks like
Kitura, for example. It looks like the code you have go all the way up
there.. The WebApp type says it all. I don't think we should have this at
all. At least not for now.. I believe we should only go up to the HTTP
server/client. I think the types we should implement are in order (naming
aside, adding prefixes or not, Sync, Async etc)

Week 1

Version
Headers
Message
Request
Response

Week 2

SyncBody
AsyncBody

Week 3

SyncRequestParser
AsyncRequestParser
SyncResponseParser
AsyncResponseParser

Week 4

SyncRequestSerializer
AsyncRequestSerializer
SyncResponseSerializer
AsyncResponseSerializer

Week 5

SyncServer
AsyncServer

Week 6

SyncClient
AsyncClient

We would have one week to discuss each the API of the types in each set.
After we decide on the API we implement in the following week while
discussing the next set of APIs. Once we settle on the API for each type
implementing shouldn't take more than 2 days based on our experience with
the subject.

Using this scheme we would have the API ready in mid-july. What do you guys
think? I really think this is a good way to follow on the
discussion/implementation.

···

On 26 May 2017 at 20:40, Carl Brown <carl.brown.swift@linuxswift.com> wrote:

On May 26, 2017, at 6:07 PM, Paulo Faria via swift-server-dev < > swift-server-dev@swift.org> wrote:

Hi, Carl!

I honestly think the code is too overwhelming.

I can see that. It was important to us to get something that actually
worked end-to-end, could be profiled and had test coverage. We couldn't
get it much smaller.

I'm also a bit confused about the IBM dependencies. Is this supposed to
stay in the swift server repos or is it just momentary so we have working
code?

It's a stand-in until the Transport part of the working group has a
reference implementation of a socket/stream API and the Security part of
the working group has a reference implementation of SSL/TLS. The HTTP part
of the group is just ahead of everyone else.

I'm not very fond of the first option if it is the case, and about the
second option... If it's just to see working code.. Then we should have in
swift server org only the HTTP module and then have the actual
implementation of the framework reside elsewhere importing HTTP module as a
dependency. This way we have an view of how people will import and use the
framework. It also gives the other frameworks a chance to implement their
solutions on top of the common HTTP module.

What do you mean by "the framework" here? I can think of several things
that could be referred to by that phrase, and I'm not sure which one you
mean.

I think we should start moving small bits of code, for example, Version,
Headers, Message, Request and Response. After we have that code in the repo
working with all the frameworks which are validating the API, then we move
on to the body (which is probably going to be the hardest part to settle
for everyone). After that we go to the parser/serializer.

Again, I'm not sure what you mean by "the frameworks" here, but putting
that aside, there's the issue of timing. I can see value in asking the
group for comments on a small piece of code and not moving on to the next
small piece until after everyone has had a chance to comment, but given the
pace of progress that's been made by this group so far, don't you think
that would take a very, very long time?

-Carl

On 26 May 2017 at 19:51, Helge Heß via swift-server-dev < > swift-server-dev@swift.org> wrote:

Hi Carl,

> But the question of the moment (for you and for the group) is - do you
think this is a good enough starting point for us to move it to the
Swift on Server · GitHub organization and start iterating via
GitHub issues & Pull Requests, or do you think we're still at the stage
where we need to have more email discussions about it before we're ready to
take that step?

that seems perfectly reasonable to me.

What I personally would like best is submitting Johannes’ proposal first,
and on top of that submit your changes (w/ explanations what you changed
and why in the commits).

I have Johannes’ original proposal in code over here, in case you need it
as a basis:

  https://github.com/ApacheExpress/ApacheExpress/tree/s3wg-
api-proposal-1/S3WGAPIProposal1

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev