HTTP API Sketch v2

Hi George,

This seems interesting, but is it necessary for getting off the ground?
Unless I'm missing something (it's been known to happen!) a protocol based
approach could be implemented purely additively, on top of the concrete
design Johannes has proposed, at a later date.

The advantage I see in waiting is that we can get more feedback from real
users of the API, such as what extension points they want and need, where
their performance concerns are, etc. We would also be able to make sure it
integrates well with what the community has been doing around higher level
libraries, middleware, etc.

-C

···

On Sun, Apr 9, 2017 at 9:23 PM George Leontiev via swift-server-dev < swift-server-dev@swift.org> wrote:

I’ve fleshed out my earlier suggestion.

For responses coming back from a server or requests going to a server I
like what Helge referred to as a “parser delegate” (naming suggestions
welcome) because it allows us to parse the request or response exactly as
it comes of the wire, and make no assumptions about how the user wants that
data. Also, this API can be used with both an async API (built on top of
dispatch) but could also be leveraged to create a synchronous API.

In terms of utility, I like that clients can provide their own
HTTPResponseDecoders which can decode to the specific type they are
expecting from that endpoint. Servers, on the other hand, can perform their
routing logic while processing that path and use the results of that to
more appropriately decode the rest of the request.

As I and Helge mentioned, there is overhead to using a protocol
cross-module, but the benefit is that we no longer have to have a catch-all
type to represent a decoded request or response; this allows users who care
about performance to encode their concrete types as efficiently as they
can. We can also have a catch-all type provided by the standard library
which leaves all the fields as strings (indeed, it can be backed by the
actual request data and just point to ranges) and would benefit from
full-module optimization. Even better, we can make an opinionated concrete
implementation (which can use enums for stata and headers and enforce
things like the ContentType header value should always be a MediaType,
etc.) but allow specialized applications which need more exotic semantics
to fall back to implementing their own type (instead of having `.custom(…)`
enum cases)

Note: I specifically omitted the body implementation and we may need
something like Johannes’ *Writer API to support sending chunked things.

- George

Code:
// MARK: - Stubs

public protocol HTTPBody {
    // API omitted.
    // There are decisions about how to handle streaming bodies which are
outside the scope of this example.
}

// Definition omitted
public typealias HTTPHeader = Void

public struct PathIterator: IteratorProtocol {

    func next() -> String {
        fatalError()
    }

}

// MARK: - Body Decoder

public protocol HTTPBodyDecoder {

    mutating func decodeTrailingHeader(_ key: String, value: String)
throws

    mutating func decodeBodyChunk(_ data: Data) throws

}

// MARK: - Server API

// A type which can be created from an HTTP Request as it is coming off
the wire. Suggestions on naming are welcome!
public protocol HTTPRequestDecoder: HTTPBodyDecoder {

    associatedtype Request

    // Thrown exceptions terminate decoding

    // Called before any other methods
    mutating func decodeMethod(_ value: String) throws

    // Called after `decodeMethod` and before any other methods. Types
conforming to `HTTPRequestDecoder` can use this method to walk the
requested path and prepare to handle the rest of the request appropriately.
Implementation does not need to exhaust the iterator.
    mutating func decodePath(_ iterator: inout PathIterator) throws

    // Called after all of the above methods, and before any methods
below.
    mutating func decodeVersion(_ value: String) throws

    // Called after all of the above methods and before any methods
below. May be called multiple times
    mutating func decodeHeader(_ key: String, value: String) throws

    // HTTPBodyDecoder methods conceptually appear here

    // Returns an object to be used to represent the request
    func decoded() throws -> Request

}

public extension HTTPRequestDecoder {

    mutating func decodeVersion(_ value: String) throws {
        // Implementing decodeVersion is not necessary
    }

}

/**
A type which can be represented as an HTTP Response
*/
public protocol HTTPResponse {
    var statusCode: UInt { get }
    var headers: [HTTPHeader] { get }
    var body: HTTPBody { get }
}

// MARK: - Client API

/**
A type which can be represented as an HTTP Request
*/
public protocol HTTPRequest {
    var method: String { get }

    // Note that the target host a part of an HTTPRequest
    var path: String { get }
    var headers: [HTTPHeader] { get }
    var body: HTTPBody { get }
}

public protocol HTTPResponseDecoder: HTTPBodyDecoder {

    associatedtype Response

    // Thrown exceptions terminate decoding

    // Called before any other methods
    mutating func decodeVersion(_ value: String) throws

    // Called after `decodeVersion` and before any other methods
    mutating func decodeStatus(_ value: UInt) throws

    // Called after `decodeVersion` and before any methods below
    mutating func decodeStatusText(_ value: String) throws

    // Called after all of the above methods and before any methods
below. May be called multiple times
    mutating func decodeHeader(_ key: String, value: String) throws

    // HTTPBodyDecoder methods conceptually appear here

    // Returns an object to be used to represent the response
    func decoded() throws -> Response

}

// MARK: - Implementing concrete types.

// We can provide come concrete implementations to handle *most* cases.
For instance, a concrete type could only support IANA registered stata (as
an enum) while a separate concrete type can allow for custom stata.

// Here is an example for a very simple server

public enum GeorgeMethod: String {
    case get, post
}

public struct GeorgeHTTPRequest {

    let status: GeorgeStatus

    // Implementation omitted

}

// We can provide convenience types/protocols for generating decoders from
types like my simple enum type above
public struct GeorgeHTTPRequestDecoder: HTTPRequestDecoder {

    // Implementation omitted

    func decode() throws -> GeorgeHTTPRequest {
        fatalError()
    }

}

On Apr 9, 2017, at 4:50 AM, Helge Heß via swift-server-dev < > swift-server-dev@swift.org> wrote:

Hi,

On 9. Apr 2017, at 04:12, George Leontiev via swift-server-dev < > swift-server-dev@swift.org> wrote:

Earlier in this thread, I think someone dismissed protocols/generics
because “at some point, we need to implement a concrete type”.

I don’t think anything was ‘dismissed’. The sole meaning was that there
will be a Swift Server module and that this should/will come with a
concrete type (so that people can have a minimal running http server out of
the box by just using that module). You can still have protocols or
whatever you chose.

I think we can leverage the Swift type system to have something much
better.

I recommend having protocols for HTTP{Request,Response}{Encodable,Decoder}
types

<snip> protocols <snip>

Your `HTTPRequestEncodable` is in a way the reverse of Johannes’
HTTPResponseWriter. The advantage of the latter is that it also doesn’t
enforce a specific response type and neither requires a type at all. W/
your approach there always has to be a type implementing the protocol.

I’m not entirely sure why you would call it `HTTPRequestEncodable`. I
would just call it `HTTPRequest`? To me an `Encodable` has a method which
provides another value (the encoded representation). Like this:

protocol HTTPRequestEncodable {
   func encodeAsHTTPRequest() -> HTTPRequest
}

which may still be useful. But extra indirections :-)

Your `HTTPRequestDecoder` is like a parser delegate, kinda like the peer
to the `HTTPResponseWriter`. Could be done like this.
Though if you provide an “Encodable” I suppose the assumption would be
that you also provide a “Decodable” too ...

I can flesh out the APIs more if people are interested, but I think this
is sufficient to communicate the idea, and I’d love to hear more feedback.

A complete proposal like Johannes’ would be more useful. Or maybe even
better a revision of Johannes’ proposal with your ideas in it.

There is a performance overhead to using protocols/generics, but I believe
these can be effectively mediated by the compiler.

I don’t see how you would eliminate the protocol overhead cross-module.
But I think it should be small enough (after all the proposal also uses
Strings ;-).

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

We could, but I’m not sure it would be more performant. The parser still has to scan the string to figure out how much data there is (find ‘\r\n’, etc). This is complicated by the fact that there are already emoji in URLs, among other issues. So, if the data coming off the wire is parsed as a string I’d prefer to keep it as such. I actually wasn’t sure about using Uint for status and could be for using strings for everything.

Also, bytes do not fare well with asynchronous APIs as all of the bytes may not be in a continuous buffer (this is what dispatch_data was made to handle)

···

On Apr 10, 2017, at 2:35 AM, Joy Keys <proyb7@gmail.com> wrote:

I would like to know what if we could use bytes to accept input instead of UInt and String for performance?

On Monday, April 10, 2017, George Leontiev via swift-server-dev <swift-server-dev@swift.org <javascript:_e(%7B%7D,'cvml','swift-server-dev@swift.org');>> wrote:
I’ve fleshed out my earlier suggestion.

For responses coming back from a server or requests going to a server I like what Helge referred to as a “parser delegate” (naming suggestions welcome) because it allows us to parse the request or response exactly as it comes of the wire, and make no assumptions about how the user wants that data. Also, this API can be used with both an async API (built on top of dispatch) but could also be leveraged to create a synchronous API.

In terms of utility, I like that clients can provide their own HTTPResponseDecoders which can decode to the specific type they are expecting from that endpoint. Servers, on the other hand, can perform their routing logic while processing that path and use the results of that to more appropriately decode the rest of the request.

As I and Helge mentioned, there is overhead to using a protocol cross-module, but the benefit is that we no longer have to have a catch-all type to represent a decoded request or response; this allows users who care about performance to encode their concrete types as efficiently as they can. We can also have a catch-all type provided by the standard library which leaves all the fields as strings (indeed, it can be backed by the actual request data and just point to ranges) and would benefit from full-module optimization. Even better, we can make an opinionated concrete implementation (which can use enums for stata and headers and enforce things like the ContentType header value should always be a MediaType, etc.) but allow specialized applications which need more exotic semantics to fall back to implementing their own type (instead of having `.custom(…)` enum cases)

Note: I specifically omitted the body implementation and we may need something like Johannes’ *Writer API to support sending chunked things.

- George

Code:
// MARK: - Stubs

public protocol HTTPBody {
    // API omitted.
    // There are decisions about how to handle streaming bodies which are outside the scope of this example.
}

// Definition omitted
public typealias HTTPHeader = Void

public struct PathIterator: IteratorProtocol {
    
    func next() -> String {
        fatalError()
    }
    
}

// MARK: - Body Decoder

public protocol HTTPBodyDecoder {
    
    mutating func decodeTrailingHeader(_ key: String, value: String) throws
    
    mutating func decodeBodyChunk(_ data: Data) throws
    
}

// MARK: - Server API

// A type which can be created from an HTTP Request as it is coming off the wire. Suggestions on naming are welcome!
public protocol HTTPRequestDecoder: HTTPBodyDecoder {
    
    associatedtype Request
    
    // Thrown exceptions terminate decoding
    
    // Called before any other methods
    mutating func decodeMethod(_ value: String) throws
    
    // Called after `decodeMethod` and before any other methods. Types conforming to `HTTPRequestDecoder` can use this method to walk the requested path and prepare to handle the rest of the request appropriately. Implementation does not need to exhaust the iterator.
    mutating func decodePath(_ iterator: inout PathIterator) throws
    
    // Called after all of the above methods, and before any methods below.
    mutating func decodeVersion(_ value: String) throws
    
    // Called after all of the above methods and before any methods below. May be called multiple times
    mutating func decodeHeader(_ key: String, value: String) throws
    
    // HTTPBodyDecoder methods conceptually appear here
    
    // Returns an object to be used to represent the request
    func decoded() throws -> Request
    
}

public extension HTTPRequestDecoder {
    
    mutating func decodeVersion(_ value: String) throws {
        // Implementing decodeVersion is not necessary
    }
    
}

/**
A type which can be represented as an HTTP Response
*/
public protocol HTTPResponse {
    var statusCode: UInt { get }
    var headers: [HTTPHeader] { get }
    var body: HTTPBody { get }
}

// MARK: - Client API

/**
A type which can be represented as an HTTP Request
*/
public protocol HTTPRequest {
    var method: String { get }
    
    // Note that the target host a part of an HTTPRequest
    var path: String { get }
    var headers: [HTTPHeader] { get }
    var body: HTTPBody { get }
}

public protocol HTTPResponseDecoder: HTTPBodyDecoder {
    
    associatedtype Response
    
    // Thrown exceptions terminate decoding
    
    // Called before any other methods
    mutating func decodeVersion(_ value: String) throws
    
    // Called after `decodeVersion` and before any other methods
    mutating func decodeStatus(_ value: UInt) throws
    
    // Called after `decodeVersion` and before any methods below
    mutating func decodeStatusText(_ value: String) throws
    
    // Called after all of the above methods and before any methods below. May be called multiple times
    mutating func decodeHeader(_ key: String, value: String) throws
    
    // HTTPBodyDecoder methods conceptually appear here
    
    // Returns an object to be used to represent the response
    func decoded() throws -> Response
    
}

// MARK: - Implementing concrete types.

// We can provide come concrete implementations to handle *most* cases. For instance, a concrete type could only support IANA registered stata (as an enum) while a separate concrete type can allow for custom stata.

// Here is an example for a very simple server

public enum GeorgeMethod: String {
    case get, post
}

public struct GeorgeHTTPRequest {
    
    let status: GeorgeStatus
    
    // Implementation omitted
    
}

// We can provide convenience types/protocols for generating decoders from types like my simple enum type above
public struct GeorgeHTTPRequestDecoder: HTTPRequestDecoder {
    
    // Implementation omitted
    
    func decode() throws -> GeorgeHTTPRequest {
        fatalError()
    }
    
}

On Apr 9, 2017, at 4:50 AM, Helge Heß via swift-server-dev <swift-server-dev@swift.org <>> wrote:

Hi,

On 9. Apr 2017, at 04:12, George Leontiev via swift-server-dev <swift-server-dev@swift.org <>> wrote:

Earlier in this thread, I think someone dismissed protocols/generics because “at some point, we need to implement a concrete type”.

I don’t think anything was ‘dismissed’. The sole meaning was that there will be a Swift Server module and that this should/will come with a concrete type (so that people can have a minimal running http server out of the box by just using that module). You can still have protocols or whatever you chose.

I think we can leverage the Swift type system to have something much better.

I recommend having protocols for HTTP{Request,Response}{Encodable,Decoder} types

<snip> protocols <snip>

Your `HTTPRequestEncodable` is in a way the reverse of Johannes’ HTTPResponseWriter. The advantage of the latter is that it also doesn’t enforce a specific response type and neither requires a type at all. W/ your approach there always has to be a type implementing the protocol.

I’m not entirely sure why you would call it `HTTPRequestEncodable`. I would just call it `HTTPRequest`? To me an `Encodable` has a method which provides another value (the encoded representation). Like this:

protocol HTTPRequestEncodable {
   func encodeAsHTTPRequest() -> HTTPRequest
}

which may still be useful. But extra indirections :-)

Your `HTTPRequestDecoder` is like a parser delegate, kinda like the peer to the `HTTPResponseWriter`. Could be done like this.
Though if you provide an “Encodable” I suppose the assumption would be that you also provide a “Decodable” too ...

I can flesh out the APIs more if people are interested, but I think this is sufficient to communicate the idea, and I’d love to hear more feedback.

A complete proposal like Johannes’ would be more useful. Or maybe even better a revision of Johannes’ proposal with your ideas in it.

There is a performance overhead to using protocols/generics, but I believe these can be effectively mediated by the compiler.

I don’t see how you would eliminate the protocol overhead cross-module. But I think it should be small enough (after all the proposal also uses Strings ;-).

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <>
https://lists.swift.org/mailman/listinfo/swift-server-dev

Yes but this particular protocol-based approach has implications on how the parsing is done, which we should decide on before moving forward with a concrete implementation. In order to support something like this, the parser would need to be built as a streaming parser with delegation in mind.

- George

···

On Apr 10, 2017, at 1:27 PM, Colin Barrett <colin@springsandstruts.com> wrote:

Hi George,

This seems interesting, but is it necessary for getting off the ground? Unless I'm missing something (it's been known to happen!) a protocol based approach could be implemented purely additively, on top of the concrete design Johannes has proposed, at a later date.

The advantage I see in waiting is that we can get more feedback from real users of the API, such as what extension points they want and need, where their performance concerns are, etc. We would also be able to make sure it integrates well with what the community has been doing around higher level libraries, middleware, etc.

-C

On Sun, Apr 9, 2017 at 9:23 PM George Leontiev via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:
I’ve fleshed out my earlier suggestion.

For responses coming back from a server or requests going to a server I like what Helge referred to as a “parser delegate” (naming suggestions welcome) because it allows us to parse the request or response exactly as it comes of the wire, and make no assumptions about how the user wants that data. Also, this API can be used with both an async API (built on top of dispatch) but could also be leveraged to create a synchronous API.

In terms of utility, I like that clients can provide their own HTTPResponseDecoders which can decode to the specific type they are expecting from that endpoint. Servers, on the other hand, can perform their routing logic while processing that path and use the results of that to more appropriately decode the rest of the request.

As I and Helge mentioned, there is overhead to using a protocol cross-module, but the benefit is that we no longer have to have a catch-all type to represent a decoded request or response; this allows users who care about performance to encode their concrete types as efficiently as they can. We can also have a catch-all type provided by the standard library which leaves all the fields as strings (indeed, it can be backed by the actual request data and just point to ranges) and would benefit from full-module optimization. Even better, we can make an opinionated concrete implementation (which can use enums for stata and headers and enforce things like the ContentType header value should always be a MediaType, etc.) but allow specialized applications which need more exotic semantics to fall back to implementing their own type (instead of having `.custom(…)` enum cases)

Note: I specifically omitted the body implementation and we may need something like Johannes’ *Writer API to support sending chunked things.

- George

Code:
// MARK: - Stubs

public protocol HTTPBody {
    // API omitted.
    // There are decisions about how to handle streaming bodies which are outside the scope of this example.
}

// Definition omitted
public typealias HTTPHeader = Void

public struct PathIterator: IteratorProtocol {
    
    func next() -> String {
        fatalError()
    }
    
}

// MARK: - Body Decoder

public protocol HTTPBodyDecoder {
    
    mutating func decodeTrailingHeader(_ key: String, value: String) throws
    
    mutating func decodeBodyChunk(_ data: Data) throws
    
}

// MARK: - Server API

// A type which can be created from an HTTP Request as it is coming off the wire. Suggestions on naming are welcome!
public protocol HTTPRequestDecoder: HTTPBodyDecoder {
    
    associatedtype Request
    
    // Thrown exceptions terminate decoding
    
    // Called before any other methods
    mutating func decodeMethod(_ value: String) throws
    
    // Called after `decodeMethod` and before any other methods. Types conforming to `HTTPRequestDecoder` can use this method to walk the requested path and prepare to handle the rest of the request appropriately. Implementation does not need to exhaust the iterator.
    mutating func decodePath(_ iterator: inout PathIterator) throws
    
    // Called after all of the above methods, and before any methods below.
    mutating func decodeVersion(_ value: String) throws
    
    // Called after all of the above methods and before any methods below. May be called multiple times
    mutating func decodeHeader(_ key: String, value: String) throws
    
    // HTTPBodyDecoder methods conceptually appear here
    
    // Returns an object to be used to represent the request
    func decoded() throws -> Request
    
}

public extension HTTPRequestDecoder {
    
    mutating func decodeVersion(_ value: String) throws {
        // Implementing decodeVersion is not necessary
    }
    
}

/**
A type which can be represented as an HTTP Response
*/
public protocol HTTPResponse {
    var statusCode: UInt { get }
    var headers: [HTTPHeader] { get }
    var body: HTTPBody { get }
}

// MARK: - Client API

/**
A type which can be represented as an HTTP Request
*/
public protocol HTTPRequest {
    var method: String { get }
    
    // Note that the target host a part of an HTTPRequest
    var path: String { get }
    var headers: [HTTPHeader] { get }
    var body: HTTPBody { get }
}

public protocol HTTPResponseDecoder: HTTPBodyDecoder {
    
    associatedtype Response
    
    // Thrown exceptions terminate decoding
    
    // Called before any other methods
    mutating func decodeVersion(_ value: String) throws
    
    // Called after `decodeVersion` and before any other methods
    mutating func decodeStatus(_ value: UInt) throws
    
    // Called after `decodeVersion` and before any methods below
    mutating func decodeStatusText(_ value: String) throws
    
    // Called after all of the above methods and before any methods below. May be called multiple times
    mutating func decodeHeader(_ key: String, value: String) throws
    
    // HTTPBodyDecoder methods conceptually appear here
    
    // Returns an object to be used to represent the response
    func decoded() throws -> Response
    
}

// MARK: - Implementing concrete types.

// We can provide come concrete implementations to handle *most* cases. For instance, a concrete type could only support IANA registered stata (as an enum) while a separate concrete type can allow for custom stata.

// Here is an example for a very simple server

public enum GeorgeMethod: String {
    case get, post
}

public struct GeorgeHTTPRequest {
    
    let status: GeorgeStatus
    
    // Implementation omitted
    
}

// We can provide convenience types/protocols for generating decoders from types like my simple enum type above
public struct GeorgeHTTPRequestDecoder: HTTPRequestDecoder {
    
    // Implementation omitted
    
    func decode() throws -> GeorgeHTTPRequest {
        fatalError()
    }
    
}

On Apr 9, 2017, at 4:50 AM, Helge Heß via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:

Hi,

On 9. Apr 2017, at 04:12, George Leontiev via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:

Earlier in this thread, I think someone dismissed protocols/generics because “at some point, we need to implement a concrete type”.

I don’t think anything was ‘dismissed’. The sole meaning was that there will be a Swift Server module and that this should/will come with a concrete type (so that people can have a minimal running http server out of the box by just using that module). You can still have protocols or whatever you chose.

I think we can leverage the Swift type system to have something much better.

I recommend having protocols for HTTP{Request,Response}{Encodable,Decoder} types

<snip> protocols <snip>

Your `HTTPRequestEncodable` is in a way the reverse of Johannes’ HTTPResponseWriter. The advantage of the latter is that it also doesn’t enforce a specific response type and neither requires a type at all. W/ your approach there always has to be a type implementing the protocol.

I’m not entirely sure why you would call it `HTTPRequestEncodable`. I would just call it `HTTPRequest`? To me an `Encodable` has a method which provides another value (the encoded representation). Like this:

protocol HTTPRequestEncodable {
   func encodeAsHTTPRequest() -> HTTPRequest
}

which may still be useful. But extra indirections :-)

Your `HTTPRequestDecoder` is like a parser delegate, kinda like the peer to the `HTTPResponseWriter`. Could be done like this.
Though if you provide an “Encodable” I suppose the assumption would be that you also provide a “Decodable” too ...

I can flesh out the APIs more if people are interested, but I think this is sufficient to communicate the idea, and I’d love to hear more feedback.

A complete proposal like Johannes’ would be more useful. Or maybe even better a revision of Johannes’ proposal with your ideas in it.

There is a performance overhead to using protocols/generics, but I believe these can be effectively mediated by the compiler.

I don’t see how you would eliminate the protocol overhead cross-module. But I think it should be small enough (after all the proposal also uses Strings ;-).

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev

Thanks George on bytes. I have seen the recently Techempower
benchmark shown minihttp is performant in JSON and plain text is an
interesting finding whether if it is something we haven't explore.

Check out on JSON and Plaintext
https://www.techempower.com/benchmarks/previews/round14/#section=data-r14&hw=ph&test=json

···

On Monday, April 10, 2017, George Leontiev <georgeleontiev@gmail.com> wrote:

We could, but I’m not sure it would be more performant. The parser still
has to scan the string to figure out how much data there is (find ‘\r\n’,
etc). This is complicated by the fact that there are already emoji in URLs,
among other issues. So, if the data coming off the wire is parsed as a
string I’d prefer to keep it as such. I actually wasn’t sure about using
Uint for status and could be for using strings for everything.

Also, bytes do not fare well with asynchronous APIs as all of the bytes
may not be in a continuous buffer (this is what dispatch_data was made to
handle)

On Apr 10, 2017, at 2:35 AM, Joy Keys <proyb7@gmail.com > <javascript:_e(%7B%7D,'cvml','proyb7@gmail.com');>> wrote:

I would like to know what if we could use bytes to accept input instead of
UInt and String for performance?

On Monday, April 10, 2017, George Leontiev via swift-server-dev < > swift-server-dev@swift.org> wrote:

I’ve fleshed out my earlier suggestion.

For responses coming back from a server or requests going to a server I
like what Helge referred to as a “parser delegate” (naming suggestions
welcome) because it allows us to parse the request or response exactly as
it comes of the wire, and make no assumptions about how the user wants that
data. Also, this API can be used with both an async API (built on top of
dispatch) but could also be leveraged to create a synchronous API.

In terms of utility, I like that clients can provide their own
HTTPResponseDecoders which can decode to the specific type they are
expecting from that endpoint. Servers, on the other hand, can perform their
routing logic while processing that path and use the results of that to
more appropriately decode the rest of the request.

As I and Helge mentioned, there is overhead to using a protocol
cross-module, but the benefit is that we no longer have to have a catch-all
type to represent a decoded request or response; this allows users who care
about performance to encode their concrete types as efficiently as they
can. We can also have a catch-all type provided by the standard library
which leaves all the fields as strings (indeed, it can be backed by the
actual request data and just point to ranges) and would benefit from
full-module optimization. Even better, we can make an opinionated concrete
implementation (which can use enums for stata and headers and enforce
things like the ContentType header value should always be a MediaType,
etc.) but allow specialized applications which need more exotic semantics
to fall back to implementing their own type (instead of having `.custom(…)`
enum cases)

Note: I specifically omitted the body implementation and we may need
something like Johannes’ *Writer API to support sending chunked things.

- George

Code:
// MARK: - Stubs

public protocol HTTPBody {
    // API omitted.
    // There are decisions about how to handle streaming bodies which
are outside the scope of this example.
}

// Definition omitted
public typealias HTTPHeader = Void

public struct PathIterator: IteratorProtocol {

    func next() -> String {
        fatalError()
    }

}

// MARK: - Body Decoder

public protocol HTTPBodyDecoder {

    mutating func decodeTrailingHeader(_ key: String, value: String)
throws

    mutating func decodeBodyChunk(_ data: Data) throws

}

// MARK: - Server API

// A type which can be created from an HTTP Request as it is coming off
the wire. Suggestions on naming are welcome!
public protocol HTTPRequestDecoder: HTTPBodyDecoder {

    associatedtype Request

    // Thrown exceptions terminate decoding

    // Called before any other methods
    mutating func decodeMethod(_ value: String) throws

    // Called after `decodeMethod` and before any other methods. Types
conforming to `HTTPRequestDecoder` can use this method to walk the
requested path and prepare to handle the rest of the request appropriately.
Implementation does not need to exhaust the iterator.
    mutating func decodePath(_ iterator: inout PathIterator) throws

    // Called after all of the above methods, and before any methods
below.
    mutating func decodeVersion(_ value: String) throws

    // Called after all of the above methods and before any methods
below. May be called multiple times
    mutating func decodeHeader(_ key: String, value: String) throws

    // HTTPBodyDecoder methods conceptually appear here

    // Returns an object to be used to represent the request
    func decoded() throws -> Request

}

public extension HTTPRequestDecoder {

    mutating func decodeVersion(_ value: String) throws {
        // Implementing decodeVersion is not necessary
    }

}

/**
A type which can be represented as an HTTP Response
*/
public protocol HTTPResponse {
    var statusCode: UInt { get }
    var headers: [HTTPHeader] { get }
    var body: HTTPBody { get }
}

// MARK: - Client API

/**
A type which can be represented as an HTTP Request
*/
public protocol HTTPRequest {
    var method: String { get }

    // Note that the target host a part of an HTTPRequest
    var path: String { get }
    var headers: [HTTPHeader] { get }
    var body: HTTPBody { get }
}

public protocol HTTPResponseDecoder: HTTPBodyDecoder {

    associatedtype Response

    // Thrown exceptions terminate decoding

    // Called before any other methods
    mutating func decodeVersion(_ value: String) throws

    // Called after `decodeVersion` and before any other methods
    mutating func decodeStatus(_ value: UInt) throws

    // Called after `decodeVersion` and before any methods below
    mutating func decodeStatusText(_ value: String) throws

    // Called after all of the above methods and before any methods
below. May be called multiple times
    mutating func decodeHeader(_ key: String, value: String) throws

    // HTTPBodyDecoder methods conceptually appear here

    // Returns an object to be used to represent the response
    func decoded() throws -> Response

}

// MARK: - Implementing concrete types.

// We can provide come concrete implementations to handle *most* cases.
For instance, a concrete type could only support IANA registered stata (as
an enum) while a separate concrete type can allow for custom stata.

// Here is an example for a very simple server

public enum GeorgeMethod: String {
    case get, post
}

public struct GeorgeHTTPRequest {

    let status: GeorgeStatus

    // Implementation omitted

}

// We can provide convenience types/protocols for generating decoders
from types like my simple enum type above
public struct GeorgeHTTPRequestDecoder: HTTPRequestDecoder {

    // Implementation omitted

    func decode() throws -> GeorgeHTTPRequest {
        fatalError()
    }

}

On Apr 9, 2017, at 4:50 AM, Helge Heß via swift-server-dev < >> swift-server-dev@swift.org> wrote:

Hi,

On 9. Apr 2017, at 04:12, George Leontiev via swift-server-dev < >> swift-server-dev@swift.org> wrote:

Earlier in this thread, I think someone dismissed protocols/generics
because “at some point, we need to implement a concrete type”.

I don’t think anything was ‘dismissed’. The sole meaning was that there
will be a Swift Server module and that this should/will come with a
concrete type (so that people can have a minimal running http server out of
the box by just using that module). You can still have protocols or
whatever you chose.

I think we can leverage the Swift type system to have something much
better.

I recommend having protocols for HTTP{Request,Response}{Encodable,Decoder}
types

<snip> protocols <snip>

Your `HTTPRequestEncodable` is in a way the reverse of Johannes’
HTTPResponseWriter. The advantage of the latter is that it also doesn’t
enforce a specific response type and neither requires a type at all. W/
your approach there always has to be a type implementing the protocol.

I’m not entirely sure why you would call it `HTTPRequestEncodable`. I
would just call it `HTTPRequest`? To me an `Encodable` has a method which
provides another value (the encoded representation). Like this:

protocol HTTPRequestEncodable {
   func encodeAsHTTPRequest() -> HTTPRequest
}

which may still be useful. But extra indirections :-)

Your `HTTPRequestDecoder` is like a parser delegate, kinda like the peer
to the `HTTPResponseWriter`. Could be done like this.
Though if you provide an “Encodable” I suppose the assumption would be
that you also provide a “Decodable” too ...

I can flesh out the APIs more if people are interested, but I think this
is sufficient to communicate the idea, and I’d love to hear more feedback.

A complete proposal like Johannes’ would be more useful. Or maybe even
better a revision of Johannes’ proposal with your ideas in it.

There is a performance overhead to using protocols/generics, but I
believe these can be effectively mediated by the compiler.

I don’t see how you would eliminate the protocol overhead cross-module.
But I think it should be small enough (after all the proposal also uses
Strings ;-).

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

If we want to change the implementation, we can change it. Much much more
important not to ship API we'll regret—API is forever.

···

On Mon, Apr 10, 2017 at 9:34 PM George Leontiev <georgeleontiev@gmail.com> wrote:

Yes but this particular protocol-based approach has implications on how
the parsing is done, which we should decide on before moving forward with a
concrete implementation. In order to support something like this, the
parser would need to be built as a streaming parser with delegation in mind.

- George

On Apr 10, 2017, at 1:27 PM, Colin Barrett <colin@springsandstruts.com> > wrote:

Hi George,

This seems interesting, but is it necessary for getting off the ground?
Unless I'm missing something (it's been known to happen!) a protocol based
approach could be implemented purely additively, on top of the concrete
design Johannes has proposed, at a later date.

The advantage I see in waiting is that we can get more feedback from real
users of the API, such as what extension points they want and need, where
their performance concerns are, etc. We would also be able to make sure it
integrates well with what the community has been doing around higher level
libraries, middleware, etc.

-C

On Sun, Apr 9, 2017 at 9:23 PM George Leontiev via swift-server-dev < > swift-server-dev@swift.org> wrote:

I’ve fleshed out my earlier suggestion.

For responses coming back from a server or requests going to a server I
like what Helge referred to as a “parser delegate” (naming suggestions
welcome) because it allows us to parse the request or response exactly as
it comes of the wire, and make no assumptions about how the user wants that
data. Also, this API can be used with both an async API (built on top of
dispatch) but could also be leveraged to create a synchronous API.

In terms of utility, I like that clients can provide their own
HTTPResponseDecoders which can decode to the specific type they are
expecting from that endpoint. Servers, on the other hand, can perform their
routing logic while processing that path and use the results of that to
more appropriately decode the rest of the request.

As I and Helge mentioned, there is overhead to using a protocol
cross-module, but the benefit is that we no longer have to have a catch-all
type to represent a decoded request or response; this allows users who care
about performance to encode their concrete types as efficiently as they
can. We can also have a catch-all type provided by the standard library
which leaves all the fields as strings (indeed, it can be backed by the
actual request data and just point to ranges) and would benefit from
full-module optimization. Even better, we can make an opinionated concrete
implementation (which can use enums for stata and headers and enforce
things like the ContentType header value should always be a MediaType,
etc.) but allow specialized applications which need more exotic semantics
to fall back to implementing their own type (instead of having `.custom(…)`
enum cases)

Note: I specifically omitted the body implementation and we may need
something like Johannes’ *Writer API to support sending chunked things.

- George

Code:
// MARK: - Stubs

public protocol HTTPBody {
    // API omitted.
    // There are decisions about how to handle streaming bodies which are
outside the scope of this example.
}

// Definition omitted
public typealias HTTPHeader = Void

public struct PathIterator: IteratorProtocol {

    func next() -> String {
        fatalError()
    }

}

// MARK: - Body Decoder

public protocol HTTPBodyDecoder {

    mutating func decodeTrailingHeader(_ key: String, value: String)
throws

    mutating func decodeBodyChunk(_ data: Data) throws

}

// MARK: - Server API

// A type which can be created from an HTTP Request as it is coming off
the wire. Suggestions on naming are welcome!
public protocol HTTPRequestDecoder: HTTPBodyDecoder {

    associatedtype Request

    // Thrown exceptions terminate decoding

    // Called before any other methods
    mutating func decodeMethod(_ value: String) throws

    // Called after `decodeMethod` and before any other methods. Types
conforming to `HTTPRequestDecoder` can use this method to walk the
requested path and prepare to handle the rest of the request appropriately.
Implementation does not need to exhaust the iterator.
    mutating func decodePath(_ iterator: inout PathIterator) throws

    // Called after all of the above methods, and before any methods
below.
    mutating func decodeVersion(_ value: String) throws

    // Called after all of the above methods and before any methods
below. May be called multiple times
    mutating func decodeHeader(_ key: String, value: String) throws

    // HTTPBodyDecoder methods conceptually appear here

    // Returns an object to be used to represent the request
    func decoded() throws -> Request

}

public extension HTTPRequestDecoder {

    mutating func decodeVersion(_ value: String) throws {
        // Implementing decodeVersion is not necessary
    }

}

/**
A type which can be represented as an HTTP Response
*/
public protocol HTTPResponse {
    var statusCode: UInt { get }
    var headers: [HTTPHeader] { get }
    var body: HTTPBody { get }
}

// MARK: - Client API

/**
A type which can be represented as an HTTP Request
*/
public protocol HTTPRequest {
    var method: String { get }

    // Note that the target host a part of an HTTPRequest
    var path: String { get }
    var headers: [HTTPHeader] { get }
    var body: HTTPBody { get }
}

public protocol HTTPResponseDecoder: HTTPBodyDecoder {

    associatedtype Response

    // Thrown exceptions terminate decoding

    // Called before any other methods
    mutating func decodeVersion(_ value: String) throws

    // Called after `decodeVersion` and before any other methods
    mutating func decodeStatus(_ value: UInt) throws

    // Called after `decodeVersion` and before any methods below
    mutating func decodeStatusText(_ value: String) throws

    // Called after all of the above methods and before any methods
below. May be called multiple times
    mutating func decodeHeader(_ key: String, value: String) throws

    // HTTPBodyDecoder methods conceptually appear here

    // Returns an object to be used to represent the response
    func decoded() throws -> Response

}

// MARK: - Implementing concrete types.

// We can provide come concrete implementations to handle *most* cases.
For instance, a concrete type could only support IANA registered stata (as
an enum) while a separate concrete type can allow for custom stata.

// Here is an example for a very simple server

public enum GeorgeMethod: String {
    case get, post
}

public struct GeorgeHTTPRequest {

    let status: GeorgeStatus

    // Implementation omitted

}

// We can provide convenience types/protocols for generating decoders from
types like my simple enum type above
public struct GeorgeHTTPRequestDecoder: HTTPRequestDecoder {

    // Implementation omitted

    func decode() throws -> GeorgeHTTPRequest {
        fatalError()
    }

}

On Apr 9, 2017, at 4:50 AM, Helge Heß via swift-server-dev < > swift-server-dev@swift.org> wrote:

Hi,

On 9. Apr 2017, at 04:12, George Leontiev via swift-server-dev < > swift-server-dev@swift.org> wrote:

Earlier in this thread, I think someone dismissed protocols/generics
because “at some point, we need to implement a concrete type”.

I don’t think anything was ‘dismissed’. The sole meaning was that there
will be a Swift Server module and that this should/will come with a
concrete type (so that people can have a minimal running http server out of
the box by just using that module). You can still have protocols or
whatever you chose.

I think we can leverage the Swift type system to have something much
better.

I recommend having protocols for HTTP{Request,Response}{Encodable,Decoder}
types

<snip> protocols <snip>

Your `HTTPRequestEncodable` is in a way the reverse of Johannes’
HTTPResponseWriter. The advantage of the latter is that it also doesn’t
enforce a specific response type and neither requires a type at all. W/
your approach there always has to be a type implementing the protocol.

I’m not entirely sure why you would call it `HTTPRequestEncodable`. I
would just call it `HTTPRequest`? To me an `Encodable` has a method which
provides another value (the encoded representation). Like this:

protocol HTTPRequestEncodable {
   func encodeAsHTTPRequest() -> HTTPRequest
}

which may still be useful. But extra indirections :-)

Your `HTTPRequestDecoder` is like a parser delegate, kinda like the peer
to the `HTTPResponseWriter`. Could be done like this.
Though if you provide an “Encodable” I suppose the assumption would be
that you also provide a “Decodable” too ...

I can flesh out the APIs more if people are interested, but I think this
is sufficient to communicate the idea, and I’d love to hear more feedback.

A complete proposal like Johannes’ would be more useful. Or maybe even
better a revision of Johannes’ proposal with your ideas in it.

There is a performance overhead to using protocols/generics, but I believe
these can be effectively mediated by the compiler.

I don’t see how you would eliminate the protocol overhead cross-module.
But I think it should be small enough (after all the proposal also uses
Strings ;-).

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

The latter is far from being true in the Swift universe. In the Swift universe API is good for about a year. Then the Xcode migrator kicks in.

That doesn’t mean you should ship something you’ll regret of course :-)

hh

···

On 12. Apr 2017, at 01:01, Colin Barrett <colin@springsandstruts.com> wrote:

Much much more important not to ship API we'll regret—API is forever.

Assuming you are running on xcode that is.

···

On Wed, Apr 12, 2017, 07:35 Helge Heß via swift-server-dev < swift-server-dev@swift.org> wrote:

On 12. Apr 2017, at 01:01, Colin Barrett <colin@springsandstruts.com> > wrote:
> Much much more important not to ship API we'll regret—API is forever.

The latter is far from being true in the Swift universe. In the Swift
universe API is good for about a year. Then the Xcode migrator kicks in.

That doesn’t mean you should ship something you’ll regret of course :-)

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev