Next HTTP API meeting

Server Development
1

We're now at the point that we have broad agreement on the general
approach to building a set of HTTP APIs, and we're starting to get down to
the next level of detail which is likely to be more iterative. As such
we'll be having more regular meetings and starting to have working designs
in Google Docs and GitHub.

We'll therefore start scheduling the next meeting immediately after the
previous one. Please use the Doodle Poll below to show your availability -
dates between March 27th and April 7th are available.
        http://doodle.com/poll/nfmepruih5nr9p2g

Thanks,

Chris

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

2

Hi,

unsurprisingly for me there is a heated discussion on this in the pull request. I thought I bring it back over here:

https://github.com/swift-server/work-group/pull/76

I wonder where this is going. I mean all those discussions are nice and valuable, but someone eventually has to do the work, right? :-)

My understanding is that there is a goal to have that essentially ready in summer / for Swift 4, right? How is this going to happen? Is IBM and/or Apple going to provide a reference implementation and the others can use it or not?

hh

···

On 16. Mar 2017, at 21:56, Chris Bailey via swift-server-dev <swift-server-dev@swift.org> wrote:

We're now at the point that we have broad agreement on the general approach to building a set of HTTP APIs, and we're starting to get down to the next level of detail which is likely to be more iterative. As such we'll be having more regular meetings and starting to have working designs in Google Docs and GitHub.

We'll therefore start scheduling the next meeting immediately after the previous one. Please use the Doodle Poll below to show your availability - dates between March 27th and April 7th are available.
       Expired Group Poll - Create a New Poll or Contact the Owner

Thanks,

Chris

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

3

Helge,

At some point someone has to do the work, yes, but ultimately there are a
lot of stakeholders to consider and if there's a chance that they can all
find common ground, then it must be pursued. The point of this group is to
provide base level APIs that can work with the existing frameworks in a way
that adds value. Apple isn't going to do this (or has at least made no
indication of any interest whatsoever), and for IBM to just do it, I don't
think choosing one framework to steamroll is the answer and defeats the
purpose of the group. Technically there are many reference implementations
already, Kitura, Perfect, Vapor, Foundation to some degree.

Things like sockets/security/crypto are easily agreed upon and moved
forward with little impact to existing frameworks. The HTTP models and
architecture have by far the greatest impact of any other feature of the
server side group and so the discussion will take as long as it needs to in
order to get it right. The existing frameworks have a lot of experience in
these areas for what has worked and what hasn't, and they all have existing
architectures and paradigms that need to be accounted for as much as
possible.

There has already been a great deal of compromise on the parsing and
serialization layer, and I'm confident we can get somewhere with models as
well. I'd personally like to see us more explicitly explore protocols, I
haven't seen anything so far that actually requires a concrete model and
would perhaps be an easier way for us to reach consensus. In the meantime,
I think it's important to get something that works for everyone the first
time as much as possible. If that means more discussion, then that is what
is required. Eventually, as with other things, I'm sure we can come to an
agreement with time.

- Logan

Hi,

unsurprisingly for me there is a heated discussion on this in the pull
request. I thought I bring it back over here:

https://github.com/swift-server/work-group/pull/76

I wonder where this is going. I mean all those discussions are nice and
valuable, but someone eventually has to do the work, right? :-)

My understanding is that there is a goal to have that essentially ready in
summer / for Swift 4, right? How is this going to happen? Is IBM and/or
Apple going to provide a reference implementation and the others can use it
or not?

hh

We're now at the point that we have broad agreement on the general

approach to building a set of HTTP APIs, and we're starting to get down to
the next level of detail which is likely to be more iterative. As such
we'll be having more regular meetings and starting to have working designs
in Google Docs and GitHub.

We'll therefore start scheduling the next meeting immediately after the

previous one. Please use the Doodle Poll below to show your availability -
dates between March 27th and April 7th are available.

       http://doodle.com/poll/nfmepruih5nr9p2g

Thanks,

Chris

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number

741598.

···

On Mon, Mar 20, 2017 at 9:07 PM Helge Heß via swift-server-dev < swift-server-dev@swift.org> wrote:

On 16. Mar 2017, at 21:56, Chris Bailey via swift-server-dev < swift-server-dev@swift.org> wrote:
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

4

Hi Logan,

What is it about protocols that you feel would help here? In general, the relationship between concrete implementations and protocols should be that the former comes first, then the latter. Start with a concrete type, then as you move to multiple diverse implementations, factor out the common things that unify those types.

Protocols in Swift exist primarily to allow code to operate on multiple different concrete implementations simultaneously – either with generics as constraints, or as existentials. In this case, there is no clear reason why someone would want to write code that handles two separate implementations of an HTTP API at the same time within a single program. Protocols are not necessary in order to “swap” implementation A for implementation B. All that is needed for this is that the two implementations be source compatible i.e. have the same method signatures (protocols are a way of helping enforce that, but there are other less performance-consequential ways of doing it too).

Now, it may be that a need does emerge as the design takes shape to have separate types, say, for different kinds of HTTP connections, that might need to be handled by the same program and have to be unified via a protocol to enable handling code to be shared across both. But this protocol should be discovered through the process of looking at the common features the different concrete types have, rather than designed up front.

Cheers,
Ben

···

On Mar 20, 2017, at 11:11 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

I'd personally like to see us more explicitly explore protocols, I haven't seen anything so far that actually requires a concrete model and would perhaps be an easier way for us to reach consensus.

5

Protocols in Swift exist primarily to allow code to operate on multiple

different concrete implementations simultaneously – either with generics as
constraints, or as existentials. In this case, there is no clear reason why
someone would want to write code that handles two separate implementations
of an HTTP API at the same time within a single program.

I completely agree with this. Using protocols to allow multiple
implementations in this scenario would not be a solution for a technical
problem. It would be, instead, a solution for a "human" problem which is
basically the difficulty of reaching an agreement. In the end what we all
want is the best solution technically.

···

On 22 March 2017 at 10:19, Ben Cohen via swift-server-dev < swift-server-dev@swift.org> wrote:

On Mar 20, 2017, at 11:11 PM, Logan Wright via swift-server-dev < > swift-server-dev@swift.org> wrote:

I'd personally like to see us more explicitly explore protocols, I haven't
seen anything so far that actually requires a concrete model and would
perhaps be an easier way for us to reach consensus.

Hi Logan,

What is it about protocols that you feel would help here? In general, the
relationship between concrete implementations and protocols should be that
the former comes first, then the latter. Start with a concrete type, then
as you move to multiple diverse implementations, factor out the common
things that unify those types.

Protocols in Swift exist primarily to allow code to operate on multiple
different concrete implementations simultaneously – either with generics as
constraints, or as existentials. In this case, there is no clear reason why
someone would want to write code that handles two separate implementations
of an HTTP API at the same time within a single program. Protocols are not
necessary in order to “swap” implementation A for implementation B. All
that is needed for this is that the two implementations be source
compatible i.e. have the same method signatures (protocols are a way of
helping enforce that, but there are other less performance-consequential
ways of doing it too).

Now, it may be that a need does emerge as the design takes shape to have
separate types, say, for different kinds of HTTP connections, that might
need to be handled by the same program and have to be unified via a
protocol to enable handling code to be shared across both. But this
protocol should be discovered through the process of looking at the common
features the different concrete types have, rather than designed up front.

Cheers,
Ben

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

6

Hi Ben,

I guess I can give an example of what I think Logan may also have in mind.

For mod_swift I ported a trimmed down version of the Noze.io Express module to Apache. Originally that was using a concrete implementation for the incoming request (the ApacheExpress.IncomingMessage class).
After doing the work I found it stupid to have two Express implementations flying around. The `ApacheExpress` module (middleware etc) didn't really have many Apache dependencies, it still just works on top of the request and response. So I figured I can separate out the whole Express functionality just by making IncomingMessage/ServerResponse protocols:

  https://github.com/AlwaysRightInstitute/mod_swift/blob/master/ThirdParty/ExExpress/Sources/http/IncomingMessage.swift#L6

Now I have a server agnostic `ExExpress` module which has zero Apache dependencies and could be used by any other framework implementing the `ExExpress.IncomingMessage` protocol.

And the `ApacheExpress` is now a pretty small module which is just concerned with the Apache mapping of the IncomingMessage protocol:

  https://github.com/AlwaysRightInstitute/mod_swift/blob/master/ApacheExpress/Sources/ApacheIncomingMessage.swift#L11

Now I think there is the hope that the Swift Server Project would provide server agnostic protocols for Request/Response which can be used as a common basis by module writers. And which would then work for Kitura or Perfect etc.

Do you have another suggestion on how this could be solved w/o protocols? Just switch implementations by switching packages? ;-]

Note that there may be a difference between the types used for higher level middleware-like processing and the entities produced by a low level parser. The low level parser could still e.g. produce a struct representing an HTTP message head, which is then used as a value by the higher level object.
Or you could give the parser a factory which directly feeds in parsed fields etc, but that may not make a lot of sense from a performance perspective (then rather have a `init(head: HTTPParser.RequestHead)`).

hh

P.S.: The Noze.io vs mod_swift ‘Express module' is also a pretty good demo on the difficulties sharing an implementation between different concurrency models (ExExpress is completely sync and doesn’t require any @escaping callbacks, while Noze.io is the exact opposite of that).

···

On 22 Mar 2017, at 14:19, Ben Cohen <ben_cohen@apple.com> wrote:

On Mar 20, 2017, at 11:11 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

I'd personally like to see us more explicitly explore protocols, I haven't seen anything so far that actually requires a concrete model and would perhaps be an easier way for us to reach consensus.

Hi Logan,

What is it about protocols that you feel would help here? In general, the relationship between concrete implementations and protocols should be that the former comes first, then the latter. Start with a concrete type, then as you move to multiple diverse implementations, factor out the common things that unify those types.

Protocols in Swift exist primarily to allow code to operate on multiple different concrete implementations simultaneously – either with generics as constraints, or as existentials. In this case, there is no clear reason why someone would want to write code that handles two separate implementations of an HTTP API at the same time within a single program. Protocols are not necessary in order to “swap” implementation A for implementation B. All that is needed for this is that the two implementations be source compatible i.e. have the same method signatures (protocols are a way of helping enforce that, but there are other less performance-consequential ways of doing it too).

Now, it may be that a need does emerge as the design takes shape to have separate types, say, for different kinds of HTTP connections, that might need to be handled by the same program and have to be unified via a protocol to enable handling code to be shared across both. But this protocol should be discovered through the process of looking at the common features the different concrete types have, rather than designed up front.

Cheers,
Ben

7

I think protocols by their self evident definition of an interface only will allow the HTTP library implementors to provide a consistent and stable interface that we can internally reimplement (for instance, upgrade). In the face of ABI stability and resilience, protocols are going to live longest, I would think.

On the subject of middlewares, let's remember that one framework's middleware is another's shared data nightmare. I would fully expect many frameworks to wrap the raw HTTP requests or response streams in opinionated APIs of their own design.

Tom

···

Sent from my iPhone

On 22 Mar 2017, at 13:55, Paulo Faria via swift-server-dev <swift-server-dev@swift.org> wrote:

> Protocols in Swift exist primarily to allow code to operate on multiple different concrete implementations simultaneously – either with generics as constraints, or as existentials. In this case, there is no clear reason why someone would want to write code that handles two separate implementations of an HTTP API at the same time within a single program.

I completely agree with this. Using protocols to allow multiple implementations in this scenario would not be a solution for a technical problem. It would be, instead, a solution for a "human" problem which is basically the difficulty of reaching an agreement. In the end what we all want is the best solution technically.

On 22 March 2017 at 10:19, Ben Cohen via swift-server-dev <swift-server-dev@swift.org> wrote:

On Mar 20, 2017, at 11:11 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

I'd personally like to see us more explicitly explore protocols, I haven't seen anything so far that actually requires a concrete model and would perhaps be an easier way for us to reach consensus.

Hi Logan,

What is it about protocols that you feel would help here? In general, the relationship between concrete implementations and protocols should be that the former comes first, then the latter. Start with a concrete type, then as you move to multiple diverse implementations, factor out the common things that unify those types.

Protocols in Swift exist primarily to allow code to operate on multiple different concrete implementations simultaneously – either with generics as constraints, or as existentials. In this case, there is no clear reason why someone would want to write code that handles two separate implementations of an HTTP API at the same time within a single program. Protocols are not necessary in order to “swap” implementation A for implementation B. All that is needed for this is that the two implementations be source compatible i.e. have the same method signatures (protocols are a way of helping enforce that, but there are other less performance-consequential ways of doing it too).

Now, it may be that a need does emerge as the design takes shape to have separate types, say, for different kinds of HTTP connections, that might need to be handled by the same program and have to be unified via a protocol to enable handling code to be shared across both. But this protocol should be discovered through the process of looking at the common features the different concrete types have, rather than designed up front.

Cheers,
Ben

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

8

Ben and others,

One thing I'd like to address first is middleware. People keep bringing it
up, and I see this as absolutely something that the serverside group has no
responsibility in creating or defining paradigms for. The focus of this
group should be lower and allow frameworks to define how their middleware
should be defined and act. Let's remember that many frameworks exist
because there are many ideas on how a framework should look, for now, let's
focus on parsing and serializing http. There's no reason to add additional
responsibilities to a group already in disagreement.

On protocols, if this was a private project, I'd say yes, let's use
something concrete and move to protocols when there are multiple use cases.
In this case it's not, it's a library, and there are already multiple use
cases. It is intended to be consumed by multiple different
consumers/frameworks immediately. Already we have multiple stakeholders in
the group with very different ideas and use cases for the model,
protocolizing this would allow us to focus on parsing and let frameworks
choose how to model. By opting for generics constrained to our protocol
internally, the compiler will inline the code and give us pretty good
performance, in our generic constrained code, it was just as fast, or
faster than it's model counterparts.

Here's an example of a protocol required to parse a request

protocol HTTPMessage {
    init(status: String, version: String, method: String, headers: [String:
String], etc...
}

It doesn't need to be anything particularly complex, and we could include
easily the inverse as well for the serializer. This could be a distinct
protocol even for use cases we might not consider.

I'd like to just put up a quick reminder that we're not building a
framework here, we're building core apis that are intended to be flexible
and useful for framework developers. With something like this, we could
easily conform foundation's URLRequest or NSURLRequest by default so basic
users could use those built in without much effort, but framework
developers wouldn't be stuck constantly dealing with non-native objects or
doing unnecessary conversions.

This doesn't take away from anything that has been proposed so far, and
creates additional capabilities and flexibilities for a system that is
intended to be just that ... flexible. In general, I think it's important
for us to remind ourselves what we're building and that creating the most
flexible possible code will make the library more lasting and allow users
to create unique and interesting interactions that we might not have even
considered in our narrow scope. We want to support and encourage this
creativity, not stifle it, wherever possible. Looking forward to working
through more of this.

- Logan

···

On Wed, Mar 22, 2017 at 6:41 PM Swizzlr via swift-server-dev < swift-server-dev@swift.org> wrote:

I think protocols by their self evident definition of an interface only
will allow the HTTP library implementors to provide a consistent and stable
interface that we can internally reimplement (for instance, upgrade). In
the face of ABI stability and resilience, protocols are going to live
longest, I would think.

On the subject of middlewares, let's remember that one framework's
middleware is another's shared data nightmare. I would fully expect many
frameworks to wrap the raw HTTP requests or response streams in opinionated
APIs of their own design.

Tom

Sent from my iPhone

On 22 Mar 2017, at 13:55, Paulo Faria via swift-server-dev < > swift-server-dev@swift.org> wrote:

> Protocols in Swift exist primarily to allow code to operate on multiple
different concrete implementations simultaneously – either with generics as
constraints, or as existentials. In this case, there is no clear reason why
someone would want to write code that handles two separate implementations
of an HTTP API at the same time within a single program.

I completely agree with this. Using protocols to allow multiple
implementations in this scenario would not be a solution for a technical
problem. It would be, instead, a solution for a "human" problem which is
basically the difficulty of reaching an agreement. In the end what we all
want is the best solution technically.

On 22 March 2017 at 10:19, Ben Cohen via swift-server-dev < > swift-server-dev@swift.org> wrote:

On Mar 20, 2017, at 11:11 PM, Logan Wright via swift-server-dev < > swift-server-dev@swift.org> wrote:

I'd personally like to see us more explicitly explore protocols, I haven't
seen anything so far that actually requires a concrete model and would
perhaps be an easier way for us to reach consensus.

Hi Logan,

What is it about protocols that you feel would help here? In general, the
relationship between concrete implementations and protocols should be that
the former comes first, then the latter. Start with a concrete type, then
as you move to multiple diverse implementations, factor out the common
things that unify those types.

Protocols in Swift exist primarily to allow code to operate on multiple
different concrete implementations simultaneously – either with generics as
constraints, or as existentials. In this case, there is no clear reason why
someone would want to write code that handles two separate implementations
of an HTTP API at the same time within a single program. Protocols are not
necessary in order to “swap” implementation A for implementation B. All
that is needed for this is that the two implementations be source
compatible i.e. have the same method signatures (protocols are a way of
helping enforce that, but there are other less performance-consequential
ways of doing it too).

Now, it may be that a need does emerge as the design takes shape to have
separate types, say, for different kinds of HTTP connections, that might
need to be handled by the same program and have to be unified via a
protocol to enable handling code to be shared across both. But this
protocol should be discovered through the process of looking at the common
features the different concrete types have, rather than designed up front.

Cheers,
Ben

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

9

Here's an example of a protocol required to parse a request

I agree in principle, thinking in protocols is a good way of defining what an API should look like and how it should interact.

protocol HTTPMessage {
    init(status: String, version: String, method: String, headers: [String: String], etc...
}

This is a good example, in that it brings up a number of other questions. For example: the 'status' is a String here. Why not an enum or an int?

In addition, the headers are a dictionary of [String:String]. However, HTTP headers can be repeated and it's possible to get the first or a list of values for the repeated header, so using a dictionary isn't sufficient. In addition the headers dictionary has an implicit assumption about the fact they're stored as values (for example). You'd need to have a separate protocol for representing the headers in order to permit multiple implementations.

Discussing the protocols helps understand the problems from two perspectives; from an end user's view of the software, and from a framework implementor's view of the software.

It doesn't need to be anything particularly complex, and we could include easily the inverse as well for the serializer. This could be a distinct protocol even for use cases we might not consider.

I'd like to just put up a quick reminder that we're not building a framework here, we're building core apis that are intended to be flexible and useful for framework developers. With something like this, we could easily conform foundation's URLRequest or NSURLRequest by default so basic users could use those built in without much effort, but framework developers wouldn't be stuck constantly dealing with non-native objects or doing unnecessary conversions.

This doesn't take away from anything that has been proposed so far, and creates additional capabilities and flexibilities for a system that is intended to be just that ... flexible. In general, I think it's important for us to remind ourselves what we're building and that creating the most flexible possible code will make the library more lasting and allow users to create unique and interesting interactions that we might not have even considered in our narrow scope. We want to support and encourage this creativity, not stifle it, wherever possible. Looking forward to working through more of this.

Alex

···

On 23 Mar 2017, at 09:51, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

10

Please remove me from this mailing list.

Thanks.

My GitHub: https://github.com/iGaoMing/

Local Time: March 23, 2017 5:51 PM
UTC Time: March 23, 2017 9:51 AM
swift-server-dev <swift-server-dev@swift.org>

Ben and others,

One thing I'd like to address first is middleware. People keep bringing it up, and I see this as absolutely something that the serverside group has no responsibility in creating or defining paradigms for. The focus of this group should be lower and allow frameworks to define how their middleware should be defined and act. Let's remember that many frameworks exist because there are many ideas on how a framework should look, for now, let's focus on parsing and serializing http. There's no reason to add additional responsibilities to a group already in disagreement.

On protocols, if this was a private project, I'd say yes, let's use something concrete and move to protocols when there are multiple use cases. In this case it's not, it's a library, and there are already multiple use cases. It is intended to be consumed by multiple different consumers/frameworks immediately. Already we have multiple stakeholders in the group with very different ideas and use cases for the model, protocolizing this would allow us to focus on parsing and let frameworks choose how to model. By opting for generics constrained to our protocol internally, the compiler will inline the code and give us pretty good performance, in our generic constrained code, it was just as fast, or faster than it's model counterparts.

Here's an example of a protocol required to parse a request


protocol HTTPMessage {
init(status: String, version: String, method: String, headers: [String: String], etc...
}

It doesn't need to be anything particularly complex, and we could include easily the inverse as well for the serializer. This could be a distinct protocol even for use cases we might not consider.

I'd like to just put up a quick reminder that we're not building a framework here, we're building core apis that are intended to be flexible and useful for framework developers. With something like this, we could easily conform foundation's URLRequest or NSURLRequest by default so basic users could use those built in without much effort, but framework developers wouldn't be stuck constantly dealing with non-native objects or doing unnecessary conversions.

This doesn't take away from anything that has been proposed so far, and creates additional capabilities and flexibilities for a system that is intended to be just that ... flexible. In general, I think it's important for us to remind ourselves what we're building and that creating the most flexible possible code will make the library more lasting and allow users to create unique and interesting interactions that we might not have even considered in our narrow scope. We want to support and encourage this creativity, not stifle it, wherever possible. Looking forward to working through more of this.

- Logan

I think protocols by their self evident definition of an interface only will allow the HTTP library implementors to provide a consistent and stable interface that we can internally reimplement (for instance, upgrade). In the face of ABI stability and resilience, protocols are going to live longest, I would think.

On the subject of middlewares, let's remember that one framework's middleware is another's shared data nightmare. I would fully expect many frameworks to wrap the raw HTTP requests or response streams in opinionated APIs of their own design.

Tom

···

-------- Original Message --------
Subject: Re: [swift-server-dev] Next HTTP API meeting
From: swift-server-dev@swift.org
To: Swizzlr <me@swizzlr.co>, Paulo Faria <paulo@zewo.io>
On Wed, Mar 22, 2017 at 6:41 PM Swizzlr via swift-server-dev <swift-server-dev@swift.org> wrote:

Sent from my iPhone

On 22 Mar 2017, at 13:55, Paulo Faria via swift-server-dev <swift-server-dev@swift.org> wrote:

Protocols in Swift exist primarily to allow code to operate on multiple different concrete implementations simultaneously – either with generics as constraints, or as existentials. In this case, there is no clear reason why someone would want to write code that handles two separate implementations of an HTTP API at the same time within a single program.

I completely agree with this. Using protocols to allow multiple implementations in this scenario would not be a solution for a technical problem. It would be, instead, a solution for a "human" problem which is basically the difficulty of reaching an agreement. In the end what we all want is the best solution technically.

On 22 March 2017 at 10:19, Ben Cohen via swift-server-dev <swift-server-dev@swift.org> wrote:

On Mar 20, 2017, at 11:11 PM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

I'd personally like to see us more explicitly explore protocols, I haven't seen anything so far that actually requires a concrete model and would perhaps be an easier way for us to reach consensus.

Hi Logan,

What is it about protocols that you feel would help here? In general, the relationship between concrete implementations and protocols should be that the former comes first, then the latter. Start with a concrete type, then as you move to multiple diverse implementations, factor out the common things that unify those types.

Protocols in Swift exist primarily to allow code to operate on multiple different concrete implementations simultaneously – either with generics as constraints, or as existentials. In this case, there is no clear reason why someone would want to write code that handles two separate implementations of an HTTP API at the same time within a single program. Protocols are not necessary in order to “swap” implementation A for implementation B. All that is needed for this is that the two implementations be source compatible i.e. have the same method signatures (protocols are a way of helping enforce that, but there are other less performance-consequential ways of doing it too).

Now, it may be that a need does emerge as the design takes shape to have separate types, say, for different kinds of HTTP connections, that might need to be handled by the same program and have to be unified via a protocol to enable handling code to be shared across both. But this protocol should be discovered through the process of looking at the common features the different concrete types have, rather than designed up front.

Cheers,
Ben

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

11

Okay, but one of the major complaints I see in the GitHub thread is that some people want to use a reference type, and it seems like some people may want protocols so they can make that decision for themselves. I don't think that's going to work, though. If the protocol has any setters or mutating methods, then it will need to document whether it expects value or reference semantics; otherwise nobody will be able to know how to use those members properly.

(Also, if that really is the entire `HTTPMessage` protocol—just an initializer—why not have users pass in a closure with that signature instead? They could easily pass `MyFramework.HTTPMessage.init` if they want, but they would have other choices too.)

As for the performance aspect of value types, I don't quite see the concern. If we find that performance is a problem, we can switch to a copy-on-write implementation, where a public struct wraps a private class which actually contains the data; that should be exactly equivalent for ARC purposes. Choose value or reference based on the semantics you want; if the performance characteristics are wrong, you can then begin to optimize.

···

On Mar 23, 2017, at 2:51 AM, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

Already we have multiple stakeholders in the group with very different ideas and use cases for the model, protocolizing this would allow us to focus on parsing and let frameworks choose how to model. By opting for generics constrained to our protocol internally, the compiler will inline the code and give us pretty good performance, in our generic constrained code, it was just as fast, or faster than it's model counterparts.

Here's an example of a protocol required to parse a request

protocol HTTPMessage {
    init(status: String, version: String, method: String, headers: [String: String], etc...
}

It doesn't need to be anything particularly complex, and we could include easily the inverse as well for the serializer. This could be a distinct protocol even for use cases we might not consider.

--
Brent Royal-Gordon
Architechies

12

I agree. Let's not be too picky on Logan's example. It's just an example. :)

I personally think that http header/components are better define as Collection<HTTPHeader>. In fact it can even be Collection<HTTPComponent> where HTTPComponent is an enum of .statusline, .header, .data but whatever, detail implementation is not important at this point.

In general, I think protocol is the best for the job. In fact if we use protocol we probably don't even worry about value/reference semantic problem here.

Just leave it and let the user decide what they want to use. We really only need an abstract interface to implement a HTTP parser, this is what protocol for in the first place.

Someone might prefer an array to store headers, someone might prefer a linked list (or other custom data structures). We really don't have to care.

Michael

···

Sent from my iPhone

On Mar 23, 2017, at 3:00 AM, Alex Blewitt via swift-server-dev <swift-server-dev@swift.org> wrote:

On 23 Mar 2017, at 09:51, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

Here's an example of a protocol required to parse a request

I agree in principle, thinking in protocols is a good way of defining what an API should look like and how it should interact.

protocol HTTPMessage {
   init(status: String, version: String, method: String, headers: [String: String], etc...
}

This is a good example, in that it brings up a number of other questions. For example: the 'status' is a String here. Why not an enum or an int?

In addition, the headers are a dictionary of [String:String]. However, HTTP headers can be repeated and it's possible to get the first or a list of values for the repeated header, so using a dictionary isn't sufficient. In addition the headers dictionary has an implicit assumption about the fact they're stored as values (for example). You'd need to have a separate protocol for representing the headers in order to permit multiple implementations.

Discussing the protocols helps understand the problems from two perspectives; from an end user's view of the software, and from a framework implementor's view of the software.

It doesn't need to be anything particularly complex, and we could include easily the inverse as well for the serializer. This could be a distinct protocol even for use cases we might not consider.

I'd like to just put up a quick reminder that we're not building a framework here, we're building core apis that are intended to be flexible and useful for framework developers. With something like this, we could easily conform foundation's URLRequest or NSURLRequest by default so basic users could use those built in without much effort, but framework developers wouldn't be stuck constantly dealing with non-native objects or doing unnecessary conversions.

This doesn't take away from anything that has been proposed so far, and creates additional capabilities and flexibilities for a system that is intended to be just that ... flexible. In general, I think it's important for us to remind ourselves what we're building and that creating the most flexible possible code will make the library more lasting and allow users to create unique and interesting interactions that we might not have even considered in our narrow scope. We want to support and encourage this creativity, not stifle it, wherever possible. Looking forward to working through more of this.

Alex
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

13

Alex,

Great discussion points! I just churned out some quick types that could be
used, there's some types that I think are pretty unambiguous like
HTTPMethod, but something like headers gets more complex very quickly.

Historically swift frameworks have tried tons of iterations of HTTPHeaders,
but at the end of the day, every single one of them created significantly
more problems than they solved. They were either unbearably slow, or so
confusing to end users that every day was spent explaining them. URLRequest
in Foundation has also opted for what amounts to [String: String] as well.
In practice, it's probably something closer to [caseInsensitive:
[String?]?] I believe. At vapor we have [HeaderKey: String] with some
literal convertibles which has been pretty easy to interact with.

The reason for the header model was a consideration of how HTTPHeaders are
parsed. We could debate about usefulness to end user and best model for
that, but instead let's leave that up to frameworks. Let's not focus on
that.

Since this is a parser, I'd opt for something more general like `[String:
String?]` (would need to check on the optional there), because that's how
the parser will likely receive it, ie:

Key: Value\r\n
Other-Key: OtherValue\r\n

Then inside the initializer, the model would modify it, create arrays or
custom headers models as they see fit. This allows the server side apis to
opt out out of taking too many opinions and lets frameworks decide what's
best for them. Many want simple [String: String], at Vapor we use
[HeaderKey: String] for case insensitivity, some might want something
that's considered more "correct" as [CaseInsensitive: [String?]?] or
something.

All of these are arguably valid, so I think taking the approach of Parser's
perspective and how things are received would be a good paradigm to strive
towards.

The one thing missing in my example is order, which perhaps `[(String,
String?)]` would be better if there are significant implications for order
here to an end model.

- Logan

···

On Thu, Mar 23, 2017 at 11:00 AM Alex Blewitt <alblue@apple.com> wrote:

> On 23 Mar 2017, at 09:51, Logan Wright via swift-server-dev < > swift-server-dev@swift.org> wrote:
>
> Here's an example of a protocol required to parse a request

I agree in principle, thinking in protocols is a good way of defining what
an API should look like and how it should interact.

> ```
> protocol HTTPMessage {
> init(status: String, version: String, method: String, headers:
[String: String], etc...
> }
> ```

This is a good example, in that it brings up a number of other questions.
For example: the 'status' is a String here. Why not an enum or an int?

In addition, the headers are a dictionary of [String:String]. However,
HTTP headers can be repeated and it's possible to get the first or a list
of values for the repeated header, so using a dictionary isn't sufficient.
In addition the headers dictionary has an implicit assumption about the
fact they're stored as values (for example). You'd need to have a separate
protocol for representing the headers in order to permit multiple
implementations.

Discussing the protocols helps understand the problems from two
perspectives; from an end user's view of the software, and from a framework
implementor's view of the software.

> It doesn't need to be anything particularly complex, and we could
include easily the inverse as well for the serializer. This could be a
distinct protocol even for use cases we might not consider.
>
> I'd like to just put up a quick reminder that we're not building a
framework here, we're building core apis that are intended to be flexible
and useful for framework developers. With something like this, we could
easily conform foundation's URLRequest or NSURLRequest by default so basic
users could use those built in without much effort, but framework
developers wouldn't be stuck constantly dealing with non-native objects or
doing unnecessary conversions.
>
> This doesn't take away from anything that has been proposed so far, and
creates additional capabilities and flexibilities for a system that is
intended to be just that ... flexible. In general, I think it's important
for us to remind ourselves what we're building and that creating the most
flexible possible code will make the library more lasting and allow users
to create unique and interesting interactions that we might not have even
considered in our narrow scope. We want to support and encourage this
creativity, not stifle it, wherever possible. Looking forward to working
through more of this.

Alex

14

I'm with Brent a 100%. As discussed before in Open Swift, the embryo that
lead to the Swift Server APIs Work Group, using a protocol for the HTTP
request/response is not the best option because of the value type/reference
type duality. People implementing an HTTP middleware, for example, would
have to decide themselves which type semantics they're expecting the
underlying implementation to provide. This means that code written with one
semantics in mind would present bugs if used with an implementation that
uses the other implementation. Of course we could restrict the protocol to
reference types by making it a class protocol, but if we're doing that we
might as well just go with classes. So the best solution is to go with
concrete types. I prefer value type semantics, and to be honest, after
reading the memory ownership manifesto, maybe move semantics would be the
best option for HTTP request/response, but since we don't have it yet, I
think we should go with value types for now.

···

On Mon, Mar 27, 2017, 07:40 Brent Royal-Gordon <brent@architechies.com> wrote:

On Mar 23, 2017, at 2:51 AM, Logan Wright via swift-server-dev < > swift-server-dev@swift.org> wrote:

Already we have multiple stakeholders in the group with very different
ideas and use cases for the model, protocolizing this would allow us to
focus on parsing and let frameworks choose how to model. By opting for
generics constrained to our protocol internally, the compiler will inline
the code and give us pretty good performance, in our generic constrained
code, it was just as fast, or faster than it's model counterparts.

Here's an example of a protocol required to parse a request

protocol HTTPMessage {
    init(status: String, version: String, method: String, headers:
[String: String], etc...
}

It doesn't need to be anything particularly complex, and we could include
easily the inverse as well for the serializer. This could be a distinct
protocol even for use cases we might not consider.

Okay, but one of the major complaints I see in the GitHub thread is that
some people want to use a reference type, and it seems like some people may
want protocols so they can make that decision for themselves. I don't think
that's going to work, though. If the protocol has any setters or mutating
methods, then it will need to document whether it expects value or
reference semantics; otherwise nobody will be able to know how to use those
members properly.

(Also, if that really is the entire `HTTPMessage` protocol—just an
initializer—why not have users pass in a closure with that signature
instead? They could easily pass `MyFramework.HTTPMessage.init` if they
want, but they would have other choices too.)

As for the performance aspect of value types, I don't quite see the
concern. If we find that performance is a problem, we can switch to a
copy-on-write implementation, where a public struct wraps a private class
which actually contains the data; that should be exactly equivalent for ARC
purposes. Choose value or reference based on the semantics you want; if the
performance characteristics are wrong, you can then begin to optimize.

--
Brent Royal-Gordon
Architechies

15

Alex,

Since this is a parser, I'd opt for something more general like `[String: String?]` (would need to check on the optional there), because that's how the parser will likely receive it, ie:

Key: Value\r\n
Other-Key: OtherValue\r\n

The problem is that the HTTP spec permits:

Key: Value\r\n
Other-Key: OtherValue\r\n
Key: Different Value\r\n

and this isn't possible with a single dictionary as an implementation. So instead of constraining the implementation to a particular assumption about how implementations may need to use it, define the API for how you want to interact with it, e.g.:

HttpServletRequest (Java EE 6 ) <HttpServletRequest (Java EE 6 );
HttpServletRequest (Java EE 6 ) <HttpServletRequest (Java EE 6 );
HttpServletRequest (Java EE 6 ) <HttpServletRequest (Java EE 6 );

In other words, make the simple case easy but permit the additional cases as well.

Then inside the initializer, the model would modify it, create arrays or custom headers models as they see fit. This allows the server side apis to opt out out of taking too many opinions and lets frameworks decide what's best for them. Many want simple [String: String], at Vapor we use [HeaderKey: String] for case insensitivity, some might want something that's considered more "correct" as [CaseInsensitive: [String?]?] or something.

Do you want/need [String:String?], or do you just want something as simple as a data structure that takes a String subscript and returns a String? In fact you can have two views (much like different views exist on strings for e.g. utf8 and utf16) that provide you the case sensitivity or not:

headers.caseSensitive["Cache-Control"]
headers.caseInsensitive["Cache-Control"]

One could always provide the default case-sensitive view on the headers as well:

headers["Cache-Control"]

That API, while it would be satisfied by [String:String?], doesn't imply that it must be [String:String?].

Furthermore it's worth noting that [String:String?] is not order preserving on the headers, while HTTP is an ordered sequence of such headers. I doubt there's anyone who would need to know that but it's an example of something that an API may need to provide that can't be done with a trivial implementation.

All of these are arguably valid, so I think taking the approach of Parser's perspective and how things are received would be a good paradigm to strive towards.

I'm not clear what you're referring to here; do you have an example?

The one thing missing in my example is order, which perhaps `[(String, String?)]` would be better if there are significant implications for order here to an end model.

This is why I think defining the headers to be <a specific data structure> is impractical, and using a protocol to define how you interact with them is more appropriate.

Alex

···

On 23 Mar 2017, at 10:18, Logan Wright <logan@qutheory.io> wrote:

16

May I ask how that would look like in your opinion? I was thinking about that too, and I find it rather hard to come up with a really good solution.

The desirable thing would be an enum, but such can’t be extended in Swift w/o an associated value. Like so

  enum HTTPMethod {
    case GET, REPORT, MKCALENDAR, CHECKOUT, …
    case ExtendedMethod(String)
  }

But I think this would be pretty weird for ‘late’ methods and you’d end up with code like

  switch method {
    case .get: ...
    case ExtendedMethod(“PATCH”): ...
  }

which will look really weird quickly. New methods do not come around every day, but more often than you might think (note explicit support for arbitrary ones in todays XHR, this used to support only GET/POST, hence the xhr-method header thing).

hh

···

On 23 Mar 2017, at 11:18, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

some types that I think are pretty unambiguous like HTTPMethod

17

If we have to go for concrete type, I will go for value type. But I’m not quite understand how the value/reference duality of protocol will led to bugs, in fact I think protocol implementation should be much safer because it explicitly mark if a member or method is mutating or not.

For the concern of middleware, I believe the most efficient and the best way is to pass the req/res as inout argument, this should make no much difference between passing a protocol/struct/class, even better, if the middleware does not require mutability of the res/req, for example a statistics middleware, we can simply not make the argument inout to prevent accidental mutation. In the end, It results a much safer api.

Just an example to show the idea:
…
protocol Request { … }

// add middleware that can mutate the content
func addMutatingMiddleware(middleware: (inout Request) -> ())

// add middleware that cannot mutate the content
// if Request is a class, there’s no way to prevent the middleware from mutating the content
fun addConstMiddleware(middleware: (Request) -> ())
...

···

On Mar 27, 2017, at 4:52 AM, Paulo Faria via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:

using a protocol for the HTTP request/response is not the best option because of the value type/reference type duality. People implementing an HTTP middleware, for example, would have to decide themselves which type semantics they're expecting the underlying implementation to provide. This means that code written with one semantics in mind would present bugs if used with an implementation that uses the other implementation. Of course we could restrict the protocol to reference types by making it a class protocol, but if we're doing that we might as well just go with classes. So the best solution is to go with concrete types. I prefer value type semantics, and to be honest, after reading the memory ownership manifesto, maybe move semantics would be the best option for HTTP request/response, but since we don't have it yet, I think we should go with value types for now

18

Hey Everyone,

I disagree with the premise that protocols inherently create bugs, but
would be open to constraining to classes if this alleviates concerns. If we
create protocols, this allows frameworks to implement concrete models with
whatever semantics they see fit. Any subsequent interaction from the
framework would have a concrete type to rely on.

In terms of what we're building here, I don't see middleware as much of a
responsibility since this should be the tools that a framework would use. I
think it's very important for all of us to remember that we're not building
a web framework. We're building the tooling to make a good starting point
for others to use in development of a web framework.

If people feel extremely strong that there needs to be a concrete type,
then I'd like to push for reference type as much as possible. As far as
reference vs value type, I haven't really heard an argument for value types
beyond what feels like a reaction to value types being the hip new hotness.
While yes, they're great in Swift, and there's tons of places that should
absolutely be modeled with value semantics, a request/response interaction
represents a single request and should definitely be a reference based
interaction.

In practice, we went through several model iterations and the value type
created very high levels of bugs and confusion for end users. The three
biggest problems we had were as follows:

- Greatly increased bug levels and confusion related to unexpected mutation
- Unnecessary code requirements added to every single passive access (ie:
middleware) increasing code bloat unnecessarily
- Extreme performance loss due to massive copy overhead

Each of these problems evaporated pretty instantaneously when moving to
reference types; it made it significantly easier to reason about for end
users.

Summary:

Would like to remind again for those that skipped above reading that our
goal is not to build a web framework here, but rather to build small tools
that make building frameworks slightly easier for library maintainers and
creators.

Ideally, we could abstract the memory semantics away and leave them up to
frameworks by utilizing protocols .. this is my preference. If however,
concrete models are required, I feel reference type is the ideal choice for
the reasons outlined above. I believe that a request/response interaction
is representative of singular instances that should be referenced by
various interactors as opposed to introducing more copy/mutation complexity
and a model that doesn't fit with the concept.

Looking forward to more discussion, thanks for participating everyone!

Logan

···

On Mon, Mar 27, 2017 at 12:52 PM Paulo Faria <paulo@zewo.io> wrote:

I'm with Brent a 100%. As discussed before in Open Swift, the embryo that
lead to the Swift Server APIs Work Group, using a protocol for the HTTP
request/response is not the best option because of the value type/reference
type duality. People implementing an HTTP middleware, for example, would
have to decide themselves which type semantics they're expecting the
underlying implementation to provide. This means that code written with one
semantics in mind would present bugs if used with an implementation that
uses the other implementation. Of course we could restrict the protocol to
reference types by making it a class protocol, but if we're doing that we
might as well just go with classes. So the best solution is to go with
concrete types. I prefer value type semantics, and to be honest, after
reading the memory ownership manifesto, maybe move semantics would be the
best option for HTTP request/response, but since we don't have it yet, I
think we should go with value types for now.

On Mon, Mar 27, 2017, 07:40 Brent Royal-Gordon <brent@architechies.com> > wrote:

On Mar 23, 2017, at 2:51 AM, Logan Wright via swift-server-dev < > swift-server-dev@swift.org> wrote:

Already we have multiple stakeholders in the group with very different
ideas and use cases for the model, protocolizing this would allow us to
focus on parsing and let frameworks choose how to model. By opting for
generics constrained to our protocol internally, the compiler will inline
the code and give us pretty good performance, in our generic constrained
code, it was just as fast, or faster than it's model counterparts.

Here's an example of a protocol required to parse a request

protocol HTTPMessage {
    init(status: String, version: String, method: String, headers:
[String: String], etc...
}

It doesn't need to be anything particularly complex, and we could include
easily the inverse as well for the serializer. This could be a distinct
protocol even for use cases we might not consider.

Okay, but one of the major complaints I see in the GitHub thread is that
some people want to use a reference type, and it seems like some people may
want protocols so they can make that decision for themselves. I don't think
that's going to work, though. If the protocol has any setters or mutating
methods, then it will need to document whether it expects value or
reference semantics; otherwise nobody will be able to know how to use those
members properly.

(Also, if that really is the entire `HTTPMessage` protocol—just an
initializer—why not have users pass in a closure with that signature
instead? They could easily pass `MyFramework.HTTPMessage.init` if they
want, but they would have other choices too.)

As for the performance aspect of value types, I don't quite see the
concern. If we find that performance is a problem, we can switch to a
copy-on-write implementation, where a public struct wraps a private class
which actually contains the data; that should be exactly equivalent for ARC
purposes. Choose value or reference based on the semantics you want; if the
performance characteristics are wrong, you can then begin to optimize.

--
Brent Royal-Gordon
Architechies

19

I think it might be lowercase to stick w/ the official Apple swift 3
guidelines, but definitely agree on some type of extensibility. Currently
I've been naming that 'other' in general. With that said, that's kind of
why I included a String in my proposal. If we can focus on the actual
components of the parser and let users define models as much as possible,
then it offloads a lot of our opinions into respective frameworks. In terms
of Method, here's how I currently have it for discussion:

enum Method {
    case get, post, put, patch
    case other(String)
}

I think in addition to adding new official methods (I might be wrong here,
need to double check) but I believe server/clients can define any method
they might want to use between them, so I completely agree with you that
whatever we have will definitely need some type of extensibility (whatever
we call it) to conform to spec.

···

On Thu, Mar 23, 2017 at 12:12 PM Helge Heß via swift-server-dev < swift-server-dev@swift.org> wrote:

On 23 Mar 2017, at 11:18, Logan Wright via swift-server-dev < > swift-server-dev@swift.org> wrote:
> some types that I think are pretty unambiguous like HTTPMethod

May I ask how that would look like in your opinion? I was thinking about
that too, and I find it rather hard to come up with a really good solution.

The desirable thing would be an enum, but such can’t be extended in Swift
w/o an associated value. Like so

  enum HTTPMethod {
    case GET, REPORT, MKCALENDAR, CHECKOUT, …
    case ExtendedMethod(String)
  }

But I think this would be pretty weird for ‘late’ methods and you’d end up
with code like

  switch method {
    case .get: ...
    case ExtendedMethod(“PATCH”): ...
  }

which will look really weird quickly. New methods do not come around every
day, but more often than you might think (note explicit support for
arbitrary ones in todays XHR, this used to support only GET/POST, hence the
xhr-method header thing).

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

20

Do you want/need [String:String?], or do you just want something as simple as a data structure that takes a String subscript and returns a String?

+1 for the latter (do not use a Dictionary, or if you do, do not use String as the key, as it has to be case insensitive)

A protocol could also just require the complex variant, say:

  func iterateValues(for key: String, valueCallback: (String)->Void)

and provide the convenience things, like

  subscript[key: String] -> String { … }

as extensions based on top of that.

In fact you can have two views (much like different views exist on strings for e.g. utf8 and utf16) that provide you the case sensitivity or not

-1 HTTP headers are always case insensitive:

  HTTP/1.1: HTTP Message
  "Field names are case-insensitive."

This is why I think defining the headers to be <a specific data structure> is impractical, and using a protocol to define how you interact with them is more appropriate.

+1. Well, I think a specific data structure may be OK, but it should not be a `Dictionary<String, String>`, just behave like one as a convenience.

hh

P.S.: I really don’t want to complicate the discussion, but doesn’t someone else feel that the overhead of String is way too big here (for such a core library) and that significant space and time savings could be accomplished by using enums and such? (Swift has no String.intern, right? - but even just a 64bit ptr could often almost fit all header keys + method)

···

On 23 Mar 2017, at 11:55, Alex Blewitt via swift-server-dev <swift-server-dev@swift.org> wrote: