[HTTP] Value vs Reference Types

Danappelxx · November 23, 2016, 9:07pm

Hello everyone!

I was unable to make the kick-off meeting for the HTTP sub-team, but I
looked over the meeting notes
<https://docs.google.com/document/d/1SWK0qBDi-9DeLJwHlcXcPU7h22JU-DWW8HTVuHbTQiw/edit>
and
found some topics that I think could use some more on-the-record discussion.

A few questions that I wanted to raise:

1. Do we want to use concrete types or protocols for Request/Response?
2. If we use concrete types, do we want value or reference semantics?
3. When is it more convenient to have reference semantics?
4. Are there problems that can't be solved with value semantics?

I would like to avoid bike-shedding, and I think this can be done by
providing real examples rather than just talking about the pros and cons.

···

--
Dan Appel

Danappelxx · November 23, 2016, 9:15pm

Hello everyone!

I was unable to make the kick-off meeting for the HTTP sub-team, but I looked over themeeting notes(https://docs.google.com/document/d/1SWK0qBDi-9DeLJwHlcXcPU7h22JU-DWW8HTVuHbTQiw/edit\)and found some topics that I think could use some more on-the-record discussion.

A few questions that I wanted to raise:

1. Do we want to use concrete types or protocols for Request/Response?
2. If we use concrete types, do we want value or reference semantics?
3. When is it more convenient to have reference semantics?
4. Are there problems that can't be solved with value semantics?

I would like to avoid bike-shedding, and I think this can be done by providing real examples rather than just talking about the pros and cons.
--
Dan Appel

My own responses:

1. Do we want to use concrete types or protocols for Request/Response?

When working on Open Swift <https://github.com/open-swift>, this was a hot topic since we believed that it would be unsafe to have a protocol that would allow both value and reference types. We arrived upon the `{Request|Response}Representable` pattern which worked but was a bit of a mess. Because of this, I would prefer concrete Request/Response types.

2. If we use concrete types, do we want value or reference semantics?

What I think makes this easier is that the "big four" have each taken a slightly different approach that can be used as a reference.
Zewo <https://github.com/Zewo/Zewo/blob/master/Modules/HTTP/Sources/HTTP/Message/Request.swift#L3> - struct, value semantics
Vapor <https://github.com/vapor/engine/blob/master/Sources/HTTP/Models/Request/Request.swift#L4> - closed class, reference semantics
Kitura <https://github.com/IBM-Swift/Kitura/blob/master/Sources/Kitura/RouterRequest.swift#L26> - closed class + has-a pattern, reference semantics
Perfect <https://github.com/PerfectlySoft/Perfect-HTTP/blob/master/Sources/HTTPRequest.swift#L25> - class protocol, reference semantics

Zewo is the outlier here, but I would like to note as a contributor to Zewo that we have not ran into situations where value semantics create an impassable roadblock.

To me, it makes sense to pass them around as values since they don't have any logic of their own. Requests/Responses can't send themselves, they can only read and modified. It also gives me as a user more safety to pass them around since I know that they won't be modified implicitly.

Take the following pseudo-code as an example:

HTTPServer.onRequest { request in
    print(request.sourceIp)
    HTTPClient.send(request)
    print(request.sourceIp)

}

With reference semantics, there is no guarantee that sourceIp will be the same before and after sending off the request. After all, it could make sense for the HTTPClient to modify the sourceIp before sending off the request. This of course a contrived example, but the point stands.

Anyway, I think it would be great if we could have people talk about their own experiences.

3. When is it more convenient to have reference semantics?

In the middleware chain architecture that we decided on in Zewo (the other ones have something similar), it can be convenient to modify requests in the responder and have that reflect in the middleware. I think this problem is best solved with `inout` parameters rather than reference types, but that is my personal opinion.

4. Are there problems that can't be solved with value semantics?

I haven't found one that we can't solve, but I'm sure others can bring something interesting to the table.

···

--

Dan Appel

JetForMe · November 23, 2016, 10:04pm

Have you looked at Swifter (https://github.com/httpswift/swifter\)? It uses value semantics, and interestingly, uses an enum for HttpResponse.

Not sure if it's the best, but I'm using it to embed a server in an iOS app, and it was very easy to integrate. I had to make some modifications to the source to support a few extra response types and behaviors, but that may have been due in part to my own relative lack of expertise in Swift.

My server-side experience is primarily Java, which has no value semantics. I've also written servers in Swift and node.js. So far, I much prefer the Java/Spring environment for writing anything but the most trivial web apps. As long as we can get that level of expressiveness, performance, flexibility, and ease, I'll be happy. Not sure it's possible without a great deal more language support for introspection (@Annotations).

···

On Nov 23, 2016, at 13:15 , Dan Appel via swift-server-dev <swift-server-dev@swift.org> wrote:

>2. If we use concrete types, do we want value or reference semantics?

What I think makes this easier is that the "big four" have each taken a slightly different approach that can be used as a reference.
Zewo - struct, value semantics
Vapor - closed class, reference semantics
Kitura - closed class + has-a pattern, reference semantics
Perfect - class protocol, reference semantics

--
Rick Mann
rmann@latencyzero.com

Danappelxx · November 23, 2016, 9:19pm

Woah, that formatting got really messed up. Let me try again...

My own responses:

1. Do we want to use concrete types or protocols for Request/Response?

When working on Open Swift <https://github.com/open-swift>, this was a hot
topic since we believed that it would be unsafe to have a protocol that
would allow both value and reference types. We arrived upon the
`{Request|Response}Representable` pattern which worked but was a bit of a
mess. Because of this, I would prefer concrete Request/Response types.

2. If we use concrete types, do we want value or reference semantics?

What I think makes this easier is that the "big four" have each taken a
slightly different approach that can be used as a reference.

Zewo
<https://github.com/Zewo/Zewo/blob/master/Modules/HTTP/Sources/HTTP/Message/Request.swift#L3>
- struct,
value semantics

Vapor
<https://github.com/vapor/engine/blob/master/Sources/HTTP/Models/Request/Request.swift#L4>
- closed
class, reference semantics

Kitura
<https://github.com/IBM-Swift/Kitura/blob/master/Sources/Kitura/RouterRequest.swift#L26>

···

-
closed class + has-a pattern, reference semantics

Perfect
<https://github.com/PerfectlySoft/Perfect-HTTP/blob/master/Sources/HTTPRequest.swift#L25>
-
class protocol, reference semantics

Zewo is the outlier here, but I would like to note as a contributor to Zewo
that we have not ran into situations where value semantics create an
impassable roadblock.

To me, it makes sense to pass them around as values since they don't have
any logic of their own. Requests/Responses can't send themselves, they can
only read and modified. It also gives me as a user more safety to pass them
around since I know that they won't be modified implicitly.

Take the following pseudo-code as an example:

HTTPServer.onRequest { request in

print(request.sourceIp)

HTTPClient.send(request)

print(request.sourceIp)

}

With reference semantics, there is no guarantee that sourceIp will be the
same before and after sending off the request. After all, it *could* make
sense for the HTTPClient to modify the sourceIp before sending off the
request. This of course a contrived example, but the point stands.

Anyway, I think it would be great if we could have people talk about their
own experiences.

3. When is it more convenient to have reference semantics?

In the middleware chain architecture that we decided on in Zewo (the other
ones have something similar), it can be convenient to modify requests in
the responder and have that reflect in the middleware. I think this problem
is best solved with `inout` parameters rather than reference types, but
that is my personal opinion.

4. Are there problems that can't be solved with value semantics?

I haven't found any, but I'm sure others can bring something interesting to
the table.

On Wed, Nov 23, 2016 at 1:07 PM Dan Appel <dan.appel00@gmail.com> wrote:

Hello everyone!

I was unable to make the kick-off meeting for the HTTP sub-team, but I
looked over the meeting notes
<https://docs.google.com/document/d/1SWK0qBDi-9DeLJwHlcXcPU7h22JU-DWW8HTVuHbTQiw/edit> and
found some topics that I think could use some more on-the-record discussion.

A few questions that I wanted to raise:

1. Do we want to use concrete types or protocols for Request/Response?
2. If we use concrete types, do we want value or reference semantics?
3. When is it more convenient to have reference semantics?
4. Are there problems that can't be solved with value semantics?

I would like to avoid bike-shedding, and I think this can be done by
providing real examples rather than just talking about the pros and cons.
--
Dan Appel

--
Dan Appel

Helge_Hess1 · November 23, 2016, 10:45pm

1. Do we want to use concrete types or protocols for Request/Response?

When working on Open Swift, this was a hot topic since we believed that it would be unsafe to have a protocol that would allow both value and reference types.

...

2. If we use concrete types, do we want value or reference semantics?

Kinda what you said before this decision is unrelated to protocol vs concrete type. I suppose protocols may make some sense, so that they can be backed by different mechanisms (say libcurl or http_parser, etc).

E.g. quite often you don’t really need to decode all fields of an HTTP header but just specific fields of it, or you need the fields just once. In such cases it may not be necessary to proactively waste space on a hash-map and actual strings for such and instead just keep the buffer containing the data and do things on demand (aka don’t parse/load stuff you don’t use).
All I’m saying is that there may be different implementations for different uses cases.

BTW: It was also mentioned that it may be highly desirable to use Foundation’s NSHTTPURLRequest/NSHTTPResponse.

To me, it makes sense to pass them around as values

Unless you are talking about just the HTTP message head (which is not much more than a specialised dictionary), it doesn’t make any sense to me to pass them around as values.

The body's of the HTTP messages can be gigabytes big and won’t usually be stored in-memory at all or at least as-is. Processing state on the body stream needs to be passed around as a reference.

hh

···

On 23 Nov 2016, at 22:19, Dan Appel via swift-server-dev <swift-server-dev@swift.org> wrote:

Ben_Cohen · December 8, 2016, 2:04am

Hi Dan,

Thanks for spelling out these questions, I think they are a great starting point for a discussion. A few comments inline.

My own responses:

>1. Do we want to use concrete types or protocols for Request/Response?

When working on Open Swift <https://github.com/open-swift>, this was a hot topic since we believed that it would be unsafe to have a protocol that would allow both value and reference types.

Bear in mind that a protocol is more than just the methods and types it declares – it is also its documentation. For example, a number of protocols in the standard library state things like complexity requirements in their documenting comments. The language has no way of enforcing these, but your type does not truly “conform” to the protocol unless you adhere to them. Value semantics are similar – you can document that it is invalid to implement a protocol with reference semantics. So I don’t think this is a blocker to wanting to use protocols.

We arrived upon the `{Request|Response}Representable` pattern which worked but was a bit of a mess. Because of this, I would prefer concrete Request/Response types.

You could think of there as being 3 purposes to using protocols here, roughly in order of importance:

being able to write generic code
allowing different frameworks to interoperate
documenting what you need to implement

The first one is the only reason why a protocol must be included in a library, and the key question to ask when considering defining a protocol like this is “What common algorithms do you want to write across multiple different conforming types in your program”? (such as generic functions, including protocol extensions, or functions that take an existential if the protocol has no associated types).

This is distinct from wanting to be able to be able to choose from different library implementations of Request. You might want to choose between the Acme Inc Web Framework’s Request type, or some Swift-Server “official" Request type, but you never need to use both at once and write code spanning them. You just want to make sure that one can serve as a source-compatible “drop-in” replacement for the other in your code. This doesn’t mean you can’t write your own extensions – but you would extend the concrete Request not a RequestRepresentable protocol.

Next, it’s possible that there might be a collection of 3rd-party frameworks out there that don’t define Request, but want to be able to write methods that take or extend multiple possible Request implementations. This seems a bit unlikely in the case of these types, more likely in other cases like networking, so it’s kind of a what-if scenario where there are both multiple popular implementations of Request, and various frameworks that want to interact with them. Anyone can add a conformance to anything, so those frameworks can define a protocol of their own with a subset of the functionality they need, and then just extend the popular implementations to conform to it. If this gets really common, at that point it might be worth creating an official protocol for everyone to share – but this can be done later, doesn’t have to be done up-front.

Finally, if you do expect multiple implementations and want people to be able to swap them in and out when they choose, the protocol can serve to document what methods and properties you are expected to implement to be “source compatible". This can be done in documentation instead, the benefit of the protocol being it helps the library developer ensure they’ve got all the signatures right etc. But this isn’t something you expose to users, it’s something on the side to help implementors.

Based on all the above, it seems like there isn’t a pressing need for a protocol for these types right now and initial designs should focus on a concrete implementation until one emerges, if only to avoid premature generalization. Useful protocols tend to be discovered, rather than designed, through a desire to share common operations on different concrete types.

>2. If we use concrete types, do we want value or reference semantics?

What I think makes this easier is that the "big four" have each taken a slightly different approach that can be used as a reference.
Zewo <https://github.com/Zewo/Zewo/blob/master/Modules/HTTP/Sources/HTTP/Message/Request.swift#L3> - struct, value semantics
Vapor <https://github.com/vapor/engine/blob/master/Sources/HTTP/Models/Request/Request.swift#L4> - closed class, reference semantics
Kitura <https://github.com/IBM-Swift/Kitura/blob/master/Sources/Kitura/RouterRequest.swift#L26> - closed class + has-a pattern, reference semantics
Perfect <https://github.com/PerfectlySoft/Perfect-HTTP/blob/master/Sources/HTTPRequest.swift#L25> - class protocol, reference semantics

Zewo is the outlier here, but I would like to note as a contributor to Zewo that we have not ran into situations where value semantics create an impassable roadblock.

To me, it makes sense to pass them around as values since they don't have any logic of their own. Requests/Responses can't send themselves, they can only read and modified. It also gives me as a user more safety to pass them around since I know that they won't be modified implicitly.

Take the following pseudo-code as an example:

HTTPServer.onRequest { request in
    print(request.sourceIp)
    HTTPClient.send(request)
    print(request.sourceIp)

}

With reference semantics, there is no guarantee that sourceIp will be the same before and after sending off the request. After all, it could make sense for the HTTPClient to modify the sourceIp before sending off the request. This of course a contrived example, but the point stands.

Not contrived at all, this is a perfect illustration of why reference semantics make it harder to reason about your code and identify the cause of bugs.

Anyway, I think it would be great if we could have people talk about their own experiences.

>3. When is it more convenient to have reference semantics?

Convenience is a double-edged thing. Pointers with possibly-null values, or integer indexes into Unicode strings, are often considered convenient. But that convenience comes with a hidden cost to correctness – unexpected nulls, accidentally indexing into the middle of a grapheme cluster etc. When making a proper effort to handle these things correctly, code quickly becomes less convenient, and less readable, compared to the alternatives.

It’s generally the style in Swift that correctness shouldn't be sacrificed for convenience, but when things work out well, convenience and ergonomics can be mutually reinforcing – the code is nice to use correctly, awkward to use incorrectly. For example, optionals that force you to handle nil help with correctness, but they have sugar like ?? or optional chaining to handle common patterns clearly and idiomatically, ! as a shorthand for asserting something is non-nil etc.

In the middleware chain architecture that we decided on in Zewo (the other ones have something similar), it can be convenient to modify requests in the responder and have that reflect in the middleware. I think this problem is best solved with `inout` parameters rather than reference types, but that is my personal opinion.

FWIW, this design view is also strongly held by those of us working on the Swift Standard Library. I also brought this up with several members of the Core Team and they also strongly felt that inout and value types was the general approach we should take with such types in Swift. The consensus there was that reference types really should be mostly used when identity of the value is important.

>4. Are there problems that can't be solved with value semantics?

I haven't found any, but I'm sure others can bring something interesting to the table.

Shared mutable state is one. With an unavoidably-shared resource, like a network connection or a handle to a window on a screen, reference semantics are often what you want.

···

On Nov 23, 2016, at 1:19 PM, Dan Appel via swift-server-dev <swift-server-dev@swift.org> wrote:

On Wed, Nov 23, 2016 at 1:07 PM Dan Appel <dan.appel00@gmail.com <mailto:dan.appel00@gmail.com>> wrote:
Hello everyone!

I was unable to make the kick-off meeting for the HTTP sub-team, but I looked over the meeting notes <https://docs.google.com/document/d/1SWK0qBDi-9DeLJwHlcXcPU7h22JU-DWW8HTVuHbTQiw/edit> and found some topics that I think could use some more on-the-record discussion.

A few questions that I wanted to raise:

1. Do we want to use concrete types or protocols for Request/Response?
2. If we use concrete types, do we want value or reference semantics?
3. When is it more convenient to have reference semantics?
4. Are there problems that can't be solved with value semantics?

I would like to avoid bike-shedding, and I think this can be done by providing real examples rather than just talking about the pros and cons.
--
Dan Appel
--
Dan Appel
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev

Danappelxx · November 23, 2016, 11:09pm

I think it's important to make the distinction between performance and
semantics, but just to address this...

Unless you are talking about just the HTTP message head (which is not much

more than a specialised dictionary), it doesn’t make any sense to me to
pass them around as values.

The body's of the HTTP messages can be gigabytes big and won’t usually be

stored in-memory at all or at least as-is. Processing state on the body
stream needs to be passed around as a reference.

The model we chose for Zewo is to have the request store both the head and
the body. I understand that you are worried about copying around requests
with large bodies. To that, I want to remind you that Swift's preferred way
to address this is through Copy-on-Write, which is taken advantage of
automatically when using arrays.

The other point you raise is that the body may be a stream, at which point
it is impossible to copy around. What we did here for Zewo is make the body
an enum
<https://github.com/Zewo/Zewo/blob/master/Modules/HTTP/Sources/HTTP/Message/Body.swift#L1>\.
So, if you want to return a file stream, you just do that. Copying the
message around will keep a reference to the same stream, but since its
immutable this isn't an issue.

I urge you to reconsider your stance. Just because something can be very
large doesn't immediately mean it should be a reference type.

···

On Wed, Nov 23, 2016 at 2:45 PM Helge Heß via swift-server-dev < swift-server-dev@swift.org> wrote:

On 23 Nov 2016, at 22:19, Dan Appel via swift-server-dev < > swift-server-dev@swift.org> wrote:
>> 1. Do we want to use concrete types or protocols for Request/Response?
>
> When working on Open Swift, this was a hot topic since we believed that
it would be unsafe to have a protocol that would allow both value and
reference types.
...
>> 2. If we use concrete types, do we want value or reference semantics?

Kinda what you said before this decision is unrelated to protocol vs
concrete type. I suppose protocols may make some sense, so that they can be
backed by different mechanisms (say libcurl or http_parser, etc).

E.g. quite often you don’t really need to decode all fields of an HTTP
header but just specific fields of it, or you need the fields just once. In
such cases it may not be necessary to proactively waste space on a hash-map
and actual strings for such and instead just keep the buffer containing the
data and do things on demand (aka don’t parse/load stuff you don’t use).
All I’m saying is that there may be different implementations for
different uses cases.

BTW: It was also mentioned that it may be highly desirable to use
Foundation’s NSHTTPURLRequest/NSHTTPResponse.

> To me, it makes sense to pass them around as values

Unless you are talking about just the HTTP message head (which is not much
more than a specialised dictionary), it doesn’t make any sense to me to
pass them around as values.

The body's of the HTTP messages can be gigabytes big and won’t usually be
stored in-memory at all or at least as-is. Processing state on the body
stream needs to be passed around as a reference.

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

--
Dan Appel

Helge_Hess1 · November 23, 2016, 11:29pm

I think it's important to make the distinction between performance and semantics, but just to address this...

>Unless you are talking about just the HTTP message head (which is not much more than a specialised dictionary), it doesn’t make any sense to me to pass them around as values.
>
>The body's of the HTTP messages can be gigabytes big and won’t usually be stored in-memory at all or at least as-is. Processing state on the body stream needs to be passed around as a reference.

...

The other point you raise is that the body may be a stream, at which point it is impossible to copy around.

The body of a HTTP message is by definition always a stream. You can opt to load the stream into a buffer (either memory or disk), but that is rarely what you want in server applications.
Almost certainly not for responses, but the requests are not really that different either (you want to support uploading photos/videos and such ...)

Copying the message around will keep a reference to the same stream, but since its immutable this isn't an issue.

Elaborate. If a stream is attached the immutability property of the message is essentially void and rendered pretty much useless? (as you can’t pass it around anymore as-if-new, that only makes sense if the body too is part of the immutability contract).

You also seem to ignore trailing headers which are only coming in while the HTTP body stream is being processed.

I urge you to reconsider your stance. Just because something can be very large doesn't immediately mean it should be a reference type.

Well, but this is what you are doing to? You are keeping the streams as a reference type. Why? ;-)

hh

···

On 24 Nov 2016, at 00:09, Dan Appel <dan.appel00@gmail.com> wrote:

steve_algernon · November 28, 2016, 8:32pm

A few notes in my head here:

- The NSURL* APIs lack the ability to generate an NSHTTPURLRequest / NSHTTPURLResponse from bytes - the http parser is buried in a CFNetwork API. (I'd like this to change and have an NSHTTPMessage -> NSHTTP(Request | Response)Message available.)

- NSURLResponse is the base class since the NSURL* APis are protocol agnostitic, so the NSURLSession APIs expose this - and everyone casts because in the end, they want an HTTP response. We should avoid this from the get-go.

- The underlying dictionary of (case insensitive) key to value strings of these objects has a major design flaw in that it does not deal with multiple headers with the same key (Set-Cookie) It does this by arbitrarily combining value strings with some deliminter (I think its a ",") The correct API would return a value as an array even if it only contains 1 element.

- The req/resp objects do not preserve parsed header order, which may be desirable. I'm considering adding:

-(NSArray *)headerKeyOrder;
-(NSArray *)headerValueArrayForKey:(NSString *)key;

- NSURLSession attempts to separate the NSURLRequest API from the existing -bodyData or -bodyStream methods. When performing an upload the bytes of the body may be sent multiple times (in the case where we never receive a response, or the connection closes during an upload, or a proxy is involved) and long ago we had to add a delegate to request a new body stream. The NSURLSession async convenience APIs work with data and stream objects external to the NSURLRequest, and in the stream case always defer to the delegate for an upload stream if data was not provided. This separation of header and body is consistent with an NSURLResponse being distinct from the received data. (Always requiring a stream be provided by a delegate guarantees that developers implement the method correctly so they continue to work in the edge cases that they can never actually test with!)

- NSHTTPURLRequest and NSHTTPURLResponse mutability is an issue that just makes for a messy API. Making these value classes makes sense - the headers are never large enough to warrant even COW semantics for the internal dictionary.

- The client headers are not complete until they're about to be put on the wire; the URL of a request is added as a Host: very late and it is not possible to see the entire header before it is sent. (This is a limitation of the NSURLSession / NSURLConnection APIs as it necessarily involved a delegate callout and subsequent performance hit.). This may be true of a server API as well, where configuration options from the server are added immediately before the response is written.

--sma

···

On Nov 23, 2016, at 2:45 PM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

BTW: It was also mentioned that it may be highly desirable to use Foundation’s NSHTTPURLRequest/NSHTTPResponse.

Chris_Bailey · December 12, 2016, 12:08pm

Thanks Ben. This is really useful.

Its probably worth also adding some information on the approach taken by
Foundation today for URLRequest/Response.

For outbound requests, Foundation provides both NSURLRequest (class) and
URLRequest (struct). In the case of URLRequest (struct), there is a mix of
types as it contains a getter that returns a stream class for the body -
and the with a number of warnings commented in the code:
"The stream is returned for examination only; it is not safe for
the caller to manipulate the stream in any way"
(

github.com

apple/swift-corelibs-foundation/blob/5ab46791996cb65079d02f98115cef6562d4819b/Foundation/URLRequest.swift#L187


      
          /// in done in an HTTP POST request.
          public var httpBody: Data? {
              get {
                  return _handle.map { $0.httpBody }
              }
              set {
                  _applyMutation { $0.httpBody = newValue }
              }
          }
          
          /// The stream is returned for examination only; it is
          /// not safe for the caller to manipulate the stream in any way.  Also
          /// note that the HTTPBodyStream and HTTPBody are mutually exclusive - only
          /// one can be set on a given request.  Also note that the body stream is
          /// preserved across copies, but is LOST when the request is coded via the
          /// NSCoding protocol
          public var httpBodyStream: InputStream? {
              get {
                  return _handle.map { $0.httpBodyStream }
              }
              set {

)
This is presumably because you shouldn't need to modify the stream for an
outbound request

For responses from outbound requests, only NSURLResponse (class) is
provided as the body is read from the stream.

This effectively give us a model where:
outbound can be a struct (you shouldn't need to touch the
stream)
inbound class (reading from stream)
For the server case, we need the mirror opposite of this - with incoming
requests being a class and the outgoing response potentially being a
struct.

This leads to two questions:
1. Do we think the Foundation model is correct?
2. Do we think it would be confusing to switch been classes and structs
when working with inbound vs. outbound requests, or do we think it would
enforce correct behaviour?

Chris

···

From: Ben Cohen via swift-server-dev <swift-server-dev@swift.org>
To: Dan Appel <dan.appel00@gmail.com>
Cc: "swift-server-dev@swift.org" <swift-server-dev@swift.org>
Date: 08/12/2016 02:04
Subject: Re: [swift-server-dev] [HTTP] Value vs Reference Types
Sent by: swift-server-dev-bounces@swift.org

Hi Dan,

Thanks for spelling out these questions, I think they are a great starting
point for a discussion. A few comments inline.

On Nov 23, 2016, at 1:19 PM, Dan Appel via swift-server-dev < swift-server-dev@swift.org> wrote:

My own responses:

1. Do we want to use concrete types or protocols for Request/Response?

When working on Open Swift, this was a hot topic since we believed that it
would be unsafe to have a protocol that would allow both value and
reference types.

Bear in mind that a protocol is more than just the methods and types it
declares – it is also its documentation. For example, a number of
protocols in the standard library state things like complexity
requirements in their documenting comments. The language has no way of
enforcing these, but your type does not truly “conform” to the protocol
unless you adhere to them. Value semantics are similar – you can document
that it is invalid to implement a protocol with reference semantics. So I
don’t think this is a blocker to wanting to use protocols.

We arrived upon the `{Request|Response}Representable` pattern which worked
but was a bit of a mess. Because of this, I would prefer concrete
Request/Response types.

You could think of there as being 3 purposes to using protocols here,
roughly in order of importance:

being able to write generic code
allowing different frameworks to interoperate
documenting what you need to implement

The first one is the only reason why a protocol must be included in a
library, and the key question to ask when considering defining a protocol
like this is “What common algorithms do you want to write across multiple
different conforming types in your program”? (such as generic functions,
including protocol extensions, or functions that take an existential if
the protocol has no associated types).

This is distinct from wanting to be able to be able to choose from
different library implementations of Request. You might want to choose
between the Acme Inc Web Framework’s Request type, or some Swift-Server
“official" Request type, but you never need to use both at once and write
code spanning them. You just want to make sure that one can serve as a
source-compatible “drop-in” replacement for the other in your code. This
doesn’t mean you can’t write your own extensions – but you would extend
the concrete Request not a RequestRepresentable protocol.

Next, it’s possible that there might be a collection of 3rd-party
frameworks out there that don’t define Request, but want to be able to
write methods that take or extend multiple possible Request
implementations. This seems a bit unlikely in the case of these types,
more likely in other cases like networking, so it’s kind of a what-if
scenario where there are both multiple popular implementations of Request,
and various frameworks that want to interact with them. Anyone can add a
conformance to anything, so those frameworks can define a protocol of
their own with a subset of the functionality they need, and then just
extend the popular implementations to conform to it. If this gets really
common, at that point it might be worth creating an official protocol for
everyone to share – but this can be done later, doesn’t have to be done
up-front.

Finally, if you do expect multiple implementations and want people to be
able to swap them in and out when they choose, the protocol can serve to
document what methods and properties you are expected to implement to be
“source compatible". This can be done in documentation instead, the
benefit of the protocol being it helps the library developer ensure
they’ve got all the signatures right etc. But this isn’t something you
expose to users, it’s something on the side to help implementors.

Based on all the above, it seems like there isn’t a pressing need for a
protocol for these types right now and initial designs should focus on a
concrete implementation until one emerges, if only to avoid premature
generalization. Useful protocols tend to be discovered, rather than
designed, through a desire to share common operations on different
concrete types.

2. If we use concrete types, do we want value or reference semantics?

What I think makes this easier is that the "big four" have each taken a
slightly different approach that can be used as a reference.
Zewo - struct, value semantics
Vapor - closed class, reference semantics
Kitura - closed class + has-a pattern, reference semantics
Perfect - class protocol, reference semantics

Zewo is the outlier here, but I would like to note as a contributor to
Zewo that we have not ran into situations where value semantics create an
impassable roadblock.

To me, it makes sense to pass them around as values since they don't have
any logic of their own. Requests/Responses can't send themselves, they can
only read and modified. It also gives me as a user more safety to pass
them around since I know that they won't be modified implicitly.

Take the following pseudo-code as an example:

HTTPServer.onRequest { request in
    print(request.sourceIp)
    HTTPClient.send(request)
    print(request.sourceIp)

}

With reference semantics, there is no guarantee that sourceIp will be the
same before and after sending off the request. After all, it could make
sense for the HTTPClient to modify the sourceIp before sending off the
request. This of course a contrived example, but the point stands.

Not contrived at all, this is a perfect illustration of why reference
semantics make it harder to reason about your code and identify the cause
of bugs.

Anyway, I think it would be great if we could have people talk about their
own experiences.

3. When is it more convenient to have reference semantics?

Convenience is a double-edged thing. Pointers with possibly-null values,
or integer indexes into Unicode strings, are often considered convenient.
But that convenience comes with a hidden cost to correctness – unexpected
nulls, accidentally indexing into the middle of a grapheme cluster etc.
When making a proper effort to handle these things correctly, code quickly
becomes less convenient, and less readable, compared to the alternatives.

It’s generally the style in Swift that correctness shouldn't be sacrificed
for convenience, but when things work out well, convenience and ergonomics
can be mutually reinforcing – the code is nice to use correctly, awkward
to use incorrectly. For example, optionals that force you to handle nil
help with correctness, but they have sugar like ?? or optional chaining to
handle common patterns clearly and idiomatically, ! as a shorthand for
asserting something is non-nil etc.

In the middleware chain architecture that we decided on in Zewo (the other
ones have something similar), it can be convenient to modify requests in
the responder and have that reflect in the middleware. I think this
problem is best solved with `inout` parameters rather than reference
types, but that is my personal opinion.

FWIW, this design view is also strongly held by those of us working on the
Swift Standard Library. I also brought this up with several members of the
Core Team and they also strongly felt that inout and value types was the
general approach we should take with such types in Swift. The consensus
there was that reference types really should be mostly used when identity
of the value is important.

4. Are there problems that can't be solved with value semantics?

I haven't found any, but I'm sure others can bring something interesting
to the table.

Shared mutable state is one. With an unavoidably-shared resource, like a
network connection or a handle to a window on a screen, reference
semantics are often what you want.

On Wed, Nov 23, 2016 at 1:07 PM Dan Appel <dan.appel00@gmail.com> wrote:
Hello everyone!

I was unable to make the kick-off meeting for the HTTP sub-team, but I
looked over the meeting notes and found some topics that I think could use
some more on-the-record discussion.

A few questions that I wanted to raise:

1. Do we want to use concrete types or protocols for Request/Response?
2. If we use concrete types, do we want value or reference semantics?
3. When is it more convenient to have reference semantics?
4. Are there problems that can't be solved with value semantics?

I would like to avoid bike-shedding, and I think this can be done by
providing real examples rather than just talking about the pros and cons.
--
Dan Appel
--
Dan Appel
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU

Rob_Allen · November 24, 2016, 8:52am

I've worked on systems in other languages where the message object is immutable, but the body is reference to a stream. This is a very confusing scenario and I'm not a fan.

Regards,

Rob...

···

On 23 Nov 2016, at 23:29, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

On 24 Nov 2016, at 00:09, Dan Appel <dan.appel00@gmail.com> wrote:

Copying the message around will keep a reference to the same stream, but since its immutable this isn't an issue.

Elaborate. If a stream is attached the immutability property of the message is essentially void and rendered pretty much useless? (as you can’t pass it around anymore as-if-new, that only makes sense if the body too is part of the immutability contract).

Helge_Hess1 · November 28, 2016, 8:43pm

I think that no one debates that they have issues. I think the key point was that most/all of those issues need to be solved in Foundation too and that it doesn’t make sense to have two variants of them.

Also see:

[swift-server-dev] HTTP Parser
[swift-server-dev] HTTP Parser

and

[swift-server-dev] HTTP Parser

hh

···

On 28 Nov 2016, at 21:32, steve algernon <salgernon@apple.com> wrote:

On Nov 23, 2016, at 2:45 PM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

BTW: It was also mentioned that it may be highly desirable to use Foundation’s NSHTTPURLRequest/NSHTTPResponse.

A few notes in my head here:

Helge_Hess1 · December 12, 2016, 2:14pm

Its probably worth also adding some information on the approach taken by Foundation today for URLRequest/Response.

For outbound requests, Foundation provides both NSURLRequest (class) and URLRequest (struct).

I thought that both are available on Linux might be more of an oversight aka a bug (not removing NSURLRequest when URLRequest was introduced).

This is presumably because you shouldn't need to modify the stream for an outbound request

Well, somehow I need to send my data aka write to the stream? This obviously doesn’t matter for a HEAD or GET request, or a small JSON payload, but if you are uploading a photo or video, you somehow need to stream the data.

Also note that in many frameworks the stream itself can change (both in- and outbound). The common example is deflate style filters or middleware.
Now in the context of the discussion that middleware could of course also create a new request with the respective encoding headers removed and the new wrapped stream being attached.

For responses from outbound requests, only NSURLResponse (class) is provided as the body is read from the stream.

This effectively give us a model where:
outbound can be a struct (you shouldn't need to touch the stream)
inbound class (reading from stream)
For the server case, we need the mirror opposite of this - with incoming requests being a class and the outgoing response potentially being a struct.

Is this confusing immutability with reference types here? A class’ ivars can be all r/o and a struct can be passed around by reference (inout) and have r/w fields. Maybe I’m just stating the obvious :-)

I think we all agree that while a request or response is passed through the server, separate modules (middleware or other approaches) are going to change the data (changing headers, adding cookies, decoding payloads, etc etc)?
Now this can be done by creating a new, derived object representing the HTTP message, or it can be done by modifying the object which is passed in (either struct or class).
This goes both directions, client and server.

In my opinion an `HTTPMessage` (regardless which) is w/o question a reference type. It is a living object which progresses as the HTTP stream is being processed. There are events associated with it (like shutdown, ‘EOF' etc) and the meta data associated with it can change, say headers (chunked trailing headers, or via middleware, e.g. a cookie based proxy routing one). Even the method can change (e.g. X-HTTP-Method-Override …).
An interesting question is whether that `HTTPMessage` object should be provided by the Swift server lib or whether that is part of the framework on top. If it is being wrapped anyways, it may make sense to just provide lower level primitives.

There could be immutable structs called `HTTPRequestHead` and `HTTPResponseHead`, but I would definitely call them like that. Not `HTTPRequest`. Putting (user visible) reference types into a struct seems like a strong anti pattern to me (an immutable thing annotating a thing that can change).

This leads to two questions:
1. Do we think the Foundation model is correct?

To me the Foundation model is more like a framework on top. If you are developing an iOS app you just fire off a request like ‘download that resource and tell me when you have it fully cached for me, deal with all else’.
But that is not what you (usually) want on the server where (per user) memory and disk is scarce.

(Presumably this is where the "don’t touch the stream” is coming from. Its details are handled by the ‘client framework’ aka Foundation for you. You don’t usually want that in a server side HTTP client.)

IMO making `URLRequest` a struct is incorrect. As outlined above I think this should either be called `URLRequestHead` and have the stream removed, or it should be a reference type (if only because the headers can change while a chunked HTTP stream is read).

Answer: No. But this should be fixed in Foundation too :-)

2. Do we think it would be confusing to switch been classes and structs when working with inbound vs. outbound requests, or do we think it would enforce correct behaviour?

To me there is really no difference between in- and outbound HTTP requests. Both are streams of arbitrary size with some meta data attached.

Say you write a BlueMix document translation service. The client would `POST` a jigabytes big Word document in language A to your service, which would decode and translate it on the fly and stream out jigabytes of translated Word in language B.

Answer: Yes that would be confusing. They are both the same kind of streams.

hh

···

On 12 Dec 2016, at 13:08, Chris Bailey via swift-server-dev <swift-server-dev@swift.org> wrote:

Chris

From: Ben Cohen via swift-server-dev <swift-server-dev@swift.org>
To: Dan Appel <dan.appel00@gmail.com>
Cc: "swift-server-dev@swift.org" <swift-server-dev@swift.org>
Date: 08/12/2016 02:04
Subject: Re: [swift-server-dev] [HTTP] Value vs Reference Types
Sent by: swift-server-dev-bounces@swift.org

Hi Dan,

Thanks for spelling out these questions, I think they are a great starting point for a discussion. A few comments inline.

On Nov 23, 2016, at 1:19 PM, Dan Appel via swift-server-dev <swift-server-dev@swift.org> wrote:

My own responses:

>1. Do we want to use concrete types or protocols for Request/Response?

When working on Open Swift, this was a hot topic since we believed that it would be unsafe to have a protocol that would allow both value and reference types.

Bear in mind that a protocol is more than just the methods and types it declares – it is also its documentation. For example, a number of protocols in the standard library state things like complexity requirements in their documenting comments. The language has no way of enforcing these, but your type does not truly “conform” to the protocol unless you adhere to them. Value semantics are similar – you can document that it is invalid to implement a protocol with reference semantics. So I don’t think this is a blocker to wanting to use protocols.

We arrived upon the `{Request|Response}Representable` pattern which worked but was a bit of a mess. Because of this, I would prefer concrete Request/Response types.

You could think of there as being 3 purposes to using protocols here, roughly in order of importance:
  • being able to write generic code
  • allowing different frameworks to interoperate
  • documenting what you need to implement

The first one is the only reason why a protocol must be included in a library, and the key question to ask when considering defining a protocol like this is “What common algorithms do you want to write across multiple different conforming types in your program”? (such as generic functions, including protocol extensions, or functions that take an existential if the protocol has no associated types).

This is distinct from wanting to be able to be able to choose from different library implementations of Request. You might want to choose between the Acme Inc Web Framework’s Request type, or some Swift-Server “official" Request type, but you never need to use both at once and write code spanning them. You just want to make sure that one can serve as a source-compatible “drop-in” replacement for the other in your code. This doesn’t mean you can’t write your own extensions – but you would extend the concrete Request not a RequestRepresentable protocol.

Next, it’s possible that there might be a collection of 3rd-party frameworks out there that don’t define Request, but want to be able to write methods that take or extend multiple possible Request implementations. This seems a bit unlikely in the case of these types, more likely in other cases like networking, so it’s kind of a what-if scenario where there are both multiple popular implementations of Request, and various frameworks that want to interact with them. Anyone can add a conformance to anything, so those frameworks can define a protocol of their own with a subset of the functionality they need, and then just extend the popular implementations to conform to it. If this gets really common, at that point it might be worth creating an official protocol for everyone to share – but this can be done later, doesn’t have to be done up-front.

Finally, if you do expect multiple implementations and want people to be able to swap them in and out when they choose, the protocol can serve to document what methods and properties you are expected to implement to be “source compatible". This can be done in documentation instead, the benefit of the protocol being it helps the library developer ensure they’ve got all the signatures right etc. But this isn’t something you expose to users, it’s something on the side to help implementors.

Based on all the above, it seems like there isn’t a pressing need for a protocol for these types right now and initial designs should focus on a concrete implementation until one emerges, if only to avoid premature generalization. Useful protocols tend to be discovered, rather than designed, through a desire to share common operations on different concrete types.

>2. If we use concrete types, do we want value or reference semantics?

What I think makes this easier is that the "big four" have each taken a slightly different approach that can be used as a reference.
Zewo - struct, value semantics
Vapor - closed class, reference semantics
Kitura - closed class + has-a pattern, reference semantics
Perfect - class protocol, reference semantics

Zewo is the outlier here, but I would like to note as a contributor to Zewo that we have not ran into situations where value semantics create an impassable roadblock.

To me, it makes sense to pass them around as values since they don't have any logic of their own. Requests/Responses can't send themselves, they can only read and modified. It also gives me as a user more safety to pass them around since I know that they won't be modified implicitly.

Take the following pseudo-code as an example:

HTTPServer.onRequest { request in
    print(request.sourceIp)
    HTTPClient.send(request)
    print(request.sourceIp)

}

With reference semantics, there is no guarantee that sourceIp will be the same before and after sending off the request. After all, it could make sense for the HTTPClient to modify the sourceIp before sending off the request. This of course a contrived example, but the point stands.

Not contrived at all, this is a perfect illustration of why reference semantics make it harder to reason about your code and identify the cause of bugs.

Anyway, I think it would be great if we could have people talk about their own experiences.

>3. When is it more convenient to have reference semantics?

Convenience is a double-edged thing. Pointers with possibly-null values, or integer indexes into Unicode strings, are often considered convenient. But that convenience comes with a hidden cost to correctness – unexpected nulls, accidentally indexing into the middle of a grapheme cluster etc. When making a proper effort to handle these things correctly, code quickly becomes less convenient, and less readable, compared to the alternatives.

It’s generally the style in Swift that correctness shouldn't be sacrificed for convenience, but when things work out well, convenience and ergonomics can be mutually reinforcing – the code is nice to use correctly, awkward to use incorrectly. For example, optionals that force you to handle nil help with correctness, but they have sugar like ?? or optional chaining to handle common patterns clearly and idiomatically, ! as a shorthand for asserting something is non-nil etc.

In the middleware chain architecture that we decided on in Zewo (the other ones have something similar), it can be convenient to modify requests in the responder and have that reflect in the middleware. I think this problem is best solved with `inout` parameters rather than reference types, but that is my personal opinion.

FWIW, this design view is also strongly held by those of us working on the Swift Standard Library. I also brought this up with several members of the Core Team and they also strongly felt that inout and value types was the general approach we should take with such types in Swift. The consensus there was that reference types really should be mostly used when identity of the value is important.

>4. Are there problems that can't be solved with value semantics?

I haven't found any, but I'm sure others can bring something interesting to the table.

Shared mutable state is one. With an unavoidably-shared resource, like a network connection or a handle to a window on a screen, reference semantics are often what you want.

On Wed, Nov 23, 2016 at 1:07 PM Dan Appel <dan.appel00@gmail.com> wrote:
Hello everyone!

I was unable to make the kick-off meeting for the HTTP sub-team, but I looked over the meeting notes and found some topics that I think could use some more on-the-record discussion.

A few questions that I wanted to raise:

1. Do we want to use concrete types or protocols for Request/Response?
2. If we use concrete types, do we want value or reference semantics?
3. When is it more convenient to have reference semantics?
4. Are there problems that can't be solved with value semantics?

I would like to avoid bike-shedding, and I think this can be done by providing real examples rather than just talking about the pros and cons.
--
Dan Appel
--
Dan Appel
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Samuel_Kallner · December 12, 2016, 1:45pm

My biggest problem with responses being structs is that it makes it much
harder to have a server in which multiple independent pieces of code need
to run in order for the request to be processed. Why is this useful? It's
nice to have all sorts of things that "annotate" the Request and the
Response, such as Session support, Compression support, Authentication
support, and others.

If one's handlers, as in the example in Ben's e-mail are designed to
simply return a Response struct, the only way subsequent handlers can
modify the response is by wrapping the call to the next handler in the
chain, creating a new response based on the response received, and
returning this new response.

This causes one of two things.

    1. There are two types of handlers, one type that simply gets invoked
and returns a Response struct and a second type that gets a
        chain to invoke the next handler in the chain.
    2. The nice and clean API in the example needs to be changed. And one
always needs to deal with the results of the chain......

The problem I have is that it seems to be a very artificial need for two
different types of handlers. You also then need an opinionated definition
of when each kind of handler runs.

To me things become much simpler if there is a chain of handlers that need
to be run, with simple straight forward order. To do that I believe Both
the Request and the Response need to be reference types.

Shmuel Kallner
STSM Smart Client Platforms group
Tel: +972-4829-6430
e-mail: kallner@il.ibm.com

···

From: Chris Bailey via swift-server-dev <swift-server-dev@swift.org>
To: Ben Cohen <ben_cohen@apple.com>
Cc: "swift-server-dev@swift.org" <swift-server-dev@swift.org>
Date: 12/12/2016 14:11
Subject: Re: [swift-server-dev] [HTTP] Value vs Reference Types
Sent by: swift-server-dev-bounces@swift.org

Thanks Ben. This is really useful.

Its probably worth also adding some information on the approach taken by
Foundation today for URLRequest/Response.

For outbound requests, Foundation provides both NSURLRequest (class) and
URLRequest (struct). In the case of URLRequest (struct), there is a mix of
types as it contains a getter that returns a stream class for the body -
and the with a number of warnings commented in the code:
"The stream is returned for examination only; it is not safe for
the caller to manipulate the stream in any way"
(

github.com

apple/swift-corelibs-foundation/blob/5ab46791996cb65079d02f98115cef6562d4819b/Foundation/URLRequest.swift#L187


      
          /// in done in an HTTP POST request.
          public var httpBody: Data? {
              get {
                  return _handle.map { $0.httpBody }
              }
              set {
                  _applyMutation { $0.httpBody = newValue }
              }
          }
          
          /// The stream is returned for examination only; it is
          /// not safe for the caller to manipulate the stream in any way.  Also
          /// note that the HTTPBodyStream and HTTPBody are mutually exclusive - only
          /// one can be set on a given request.  Also note that the body stream is
          /// preserved across copies, but is LOST when the request is coded via the
          /// NSCoding protocol
          public var httpBodyStream: InputStream? {
              get {
                  return _handle.map { $0.httpBodyStream }
              }
              set {

)
This is presumably because you shouldn't need to modify the stream for an
outbound request

For responses from outbound requests, only NSURLResponse (class) is
provided as the body is read from the stream.

This effectively give us a model where:
outbound can be a struct (you shouldn't need to touch the
stream)
inbound class (reading from stream)
For the server case, we need the mirror opposite of this - with incoming
requests being a class and the outgoing response potentially being a
struct.

This leads to two questions:
1. Do we think the Foundation model is correct?
2. Do we think it would be confusing to switch been classes and structs
when working with inbound vs. outbound requests, or do we think it would
enforce correct behaviour?

Chris

From: Ben Cohen via swift-server-dev <swift-server-dev@swift.org>
To: Dan Appel <dan.appel00@gmail.com>
Cc: "swift-server-dev@swift.org" <swift-server-dev@swift.org>
Date: 08/12/2016 02:04
Subject: Re: [swift-server-dev] [HTTP] Value vs Reference Types
Sent by: swift-server-dev-bounces@swift.org

Hi Dan,

Thanks for spelling out these questions, I think they are a great starting
point for a discussion. A few comments inline.

On Nov 23, 2016, at 1:19 PM, Dan Appel via swift-server-dev < swift-server-dev@swift.org> wrote:

My own responses:

1. Do we want to use concrete types or protocols for Request/Response?

When working on Open Swift, this was a hot topic since we believed that it
would be unsafe to have a protocol that would allow both value and
reference types.

Bear in mind that a protocol is more than just the methods and types it
declares ? it is also its documentation. For example, a number of
protocols in the standard library state things like complexity
requirements in their documenting comments. The language has no way of
enforcing these, but your type does not truly ?conform? to the protocol
unless you adhere to them. Value semantics are similar ? you can document
that it is invalid to implement a protocol with reference semantics. So I
don?t think this is a blocker to wanting to use protocols.

We arrived upon the `{Request|Response}Representable` pattern which worked
but was a bit of a mess. Because of this, I would prefer concrete
Request/Response types.

You could think of there as being 3 purposes to using protocols here,
roughly in order of importance:
being able to write generic code
allowing different frameworks to interoperate
documenting what you need to implement

The first one is the only reason why a protocol must be included in a
library, and the key question to ask when considering defining a protocol
like this is ?What common algorithms do you want to write across multiple
different conforming types in your program?? (such as generic functions,
including protocol extensions, or functions that take an existential if
the protocol has no associated types).

This is distinct from wanting to be able to be able to choose from
different library implementations of Request. You might want to choose
between the Acme Inc Web Framework?s Request type, or some Swift-Server
?official" Request type, but you never need to use both at once and write
code spanning them. You just want to make sure that one can serve as a
source-compatible ?drop-in? replacement for the other in your code. This
doesn?t mean you can?t write your own extensions ? but you would extend
the concrete Request not a RequestRepresentable protocol.

Next, it?s possible that there might be a collection of 3rd-party
frameworks out there that don?t define Request, but want to be able to
write methods that take or extend multiple possible Request
implementations. This seems a bit unlikely in the case of these types,
more likely in other cases like networking, so it?s kind of a what-if
scenario where there are both multiple popular implementations of Request,
and various frameworks that want to interact with them. Anyone can add a
conformance to anything, so those frameworks can define a protocol of
their own with a subset of the functionality they need, and then just
extend the popular implementations to conform to it. If this gets really
common, at that point it might be worth creating an official protocol for
everyone to share ? but this can be done later, doesn?t have to be done
up-front.

Finally, if you do expect multiple implementations and want people to be
able to swap them in and out when they choose, the protocol can serve to
document what methods and properties you are expected to implement to be
?source compatible". This can be done in documentation instead, the
benefit of the protocol being it helps the library developer ensure
they?ve got all the signatures right etc. But this isn?t something you
expose to users, it?s something on the side to help implementors.

Based on all the above, it seems like there isn?t a pressing need for a
protocol for these types right now and initial designs should focus on a
concrete implementation until one emerges, if only to avoid premature
generalization. Useful protocols tend to be discovered, rather than
designed, through a desire to share common operations on different
concrete types.

2. If we use concrete types, do we want value or reference semantics?

What I think makes this easier is that the "big four" have each taken a
slightly different approach that can be used as a reference.
Zewo - struct, value semantics
Vapor - closed class, reference semantics
Kitura - closed class + has-a pattern, reference semantics
Perfect - class protocol, reference semantics

Zewo is the outlier here, but I would like to note as a contributor to
Zewo that we have not ran into situations where value semantics create an
impassable roadblock.

To me, it makes sense to pass them around as values since they don't have
any logic of their own. Requests/Responses can't send themselves, they can
only read and modified. It also gives me as a user more safety to pass
them around since I know that they won't be modified implicitly.

Take the following pseudo-code as an example:

HTTPServer.onRequest { request in
    print(request.sourceIp)
    HTTPClient.send(request)
    print(request.sourceIp)

}

With reference semantics, there is no guarantee that sourceIp will be the
same before and after sending off the request. After all, it could make
sense for the HTTPClient to modify the sourceIp before sending off the
request. This of course a contrived example, but the point stands.

Not contrived at all, this is a perfect illustration of why reference
semantics make it harder to reason about your code and identify the cause
of bugs.

Anyway, I think it would be great if we could have people talk about their
own experiences.

3. When is it more convenient to have reference semantics?

Convenience is a double-edged thing. Pointers with possibly-null values,
or integer indexes into Unicode strings, are often considered convenient.
But that convenience comes with a hidden cost to correctness ? unexpected
nulls, accidentally indexing into the middle of a grapheme cluster etc.
When making a proper effort to handle these things correctly, code quickly
becomes less convenient, and less readable, compared to the alternatives.

It?s generally the style in Swift that correctness shouldn't be sacrificed
for convenience, but when things work out well, convenience and ergonomics
can be mutually reinforcing ? the code is nice to use correctly, awkward
to use incorrectly. For example, optionals that force you to handle nil
help with correctness, but they have sugar like ?? or optional chaining to
handle common patterns clearly and idiomatically, ! as a shorthand for
asserting something is non-nil etc.

In the middleware chain architecture that we decided on in Zewo (the other
ones have something similar), it can be convenient to modify requests in
the responder and have that reflect in the middleware. I think this
problem is best solved with `inout` parameters rather than reference
types, but that is my personal opinion.

FWIW, this design view is also strongly held by those of us working on the
Swift Standard Library. I also brought this up with several members of the
Core Team and they also strongly felt that inout and value types was the
general approach we should take with such types in Swift. The consensus
there was that reference types really should be mostly used when identity
of the value is important.

4. Are there problems that can't be solved with value semantics?

I haven't found any, but I'm sure others can bring something interesting
to the table.

Shared mutable state is one. With an unavoidably-shared resource, like a
network connection or a handle to a window on a screen, reference
semantics are often what you want.

On Wed, Nov 23, 2016 at 1:07 PM Dan Appel <dan.appel00@gmail.com> wrote:
Hello everyone!

I was unable to make the kick-off meeting for the HTTP sub-team, but I
looked over the meeting notes and found some topics that I think could use
some more on-the-record discussion.

A few questions that I wanted to raise:

1. Do we want to use concrete types or protocols for Request/Response?
2. If we use concrete types, do we want value or reference semantics?
3. When is it more convenient to have reference semantics?
4. Are there problems that can't be solved with value semantics?

I would like to avoid bike-shedding, and I think this can be done by
providing real examples rather than just talking about the pros and cons.
--
Dan Appel
--
Dan Appel
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

steve_algernon · November 28, 2016, 8:59pm

Sorry I didn't see those threads earlier, but I'm in complete agreement. I've been holding off proposing new API internally pending the outcome of these discussions.

For my (selfish) purposes, I'd prefer a new set of objects that are almost-but-not-quite the existing objects. (I prefer policy (caching, timeouts) be tied to a session configuration rather than to the request object itself.).

--sma

···

On Nov 28, 2016, at 12:43 PM, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:

I think that no one debates that they have issues. I think the key point was that most/all of those issues need to be solved in Foundation too and that it doesn’t make sense to have two variants of them.

Ben_Cohen · December 12, 2016, 4:07pm

My biggest problem with responses being structs is that it makes it much harder to have a server in which multiple independent pieces of code need to run in order for the request to be processed. Why is this useful? It's nice to have all sorts of things that "annotate" the Request and the Response, such as Session support, Compression support, Authentication support, and others.

This is an argument for operations on types to be pass-by-reference, rather than for the types themselves to be reference types. There are various different ways to handle this without resorting to reference types.

Swift supports this via the inout keyword. It isn’t quite pass-by-reference (see TSPL <The Swift Programming Language: Redirect; for details) but it is similar enough for our purposes here from a performance/usage perspective. It also forces the caller to acknowledge the mutation by adding the & sigil at the call site, helping with the problem Dan outlined where a user can be surprised by a function mutating its argument.

If appropriate, you can also add mutating methods as extensions e.g. request.compress(). You can think of this as like passing self inout. Indication of mutation here should be via appropriate naming of the method.

There are also more elaborate techniques, such as wrapping adapters that maybe perform tasks lazily. For example myArray.reversed() doesn’t actually reverse an array, it just wraps the array in a ReversedCollection “view” that presents the data in a reversed fashion. This is a lot more complicated, and is only a win if it significantly improves the user experience for both readability and performance, but when it works, it can be a big win.

If one's handlers, as in the example in Ben's e-mail are designed to simply return a Response struct, the only way subsequent handlers can modify the response is by wrapping the call to the next handler in the chain, creating a new response based on the response received, and returning this new response.

This causes one of two things.

    1. There are two types of handlers, one type that simply gets invoked and returns a Response struct and a second type that gets a
        chain to invoke the next handler in the chain.
    2. The nice and clean API in the example needs to be changed. And one always needs to deal with the results of the chain......

The problem I have is that it seems to be a very artificial need for two different types of handlers. You also then need an opinionated definition of when each kind of handler runs.

Bear in mind with value types, it is straightforward to implement the return-a-fresh-value implementation in terms of the mutate-in-place one. The mutating/non-mutating pair pattern is adopted widely, is covered in the API naming guidelines, and its implications are well understood by the community. So the definition of when each kind of handler is needed should be clear.

To me things become much simpler if there is a chain of handlers that need to be run, with simple straight forward order. To do that I believe Both the Request and the Response need to be reference types.

This argument in favor of reference types and against the value-type approach does not to me seem like it’s specific to this particular use case. It is common to perform chains of operations on arrays, for example. The need to implement these operations efficiently is what leads to both the in-place/new-value method pair approach, as well as techniques such as .lazy. But the clarity/correctness benefits of value types are remain and are an important aspect of the Swift style of programming.

Not adopting these techniques would put the style of this library at odds with the other core Swift libraries from a style perspective, something that I feel would be very hard to defend when considering this library for adoption by the community as a standard.

···

On Dec 12, 2016, at 5:45 AM, Samuel Kallner <KALLNER@il.ibm.com> wrote:

Shmuel Kallner
STSM Smart Client Platforms group
Tel: +972-4829-6430
e-mail: kallner@il.ibm.com

From: Chris Bailey via swift-server-dev <swift-server-dev@swift.org>
To: Ben Cohen <ben_cohen@apple.com>
Cc: "swift-server-dev@swift.org" <swift-server-dev@swift.org>
Date: 12/12/2016 14:11
Subject: Re: [swift-server-dev] [HTTP] Value vs Reference Types
Sent by: swift-server-dev-bounces@swift.org

Thanks Ben. This is really useful.

Its probably worth also adding some information on the approach taken by Foundation today for URLRequest/Response.

For outbound requests, Foundation provides both NSURLRequest (class) and URLRequest (struct). In the case of URLRequest (struct), there is a mix of types as it contains a getter that returns a stream class for the body - and the with a number of warnings commented in the code:
       "The stream is returned for examination only; it is not safe for the caller to manipulate the stream in any way"
(https://github.com/apple/swift-corelibs-foundation/blob/5ab46791996cb65079d02f98115cef6562d4819b/Foundation/URLRequest.swift#L187\)
This is presumably because you shouldn't need to modify the stream for an outbound request

For responses from outbound requests, only NSURLResponse (class) is provided as the body is read from the stream.

This effectively give us a model where:
       outbound can be a struct (you shouldn't need to touch the stream)
       inbound class (reading from stream)
For the server case, we need the mirror opposite of this - with incoming requests being a class and the outgoing response potentially being a struct.

This leads to two questions:
1. Do we think the Foundation model is correct?
2. Do we think it would be confusing to switch been classes and structs when working with inbound vs. outbound requests, or do we think it would enforce correct behaviour?

Chris

From: Ben Cohen via swift-server-dev <swift-server-dev@swift.org>
To: Dan Appel <dan.appel00@gmail.com>
Cc: "swift-server-dev@swift.org" <swift-server-dev@swift.org>
Date: 08/12/2016 02:04
Subject: Re: [swift-server-dev] [HTTP] Value vs Reference Types
Sent by: swift-server-dev-bounces@swift.org

Hi Dan,

Thanks for spelling out these questions, I think they are a great starting point for a discussion. A few comments inline.

On Nov 23, 2016, at 1:19 PM, Dan Appel via swift-server-dev <swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>> wrote:

My own responses:

>1. Do we want to use concrete types or protocols for Request/Response?

When working on Open Swift <https://github.com/open-swift>, this was a hot topic since we believed that it would be unsafe to have a protocol that would allow both value and reference types.

Bear in mind that a protocol is more than just the methods and types it declares – it is also its documentation. For example, a number of protocols in the standard library state things like complexity requirements in their documenting comments. The language has no way of enforcing these, but your type does not truly “conform” to the protocol unless you adhere to them. Value semantics are similar – you can document that it is invalid to implement a protocol with reference semantics. So I don’t think this is a blocker to wanting to use protocols.

We arrived upon the `{Request|Response}Representable` pattern which worked but was a bit of a mess. Because of this, I would prefer concrete Request/Response types.

You could think of there as being 3 purposes to using protocols here, roughly in order of importance:
being able to write generic code
allowing different frameworks to interoperate
documenting what you need to implement

The first one is the only reason why a protocol must be included in a library, and the key question to ask when considering defining a protocol like this is “What common algorithms do you want to write across multiple different conforming types in your program”? (such as generic functions, including protocol extensions, or functions that take an existential if the protocol has no associated types).

This is distinct from wanting to be able to be able to choose from different library implementations of Request. You might want to choose between the Acme Inc Web Framework’s Request type, or some Swift-Server “official" Request type, but you never need to use both at once and write code spanning them. You just want to make sure that one can serve as a source-compatible “drop-in” replacement for the other in your code. This doesn’t mean you can’t write your own extensions – but you would extend the concrete Request not a RequestRepresentable protocol.

Next, it’s possible that there might be a collection of 3rd-party frameworks out there that don’t define Request, but want to be able to write methods that take or extend multiple possible Request implementations. This seems a bit unlikely in the case of these types, more likely in other cases like networking, so it’s kind of a what-if scenario where there are both multiple popular implementations of Request, and various frameworks that want to interact with them. Anyone can add a conformance to anything, so those frameworks can define a protocol of their own with a subset of the functionality they need, and then just extend the popular implementations to conform to it. If this gets really common, at that point it might be worth creating an official protocol for everyone to share – but this can be done later, doesn’t have to be done up-front.

Finally, if you do expect multiple implementations and want people to be able to swap them in and out when they choose, the protocol can serve to document what methods and properties you are expected to implement to be “source compatible". This can be done in documentation instead, the benefit of the protocol being it helps the library developer ensure they’ve got all the signatures right etc. But this isn’t something you expose to users, it’s something on the side to help implementors.

Based on all the above, it seems like there isn’t a pressing need for a protocol for these types right now and initial designs should focus on a concrete implementation until one emerges, if only to avoid premature generalization. Useful protocols tend to be discovered, rather than designed, through a desire to share common operations on different concrete types.

>2. If we use concrete types, do we want value or reference semantics?

What I think makes this easier is that the "big four" have each taken a slightly different approach that can be used as a reference.
Zewo <https://github.com/Zewo/Zewo/blob/master/Modules/HTTP/Sources/HTTP/Message/Request.swift#L3>\- struct, value semantics
Vapor <https://github.com/vapor/engine/blob/master/Sources/HTTP/Models/Request/Request.swift#L4>\- closed class, reference semantics
Kitura <https://github.com/IBM-Swift/Kitura/blob/master/Sources/Kitura/RouterRequest.swift#L26>\- closed class + has-a pattern, reference semantics
Perfect <https://github.com/PerfectlySoft/Perfect-HTTP/blob/master/Sources/HTTPRequest.swift#L25>\- class protocol, reference semantics

Zewo is the outlier here, but I would like to note as a contributor to Zewo that we have not ran into situations where value semantics create an impassable roadblock.

To me, it makes sense to pass them around as values since they don't have any logic of their own. Requests/Responses can't send themselves, they can only read and modified. It also gives me as a user more safety to pass them around since I know that they won't be modified implicitly.

Take the following pseudo-code as an example:

HTTPServer.onRequest { request in
   print(request.sourceIp)
   HTTPClient.send(request)
   print(request.sourceIp)

}

With reference semantics, there is no guarantee that sourceIp will be the same before and after sending off the request. After all, it couldmake sense for the HTTPClient to modify the sourceIp before sending off the request. This of course a contrived example, but the point stands.

Not contrived at all, this is a perfect illustration of why reference semantics make it harder to reason about your code and identify the cause of bugs.

Anyway, I think it would be great if we could have people talk about their own experiences.

>3. When is it more convenient to have reference semantics?

Convenience is a double-edged thing. Pointers with possibly-null values, or integer indexes into Unicode strings, are often considered convenient. But that convenience comes with a hidden cost to correctness – unexpected nulls, accidentally indexing into the middle of a grapheme cluster etc. When making a proper effort to handle these things correctly, code quickly becomes less convenient, and less readable, compared to the alternatives.

It’s generally the style in Swift that correctness shouldn't be sacrificed for convenience, but when things work out well, convenience and ergonomics can be mutually reinforcing – the code is nice to use correctly, awkward to use incorrectly. For example, optionals that force you to handle nil help with correctness, but they have sugar like ?? or optional chaining to handle common patterns clearly and idiomatically, ! as a shorthand for asserting something is non-nil etc.

In the middleware chain architecture that we decided on in Zewo (the other ones have something similar), it can be convenient to modify requests in the responder and have that reflect in the middleware. I think this problem is best solved with `inout` parameters rather than reference types, but that is my personal opinion.

FWIW, this design view is also strongly held by those of us working on the Swift Standard Library. I also brought this up with several members of the Core Team and they also strongly felt that inout and value types was the general approach we should take with such types in Swift. The consensus there was that reference types really should be mostly used when identity of the value is important.

>4. Are there problems that can't be solved with value semantics?

I haven't found any, but I'm sure others can bring something interesting to the table.

Shared mutable state is one. With an unavoidably-shared resource, like a network connection or a handle to a window on a screen, reference semantics are often what you want.

On Wed, Nov 23, 2016 at 1:07 PM Dan Appel <dan.appel00@gmail.com <mailto:dan.appel00@gmail.com>> wrote:
Hello everyone!

I was unable to make the kick-off meeting for the HTTP sub-team, but I looked over the meeting notes <https://docs.google.com/document/d/1SWK0qBDi-9DeLJwHlcXcPU7h22JU-DWW8HTVuHbTQiw/edit> and found some topics that I think could use some more on-the-record discussion.

A few questions that I wanted to raise:

1. Do we want to use concrete types or protocols for Request/Response?
2. If we use concrete types, do we want value or reference semantics?
3. When is it more convenient to have reference semantics?
4. Are there problems that can't be solved with value semantics?

I would like to avoid bike-shedding, and I think this can be done by providing real examples rather than just talking about the pros and cons.
--
Dan Appel
--
Dan Appel
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org <mailto:swift-server-dev@swift.org>
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Helge_Hess1 · December 12, 2016, 2:29pm

Hi Samuel,

To me things become much simpler if there is a chain of handlers that need to be run, with simple straight forward order. To do that I believe Both the Request and the Response need to be reference types.

well, no. Even if they are structs the thing you want can be built naturally. If the handler stack is run synchronously you can use `inout`, like:

    func processMethodOverride(in request : inout Request) {
      guard let methodOverride = request[header: “X-HTTP-Method-Override”]
       else { return }

request.method = methodOverride
}

The request is passed by reference and will be modified on the calling side.

And of course there is the other, more functional approach:

    func processMethodOverride(in request : Request) -> Request {
      guard let methodOverride = request[header: “X-HTTP-Method-Override”]
       else { return request }

      var newRequest = request
      newRequest.method = methodOverride
      return newRequest
    }

The calling side would then pass on the new, modified Request. (A nice thing is that it can easily keep a log of how the Request changed while it traverses the handler stack.)

Summary: I think having a handler stack doesn’t necessarily imply reference types.

It is kinda desirable to have those transformations based on immutable types. BUT: the problem is that an HTTP message is primarily a stream, headers are just metadata, and the thing being transformed is not usually available ‘in full’. Plus the fact that the meta data of the streams can change while the stream is read.

hh

···

On 12 Dec 2016, at 14:45, Samuel Kallner via swift-server-dev <swift-server-dev@swift.org> wrote:

My biggest problem with responses being structs is that it makes it much harder to have a server in which multiple independent pieces of code need to run in order for the request to be processed. Why is this useful? It's nice to have all sorts of things that "annotate" the Request and the Response, such as Session support, Compression support, Authentication support, and others.

If one's handlers, as in the example in Ben's e-mail are designed to simply return a Response struct, the only way subsequent handlers can modify the response is by wrapping the call to the next handler in the chain, creating a new response based on the response received, and returning this new response.

This causes one of two things.

    1. There are two types of handlers, one type that simply gets invoked and returns a Response struct and a second type that gets a
        chain to invoke the next handler in the chain.
    2. The nice and clean API in the example needs to be changed. And one always needs to deal with the results of the chain......

The problem I have is that it seems to be a very artificial need for two different types of handlers. You also then need an opinionated definition of when each kind of handler runs.

To me things become much simpler if there is a chain of handlers that need to be run, with simple straight forward order. To do that I believe Both the Request and the Response need to be reference types.

Shmuel Kallner
STSM Smart Client Platforms group
Tel: +972-4829-6430
e-mail: kallner@il.ibm.com

From: Chris Bailey via swift-server-dev <swift-server-dev@swift.org>
To: Ben Cohen <ben_cohen@apple.com>
Cc: "swift-server-dev@swift.org" <swift-server-dev@swift.org>
Date: 12/12/2016 14:11
Subject: Re: [swift-server-dev] [HTTP] Value vs Reference Types
Sent by: swift-server-dev-bounces@swift.org

Thanks Ben. This is really useful.

Its probably worth also adding some information on the approach taken by Foundation today for URLRequest/Response.

For outbound requests, Foundation provides both NSURLRequest (class) and URLRequest (struct). In the case of URLRequest (struct), there is a mix of types as it contains a getter that returns a stream class for the body - and the with a number of warnings commented in the code:
       "The stream is returned for examination only; it is not safe for the caller to manipulate the stream in any way"
(https://github.com/apple/swift-corelibs-foundation/blob/5ab46791996cb65079d02f98115cef6562d4819b/Foundation/URLRequest.swift#L187\)
This is presumably because you shouldn't need to modify the stream for an outbound request

For responses from outbound requests, only NSURLResponse (class) is provided as the body is read from the stream.

This effectively give us a model where:
       outbound can be a struct (you shouldn't need to touch the stream)
       inbound class (reading from stream)
For the server case, we need the mirror opposite of this - with incoming requests being a class and the outgoing response potentially being a struct.

This leads to two questions:
1. Do we think the Foundation model is correct?
2. Do we think it would be confusing to switch been classes and structs when working with inbound vs. outbound requests, or do we think it would enforce correct behaviour?

Chris

From: Ben Cohen via swift-server-dev <swift-server-dev@swift.org>
To: Dan Appel <dan.appel00@gmail.com>
Cc: "swift-server-dev@swift.org" <swift-server-dev@swift.org>
Date: 08/12/2016 02:04
Subject: Re: [swift-server-dev] [HTTP] Value vs Reference Types
Sent by: swift-server-dev-bounces@swift.org

Hi Dan,

Thanks for spelling out these questions, I think they are a great starting point for a discussion. A few comments inline.

On Nov 23, 2016, at 1:19 PM, Dan Appel via swift-server-dev <swift-server-dev@swift.org> wrote:

My own responses:

>1. Do we want to use concrete types or protocols for Request/Response?

When working on Open Swift, this was a hot topic since we believed that it would be unsafe to have a protocol that would allow both value and reference types.

Bear in mind that a protocol is more than just the methods and types it declares – it is also its documentation. For example, a number of protocols in the standard library state things like complexity requirements in their documenting comments. The language has no way of enforcing these, but your type does not truly “conform” to the protocol unless you adhere to them. Value semantics are similar – you can document that it is invalid to implement a protocol with reference semantics. So I don’t think this is a blocker to wanting to use protocols.

We arrived upon the `{Request|Response}Representable` pattern which worked but was a bit of a mess. Because of this, I would prefer concrete Request/Response types.

You could think of there as being 3 purposes to using protocols here, roughly in order of importance:
  • being able to write generic code
  • allowing different frameworks to interoperate
  • documenting what you need to implement

The first one is the only reason why a protocol must be included in a library, and the key question to ask when considering defining a protocol like this is “What common algorithms do you want to write across multiple different conforming types in your program”? (such as generic functions, including protocol extensions, or functions that take an existential if the protocol has no associated types).

This is distinct from wanting to be able to be able to choose from different library implementations of Request. You might want to choose between the Acme Inc Web Framework’s Request type, or some Swift-Server “official" Request type, but you never need to use both at once and write code spanning them. You just want to make sure that one can serve as a source-compatible “drop-in” replacement for the other in your code. This doesn’t mean you can’t write your own extensions – but you would extend the concrete Request not a RequestRepresentable protocol.

Next, it’s possible that there might be a collection of 3rd-party frameworks out there that don’t define Request, but want to be able to write methods that take or extend multiple possible Request implementations. This seems a bit unlikely in the case of these types, more likely in other cases like networking, so it’s kind of a what-if scenario where there are both multiple popular implementations of Request, and various frameworks that want to interact with them. Anyone can add a conformance to anything, so those frameworks can define a protocol of their own with a subset of the functionality they need, and then just extend the popular implementations to conform to it. If this gets really common, at that point it might be worth creating an official protocol for everyone to share – but this can be done later, doesn’t have to be done up-front.

Finally, if you do expect multiple implementations and want people to be able to swap them in and out when they choose, the protocol can serve to document what methods and properties you are expected to implement to be “source compatible". This can be done in documentation instead, the benefit of the protocol being it helps the library developer ensure they’ve got all the signatures right etc. But this isn’t something you expose to users, it’s something on the side to help implementors.

Based on all the above, it seems like there isn’t a pressing need for a protocol for these types right now and initial designs should focus on a concrete implementation until one emerges, if only to avoid premature generalization. Useful protocols tend to be discovered, rather than designed, through a desire to share common operations on different concrete types.

>2. If we use concrete types, do we want value or reference semantics?

What I think makes this easier is that the "big four" have each taken a slightly different approach that can be used as a reference.
Zewo- struct, value semantics
Vapor- closed class, reference semantics
Kitura- closed class + has-a pattern, reference semantics
Perfect- class protocol, reference semantics

Zewo is the outlier here, but I would like to note as a contributor to Zewo that we have not ran into situations where value semantics create an impassable roadblock.

To me, it makes sense to pass them around as values since they don't have any logic of their own. Requests/Responses can't send themselves, they can only read and modified. It also gives me as a user more safety to pass them around since I know that they won't be modified implicitly.

Take the following pseudo-code as an example:

HTTPServer.onRequest { request in
   print(request.sourceIp)
   HTTPClient.send(request)
   print(request.sourceIp)

}

With reference semantics, there is no guarantee that sourceIp will be the same before and after sending off the request. After all, it couldmake sense for the HTTPClient to modify the sourceIp before sending off the request. This of course a contrived example, but the point stands.

Not contrived at all, this is a perfect illustration of why reference semantics make it harder to reason about your code and identify the cause of bugs.

Anyway, I think it would be great if we could have people talk about their own experiences.

>3. When is it more convenient to have reference semantics?

Convenience is a double-edged thing. Pointers with possibly-null values, or integer indexes into Unicode strings, are often considered convenient. But that convenience comes with a hidden cost to correctness – unexpected nulls, accidentally indexing into the middle of a grapheme cluster etc. When making a proper effort to handle these things correctly, code quickly becomes less convenient, and less readable, compared to the alternatives.

It’s generally the style in Swift that correctness shouldn't be sacrificed for convenience, but when things work out well, convenience and ergonomics can be mutually reinforcing – the code is nice to use correctly, awkward to use incorrectly. For example, optionals that force you to handle nil help with correctness, but they have sugar like ?? or optional chaining to handle common patterns clearly and idiomatically, ! as a shorthand for asserting something is non-nil etc.

In the middleware chain architecture that we decided on in Zewo (the other ones have something similar), it can be convenient to modify requests in the responder and have that reflect in the middleware. I think this problem is best solved with `inout` parameters rather than reference types, but that is my personal opinion.

FWIW, this design view is also strongly held by those of us working on the Swift Standard Library. I also brought this up with several members of the Core Team and they also strongly felt that inout and value types was the general approach we should take with such types in Swift. The consensus there was that reference types really should be mostly used when identity of the value is important.

>4. Are there problems that can't be solved with value semantics?

I haven't found any, but I'm sure others can bring something interesting to the table.

Shared mutable state is one. With an unavoidably-shared resource, like a network connection or a handle to a window on a screen, reference semantics are often what you want.

On Wed, Nov 23, 2016 at 1:07 PM Dan Appel <dan.appel00@gmail.com> wrote:
Hello everyone!

I was unable to make the kick-off meeting for the HTTP sub-team, but I looked over the meeting notes and found some topics that I think could use some more on-the-record discussion.

A few questions that I wanted to raise:

1. Do we want to use concrete types or protocols for Request/Response?
2. If we use concrete types, do we want value or reference semantics?
3. When is it more convenient to have reference semantics?
4. Are there problems that can't be solved with value semantics?

I would like to avoid bike-shedding, and I think this can be done by providing real examples rather than just talking about the pros and cons.
--
Dan Appel
--
Dan Appel
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number 741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev