Swift-server-dev Digest, Vol 1, Issue 1

Server Development
1

I think there's a lot of good discussion here for different potential
directions and various areas that people are excited about. I think while
this is valuable, eventually it'd probably be beneficial to put some focus
on one or two specific areas. Sticking w/ the direction of the doc and
working upwards, TCP and general socket work seems to be a natural starting
point to work with.

- Logan

···

On Wed, Oct 26, 2016 at 1:01 PM <swift-server-dev-request@swift.org> wrote:

Send swift-server-dev mailing list submissions to
        swift-server-dev@swift.org

To subscribe or unsubscribe via the World Wide Web, visit
        https://lists.swift.org/mailman/listinfo/swift-server-dev
or, via email, send a message with subject or body 'help' to
        swift-server-dev-request@swift.org

You can reach the person managing the list at
        swift-server-dev-owner@swift.org

When replying, please edit your Subject line so it is more specific
than "Re: Contents of swift-server-dev digest..."

Today's Topics:

   1. Developing Server APIs for Swift (Chris Bailey)
   2. Database API/Framework (Ron Olson)
   3. Re: Database API/Framework (Chris Bailey)
   4. Re: Database API/Framework (Helge Heß)
   5. [Bug] Foundation.URL should support semicolon(; ) appearing
      in URL path (frogcjn@163.com)
   6. Re: Database API/Framework (Paulo Faria)
   7. Re: Database API/Framework (Haris Amin)
   8. Re: Database API/Framework (Helge Heß)
   9. Re: Database API/Framework (Ron Olson)
  10. Re: [Bug] Foundation.URL should support semicolon(; )
      appearing in URL path (Alex Blewitt)
  11. Spring and Baratine (Java frameworks) (Rick Mann)
  12. Lib dispatch and Edge (Tyler Cloutier)
  13. Why not rely on existing models? (Yoann Gini)
  14. Re: Why not rely on existing models? (Helge Heß)
  15. Re: Database API/Framework (Brent Royal-Gordon)
  16. Re: Why not rely on existing models? (Yoann Gini)
  17. Re: Why not rely on existing models? (Brent Royal-Gordon)
  18. Re: Database API/Framework (Helge Heß)
  19. Re: Why not rely on existing models? (Yoann Gini)
  20. Re: Database API/Framework (Chris Bailey)
  21. Re: Database API/Framework (Gregor Milos)
  22. Re: Database API/Framework (Gregor Milos)
  23. Re: Database API/Framework (Helge Heß)
  24. Async I/O in Rust Tokio (Benedikt Terhechte)
  25. Re: Database API/Framework (Gregor Milos)

----------------------------------------------------------------------

Message: 1
Date: Tue, 25 Oct 2016 18:18:40 +0100
From: Chris Bailey <BAILEYC@uk.ibm.com>
To: swift-server-dev@swift.org
Subject: [swift-server-dev] Developing Server APIs for Swift
Message-ID:
        <
OFF7B5F3A2.286E27A8-ON80258057.005E33D7-80258057.005F17AB@notes.na.collabserv.com
>

Content-Type: text/plain; charset="utf-8"

A new project and work group has just been announced, aimed at providing a
framework for participants in the community with an interest in building
server applications and frameworks to come together to work on providing
new Swift APIs.
Swift.org - Server APIs Work Group

Whilst there is already strong participation from some of the more well
know web frameworks, including Kitura, Vapor, Perfect, and Zewo,
participation and contribution is open to everyone - whether that's having
a requirement or use case you'd like to share, bugs you want to report
(once we have code!), code your willing to write, or just your thoughts on
what's needed.

Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
https://lists.swift.org/pipermail/swift-server-dev/attachments/20161025/89b49b87/attachment-0001.html
>

------------------------------

Message: 2
Date: Tue, 25 Oct 2016 12:53:56 -0500
From: "Ron Olson" <tachoknight@gmail.com>
To: swift-server-dev@swift.org
Subject: [swift-server-dev] Database API/Framework
Message-ID: <014CED77-5B93-4EF1-A089-EF4926CCF47C@gmail.com>
Content-Type: text/plain; format=flowed

Hi all-

I've been working with Swift more on Linux than macOS and am very
excited to see the formation of this group. Looking at the server api
workgroup's focus, I notice it's primarily low-level interaction with
the operating system. When might it be a good time to bring up the
possibility of creating a database-specific framework for those folks
who want to work directly with Postgres, MySQL, even DB2 and Oracle; I'm
thinking a JDBC-inspired framework that drivers could be written
against.

Thanks,

Ron

------------------------------

Message: 3
Date: Tue, 25 Oct 2016 19:41:32 +0100
From: Chris Bailey <BAILEYC@uk.ibm.com>
To: Ron Olson <tachoknight@gmail.com>
Cc: swift-server-dev@swift.org
Subject: Re: [swift-server-dev] Database API/Framework
Message-ID:
        <
OFCED025F9.33D57EE8-ON80258057.00662CA1-80258057.0066ADD2@notes.na.collabserv.com
>

Content-Type: text/plain; charset="utf-8"

Hi Ron:

As you say, that's outside of the scope of the work group, however I do
think its a valuable area that the various existing web framework groups
might be interested in collaborating on. Having a common, recognized
framework that the database providers or third parties would be interested
in writing database drivers for would be a huge step forward.

I know that Vapor has its Fluent ORM framework, and that its an area that
the Kitura team are looking at as well, so your post might kick-start some
wider discussion.

Chris

From: Ron Olson via swift-server-dev <swift-server-dev@swift.org>
To: swift-server-dev@swift.org
Date: 25/10/2016 18:54
Subject: [swift-server-dev] Database API/Framework
Sent by: swift-server-dev-bounces@swift.org

Hi all-

I've been working with Swift more on Linux than macOS and am very
excited to see the formation of this group. Looking at the server api
workgroup's focus, I notice it's primarily low-level interaction with
the operating system. When might it be a good time to bring up the
possibility of creating a database-specific framework for those folks
who want to work directly with Postgres, MySQL, even DB2 and Oracle; I'm
thinking a JDBC-inspired framework that drivers could be written
against.

Thanks,

Ron
_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
https://lists.swift.org/pipermail/swift-server-dev/attachments/20161025/59d5ff21/attachment-0001.html
>

------------------------------

Message: 4
Date: Tue, 25 Oct 2016 20:56:13 +0200
From: Helge Heß <me@helgehess.eu>
To: swift-server-dev@swift.org
Subject: Re: [swift-server-dev] Database API/Framework
Message-ID: <0A3EC83B-E26E-49F3-8EDE-E2B85D6836D1@helgehess.eu>
Content-Type: text/plain; charset=utf-8

Hi Chris,

it is a little unclear what the scope of the group is. Presumably you have
some very specific things in mind?

Of course we’ve seen https://swift.org/server-apis/#focus-areas\. But what
does it actually mean? :-) Looking at it:
- base networking: TCP/UDP, DNS. Covered already by libdispatch and the
OS’ses?
- security: OpenSSL? or some wrapper around both, CC and OpenSSL?
- provide low level HTTP parsing: http_parser? Is anyone using something
  else?
All that seems kinda covered/standard already? Unless maybe you include
non-Unix stuff like Windows.

Is this primarily for things like providing a standardised ’swiftier'
OpenSSL wrapper (similar to how Objective-GCD wraps libdispatch)?
Or would that just be module maps which are shipped by default with Swift
and hence can be relied on by all frameworks?

But maybe I’m just a little too impatient :->

hh

> On 25 Oct 2016, at 20:41, Chris Bailey via swift-server-dev < > swift-server-dev@swift.org> wrote:
>
> Hi Ron:
>
> As you say, that's outside of the scope of the work group, however I do
think its a valuable area that the various existing web framework groups
might be interested in collaborating on. Having a common, recognized
framework that the database providers or third parties would be interested
in writing database drivers for would be a huge step forward.
>
> I know that Vapor has its Fluent ORM framework, and that its an area
that the Kitura team are looking at as well, so your post might kick-start
some wider discussion.
>
> Chris
>
>
>
>
> From: Ron Olson via swift-server-dev <swift-server-dev@swift.org>
> To: swift-server-dev@swift.org
> Date: 25/10/2016 18:54
> Subject: [swift-server-dev] Database API/Framework
> Sent by: swift-server-dev-bounces@swift.org
>
>
>
> Hi all-
>
> I've been working with Swift more on Linux than macOS and am very
> excited to see the formation of this group. Looking at the server api
> workgroup's focus, I notice it's primarily low-level interaction with
> the operating system. When might it be a good time to bring up the
> possibility of creating a database-specific framework for those folks
> who want to work directly with Postgres, MySQL, even DB2 and Oracle; I'm
> thinking a JDBC-inspired framework that drivers could be written
> against.
>
> Thanks,
>
> Ron
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev
>
>
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev

------------------------------

Message: 5
Date: Wed, 26 Oct 2016 03:09:33 +0800
From: frogcjn@163.com
To: swift-server-dev@swift.org
Subject: [swift-server-dev] [Bug] Foundation.URL should support
        semicolon(; ) appearing in URL path
Message-ID: <22A5614E-C6ED-47DD-A190-917C49422D02@163.com>
Content-Type: text/plain; charset=us-ascii

http://host/book;id=1/page;2/

this URL is a valid URL, but URLComponents will ruin it with %3B
http://host/book%3Bid=1/page%3B2/

Since Swift Server is coming out, this bug should be solved.

------------------------------

Message: 6
Date: Tue, 25 Oct 2016 17:20:23 -0200
From: Paulo Faria <paulo@zewo.io>
To: Helge Heß <me@helgehess.eu>
Cc: swift-server-dev@swift.org
Subject: Re: [swift-server-dev] Database API/Framework
Message-ID: <78032B27-2CF6-480A-8B21-9EE4C10B8161@zewo.io>
Content-Type: text/plain; charset=utf-8

Hello, Helge.

Yes, most of the things we’ll cover are already provided by some
frameworks, but having an official library joining efforts will be great
for the community. We’ll provide full-fledged Swift libraries for the
topics mentioned. How they’re going to be developed it’s still to be
decided.

Cheers,
Paulo

> On Oct 25, 2016, at 4:56 PM, Helge Heß via swift-server-dev < > swift-server-dev@swift.org> wrote:
>
> Hi Chris,
>
> it is a little unclear what the scope of the group is. Presumably you
have some very specific things in mind?
>
> Of course we’ve seen https://swift.org/server-apis/#focus-areas\. But
what does it actually mean? :-) Looking at it:
> - base networking: TCP/UDP, DNS. Covered already by libdispatch and the
OS’ses?
> - security: OpenSSL? or some wrapper around both, CC and OpenSSL?
> - provide low level HTTP parsing: http_parser? Is anyone using something
> else?
> All that seems kinda covered/standard already? Unless maybe you include
non-Unix stuff like Windows.
>
> Is this primarily for things like providing a standardised ’swiftier'
OpenSSL wrapper (similar to how Objective-GCD wraps libdispatch)?
> Or would that just be module maps which are shipped by default with
Swift and hence can be relied on by all frameworks?
>
> But maybe I’m just a little too impatient :->
>
> hh
>
>> On 25 Oct 2016, at 20:41, Chris Bailey via swift-server-dev < > swift-server-dev@swift.org> wrote:
>>
>> Hi Ron:
>>
>> As you say, that's outside of the scope of the work group, however I do
think its a valuable area that the various existing web framework groups
might be interested in collaborating on. Having a common, recognized
framework that the database providers or third parties would be interested
in writing database drivers for would be a huge step forward.
>>
>> I know that Vapor has its Fluent ORM framework, and that its an area
that the Kitura team are looking at as well, so your post might kick-start
some wider discussion.
>>
>> Chris
>>
>>
>>
>>
>> From: Ron Olson via swift-server-dev <swift-server-dev@swift.org
>
>> To: swift-server-dev@swift.org
>> Date: 25/10/2016 18:54
>> Subject: [swift-server-dev] Database API/Framework
>> Sent by: swift-server-dev-bounces@swift.org
>>
>>
>>
>> Hi all-
>>
>> I've been working with Swift more on Linux than macOS and am very
>> excited to see the formation of this group. Looking at the server api
>> workgroup's focus, I notice it's primarily low-level interaction with
>> the operating system. When might it be a good time to bring up the
>> possibility of creating a database-specific framework for those folks
>> who want to work directly with Postgres, MySQL, even DB2 and Oracle; I'm
>> thinking a JDBC-inspired framework that drivers could be written
>> against.
>>
>> Thanks,
>>
>> Ron
>> _______________________________________________
>> swift-server-dev mailing list
>> swift-server-dev@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-server-dev
>>
>>
>> _______________________________________________
>> swift-server-dev mailing list
>> swift-server-dev@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-server-dev
>
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev

------------------------------

Message: 7
Date: Tue, 25 Oct 2016 15:34:24 -0400
From: Haris Amin <aminharis7@gmail.com>
To: Paulo Faria <paulo@zewo.io>, Helge Heß <me@helgehess.eu>, Paulo
        Faria via swift-server-dev <swift-server-dev@swift.org>
Subject: Re: [swift-server-dev] Database API/Framework
Message-ID:
        <CAG=aSgQ_byEvJPCcqU=
kT6LRuT3xXYzk+Laabja2uYnVAaZDHg@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"

Hey everyone,

So I’m currently working on a pure Swift Postgres driver. It has no
dependency on libpq at all. I’m parsing the Postgres Wire Protocol (V3)
myself and everything seems great. Here are some problems/issues I’m facing
that db driver implementers like myself (don’t shoot me this is my first
shot at a db driver ;) ) would like to see resolved by this swift server
group or we could use some guidance on:

1. A standard socket api, there are a few right now. Currently I’m using
IBM’s BlueSocket but really we just need one socket api that works cross
platform. From the blog post it seems like that falls under this groups
umbrella/objectives

2. A lil more guidance around timestamps + time zones. These are super
important for decent compliant swift db drivers. Can we rely on Foundation
NSDate and timezones to work well cross platform without issues here?

3. A standard lightweight crypto library to handle authentication. This is
something all database driver authors will have to deal with (MD5 hashing
etc). During the summer I had been switching between a few libraries but
would be great to have this unified, cross platform support for this. Again
I think this falls under the umbrella of this organization.

4. Coming from a diverse backend/server background, I’ve seen different
lang+environments handle sql interfaces differently. Most have left it up
to third party frameworks/ecosystems (ruby/node/python) but some like
Golang provide at least a default SQL interface that driver authors can
adapt to. I’m starting to lean towards the latter of the two but I’m by no
means and expert on anything :). This probably needs more community
engagement/proposal.

In short, for me at least, these 4 points would be great to be addressed
(the 4th one perhaps is debatable and not as urgent), for database driver
authors would greatly help a lot. Pretty much every server side swift
framework right now has their own ORM-ish libs/frameworks. That’s awesome!
My hope is we can start writing some of the drivers in pure Swift and help
all those frameworks/libs get better too.

Just my 2 cents. Perhaps this deserves a thread of its own. Perhaps it
doesn’t. I apologize in advance if this is distracting from the current
thread :)

--
Haris Amin
Sent with Airmail

On October 25, 2016 at 3:21:07 PM, Paulo Faria via swift-server-dev ( > swift-server-dev@swift.org) wrote:

Hello, Helge.

Yes, most of the things we’ll cover are already provided by some
frameworks, but having an official library joining efforts will be great
for the community. We’ll provide full-fledged Swift libraries for the
topics mentioned. How they’re going to be developed it’s still to be
decided.

Cheers,
Paulo

> On Oct 25, 2016, at 4:56 PM, Helge Heß via swift-server-dev < > swift-server-dev@swift.org> wrote:
>
> Hi Chris,
>
> it is a little unclear what the scope of the group is. Presumably you
have some very specific things in mind?
>
> Of course we’ve seen https://swift.org/server-apis/#focus-areas\. But
what
does it actually mean? :-) Looking at it:
> - base networking: TCP/UDP, DNS. Covered already by libdispatch and the
OS’ses?
> - security: OpenSSL? or some wrapper around both, CC and OpenSSL?
> - provide low level HTTP parsing: http_parser? Is anyone using something
> else?
> All that seems kinda covered/standard already? Unless maybe you include
non-Unix stuff like Windows.
>
> Is this primarily for things like providing a standardised ’swiftier'
OpenSSL wrapper (similar to how Objective-GCD wraps libdispatch)?
> Or would that just be module maps which are shipped by default with Swift
and hence can be relied on by all frameworks?
>
> But maybe I’m just a little too impatient :->
>
> hh
>
>> On 25 Oct 2016, at 20:41, Chris Bailey via swift-server-dev < > swift-server-dev@swift.org> wrote:
>>
>> Hi Ron:
>>
>> As you say, that's outside of the scope of the work group, however I do
think its a valuable area that the various existing web framework groups
might be interested in collaborating on. Having a common, recognized
framework that the database providers or third parties would be interested
in writing database drivers for would be a huge step forward.
>>
>> I know that Vapor has its Fluent ORM framework, and that its an area
that the Kitura team are looking at as well, so your post might kick-start
some wider discussion.
>>
>> Chris
>>
>>
>>
>>
>> From: Ron Olson via swift-server-dev <swift-server-dev@swift.org>
>> To: swift-server-dev@swift.org
>> Date: 25/10/2016 18:54
>> Subject: [swift-server-dev] Database API/Framework
>> Sent by: swift-server-dev-bounces@swift.org
>>
>>
>>
>> Hi all-
>>
>> I've been working with Swift more on Linux than macOS and am very
>> excited to see the formation of this group. Looking at the server api
>> workgroup's focus, I notice it's primarily low-level interaction with
>> the operating system. When might it be a good time to bring up the
>> possibility of creating a database-specific framework for those folks
>> who want to work directly with Postgres, MySQL, even DB2 and Oracle; I'm
>> thinking a JDBC-inspired framework that drivers could be written
>> against.
>>
>> Thanks,
>>
>> Ron
>> _______________________________________________
>> swift-server-dev mailing list
>> swift-server-dev@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-server-dev
>>
>>
>> _______________________________________________
>> swift-server-dev mailing list
>> swift-server-dev@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-server-dev
>
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
https://lists.swift.org/pipermail/swift-server-dev/attachments/20161025/12d9cfc6/attachment-0001.html
>

------------------------------

Message: 8
Date: Tue, 25 Oct 2016 22:44:03 +0200
From: Helge Heß <me@helgehess.eu>
To: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
Subject: Re: [swift-server-dev] Database API/Framework
Message-ID: <96EBE4B7-013D-42E9-BDC4-715BB6ABD1F6@helgehess.eu>
Content-Type: text/plain; charset=utf-8

Hi,

On 25 Oct 2016, at 21:34, Haris Amin <aminharis7@gmail.com> wrote:
> So I’m currently working on a pure Swift Postgres driver.

nice, I’m working on one as well (currently can run only basic queries,
nothing fancy).

> 1. A standard socket api, there are a few right now. Currently I’m using
IBM’s BlueSocket but really we just need one socket api that works cross
platform.

I would be interested what people are really missing here (at that basic
level). GCD channels are really quite nice for that kind of stuff
(scheduling IO) and are already part of Swift 3. (I personally wonder how
they scale compared to something like libuv, but for now I assume they do
;-)

For making the Unix calls a little nicer Swift extensions on the Unix
types are IMO pretty awesome, something along those lines:

https://github.com/NozeIO/Noze.io/blob/master/Sources/net/SocketAddress.swift

I would avoid creating wrappers which are then just wrapped again by the
higher level frameworks …

The only real thing I’m missing is SSL support. But that already depends
quite a lot on how the higher level frameworks deal with streams in general.

> 2. A lil more guidance around timestamps + time zones. These are super
important for decent compliant swift db drivers. Can we rely on Foundation
NSDate and timezones to work well cross platform without issues here?

If it is part of Foundation you can assume it is supposed to work, right?
:-)

> 3. A standard lightweight crypto library to handle authentication. This
is something all database driver authors will have to deal with (MD5
hashing etc). During the summer I had been switching between a few
libraries but would be great to have this unified, cross platform support
for this. Again I think this falls under the umbrella of this organization.

+1 (And that should probably be part of Foundation as it is required just
the same on the client side).

> 4. Coming from a diverse backend/server background, I’ve seen different
lang+environments handle sql interfaces differently. Most have left it up
to third party frameworks/ecosystems (ruby/node/python) but some like
Golang provide at least a default SQL interface that driver authors can
adapt to. I’m starting to lean towards the latter of the two but I’m by no
means and expert on anything :). This probably needs more community
engagement/proposal.

IMHO that sounds more like something the higher level frameworks (ORMs)
should be concerned about (as they will define what they want to map from
and how, and probably in pretty different ways). But maybe not. Personally
I’d like to avoid something like JDBC as it provides little value at extra
overhead.

As an example: if I have received a DispatchData from the socket
containing a PG row values, I don’t want to first convert that to a JDBC or
EOAdaptor record-Dictionary<Key,Any>> which is then mapped to a
Customer.purchaseStatus enum. I would rather like to pass up that
DispatchData to the highest level and only have it converted at the very
last point and only if necessary - sometimes an ORM, but other times it may
be just directly streamed into a different format, say JSON (in such cases
you can often accomplish zero-copy).

> My hope is we can start writing some of the drivers in pure Swift and
help all those frameworks/libs get better too.

Not sure a generic ‘driver' interface will help a lot here. I think you
help most by providing a SwiftPQ library which can then be integrated by
the ORMs/frameworks in a way which fits best.

Anywayz, interesting to see what people come up with :-)

hh

------------------------------

Message: 9
Date: Tue, 25 Oct 2016 15:55:23 -0500
From: "Ron Olson" <tachoknight@gmail.com>
To: swift-server-dev@swift.org
Subject: Re: [swift-server-dev] Database API/Framework
Message-ID: <BA9D7BA9-3E1A-47C3-81E8-8DE02817B74D@gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed

I mentioned JDBC because I happened to be looking at some Java code at
the time, but I really meant 'generic-interface' to provide the
lowest-level-but-no-lower-than-necessary access to the database,
regardless of whether it was sqlite or postgres. I think Go does a
pretty good job of providing that thin layer of abstraction and it's up
the driver writer to implement. After that what somebody wants to do
with the results is their business.

Personally I don't like ORM layers; most of the time I'm writing complex
queries that don't fit neatly into an ORM paradigm and I end up fighting
with the framework.

Ron

On 25 Oct 2016, at 15:44, Helge Heß via swift-server-dev wrote:

> Hi,
>
> On 25 Oct 2016, at 21:34, Haris Amin <aminharis7@gmail.com> wrote:
>> So I’m currently working on a pure Swift Postgres driver.
>
> nice, I’m working on one as well (currently can run only basic
> queries, nothing fancy).
>
>> 1. A standard socket api, there are a few right now. Currently I’m
>> using IBM’s BlueSocket but really we just need one socket api that
>> works cross platform.
>
> I would be interested what people are really missing here (at that
> basic level). GCD channels are really quite nice for that kind of
> stuff (scheduling IO) and are already part of Swift 3. (I personally
> wonder how they scale compared to something like libuv, but for now I
> assume they do ;-)
>
> For making the Unix calls a little nicer Swift extensions on the Unix
> types are IMO pretty awesome, something along those lines:
>
>
https://github.com/NozeIO/Noze.io/blob/master/Sources/net/SocketAddress.swift
>
> I would avoid creating wrappers which are then just wrapped again by
> the higher level frameworks …
>
> The only real thing I’m missing is SSL support. But that already
> depends quite a lot on how the higher level frameworks deal with
> streams in general.
>
>> 2. A lil more guidance around timestamps + time zones. These are
>> super important for decent compliant swift db drivers. Can we rely on
>> Foundation NSDate and timezones to work well cross platform without
>> issues here?
>
> If it is part of Foundation you can assume it is supposed to work,
> right? :-)
>
>> 3. A standard lightweight crypto library to handle authentication.
>> This is something all database driver authors will have to deal with
>> (MD5 hashing etc). During the summer I had been switching between a
>> few libraries but would be great to have this unified, cross platform
>> support for this. Again I think this falls under the umbrella of this
>> organization.
>
> +1 (And that should probably be part of Foundation as it is required
> just the same on the client side).
>
>> 4. Coming from a diverse backend/server background, I’ve seen
>> different lang+environments handle sql interfaces differently. Most
>> have left it up to third party frameworks/ecosystems
>> (ruby/node/python) but some like Golang provide at least a default
>> SQL interface that driver authors can adapt to. I’m starting to
>> lean towards the latter of the two but I’m by no means and expert
>> on anything :). This probably needs more community
>> engagement/proposal.
>
> IMHO that sounds more like something the higher level frameworks
> (ORMs) should be concerned about (as they will define what they want
> to map from and how, and probably in pretty different ways). But maybe
> not. Personally I’d like to avoid something like JDBC as it provides
> little value at extra overhead.
>
> As an example: if I have received a DispatchData from the socket
> containing a PG row values, I don’t want to first convert that to a
> JDBC or EOAdaptor record-Dictionary<Key,Any>> which is then mapped to
> a Customer.purchaseStatus enum. I would rather like to pass up that
> DispatchData to the highest level and only have it converted at the
> very last point and only if necessary - sometimes an ORM, but other
> times it may be just directly streamed into a different format, say
> JSON (in such cases you can often accomplish zero-copy).
>
>> My hope is we can start writing some of the drivers in pure Swift and
>> help all those frameworks/libs get better too.
>
> Not sure a generic ‘driver' interface will help a lot here. I think
> you help most by providing a SwiftPQ library which can then be
> integrated by the ORMs/frameworks in a way which fits best.
>
> Anywayz, interesting to see what people come up with :-)
>
> hh
>
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev

------------------------------

Message: 10
Date: Wed, 26 Oct 2016 00:00:35 +0200
From: Alex Blewitt <alblue@apple.com>
To: frogcjn@163.com
Cc: swift-server-dev <swift-server-dev@swift.org>
Subject: Re: [swift-server-dev] [Bug] Foundation.URL should support
        semicolon(; ) appearing in URL path
Message-ID: <3A980DA0-E213-4DBE-B928-C36094446C00@apple.com>
Content-Type: text/plain; charset=utf-8

> On 25 Oct 2016, at 21:09, Cao, Jiannan via swift-server-dev < > swift-server-dev@swift.org> wrote:
>
>
> http://host/book;id=1/page;2/
>
> this URL is a valid URL, but URLComponents will ruin it with %3B
> http://host/book%3Bid=1/page%3B2/
>
> Since Swift Server is coming out, this bug should be solved.

Can you report this on https://bugs.swift.org so that it's tracked
appropriately?

Thanks,

Alex

Sent from my iPhone :iphone:

------------------------------

Message: 11
Date: Tue, 25 Oct 2016 16:11:55 -0700
From: Rick Mann <rmann@latencyzero.com>
To: swift-server-dev@swift.org
Subject: [swift-server-dev] Spring and Baratine (Java frameworks)
Message-ID: <1736F138-7246-460C-9CC2-781446A1C90F@latencyzero.com>
Content-Type: text/plain; charset=us-ascii

Hi all,

I'm very excited for the formation of this area of development. While I
might consider myself primarily a macOS and iOS app developer, I've done my
fair share of Java webserver development, as well as a lot of small
embedded development. You can't make a non-trivial app these days without
some kind of server-side support.

I realize the initial focus of this working group is fairly low-level
support for server-side development, but in much the same way that Swift
development is made richer by the availability of the robust Swift Standard
Library, so to is web development.

In my estimation, the most current iterations of Spring.io, and the
up-and-coming Baratine.io*, offer insights into powerful web development
paradigms that I'd like to see streamlined and Swiftified. While this group
is avoiding such a high-level, it strikes me that thinking about the high
level can greatly inform how the low-level APIs evolve.

One of the things that Java frameworks take advantage of is Java's rich
introspective ability, and particularly lately use @Annotations as a way to
simplify configuration and reduce boilerplate code. This typically expands
into dynamic creation of proxy classes, and I'm not sure how Swift's
introspection and code-generation capabilities stack up. I know Java-style
@Annotations are on the roadmap in some form, but not until after ABI
stability.

Unfortunately, I think that means any Swift webapp framework will suffer
problems similar to that of Spring: a clunky datafile-based configuration
structure that becomes obsolete when more introspection features become
available, creating a wealth of out-of-date instructional material online
and a lot of technical debt.

Hopefully I'm being overly pessimistic, and we're able to enjoy elegant,
powerful, scalable web development in the near future.

In any case, I hope I can make a positive contribution.

--
Rick Mann
rmann@latencyzero.com

*Baratine is incredibly performant and substantially relieves the
developer from the persistence burden, but IMHO currently is quite verbose
when it comes to implementing something like REST endpoints, and has a very
steep learning (grokking?) curve.

------------------------------

Message: 12
Date: Tue, 25 Oct 2016 17:30:31 -0700
From: Tyler Cloutier <cloutiertyler@aol.com>
To: swift-server-dev@swift.org
Subject: [swift-server-dev] Lib dispatch and Edge
Message-ID: <56052D8D-8D9D-43E9-B9BF-0A77A09B04D2@aol.com>
Content-Type: text/plain; charset=utf-8

Hello everyone!

I just wanted to say that I look forward to contributing to this group! I
have been working on server side Swift for a while and I’ve created the
project, Edge, on Github. GitHub - skylab-inc/Lightning: A Swift Multiplatform Single-threaded Non-blocking Web and Networking Framework

It uses the libdispatch (Dispatch) api to create a nodejs like
asynchronous server APIs. I would love to be able to contribute some of my
work to the Server APIs project.

Thank you,

Tyler

------------------------------

Message: 13
Date: Wed, 26 Oct 2016 10:00:59 +0100
From: Yoann Gini <yoann.gini@gmail.com>
To: swift-server-dev@swift.org
Subject: [swift-server-dev] Why not rely on existing models?
Message-ID: <E3E74862-0681-43F3-B526-B75DD3BE497F@gmail.com>
Content-Type: text/plain; charset="utf-8"

Hi

I’m following Swift server side initiatives really closely. I’m a
developer and sys admin, really interested in subjects linked to
performance and security.

Swift look like a perfect language for server side, especially with Cocoa
framework on the back.

Looking at all the current initiative, I’ve to say don’t understand
something: why everyone start by creating a brand new HTTP server?

From a DevOps point of view, it looks like a waste of resources to start
writing HTTP server framework in swift. No sys admin will ever allow a
small custom HTTP server to be open to the Internet without a proxy before.
Plus, we already have two really great HTTP servers in the UNIX world,
Apache and NGINX. In no way a custom HTTP language related framework can
cover as much as those two projects.

Even if I understand some people might want built-in HTTP handler in a
client/server app, I don’t really understand why all existing initiatives
take this road.

Looking at what currently exists, I found WSGI (or even old school CGI)
way more interesting for production purpose.

If we are trying to deploy Swift on the server side, CGI alike solution
look like the quick win. It avoid wasting dev time by reusing all HTTP
server already existing, so developer from this community will have to
concentrate only to create a bridge. And even more, this bridge can be
created in the first version by simply porting WSGI behavior to Swift.

This is just a simple personal opinion. Just wondering why.

Best regards,
Yoann Gini

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
https://lists.swift.org/pipermail/swift-server-dev/attachments/20161026/8124b46d/attachment-0001.html
>

------------------------------

Message: 14
Date: Wed, 26 Oct 2016 11:25:16 +0200
From: Helge Heß <me@helgehess.eu>
To: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
Subject: Re: [swift-server-dev] Why not rely on existing models?
Message-ID: <8296A4EE-B0C1-449D-8F64-3C50231CB0F9@helgehess.eu>
Content-Type: text/plain; charset=us-ascii

Is there a need for a swift-server-users mailing list?

hh

------------------------------

Message: 15
Date: Wed, 26 Oct 2016 02:34:10 -0700
From: Brent Royal-Gordon <brent@architechies.com>
To: swift-server-dev@swift.org
Subject: [swift-server-dev] Re: Database API/Framework
Message-ID: <1FED5955-F653-4D36-8C82-986695A01CBE@architechies.com>
Content-Type: text/plain; charset=utf-8

(Sorry for the fake threading; I read this thread in the web archive and
wanted to chime in.)

Helge Heß:

> > 4. Coming from a diverse backend/server background, I’ve seen
different lang+environments handle sql interfaces differently. Most have
left it up to third party frameworks/ecosystems (ruby/node/python) but some
like Golang provide at least a default SQL interface that driver authors
can adapt to. I’m starting to lean towards the latter of the two but I’m by
no means and expert on anything :). This probably needs more community
engagement/proposal.
>
>
> IMHO that sounds more like something the higher level frameworks (ORMs)
should be concerned about (as they will define what they want to map from
and how, and probably in pretty different ways). But maybe not. Personally
I’d like to avoid something like JDBC as it provides little value at extra
overhead.
>
> As an example: if I have received a DispatchData from the socket
containing a PG row values, I don’t want to first convert that to a JDBC or
EOAdaptor record-Dictionary<Key,Any>> which is then mapped to a
Customer.purchaseStatus enum. I would rather like to pass up that
DispatchData to the highest level and only have it converted at the very
last point and only if necessary - sometimes an ORM, but other times it may
be just directly streamed into a different format, say JSON (in such cases
you can often accomplish zero-copy).
>
> > My hope is we can start writing some of the drivers in pure Swift and
help all those frameworks/libs get better too.
>
>
> Not sure a generic ‘driver' interface will help a lot here. I think you
help most by providing a SwiftPQ library which can then be integrated by
the ORMs/frameworks in a way which fits best.

I think Perl has a good model we might want to pay attention to. Perl has
a largely database-independent library called DBI (Database Interface)
which basically just standardizes the interfaces to connect to databases,
run queries, iterate through results, examine schemas, and so on. A system
of plug-ins, called DBDs (Database Drivers), handle each individual
database. Applications can either use DBI directly or through a library
that functions as an ORM. (Most of these higher-level libraries are called
DBI Extensions (DBIx), though this is just a convention.)

                Drivers Interface Extension/Client
        DBD::MySQL -+ +- DBIx::Class
        DBD::PG -+----- DBI ----- +- DBIx::DataModel
        DBD::Oracle -+ +- Direct use

DBI doesn't abstract away *all* differences between databases. But it gets
most of them, and in practice, it's not very difficult to switch from one
database to another, or use different databases in development and
production. It saves individual ORMs from needing to individually implement
different database backends, and facilitates sharing of code for features
like connection pooling. If the interest is there, this can support a wide
and varied ecosystem: Perl's CPAN module repository includes 103 DBD
modules and 473 DBIx modules.

In Swift, we may also be able to use it to encourage good database
practices. The control Swift offers over string literals and interpolation
could be leveraged to encourage use of prepared statements and good
escaping (or rather, use of parameterized queries instead of escaping).
Imagine if, when you wrote something like this, it was perfectly safe:

        let userID = …
        for row in try connection.query("SELECT * FROM posts WHERE user_id
= \(userID)") {
                // `query`'s parameter is actually a `SQLStatement`, so
the string literal is actually a SQL statement literal.
                …
        }

Because `userID` would be an `Int`—that is, a type expected to contain
data—it would be automatically passed as a parameter so there were no
escaping issues. If you used a type for a table, column, or fragment of a
statement, on the other hand, it would be handled appropriately:

        let userID = …
        let sortColumnName = …
        let ascending = …

        let postsTable = try! SQLTable("posts", in: connection)
        let sortColumn = try SQLColumn(named: sortColumnName, in:
postsTable)
        let sortDirection: SQLFragment = ascending ? "ASC" : "DESC"

        for row in try connection.query("SELECT * FROM \(postsTable) WHERE
user_id = \(userID) ORDER BY \(sortColumn) \(sortDirection)") {
                // In the above, the SQLTable, SQLColumn, and
SQLStatementFragment are inserted without escaping,
                // while the non-database-specific userID is passed as a
parameter.
                …
        }

This is cool stuff that the dynamic languages don't offer, but you
probably don't want to write it for each individual database driver. You
want to write it once, either at the database-independent interface layer
or as a layer above it. To do that, you need to funnel all database use
through a single chokepoint. In other words, you need something like DBI.

I'm not saying we should adopt DBI exactly as it is—Swift would demand a
very different design. But I think we should consider using a similar
architecture. It has a lot of advantages over forcing high-level frameworks
to talk directly to low-level databases.

--
Brent Royal-Gordon
Architechies

------------------------------

Message: 16
Date: Wed, 26 Oct 2016 10:43:07 +0100
From: Yoann Gini <yoann.gini@gmail.com>
To: Helge Heß <me@helgehess.eu>
Cc: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
Subject: Re: [swift-server-dev] Why not rely on existing models?
Message-ID: <D85F7505-334D-43D0-91BF-2183416636E2@gmail.com>
Content-Type: text/plain; charset=utf-8

> Le 26 oct. 2016 à 10:25, Helge Heß via swift-server-dev < > swift-server-dev@swift.org> a écrit :
>
> Is there a need for a swift-server-users mailing list?

It depends on whether you consider a language is made for the language
itself or for production. Considering real case usage scenarios and
constraints might be a good idea in any kind of software development.

In particular, advocating for WSGI alike integration isn’t a user
consideration but a request for the server side app developers who would
like to avoid coming back to the Stone Age of Tomcat.

Best regards,
Yoann Gini

------------------------------

Message: 17
Date: Wed, 26 Oct 2016 02:59:23 -0700
From: Brent Royal-Gordon <brent@architechies.com>
To: Yoann Gini <yoann.gini@gmail.com>
Cc: swift-server-dev@swift.org
Subject: Re: [swift-server-dev] Why not rely on existing models?
Message-ID: <ABF1424E-6E00-4E3D-B68A-82A590D74143@architechies.com>
Content-Type: text/plain; charset=utf-8

> On Oct 26, 2016, at 2:00 AM, Yoann Gini via swift-server-dev < > swift-server-dev@swift.org> wrote:
>
> Looking at all the current initiative, I’ve to say don’t understand
something: why everyone start by creating a brand new HTTP server?
>
> From a DevOps point of view, it looks like a waste of resources to start
writing HTTP server framework in swift. No sys admin will ever allow a
small custom HTTP server to be open to the Internet without a proxy before.
Plus, we already have two really great HTTP servers in the UNIX world,
Apache and NGINX. In no way a custom HTTP language related framework can
cover as much as those two projects.
>
> Even if I understand some people might want built-in HTTP handler in a
client/server app, I don’t really understand why all existing initiatives
take this road.

This trend seems to have started with Rails. It has a few advantages:

1. You can use the embedded server in development and testing. This is
*way* easier than installing Apache on your development machine or CI setup
and configuring it to point to your application.

2. In production, you can configure Apache or nginx to reverse proxy for
the app; this configuration is almost entirely agnostic to the application
itself, so you can modify the application's behavior without playing with
the frontend web server.

3. The frontend proxy and backend web server communicate over a plain old
socket, so they can run as different user accounts or even in separate VMs.

Basically, embedding a web server and reverse proxying for it isolates the
application from the frontend server and simplifies the management of both.

--
Brent Royal-Gordon
Architechies

------------------------------

Message: 18
Date: Wed, 26 Oct 2016 12:26:51 +0200
From: Helge Heß <me@helgehess.eu>
To: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
Subject: Re: [swift-server-dev] Database API/Framework
Message-ID: <D7AB25BE-3BAF-4C9B-A2C4-2B0A6243A93F@helgehess.eu>
Content-Type: text/plain; charset=utf-8

On 26 Oct 2016, at 11:34, Brent Royal-Gordon via swift-server-dev < > swift-server-dev@swift.org> wrote:
> I'm not saying we should adopt DBI exactly as it is—Swift would demand a
very different design. But I think we should consider using a similar
architecture. It has a lot of advantages over forcing high-level frameworks
to talk directly to low-level databases.

This is essentially the original message of the request, the desire to
have a JDBC. Maybe people really want to have that :-) I guess a few
protocols for simple use cases do not hurt, but I would advise against over
designing this.

IMHO it doesn’t make that much sense since todays databases are so
different. If you are a PostgreSQL user, you quite likely want to make use
of its features (be it JSON columns, FTS, custom types, row permissions,
table inheritance, etc).
Even pooling has very different requirements based on the setup of the
database (e.g. in PG connections are pretty expensive, in others not at
all, and SQLite doesn’t even have them). Let alone transactions and how
they behave.
Presumably a HL framework has to have specific support to account for
dealing with such differences, it can’t just rely on ‘JDBC'.

Personally I’m more interested in comprehensive interaction modules for
the respective databases that can be used by all HL projects than in an API
trying to bundle everything into a set of basic protocols. It is the HL
project presenting a consistent API to the enduser (be it a full ORM or
something more low level [e.g. I would rather stream batches of tuples
coming from the database than building an EOF/CoreData like object graph]).

But I’m quite interested to see what proposals are coming in :-)

hh

------------------------------

Message: 19
Date: Wed, 26 Oct 2016 11:33:43 +0100
From: Yoann Gini <yoann.gini@gmail.com>
To: Brent Royal-Gordon <brent@architechies.com>
Cc: swift-server-dev@swift.org
Subject: Re: [swift-server-dev] Why not rely on existing models?
Message-ID: <7D95AB95-528E-43AC-9594-4E4A2A603E6F@gmail.com>
Content-Type: text/plain; charset=utf-8

> Le 26 oct. 2016 à 10:59, Brent Royal-Gordon <brent@architechies.com> a
écrit :
>
> This trend seems to have started with Rails. It has a few advantages:
>
> 1. You can use the embedded server in development and testing. This is
*way* easier than installing Apache on your development machine or CI setup
and configuring it to point to your application.

100% agree with that

> 2. In production, you can configure Apache or nginx to reverse proxy for
the app; this configuration is almost entirely agnostic to the application
itself, so you can modify the application's behavior without playing with
the frontend web server.
>
> 3. The frontend proxy and backend web server communicate over a plain
old socket, so they can run as different user accounts or even in separate
VMs.
>
> Basically, embedding a web server and reverse proxying for it isolates
the application from the frontend server and simplifies the management of
both.

I’m not really sure this is a pros or cons. Having to run multiple
processes and use sockets increase code complexity and surface for attacks.
It increase the number of services started so the number of processes to
watch.

In some scenario, this is better, in some others, this is worst.

As I’ve said on my first e-mail, I don’t say standalone swift HTTP server
must not exist. I’ve just said it could be interesting to think about
others patterns.

Allowing behavior like WSGI where you can use both scenario is IMHO the
best solution ever, so it can fit for all needs.

Best regards,
Yoann Gini

------------------------------

Message: 20
Date: Wed, 26 Oct 2016 12:39:01 +0100
From: Chris Bailey <BAILEYC@uk.ibm.com>
To: Helge Heß <me@helgehess.eu>
Cc: swift-server-dev@swift.org
Subject: Re: [swift-server-dev] Database API/Framework
Message-ID:
        <
OF4B342709.861A5D1A-ON80258058.00369FEC-80258058.003FFEF3@notes.na.collabserv.com
>

Content-Type: text/plain; charset="utf-8"

Hi Helge:

As the initial target areas are low-level function, yes, each of the
frameworks has an implementation of most of them today. Those
implementations tend to be limited in a number of ways:
C libraries are often used directly (via a modulemap)
This means that there's no general "Swift API" that developers can use -
you have to interface with the C APIs directly and deal with
Unsafe/OpaquePointers etc. Ideally we want Swift developers to only have
to write Swift code
Where a Swift API wraps a C library its generally for a specific use case
As the framework teams have a very specific use case - using it in their
framework - the APIs are very directed to what they need to do, rather
than being general purpose for a wider set of use cases
There's multiple, similar, packages
Most frameworks have their own server socket implementation (mostly by
calling Glibc and Darwin), HTTP parser implementation, etc. Not only does
this mean we can consolidate the efforts on a single implementation, but
it means that anyone wanting to add additional platform support (like
Windows) only has to do so once.

Chris

From: Helge Heß via swift-server-dev <swift-server-dev@swift.org>
To: swift-server-dev@swift.org
Date: 25/10/2016 19:57
Subject: Re: [swift-server-dev] Database API/Framework
Sent by: swift-server-dev-bounces@swift.org

Hi Chris,

it is a little unclear what the scope of the group is. Presumably you have
some very specific things in mind?

Of course we’ve seen https://swift.org/server-apis/#focus-areas\. But what
does it actually mean? :-) Looking at it:
- base networking: TCP/UDP, DNS. Covered already by libdispatch and the
OS’ses?
- security: OpenSSL? or some wrapper around both, CC and OpenSSL?
- provide low level HTTP parsing: http_parser? Is anyone using something
  else?
All that seems kinda covered/standard already? Unless maybe you include
non-Unix stuff like Windows.

Is this primarily for things like providing a standardised ’swiftier'
OpenSSL wrapper (similar to how Objective-GCD wraps libdispatch)?
Or would that just be module maps which are shipped by default with Swift
and hence can be relied on by all frameworks?

But maybe I’m just a little too impatient :->

hh

> On 25 Oct 2016, at 20:41, Chris Bailey via swift-server-dev > <swift-server-dev@swift.org> wrote:
>
> Hi Ron:
>
> As you say, that's outside of the scope of the work group, however I do
think its a valuable area that the various existing web framework groups
might be interested in collaborating on. Having a common, recognized
framework that the database providers or third parties would be interested
in writing database drivers for would be a huge step forward.
>
> I know that Vapor has its Fluent ORM framework, and that its an area
that the Kitura team are looking at as well, so your post might kick-start
some wider discussion.
>
> Chris
>
>
>
>
> From: Ron Olson via swift-server-dev <swift-server-dev@swift.org>

> To: swift-server-dev@swift.org
> Date: 25/10/2016 18:54
> Subject: [swift-server-dev] Database API/Framework
> Sent by: swift-server-dev-bounces@swift.org
>
>
>
> Hi all-
>
> I've been working with Swift more on Linux than macOS and am very
> excited to see the formation of this group. Looking at the server api
> workgroup's focus, I notice it's primarily low-level interaction with
> the operating system. When might it be a good time to bring up the
> possibility of creating a database-specific framework for those folks
> who want to work directly with Postgres, MySQL, even DB2 and Oracle; I'm

> thinking a JDBC-inspired framework that drivers could be written
> against.
>
> Thanks,
>
> Ron
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev
>
>
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <
https://lists.swift.org/pipermail/swift-server-dev/attachments/20161026/f87a0fbc/attachment-0001.html
>

------------------------------

Message: 21
Date: Wed, 26 Oct 2016 14:35:03 +0100
From: Gregor Milos <gmilos@apple.com>
To: Chris Bailey <BAILEYC@uk.ibm.com>
Cc: swift-server-dev@swift.org
Subject: Re: [swift-server-dev] Database API/Framework
Message-ID: <C2BA24EC-223C-4518-AD1E-057EB1585FAC@apple.com>
Content-Type: text/plain; charset=utf-8

Helge,

Expand on Chris's answer, modern languages used on server side come with
abstractions native to their language. Taking the example of of HTTP
protocol:
* Go comes with http package - net/http - Go Packages
* Netty (defacto standard Java net lib) comes with:
io.netty.handler.codec.http (Netty API Reference (4.1.101.Final))
* etc ...
Our task should be to design an API that works well in Swift. Node's
http_parser is "just" an example of an API and a possible way of
accelerating the implementation of Swift's HTTP parser.

Thanks
Gregor

> On 26 Oct 2016, at 12:39, Chris Bailey via swift-server-dev < > swift-server-dev@swift.org> wrote:
>
> Hi Helge:
>
> As the initial target areas are low-level function, yes, each of the
frameworks has an implementation of most of them today. Those
implementations tend to be limited in a number of ways:
> • C libraries are often used directly (via a modulemap)
> This means that there's no general "Swift API" that developers can use -
you have to interface with the C APIs directly and deal with
Unsafe/OpaquePointers etc. Ideally we want Swift developers to only have to
write Swift code
> • Where a Swift API wraps a C library its generally for a specific
use case
> As the framework teams have a very specific use case - using it in their
framework - the APIs are very directed to what they need to do, rather than
being general purpose for a wider set of use cases
> • There's multiple, similar, packages
> Most frameworks have their own server socket implementation (mostly by
calling Glibc and Darwin), HTTP parser implementation, etc. Not only does
this mean we can consolidate the efforts on a single implementation, but it
means that anyone wanting to add additional platform support (like Windows)
only has to do so once.
>
> Chris
>
>
>
>
> From: Helge Heß via swift-server-dev <swift-server-dev@swift.org>
> To: swift-server-dev@swift.org
> Date: 25/10/2016 19:57
> Subject: Re: [swift-server-dev] Database API/Framework
> Sent by: swift-server-dev-bounces@swift.org
>
>
>
> Hi Chris,
>
> it is a little unclear what the scope of the group is. Presumably you
have some very specific things in mind?
>
> Of course we’ve seen https://swift.org/server-apis/#focus-areas\. But
what does it actually mean? :-) Looking at it:
> - base networking: TCP/UDP, DNS. Covered already by libdispatch and the
OS’ses?
> - security: OpenSSL? or some wrapper around both, CC and OpenSSL?
> - provide low level HTTP parsing: http_parser? Is anyone using something
> else?
> All that seems kinda covered/standard already? Unless maybe you include
non-Unix stuff like Windows.
>
> Is this primarily for things like providing a standardised ’swiftier'
OpenSSL wrapper (similar to how Objective-GCD wraps libdispatch)?
> Or would that just be module maps which are shipped by default with
Swift and hence can be relied on by all frameworks?
>
> But maybe I’m just a little too impatient :->
>
> hh
>
> > On 25 Oct 2016, at 20:41, Chris Bailey via swift-server-dev < > swift-server-dev@swift.org> wrote:
> >
> > Hi Ron:
> >
> > As you say, that's outside of the scope of the work group, however I
do think its a valuable area that the various existing web framework groups
might be interested in collaborating on. Having a common, recognized
framework that the database providers or third parties would be interested
in writing database drivers for would be a huge step forward.
> >
> > I know that Vapor has its Fluent ORM framework, and that its an area
that the Kitura team are looking at as well, so your post might kick-start
some wider discussion.
> >
> > Chris
> >
> >
> >
> >
> > From: Ron Olson via swift-server-dev <
swift-server-dev@swift.org>
> > To: swift-server-dev@swift.org
> > Date: 25/10/2016 18:54
> > Subject: [swift-server-dev] Database API/Framework
> > Sent by: swift-server-dev-bounces@swift.org
> >
> >
> >
> > Hi all-
> >
> > I've been working with Swift more on Linux than macOS and am very
> > excited to see the formation of this group. Looking at the server api
> > workgroup's focus, I notice it's primarily low-level interaction with
> > the operating system. When might it be a good time to bring up the
> > possibility of creating a database-specific framework for those folks
> > who want to work directly with Postgres, MySQL, even DB2 and Oracle;
I'm
> > thinking a JDBC-inspired framework that drivers could be written
> > against.
> >
> > Thanks,
> >
> > Ron
> > _______________________________________________
> > swift-server-dev mailing list
> > swift-server-dev@swift.org
> > https://lists.swift.org/mailman/listinfo/swift-server-dev
> >
> >
> > _______________________________________________
> > swift-server-dev mailing list
> > swift-server-dev@swift.org
> > https://lists.swift.org/mailman/listinfo/swift-server-dev
>
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev
>
>
> _______________________________________________
> swift-server-dev mailing list
> swift-server-dev@swift.org
> https://lists.swift.org/mailman/listinfo/swift-server-dev

------------------------------

Message: 22
Date: Wed, 26 Oct 2016 14:41:00 +0100
From: Gregor Milos <gmilos@apple.com>
To: Gregor Milos <gmilos@apple.com>
Cc: swift-server-dev@swift.org
Subject: Re: [swift-server-dev] Database API/Framework
Message-ID: <A338CA7D-6E7E-4D1E-B10C-C81764DDD909@apple.com>
Content-Type: text/plain; CHARSET=US-ASCII

> On 26 Oct 2016, at 14:35, Gregor Milos via swift-server-dev < > swift-server-dev@swift.org> wrote:
>
> Expand
s/Expand/Expanding/

------------------------------

Message: 23
Date: Wed, 26 Oct 2016 16:39:20 +0200
From: Helge Heß <me@helgehess.eu>
To: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
Subject: Re: [swift-server-dev] Database API/Framework
Message-ID: <C7A408BB-7EAC-4573-A01F-343588116F77@helgehess.eu>
Content-Type: text/plain; charset="utf-8"

Hi Gregor,

On 26 Oct 2016, at 15:35, Gregor Milos <gmilos@apple.com> wrote:
> Expand on Chris's answer, modern languages used on server side come with
abstractions native to their language.

now that is a pretty bold claim. C, C++, Java, JavaScript, Ruby and
Objective-C, don’t. Python does, but few use that. But yes, Go seems to
come with one … :-)

> * Netty (defacto standard Java net lib)

Hu?! Since when is Netty the `defacto standard Java net lib`? Netty is a
prime example of just being one of many options coming as a 3rd party
package (being a good one for sure).

OK OK, nitpicking aside, so the goal this:

> Our task should be to design an API that works well in Swift.

Understood. So the plan is indeed to replace the kernel of the HL
frameworks with a common Swift `http` server module. A peer to
https://golang.org/pkg/net/http/\. Fair enough.

> Node's http_parser is "just" an example of an API and a possible way of
accelerating the implementation of Swift's HTTP parser.

The http_parser is not just 'an example of an API', it is an actual
implementation of a HTTP parser which can be used from Swift right away.
And not a bad one :-) I guess my original question was why people invent
their own instead of just using the implementations directly accessible
already. Same for OpenSSL, Expat, etc. I really like the Swift-C mapping
and how convenient it is to use.

Unless of course we are not talking about providing core support for HL
frameworks (which provide the nice API of their choice to backend
developers), but for directly providing nice APIs to backend developers as
part of the Swift project. (so that they don’t have to grab say Kitura just
to build a small micro service).

But I think I got you, you want to wrap that kind of thing (or rewrite in
pure Swift) in a nicer, Swiftier API. Thanks for clarifying that.

hh

>> On 26 Oct 2016, at 12:39, Chris Bailey via swift-server-dev < > swift-server-dev@swift.org> wrote:
>>
>> Hi Helge:
>>
>> As the initial target areas are low-level function, yes, each of the
frameworks has an implementation of most of them today. Those
implementations tend to be limited in a number of ways:
>> • C libraries are often used directly (via a modulemap)
>> This means that there's no general "Swift API" that developers can use
- you have to interface with the C APIs directly and deal with
Unsafe/OpaquePointers etc. Ideally we want Swift developers to only have to
write Swift code
>> • Where a Swift API wraps a C library its generally for a specific
use case
>> As the framework teams have a very specific use case - using it in
their framework - the APIs are very directed to what they need to do,
rather than being general purpose for a wider set of use cases
>> • There's multiple, similar, packages
>> Most frameworks have their own server socket implementation (mostly by
calling Glibc and Darwin), HTTP parser implementation, etc. Not only does
this mean we can consolidate the efforts on a single implementation, but it
means that anyone wanting to add additional platform support (like Windows)
only has to do so once.
>>
>> Chris
>>
>>
>>
>>
>> From: Helge Heß via swift-server-dev <swift-server-dev@swift.org
>
>> To: swift-server-dev@swift.org
>> Date: 25/10/2016 19:57
>> Subject: Re: [swift-server-dev] Database API/Framework
>> Sent by: swift-server-dev-bounces@swift.org
>>
>>
>>
>> Hi Chris,
>>
>> it is a little unclear what the scope of the group is. Presumably you
have some very specific things in mind?
>>
>> Of course we’ve seen https://swift.org/server-apis/#focus-areas\. But
what does it actually mean? :-) Looking at it:
>> - base networking: TCP/UDP, DNS. Covered already by libdispatch and the
OS’ses?
>> - security: OpenSSL? or some wrapper around both, CC and OpenSSL?
>> - provide low level HTTP parsing: http_parser? Is anyone using something
>> else?
>> All that seems kinda covered/standard already? Unless maybe you include
non-Unix stuff like Windows.
>>
>> Is this primarily for things like providing a standardised ’swiftier'
OpenSSL wrapper (similar to how Objective-GCD wraps libdispatch)?
>> Or would that just be module maps which are shipped by default with
Swift and hence can be relied on by all frameworks?
>>
>> But maybe I’m just a little too impatient :->
>>
>> hh
>>
>>> On 25 Oct 2016, at 20:41, Chris Bailey via swift-server-dev < > swift-server-dev@swift.org> wrote:
>>>
>>> Hi Ron:
>>>
>>> As you say, that's outside of the scope of the work group, however I
do think its a valuable area that the various existing web framework groups
might be interested in collaborating on. Having a common, recognized
framework that the database providers or third parties would be interested
in writing database drivers for would be a huge step forward.
>>>
>>> I know that Vapor has its Fluent ORM framework, and that its an area
that the Kitura team are looking at as well, so your post might kick-start
some wider discussion.
>>>
>>> Chris
>>>
>>>
>>>
>>>
>>> From: Ron Olson via swift-server-dev <
swift-server-dev@swift.org>
>>> To: swift-server-dev@swift.org
>>> Date: 25/10/2016 18:54
>>> Subject: [swift-server-dev] Database API/Framework
>>> Sent by: swift-server-dev-bounces@swift.org
>>>
>>>
>>>
>>> Hi all-
>>>
>>> I've been working with Swift more on Linux than macOS and am very
>>> excited to see the formation of this group. Looking at the server api
>>> workgroup's focus, I notice it's primarily low-level interaction with
>>> the operating system. When might it be a good time to bring up the
>>> possibility of creating a database-specific framework for those folks
>>> who want to work directly with Postgres, MySQL, even DB2 and Oracle;
I'm
>>> thinking a JDBC-inspired framework that drivers could be written
>>> against.
>>>
>>> Thanks,
>>>
>>> Ron
>>> _______________________________________________
>>> swift-server-dev mailing list
>>> swift-server-dev@swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-server-dev
>>>
>>>
>>> _______________________________________________
>>> swift-server-dev mailing list
>>> swift-server-dev@swift.org
>>> https://lists.swift.org/mailman/listinfo/swift-server-dev
>>
>> _______________________________________________
>> swift-server-dev mailing list
>> swift-server-dev@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-server-dev
>>
>>
>> _______________________________________________
>> swift-server-dev mailing list
>> swift-server-dev@swift.org
>> https://lists.swift.org/mailman/listinfo/swift-server-dev
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 842 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <
https://lists.swift.org/pipermail/swift-server-dev/attachments/20161026/edc8b416/attachment-0001.sig
>

------------------------------

Message: 24
Date: Wed, 26 Oct 2016 16:52:42 +0200
From: Benedikt Terhechte <terhechte@me.com>
To: swift-server-dev@swift.org
Subject: [swift-server-dev] Async I/O in Rust Tokio
Message-ID: <F8F240BF-A133-4D1B-A1CA-08B1457E591F@me.com>
Content-Type: text/plain; charset=utf-8

Hey,

I’m a huge fan of server-side Swift and I feel that there is a lot of
potential for Swift to grow in this area. So I’m really looking forward to
the outcome of this discussion.

I’m also a passive observer in the world of Rust, and one thing that
recently really impressed was their async I/O library “Tokio”. It is a
fairly low level implementation of async I/O and futures in order to be
able to build all kinds of networking implementations on top of it:

"Tokio is an async I/O stack in Rust running the full gamut in terms of
what it provides. At the very bottom you'll find the tokio-core crate with
a bare-bones event loop and Future spawning. Near the top you'll find the
tokio-proto crate with generic implementations of protocol details like
pipelining and multiplexing used to serve requests to the Service trait
intokio-service inspired by Finagle.”

GitHub - tokio-rs/tokio: A runtime for writing reliable asynchronous applications with Rust. Provides I/O, networking, scheduling, timers, ...

What I find particularly interesting here is the architecture of Tokio and
the broad applications this low level library offers for high level
frameworks. Also, the performance seems to be extraordinary good.

I wanted mainly to raise awareness for this implementation to see if
there’s something interest we could learn from it. I remember that Rust did
a lot of experimentation in the async I/O area until they implemented Tokio.

Cheers,
Benedikt

------------------------------

Message: 25
Date: Wed, 26 Oct 2016 16:31:34 +0100
From: Gregor Milos <gmilos@apple.com>
To: Helge Heß <me@helgehess.eu>
Cc: Paulo Faria via swift-server-dev <swift-server-dev@swift.org>
Subject: Re: [swift-server-dev] Database API/Framework
Message-ID: <D7BE19C8-9169-4B5A-95E2-55B19E927BA0@apple.com>
Content-Type: text/plain; charset=utf-8

Additional advantage of a standardised APIs+implementation is standardised
license. MIT (for NGINX http_parser) is not a big deal, but the more
dependencies you're pulling into your app, the harder it's to track it all.

Some more comments inline:

> On 26 Oct 2016, at 15:39, Helge Heß via swift-server-dev < > swift-server-dev@swift.org> wrote:
>
> Hi Gregor,
>
> On 26 Oct 2016, at 15:35, Gregor Milos <gmilos@apple.com> wrote:
>> Expand on Chris's answer, modern languages used on server side come
with abstractions native to their language.
>
> now that is a pretty bold claim. C, C++, Java, JavaScript, Ruby and
Objective-C, don’t. Python does, but few use that. But yes, Go seems to
come with one … :-)
>
>> * Netty (defacto standard Java net lib)
>
> Hu?! Since when is Netty the `defacto standard Java net lib`? Netty is a
prime example of just being one of many options coming as a 3rd party
package (being a good one for sure).

It's all about the definition of "modern" ;). More seriously though, the
core of my point is that good APIs are invariably designed for the language
in question, rather than foreign imports. The distribution model
(compiler/stdlib/common libraries/package manager) is orthogonal. At this
point I don't think it's clear how we are going to deliver the
implementation of APIs designed by this work-group either.
https://swift.org/server-apis/ talks about the process somewhat, but
where the code belongs will largely depend on what it evolves to look like.

>
>
> OK OK, nitpicking aside, so the goal this:
>
>> Our task should be to design an API that works well in Swift.
>
> Understood. So the plan is indeed to replace the kernel of the HL
frameworks with a common Swift `http` server module. A peer to
https://golang.org/pkg/net/http/\. Fair enough.
>
>
>> Node's http_parser is "just" an example of an API and a possible way of
accelerating the implementation of Swift's HTTP parser.
>
> The http_parser is not just 'an example of an API', it is an actual
implementation of a HTTP parser which can be used from Swift right away.
And not a bad one :-) I guess my original question was why people invent
their own instead of just using the implementations directly accessible
already. Same for OpenSSL, Expat, etc. I really like the Swift-C mapping
and how convenient it is to use.

Yes, it can be used from Swift and yes, it's a good parser. But the fact
it's not a Swift HTTP parser is immediately obvious. We always wrap it in
layer of Swift that deals with all the UnsafeMutablePointers, state
management and something that exposes API conforming to
https://swift.org/documentation/api-design-guidelines/\.

Hope this all makes sense.
Gregor

>
> Unless of course we are not talking about providing core support for HL
frameworks (which provide the nice API of their choice to backend
developers), but for directly providing nice APIs to backend developers as
part of the Swift project. (so that they don’t have to grab say Kitura just
to build a small micro service).
>
>
> But I think I got you, you want to wrap that kind of thing (or rewrite
in pure Swift) in a nicer, Swiftier API. Thanks for clarifying that.
>
> hh
>
>
>>> On 26 Oct 2016, at 12:39, Chris Bailey via swift-server-dev < > swift-server-dev@swift.org> wrote:
>>>
>>> Hi Helge:
>>>
>>> As the initial target areas are low-level function, yes, each of the
frameworks has an implementation of most of them today. Those
implementations tend to be limited in a number of ways:
>>> • C libraries are often used directly (via a modulemap)
>>> This means that there's no general "Swift API" that developers can use
- you have to interface with the C APIs directly and deal with
Unsafe/OpaquePointers etc. Ideally we want Swift developers to only have to
write Swift code
>>> • Where a Swift API wraps a C library its generally for a specific
use case
>>> As the framework teams have a very specific use case - using it in
their framework - the APIs ar

2

I think people first had to figure out what this is all about. Maybe a swift-server-discuss mailing list is still valuable.

# Starting Point: Sockets

First of all I wonder whether this really belongs into Foundation, not swift-server-dev? It sounds like a generic Foundation feature which is as useful on the client. Or is it going to be developed here and moved over when ready?

Anyways, so what do people expect from a low level sockets API. There are a lot of design choices. To name a few:

1) Should it be synchronous (potentially w/ a non-blocking mode) and
   essentially just wrapping the Posix (and later) API?

2) Should it be asynchronous?

  - does everyone agree that async I/O would be driven by
    libdispatch channels?
    - and that data would be handed back-n-forth using DispatchData?

  - if it is async, ‘Swifty’ API for error handling does not really
    exist, i.e. no throws/try etc
    - so what should error handling look like?

  - should it use escaping closures for callbacks or delegates

3) Do you really want just a Socket, or is what you really want
   a (byte) stream framework?

  - there are some specifics to sockets (timeouts, connects and
    such), but the core of reading/writing(/encrypting/hashing/..)
    byte blocks/vectors is a general concept
    - this also affects TLS support (which wraps the reading/writing)

  - in the context of a potential core HTTP module, would requests
    and responses be exposed as streams with the same API?
    - think about chunked and WebSocket/Upgrade here
    - or would the HTTP API be completely separate?

  - would you want to have piping? (say, stream a file to a socket
    and through a zip stream)
    - would it handle back pressure?
    - consider optimisations for FD based streams (to use sendfile()
      and such)

4) Does it need String support or just bytes?

  - in a way java.io.Reader/Writer vs java.io.xyzStream

I would be interested to hear some ideas on where people think this would be going :-)

hh

···

On 27 Oct 2016, at 00:12, Logan Wright via swift-server-dev <swift-server-dev@swift.org> wrote:

Sticking w/ the direction of the doc and working upwards, TCP and general socket work seems to be a natural starting point to work with.
- Logan

4

I have some thought about how socket should be implemented.

I think libdispatch is maybe too high-level if we are working on a socket framework we should work on just socket for the following reasons:

1. It take care of too many things.

Most of the framework exist today using different event-looping/threading model.

I can see (and essentially agree) with your point, but then I’m also back at wondering why there is a need for a common socket API at _such_ a low level. A framework choosing a custom event loop certainly can work just fine today with the Posix functions available?

My thinking was that GCD is the ‘Swifty’ thing for doing event-looping/threading and hence would be a basis for this (and the related HTTP server module). But I wasn’t sure, hence my point 1).

2. libdispatch make it hard to implement some protocols such as TLS, where you delegate the TLS frameworks(openssl/libressl) to handle TLS handshake, sending/writing. These frameworks require a real file descriptor to work.

I don’t want to say anything wrong as I haven’t looked at it for a very long time, but 'These frameworks require a real file descriptor to work’ sounds incorrect to me. The way I remember OpenSSL is that you can wrap the underlying transport in a BIO and have TLS sit on top of this.
Maybe you have the time to elaborate.

3. I think the a fundamental socket framework should not use either delegate nor closure, but a overridable read() -> Data and write() function instead. Socket in general has a pretty short lifetime, if we use closure and delegate it will be harder to manage the lifetime of related objects. Such as if I open a separate unix socket to connect to some database, however the network socket closed before the response from database socket even arrived.

I don’t understand what you are saying here :-) Are you just describing an abstract Socket base class which has a ‘RawSocket’ subclass in which read/write is directly invoking Posix.read/write and a ‘SSLSocket’ subclass which has another implementation of that dealing with the encryption?
Or do you really want to subclass a socket in say a PostgreSQLSocket and then override a function like `handleData(…)`. That would sound wrong to me. A PGConnection should interact using a socket object, but not inherit from one.

(Just to be clear: my delegate/closure point was only related to the case in which the socket would be _asynchronous_. In other words this was about how an async socket would deliver data back to the client.)

I think we can take advantage of swift as a protocol oriented language and make a “SocketType” protocol

I expect that there would definitely be something like that (with the note that I think Swift 3 doesn’t call protocol `xyzType` anyone, but just `xyz` or `Socket` in this case :-)

And my question in 3) was whether we would have just `SocketType` or more like a:

  protocol InputStream {} // read-like stuff
  protocol OutputStream {} // write-like stuff
  protocol DuplexStream : InputStream, OutputStream
  protocol Socket : DuplexStream {} // adds timeouts and such

Considering that a lot of OpenSSL stuff (hashing, encryption etc) and other parsing code is desirable not just for sockets, but also for files and other byte based data.

hh

···

On 27 Oct 2016, at 21:07, Michael Chiu <hatsuneyuji@icloud.com> wrote:

6

One of the primary goals is around platform support, and whilst most
platforms generally provide some level of POSIX or POSIX-like APIs, we
can't guarantee that for all platforms that Swift is ever likely to
support. We know that Windows is a platform that we'll likely need to
support in the near future, and that is generally different enough that
having a thin layer over POSIX starts to be come problematic -
particularly around error handling. If we then need/want to start
providing in async socket handling constructs, then we have the further
challenge of dealing with epoll vs machports vs Windows event waiting.

Beyond Windows, I know that there's interest in adding Swift support for
additional platforms that have further differences - support for s390
mainframes in particular would add additional complexities.

There's also the Swift API Design Guidelines to consider, with the strive
towards the use of fluent usage and plain language terms. Whilst the POSIX
APIs may make sense to existing systems programmers, or people who
understand the implementations of the underlying protocols etc, I would
argue that they're somewhat daunting to programming learning systems
programming for the first time.

That's not to say a thin layer over the POSIX socket APIs, that looks a
lot like the POSIX APIs won't drop out as the answer - more that we should
focus on the use cases and what the APIs should look like, and then look
to see how we can implement that most easily.

Chris

···

From: Michael Chiu via swift-server-dev <swift-server-dev@swift.org>
To: Helge Heß <me@helgehess.eu>
Cc: swift-server-dev@swift.org
Date: 29/10/2016 01:45
Subject: Re: [swift-server-dev] Sockets API
Sent by: swift-server-dev-bounces@swift.org

You need to elaborate a little more. “is not standardised across

platforms at all”. In my eyes the reverse is true, the socket API *is*
standardised by Posix and even Winsock2 is virtually identical to the Unix
one (being just a Windows port of the BSD one).

Yes there are a lot of small differences and various extensions, but

nothing fundamental unless you want to get really advanced.
The sockaddr part is definitely not standardized, and that is what I
really mean since sockaddr is almost unavoidable when using socket. If
sockaddr is not standardized, socket is not standardized. I apologize for
not making myself clear.

Differences in size and alignment are completely handled by the clang C

mapping and of no concern at all to the user of those structs?
Not when in when BSD has 104 bytes in sockaddr_un.sun_path and Linux has
104. Plus BSD has extra sa_len component.

Absolutely, the raw structs are hard to use as-is. But since you can

extend C structs in Swift you can make them really nice.

I agree with you, that’s why we should include in the api.
This is my implementation (sockaddr family as enums) btw:

.

Yes, agreed. There should be protocols for such stuff. What is your

opinion on my point:

> 3) Do you really want just a Socket, or is what you really want
> a (byte) stream framework?

I’ll say I think just socket, my overall approach is, make an independent
layer for socket.
Then after all if we decide to build default stream framework/event
looping as well, we can simply build it separately and make socket
compatible with it.
If we going to need socket anyway, why don’t we open up more opportunities
for developers as well?

Off topic: Can you cc me a copy as well when you reply? Since somehow I
can only see your response and reply manually from Archive, and I’d like
to discuss with you more.

Sincerely,
Michael

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

7

One of the primary goals is around platform support, and whilst most platforms generally provide some level of POSIX or POSIX-like APIs, we can't guarantee that for all platforms that Swift is ever likely to support. We know that Windows is a platform that we'll likely need to support in the near future, and that is generally different enough that having a thin layer over POSIX starts to be come problematic - particularly around error handling. If we then need/want to start providing in async socket handling constructs, then we have the further challenge of dealing with epoll vs machports vs Windows event waiting.

To be honest my assumption was that libdispatch is supposedly fully supported on all Swift platforms. And libdispatch provides an abstraction for the async event handling.

Can you confirm that you consider libdispatch inadequate as a server side async IO framework?

Beyond Windows, I know that there's interest in adding Swift support for additional platforms that have further differences - support for s390 mainframes in particular would add additional complexities.

I’ve ported a reasonably large ObjC app server/application from Linux ix86 to Linux S/390 many years ago and don’t remember any particular differences. Could you elaborate where you see the issue here?
Or are you actually talking about the native S/390 OS running Swift?

There's also the Swift API Design Guidelines to consider, with the strive towards the use of fluent usage and plain language terms. Whilst the POSIX APIs may make sense to existing systems programmers, or people who understand the implementations of the underlying protocols etc, I would argue that they're somewhat daunting to programming learning systems programming for the first time.

Absolutely. Two things:

a) I thought the idea of this group was to provide a foundation for
   authors of higher level frameworks, not end-users (developers).
   No one would assume that a Berlin or SF hipster has to deal with
   Posix, hell no! :-)
   Clearly all the existent frameworks did manage to deal with the
   Posix APIs. Or else they wouldn’t exist.

b) I still think that the Posix APIs can be made quite Swifty via
   protocols and extensions w/o the need to actually wrap them.
   (and provided examples demonstrating that ...)

That's not to say a thin layer over the POSIX socket APIs, that looks a lot like the POSIX APIs won't drop out as the answer - more that we should focus on the use cases and what the APIs should look like, and then look to see how we can implement that most easily.

Sounds good too. To be honest I’m actually a little unsure how this group is supposed to work. Is someone submitting a proposal for an API via a pull request and then this is discussed? Or do we discuss on this mailing list on what the actual goal is? Or has Apple/IBM setup a group to implement an API and will ‘just do it’? ("we actually have some of the contributors/committers for Netty signed up to the work group” sounds a litte like this :-)

hh

···

On 30 Oct 2016, at 19:50, Chris Bailey <BAILEYC@uk.ibm.com> wrote:

8

One of the primary goals is around platform support, and whilst most platforms generally provide some level of POSIX or POSIX-like APIs, we can't guarantee that for all platforms that Swift is ever likely to support. We know that Windows is a platform that we'll likely need to support in the near future, and that is generally different enough that having a thin layer over POSIX starts to be come problematic - particularly around error handling. If we then need/want to start providing in async socket handling constructs, then we have the further challenge of dealing with epoll vs machports vs Windows event waiting.

To be honest my assumption was that libdispatch is supposedly fully supported on all Swift platforms. And libdispatch provides an abstraction for the async event handling.

Can you confirm that you consider libdispatch inadequate as a server side async IO framework?

I think we need to be careful to distinguish two separate pieces of functionality in this area provided by libdispatch:

  1) dispatch sources: this is the generic eventing primitive in libdispatch, for various types of event sources including file descriptors
  2) dispatch_io: this is a asynchronous IO facility built on top of dispatch sources (and advisory reads for disk IO)

I can easily believe that 2) might be too high level/abstracted and missing functionality for a server networking framework esp. around encryption (that is not what it was designed for), but I would strongly encourage you to stay with 1) for the raw event listening/handling.

The event/source layer in corelibs-libdispatch is where we already have the abstraction layer for kevent vs epoll vs … and if additional support for another platform is required that is where it should be added.

we are continuing to put significant resources into making this layer better and more tightly integrated with the Darwin kernel in particular, as well as providing additional primitives esp for server workloads (more on that soon) and are also committed to make these APIs continue to work well as the Swift concurrency story evolves.

Daniel

···

On Oct 30, 2016, at 17:18, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:
On 30 Oct 2016, at 19:50, Chris Bailey <BAILEYC@uk.ibm.com> wrote:

Beyond Windows, I know that there's interest in adding Swift support for additional platforms that have further differences - support for s390 mainframes in particular would add additional complexities.

I’ve ported a reasonably large ObjC app server/application from Linux ix86 to Linux S/390 many years ago and don’t remember any particular differences. Could you elaborate where you see the issue here?
Or are you actually talking about the native S/390 OS running Swift?

There's also the Swift API Design Guidelines to consider, with the strive towards the use of fluent usage and plain language terms. Whilst the POSIX APIs may make sense to existing systems programmers, or people who understand the implementations of the underlying protocols etc, I would argue that they're somewhat daunting to programming learning systems programming for the first time.

Absolutely. Two things:

a) I thought the idea of this group was to provide a foundation for
  authors of higher level frameworks, not end-users (developers).
  No one would assume that a Berlin or SF hipster has to deal with
  Posix, hell no! :-)
  Clearly all the existent frameworks did manage to deal with the
  Posix APIs. Or else they wouldn’t exist.

b) I still think that the Posix APIs can be made quite Swifty via
  protocols and extensions w/o the need to actually wrap them.
  (and provided examples demonstrating that ...)

That's not to say a thin layer over the POSIX socket APIs, that looks a lot like the POSIX APIs won't drop out as the answer - more that we should focus on the use cases and what the APIs should look like, and then look to see how we can implement that most easily.

Sounds good too. To be honest I’m actually a little unsure how this group is supposed to work. Is someone submitting a proposal for an API via a pull request and then this is discussed? Or do we discuss on this mailing list on what the actual goal is? Or has Apple/IBM setup a group to implement an API and will ‘just do it’? ("we actually have some of the contributors/committers for Netty signed up to the work group” sounds a litte like this :-)

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

9

Or are you actually talking about the native S/390 OS running Swift?

Yes I was ;-)

Sounds good too. To be honest I’m actually a little unsure how this

group is supposed to work. Is someone submitting a proposal for an API via
a pull request and then this is discussed? Or do we discuss on this
mailing list on what the actual goal is? Or has Apple/IBM setup a group to
implement an API and will ‘just do it’? ("we actually have some of the
contributors/committers for Netty signed up to the work group” sounds a
litte like this :-)

At the moment, the hope is that people will express interest and opinions
on what's required, and sign up to the work group. The only decisions that
have been "pre-made" are what's documented on the Swift.org project page,
covering the goals, focus areas, and development process. You'll notice
that the development process starts with an "API proposal" which is
published as a "pitch" in the Swift Evolution process.

The aim is very much that the proposals, API designs, and resultant
libraries are driven by the whole community.

Chris

···

From: Helge Heß via swift-server-dev <swift-server-dev@swift.org>
To: Swift Server Dev <swift-server-dev@swift.org>
Date: 31/10/2016 00:19
Subject: Re: [swift-server-dev] Sockets API
Sent by: swift-server-dev-bounces@swift.org

On 30 Oct 2016, at 19:50, Chris Bailey <BAILEYC@uk.ibm.com> wrote:

One of the primary goals is around platform support, and whilst most

platforms generally provide some level of POSIX or POSIX-like APIs, we
can't guarantee that for all platforms that Swift is ever likely to
support. We know that Windows is a platform that we'll likely need to
support in the near future, and that is generally different enough that
having a thin layer over POSIX starts to be come problematic -
particularly around error handling. If we then need/want to start
providing in async socket handling constructs, then we have the further
challenge of dealing with epoll vs machports vs Windows event waiting.

To be honest my assumption was that libdispatch is supposedly fully
supported on all Swift platforms. And libdispatch provides an abstraction
for the async event handling.

Can you confirm that you consider libdispatch inadequate as a server side
async IO framework?

Beyond Windows, I know that there's interest in adding Swift support for

additional platforms that have further differences - support for s390
mainframes in particular would add additional complexities.

I’ve ported a reasonably large ObjC app server/application from Linux ix86
to Linux S/390 many years ago and don’t remember any particular
differences. Could you elaborate where you see the issue here?
Or are you actually talking about the native S/390 OS running Swift?

There's also the Swift API Design Guidelines to consider, with the

strive towards the use of fluent usage and plain language terms. Whilst
the POSIX APIs may make sense to existing systems programmers, or people
who understand the implementations of the underlying protocols etc, I
would argue that they're somewhat daunting to programming learning systems
programming for the first time.

Absolutely. Two things:

a) I thought the idea of this group was to provide a foundation for
   authors of higher level frameworks, not end-users (developers).
   No one would assume that a Berlin or SF hipster has to deal with
   Posix, hell no! :-)
   Clearly all the existent frameworks did manage to deal with the
   Posix APIs. Or else they wouldn’t exist.

b) I still think that the Posix APIs can be made quite Swifty via
   protocols and extensions w/o the need to actually wrap them.
   (and provided examples demonstrating that ...)

That's not to say a thin layer over the POSIX socket APIs, that looks a

lot like the POSIX APIs won't drop out as the answer - more that we should
focus on the use cases and what the APIs should look like, and then look
to see how we can implement that most easily.

Sounds good too. To be honest I’m actually a little unsure how this group
is supposed to work. Is someone submitting a proposal for an API via a
pull request and then this is discussed? Or do we discuss on this mailing
list on what the actual goal is? Or has Apple/IBM setup a group to
implement an API and will ‘just do it’? ("we actually have some of the
contributors/committers for Netty signed up to the work group” sounds a
litte like this :-)

hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev

10

Hi Daniel,

···

On 31 Oct 2016, at 02:55, Daniel A. Steffen <dsteffen@apple.com> wrote:

2) dispatch_io: this is a asynchronous IO facility built on top of dispatch sources (and advisory reads for disk IO)

I can easily believe that 2) might be too high level/abstracted and missing functionality for a server networking framework esp. around encryption (that is not what it was designed for)

you are the second one mentioning issues with ‘encryption’. I think I may be missing something here :-) What is the issue with layering TLS on top of channels?

Is it an actual issue with TLS/encryption itself or is all that ‘just' because integration with OpenSSL is harder?

Thanks,
  hh

11

Hi Daniel,

2) dispatch_io: this is a asynchronous IO facility built on top of dispatch sources (and advisory reads for disk IO)

I can easily believe that 2) might be too high level/abstracted and missing functionality for a server networking framework esp. around encryption (that is not what it was designed for)

you are the second one mentioning issues with ‘encryption’. I think I may be missing something here :-) What is the issue with layering TLS on top of channels?

Is it an actual issue with TLS/encryption itself or is all that ‘just' because integration with OpenSSL is harder?

I was only mentioning it because it came up in this thread earlier

the main concern I have heard in the past was around impedance mismatch with encryption APIs, which tend to want to do their own buffering & chunking (and the most efficient way to use dispatch IO on sockets is to let it do as much buffering as is useful for the application layer).

One idea to address that at the dispatch IO layer would be "stacked channels”, i.e. an IO channel that has another one as input, but that is not something that has ever gone beyond the idea stage (but if somebody wants to take on adding that to the libdispatch implementation, I would certainly be supportive! ;-)

Daniel

···

On Nov 1, 2016, at 3:48, Helge Heß via swift-server-dev <swift-server-dev@swift.org> wrote:
On 31 Oct 2016, at 02:55, Daniel A. Steffen <dsteffen@apple.com> wrote:

Thanks,
hh

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev