Sockets API


#1

I can see (and essentially agree) with your point, but then I’m also back at wondering why there is a need for a common socket API at _such_ a low level.

I think one of the reason why common socket api is necessary since socket is not standardized across platforms (WINSOCK, BSD Socket, Linux Socket…) at all.
Another reason is probably because of the sockaddr struct family. They have different sizes, different alignment(on different os), often need to cast the pointer around, and incredibly hard to use in pure swift manner.
Nevertheless I think swift is a great language for both high and low level programming, if we have some swift-like yet low level api we essentially open up the opportunities for other developers.

A framework choosing a custom event loop certainly can work just fine today with the Posix functions available?

I’m not quite sure what you mean here.

I don’t understand what you are saying here :slight_smile: Are you just describing an abstract Socket base class which has a ‘RawSocket’ subclass in which read/write is directly invoking Posix.read/write and a ‘SSLSocket’ subclass which has another implementation of that dealing with the encryption?
Or do you really want to subclass a socket in say a PostgreSQLSocket and then override a function like `handleData(…)`. That would sound wrong to me. A PGConnection should interact using a socket object, but not inherit from one.

Sorry for my bad explanation, override is definitely not a good word choice. What I mean is

    protocol Readable {
        func read() -> Data?// how do we read
   }

   protocol Writable {
        func write(data: Data) // how do we write (send, sendfile, …)
   }

   protocol Socket: Readable, Writable {}

so we can easily make something like:

class TLSSocket: Socket {
    func read() -> Data? {
    … ssl_read….
    }
    func write(data: Data) {
    … ssl_write….
    }
}

such that we can easily implement low level optimization and extent to different socket interfaces.

···

On 27 Oct 2016, at 15:27:56 CDT 2016, Helge Heß <me at helgehess.eu> wrote:


(Paulo Faria) #2

I think a very good reference for the conversation regarding concurrency is libdill:

https://github.com/sustrik/libdill

And dsock:

https://github.com/sustrik/dsock

dosck has a work-in-progress RFC:

https://github.com/sustrik/dsock/blob/master/rfc/sock-api-revamp-01.txt

Libdill's biggest concept is structured concurrency:

http://libdill.org/structured-concurrency.html

libdill is an elegant solution for one of the biggest problems of concurrency, cancelation.
It uses coroutines, procs and CSP to deal with communication.
On the other hand dsock solves the problem of protocol composition. The RFC explains
the concept in great detail. I really love the approach and I think we can get a lot of
inspiration from these sources.

···

On Oct 27, 2016, at 9:27 PM, Michael Chiu via swift-server-dev <swift-server-dev@swift.org> wrote:

On 27 Oct 2016, at 15:27:56 CDT 2016, Helge Heß <me at helgehess.eu <http://helgehess.eu/>> wrote:
> I can see (and essentially agree) with your point, but then I’m also back at wondering why there is a need for a common socket API at _such_ a low level.
I think one of the reason why common socket api is necessary since socket is not standardized across platforms (WINSOCK, BSD Socket, Linux Socket…) at all.
Another reason is probably because of the sockaddr struct family. They have different sizes, different alignment(on different os), often need to cast the pointer around, and incredibly hard to use in pure swift manner.
Nevertheless I think swift is a great language for both high and low level programming, if we have some swift-like yet low level api we essentially open up the opportunities for other developers.

> A framework choosing a custom event loop certainly can work just fine today with the Posix functions available?
I’m not quite sure what you mean here.
> I don’t understand what you are saying here :slight_smile: Are you just describing an abstract Socket base class which has a ‘RawSocket’ subclass in which read/write is directly invoking Posix.read/write and a ‘SSLSocket’ subclass which has another implementation of that dealing with the encryption?
> Or do you really want to subclass a socket in say a PostgreSQLSocket and then override a function like `handleData(…)`. That would sound wrong to me. A PGConnection should interact using a socket object, but not inherit from one.

Sorry for my bad explanation, override is definitely not a good word choice. What I mean is

    protocol Readable {
        func read() -> Data?// how do we read
   }

   protocol Writable {
        func write(data: Data) // how do we write (send, sendfile, …)
   }

   protocol Socket: Readable, Writable {}

so we can easily make something like:

class TLSSocket: Socket {
    func read() -> Data? {
    … ssl_read….
    }
    func write(data: Data) {
    … ssl_write….
    }
}

such that we can easily implement low level optimization and extent to different socket interfaces.

_______________________________________________
swift-server-dev mailing list
swift-server-dev@swift.org
https://lists.swift.org/mailman/listinfo/swift-server-dev


(Helge Heß) #3

> I can see (and essentially agree) with your point, but then I’m also back at wondering why there is a need for a common socket API at _such_ a low level.
I think one of the reason why common socket api is necessary since socket is not standardized across platforms (WINSOCK, BSD Socket, Linux Socket…) at all.

You need to elaborate a little more. “is not standardised across platforms at all”. In my eyes the reverse is true, the socket API *is* standardised by Posix and even Winsock2 is virtually identical to the Unix one (being just a Windows port of the BSD one).
Yes there are a lot of small differences and various extensions, but nothing fundamental unless you want to get really advanced.

Here is a very old socket imp which works on all three platforms:

  http://svn.opengroupware.org/SOPE/branches/sope-4.6/sope-core/NGStreams/NGSocket.m

and another one:

  https://github.com/opensource-apple/CF/blob/master/CFSocket.c

Another reason is probably because of the sockaddr struct family. They have different sizes, different alignment(on different os),

Differences in size and alignment are completely handled by the clang C mapping and of no concern at all to the user of those structs?

often need to cast the pointer around, and incredibly hard to use in pure swift manner.

Absolutely, the raw structs are hard to use as-is. But since you can extend C structs in Swift you can make them really nice. Look at this as an example:

  https://github.com/AlwaysRightInstitute/SwiftSockets/blob/master/Sources/SwiftSockets/SocketAddress.swift

it lets you do stuff like:

  let addr : sockaddr_in = “127.0.0.1:80”

or iterate through `addrinfo` since they are made conforming to the `Sequence` protocol, etc. And all that w/o actually wrapping the structs in yet another construct. Want a swiftier name for them? `typealias IPv4SocketAddress = sockaddr_in` does the trick :wink:

Note: I don’t want to push this as the ‘one’ solution the working group should use. But it actually is pretty similar to how other C-APIs (like CoreGraphics) are ‘Swifty'fied’. C based stuff doesn’t have to look bad in Swift, you can make that really nice.
Just saying (and providing demo code demonstrating the fact ;-))

> A framework choosing a custom event loop certainly can work just fine today with the Posix functions available?
I’m not quite sure what you mean here.

I meant that if you are working at such a low level that you are selecting your own runloop and scheduling mechanism (instead of using the builtin one), working with the Posix socket APIs should be no big deal.

Sorry for my bad explanation, override is definitely not a good word choice. What I mean is

    protocol Readable {
        func read() -> Data?// how do we read
   }

   protocol Writable {
        func write(data: Data) // how do we write (send, sendfile, …)
   }

   protocol Socket: Readable, Writable {}

so we can easily make something like:

class TLSSocket: Socket {
    func read() -> Data? {
    … ssl_read….
    }
    func write(data: Data) {
    … ssl_write….
    }
}

such that we can easily implement low level optimization and extent to different socket interfaces.

Yes, agreed. There should be protocols for such stuff. What is your opinion on my point:

3) Do you really want just a Socket, or is what you really want
   a (byte) stream framework?

hh

···

On 28 Oct 2016, at 01:27, Michael Chiu via swift-server-dev <swift-server-dev@swift.org> wrote:

On 27 Oct 2016, at 15:27:56 CDT 2016, Helge Heß <me at helgehess.eu> wrote: