BSD Sockets & [Musl] Linux Swift

I'm trying to use Swift for a side-project, since it kinda fits for it:
I want a UI on macOS, but some CLI for Linux, ideally using same codebase.

Kinda interested "Swift-on-Server" since I like that approach since it's handy to use same language on front-end and back-end & my background in mostly server-side things... Thus I'm actually not very good at Swift - since I avoid UIs generally – but want to learn it better since I like the closures and pattern matching. Plus I think Musl support running on Alpine Linux is pretty nifty concept, since that's a common backend these days.

One need for my little experiment-in-swift project is to listen for BSD sockets for a router's UDP broadcast message (i.e. unfortunately not everything use nDNS for device discovery). I got socket stuff working for this, or at least enough Swift compiled and can process/print parsed packets, using Static Linux SDK on X86 Alpine VM in UTM (in Virtualization Framework mode for fun).

I did run into some minor roadblocks, so I'll share since there really wasn't a lot of info on using Swift on Linux, in case it helps others:

  1. I use macOS, and perhaps if you're a pro at the xc* commands, it might be possible to compile Musl Swift. I'm not, and gave up with using Xcode and macOS Terminal for it. Instead, used a clean Ubuntu Desktop in VM with VSCode, and followed the swift.org's guide. Everything just worked for dealing with Swift on Linux, without involving Xcode.
  2. While I wrote the BSD code mainly in Xcode, I'll offer the sourcekit-LSP seem to do it's thing close enough to Xcode (and since I'm more familiar with VSCode, handy), and was one par win my tests (i.e. me thinking: maybe SourceKit-LSP is wrong, try in Xcode, to get the same error - and my problem ;) )
  3. And Sourcekit-LSP is the only way to look the generated "C" headers used by Swift, that was important trick to get the socket code to compile on Linux and Musl. This was critical for figuring out what a in_addr and in6_addr look like to Linux Swift. To see Musl, I needed to add this to .sourcekit-lsp/config.json:
{
  "swiftPM": {
    "swiftSDK": "x86_64-swift-linux-musl"
  }
}
  1. I had to use a more complex #ifdef to deal with Musl than swift.org shows, since Musl is Linux...:
#if os(macOS) || os(iOS)
    import Darwin
    import Network
#else
    // otherwise Musl ... even if Linux
    #if canImport(Musl)
        import Musl
    // now try Glibc
    #elseif os(Linux)
        import Glibc
    #elseif os(Windows)
        import ucrt
    #endif
// could error, but let build fail to know where
#endif
  1. The biggest struggle with the code was deal with IP addresses. IMO, the Networking's IPv6Address and IPv4Address structs lack support for even taking Darwin socket things. And, worse in my case, the UDP contains an IP address, so part of the packet's Data() is some [UInt8] with either IPv4 or IPv6 - the Swift IPAddress don't take [UInt8]. Basically the Networking Framework's IPAddress protocol fails to deal with either in[6]_addr or bigEndian arrays. And this one that leads to some questions...

Docs on Linux support are few. Maybe someone knows these things.

  1. It wasn't 100% clear, but I believe the Networking Framework is not supported on Linux and Static/Musl Linux Swift. But this is why I choose using sockets... So if I'm wrong β€” please let me know.
  2. Xcode and sourcekit-LSP both report that String(cString:) is deprecated, but my reading of Apple docs (init(cString:) | Apple Developer Documentation) is that it's only String.init(copyNoByte:) that's deprecated β€” but both Xcode and sourcekit-LSP disagree . This one is handy for deal with socket data... specifically from inet_ntop(), sockaddr_in6, etc. I'm dealing with warning since using Encoder isn't just one call to do that conversion, unless I'm missing something but Google turned up this, which seem inelegant to replace String(cString:):
        let validbuffer = buffer.prefix { $0 != 0 }.map {
            UInt8(bitPattern: $0)
        }
        return String(decoding: validbuffer, as: UTF8.self)
  1. On these same IPAddress things, basic Swift mem mgmt question, does this need to be wrapped in some withUnsafe*() thing, from a C/C++ POV everything is on stack, but I don't know ARC/etc specifics:
    static func getIPv4StringFromBytes(_ data: Data) -> String {
        var bytes = [UInt8](data)
        var buffer = [CChar](repeating: 0, count: Int(INET_ADDRSTRLEN))
        guard
            inet_ntop(AF_INET, &bytes, &buffer, socklen_t(INET_ADDRSTRLEN))
                != nil
        else { return "" }
        return String(cString: buffer)
    }

  1. For IPv6, I really couldn't come up with better than #ifdef's to get the bytes into the need in6_addr type. But I have no clue why such a complex – and different one – is generated. Maybe I'm missing something, but there has to be easier way to get Data() into in[6]_addr types...

    static func getIPv6StringFromBytes(_ data: Data) -> String {
        var bytes = [UInt8](data)
        if (data.count != 16) {
            print("IPV6","wrong len",data.count,"bytes",bytes)
            return ""
        }
        var buffer = [CChar](repeating: 0, count: Int(INET6_ADDRSTRLEN))
        var addr: in6_addr
        #if os(macOS) || os(iOS) || os(tvOS) || os(watchOS)
            addr = in6_addr(
                __u6_addr: in6_addr.__Unnamed_union___u6_addr(
                    __u6_addr8: (
                        bytes[0], bytes[1], bytes[2], bytes[3],
                        bytes[4], bytes[5], bytes[6], bytes[7],
                        bytes[8], bytes[9], bytes[10], bytes[11],
                        bytes[12], bytes[13], bytes[14], bytes[15]
                    )))
        #elseif canImport(Musl)
            addr = in6_addr(
                __in6_union: in6_addr.__Unnamed_union___in6_union(__s6_addr: (
                    bytes[0], bytes[1], bytes[2], bytes[3],
                    bytes[4], bytes[5], bytes[6], bytes[7],
                    bytes[8], bytes[9], bytes[10], bytes[11],
                    bytes[12], bytes[13], bytes[14], bytes[15]
                )))
        #else  // Linux and other platforms
            addr = in6_addr(
                __in6_u: in6_addr.__Unnamed_union___in6_u(__u6_addr8: (
                    bytes[0], bytes[1], bytes[2], bytes[3],
                    bytes[4], bytes[5], bytes[6], bytes[7],
                    bytes[8], bytes[9], bytes[10], bytes[11],
                    bytes[12], bytes[13], bytes[14], bytes[15]
                )))
        
        #endif

        guard
            inet_ntop(AF_INET6, &addr, &buffer, socklen_t(INET6_ADDRSTRLEN))
                != nil
        else {
            return ""
        }
        // TODO: cannot believe this is right way 
        //  but using return String(cString: buffer) gives warning
        let validbuffer = buffer.prefix { $0 != 0 }.map {
            UInt8(bitPattern: $0)
        }
        return String(decoding: validbuffer, as: UTF8.self)
    }
  1. I was trying to do this without any package - to better learn Swift - but if there is some good cross-platform wrapper for IPAddress things. Or more general cross-platform nuances, LMK – I'm sure in_addr won't be the first...

I'll note all of the above works on macOS, Linux, and using Static Linux SDK --sdk option. It just kinda ugly. And took the most time in all the socket stuff to figure out.

If anyone had better approach than above, I'd be curious to hear.

Have you looked at NIO? Even if you decide not to use it, you can look at how they deal with these base networking sockets APIs internally.

3 Likes

I don't think there's a need to go down the BSD socket API on your own, especially if you want your codebase to be multi-platform in a certain sense. I'd recommend using SwiftNIO instead, which will simplify a lot of things for you and will help you structure your code. Or use any of the libraries built on top of SwiftNIO, if such library already exists for your protocol.

3 Likes

Yeah, I was going to get to NIO ... but there is a lot to absorb there. I kinda want to see some code working across platforms, before going into the waters too deep.

It actually wasn't the sockets that fowled me up, it was all the forms an "IP address" could take, [Int8]/Int32/UInt32/String/"in6_addr"... I'd originally read this post on Apple's forums:
Calling BSD Sockets from Swift | Apple Developer Forums
so I knew it was possible. I just should have used his wrappers - I just didn't think a few IP address conversions be such PITA.

And Network Framework does not support UDP broadcasts, so it was BSD or NIO. And of your BSD Socker patterns, listening for UDP broadcast messages is one of the simplest, it's just:

fd = socket(..., SOCK_DGRAM, IPPROTO_UDP)
setsockopt(fd, ..., SO_REUSEPORT, 1)
setsockopt(fd, ..., SO_REUSEADDR, 1)
sa.addr = 0.0.0.0
sa.port = 1234
bind(fd, sa, sa.count)
DispatchQueue { while(data) { recv() } }

I did try NIOPosix β€” which I like to get work β€” but my issue is there is no enum for BSD socket option of SO_REUSEPORT. And, without that option if the code below runs, it blocks other listeners from getting the broadcast. Unlike multicast, there is no IGMP to resolve this ... only a setsockopt().

// The Swift Programming Language
// https://docs.swift.org/swift-book

import NIOCore
import NIOPosix

private final class MndpListnerNIO: ChannelInboundHandler {
    public typealias InboundIn = AddressedEnvelope<ByteBuffer>
    public typealias OutboundOut = AddressedEnvelope<ByteBuffer>
    private var numBytes = 0
    
    public func channelRead(context: ChannelHandlerContext, data: NIOAny) {
        let envelope = Self.unwrapInboundIn(data)
        print("dst:", envelope.metadata?.packetInfo as Any)
        print("src:",envelope.remoteAddress)
  
        let byteBuffer = envelope.data
        self.numBytes -= byteBuffer.readableBytes
        if self.numBytes <= 0 {
            //let string = String(buffer: byteBuffer)
            //print("Received: '\(string)' back from the server, closing channel.")
            print(byteBuffer.hexDump(format: .detailed))
            // TODO: not sure NIO schemes, but want to keep reading...not close as example:
            //context.close(promise: nil)
        }
    }
    public func errorCaught(context: ChannelHandlerContext, error: Error) {
        print("error: ", error)
        context.close(promise: nil)
    }
}

let group = MultiThreadedEventLoopGroup(numberOfThreads: 1)
let bootstrap = DatagramBootstrap(group: group)
    // TODO: big issue is there is no SO_REUSEPORT in NIOPosix...
    //   .channelOption(.socketOption(.so_reuseport), value: 1)  // DOES NOT WORK
    .channelOption(.socketOption(.so_reuseaddr), value: 1)
    .channelOption(.receivePacketInfo, value: true)
    .channelInitializer { channel in
        channel.eventLoop.makeCompletedFuture {
            try channel.pipeline.syncOperations.addHandler(MndpListnerNIO())
        }
    }
defer {
    try! group.syncShutdownGracefully()
}

let channel4 = try { () -> Channel in
    try bootstrap.bind(host: "0.0.0.0", port: 5678).wait()
}()

let channel6 = try { () -> Channel in
    try bootstrap.bind(host: "::", port: 5678).wait()
}()

try print(channel6.closeFuture.and(value: channel4).wait())

so while NIOPosix works to get UDP Broadcasts...

dst: Optional(NIOCore.NIOPacketInfo(destinationAddress: [IPv4]255.255.255.255:0, interfaceIndex: 6))
src: [IPv4]192.168.88.173/192.168.88.173:5678
.....
00000010 00 07 62 69 67 64 75 64 65 00 07 00 27 37 2e 31 |..bigdude...'7.1|
00000020 38 62 65 74 61 36 20 28 74 65 73 74 69 6e 67 29 |8beta6 (testing)|

No another app on same system, which will happen, can listen for the same UDP broadcast because the NIO code above does not set the needed SO_REUSEPORT equivalent in NIOBSDSocket.Option. In my BSD Socket version, another app can listen without issue.

Anyway, that's how ones ends up with with BSD Sockets, at least for an initial dive into swift-on-server. NIO has some helpers for parsing TLV that look promising. And NIO .hexDump() was already handy in above test. But the SO_REUSEPORT didn't look so easy to workaround without recompiling NIO.

Yes, both are CoW arrays, so you need to be a bit careful with those. But in this case, I think, the compiler essentially does the with boilerplate for you when you access the arrays with &.
You can also use UnsafeBufferPointer and companions to avoid the CoW.
Copying the data into another bytes array also seems superfluous. Or actually creating a Data for the bytes, which are, well, always just 4 for IPv4, i.e. UInt32.

Right, that's a Darwin only framework.

Thanks @Helge_Hess1 - I did want confirm it wasn't an import/package/etc thing. And, Network Framework, actually would not have help For UDP broadcasts (...other than it's IPv4Address / IPv6Address types, which I initially used)

I think I figured out a workaround to my issue with NIOPosix. Apparently there is a "long form" to specify the NIO equivalent of the setsockopt's:

ChannelOptions.socket(SocketOptionLevel(SOL_SOCKET), SO_REUSEPORT)
vs. it's short cousin for SO_REUSEADDR: .socketOption(.so_reuseaddr)

So changing the above code to included SO_REUSEPORT fix the problem the "first listener wins" in previous code.

let bootstrap = DatagramBootstrap(group: group)
    .channelOption(ChannelOptions.socket(SocketOptionLevel(SOL_SOCKET), SO_REUSEPORT), value: 1)  
    .channelOption(.socketOption(.so_reuseaddr), value: 1)
    .channelOption(.socketOption(.so_broadcast), value: 1)
    .channelOption(.receivePacketInfo, value: true)
    .channelInitializer { channel in
        channel.eventLoop.makeCompletedFuture {
            try channel.pipeline.syncOperations.addHandler(MndpListnerNIO())
        }
    }

FWIW, with UDP Broadcasts, if all senders and recievers use SO_REUSEPORT and SO_REUSEADDR, you avoid doing what NIOUDPEchoClient does with the ports:

// If the server and the client are running on the same computer, these will need to differ from each other.
let defaultServerPort: Int = 9999
let defaultListeningPort: Int = 8888

If the sample would have the above SO_REUSEPORT... those could be the same port on the NIOUDPEchoServer example. I only harp on the sample since I might have avoided a trip down memory lane with BSD Sockets ;).

But overall in my experiment here, the issue wasn't getting Swift to run on Alpine Linux (Musl)... which is great news. I'm sure NIO will do better with more common things I'd use in future, but I seem to have stumbled in hole with various APIs for UDP broadcast...none are easy, but it's also not particular common these days either (i.e. broadcast storms).

1 Like