[Pitch] Cross-Platform C Library Access

Evolution Pitches
1

In the earlier days of Swift, command-line and other apps that needed access to C library functions would simply add a one-liner import:

import Darwin

With the availability of Swift on a wide variety of platforms, there are now many different potential imports that could bring in the right libraries. On Linux, you'd import Glibc, and on Windows, you'd import ucrt. And now with Static Linux support, you might import musl. The "correct" import is becoming quite complicated and a lot of ceremony at the start of an otherwise innocuous Swift file:

#if os(macOS) || os(iOS)
import Darwin
#elseif canImport(Glibc)
import Glibc
#elseif canImport(Musl)
import Musl
#elseif os(Windows)
import ucrt
#else
#error(Unknown platform)
#endif

Yet the above code is missing support for important platforms like tvOS, watchOS, and visionOS, and will be broken on platforms like Wasm/WASI, FreeBSD, Android...

It would be interesting to consider if there's a better approach here, since packages that don't have exactly the right import formulation will break on inclusion with a static Linux app.

For example, could we offer a meta-import that is syntactic sugar for the above code snippet?

import stdlib

A quick review of open source code shows a wide variety of invocations, many of which are incorrect or at least dissimilar. Some assume Glibc, others check for os(musl), on Windows there is a variety of imports (ucrt, CRT, WinSDK). It feels like a good time to harmonize this for certain scenarios, rather than source code being bound to an assumption of what linked C libraries are available.

8 Likes
Using the C library from Swift: thoughts
2
6 Likes
3

I feel like this is a step in the wrong direction, though. Using the C standard library should be a last-ditch effort, with effort being made into replicating anything you might want from it, within Foundation instead, using a more Swift-y API.

The more advanced low level stuff you might want aren't part of the standard library, anyway, and could vary from platform to platform. Even something like using locks/mutexes is very different from POSIX to, say, Windows. The #ifs don't end at the import block.

The dream would actually be for Swift programs to compile on an environment without a C compiler or runtime. With Foundation offering low level (i.e. not based on equivalent C functions) per-platform implementations of anything you might need, under a uniform API.

Making C standard libraries a standard in Swift would being going in the opposite direction to that.

4 Likes
4

I'd love to see something like this.

Not all C libraries offer the same API surface. Something like an import libc or import clib would be great for importing the APIs from the C standard. Then if you actually do need vendor-specific extensions, say GNU extensions for example, then you can do the full import Glibc to get at the extensions.

2 Likes
5

It's actually pretty tricky. In order for the OS/SDK's C standard library module(s) to remain acyclic with the clang and C++ modules, you need multiple modules for the OS/SDK C standard library headers.

The second wrinkle is that the OS/SDK modules for the C standard library headers aren't usable. e.g. if you import the OS/SDK's limits module, you'll just get the OS/SDK limits.h which (should) just have the POSIX limits and you won't get the clang limits.h which has the C standard library limits. It gets further complicated when you add in the C++ standard library's limits.h module. Our solution for this is "cover" headers and modules such as the limits_h module in the current Apple SDKs, and the Android and musl module maps provided by Swift.

To mirror @SlugFiller 's sentiment, I don't think we want to encourage any C standard library use from Swift. Many of the interfaces don't make sense in Swift like assert.h, and many more have better APIs for clients in the Swift standard library, and in Swift System.

As we work towards fixing the rest of the module maps in Swift to be acyclic and conform to the newer system, I think that should largely alleviate the #if os(...) train and allow people to just import math_h as needed.

1 Like
6

Reviewing Vapor's codebase we have several files where we do:

#if canImport(Glibc)
import Glibc
#elseif canImport(Musl)
import Musl
#elseif canImport(Android)
import Android
#else
import Darwin.C
#endif

Which does feel like there should be something higher level just to call down to some fairly simple C APIs or offer a better abstraction on top. (It feels like Swift System should be this but the guidance isn't particularly clear and it obiously doesn't cover everything we use

Using the C library from Swift: thoughts
7

What are you actually using out of Darwin.C?

8

Quick scan shows some of the time APIs (time/gmtime_r), some legacy filesystem APIs getcwd etc

9

The time stuff would be time_h with the newer naming convention. Although I think Swift's Clock and Duration APIs are probably preferred?

The Android module has some newer conventions for basic posix stuff, I don't remember if unistd is in there though. You'd think that System would have some kind of cwd or Path(".").realpath or something like that, but I don't quickly see it. Maybe I'm just missing the obvious there.

10

See ongoing discussion about just this topic.

11

I knew there was a recent thread!