Swift System is now Open Source

I am excited to announce that Swift System is now open source! Please see the great blog post written by @Michael_Ilseman on this announcement:

Feel free to use this forum thread for questions!

55 Likes

What's not to love the the first header is Goodbye Imported C Interfaces. :two_hearts:

8 Likes

A goal is to cover all relevant system calls, superseding most uses of Darwin/GlibC (which just forward the C declarations). That would include POSIX on Darwin and Linux (or at least the subset a platform might implement) as well as additional interfaces such as kevent/epoll/io_uring.

That is a lot of interfaces, so for now we're prioritizing what existing cross-platform libraries and projects use such as SwiftPM and SwiftNIO. And of course, System is taking contributions if people have other concrete use cases.

9 Likes

I would be interested to try to add support for Android at some point when I get around to doing more Swift again. But it wouldn’t be so easy to fit with the normal POSIX paradigm there.

It would require calling into the JNI, which would require an additional dependency (the JNI itself) and also require the user to provide a Java context object in order to make the file system and other system calls. Do you see any scope for this in System?

Great News!

One thing that bothers me though with Swift System's API is the FilePath type.

My main concern with this FilePath type is its name, which I find quite misleading. I would like to discuss here if we could not rename it to something else.

Nothing in the documentation nor its usage gives a reason for it to only be tied/used for file paths: by definition and as per documentation, the type is just a null-terminated string; other low-level APIs might be requiring null-terminated strings which don't represent file paths, so why pick a name for this type that might give the wrong idea? Especially given this type doesn't have any instance methods related to manipulating file paths (†)

I propose for it to be renamed CString or SystemString or something like that, at least a name that is detached from the semantics of "file paths" and more close to the semantics of "low-level / null-terminated string", since that's this type's definition after all.


(†) PS: I've been wanting for Foundation to get a proper Path type, similar to what GitHub - kylef/PathKit: Effortless path operations in Swift or similar 3rd party libraries provides, i.e. a type dedicated to manipulating filesystem paths and containing methods to operate on those paths on the file system. It would make much more sense to me to have all APIs manipulating file paths to have a dedicated type for it in Foundation rather than using a more generic type like URL or String for that like current Foundation APIs do. When I saw that new FilePath type introduced by Swift System, I immediately thought it was this type that I was hoping for for so long, only to be disappointed by discovering that this new FilePath type has not really anything to do with file paths, but is just a type to hold a null-terminated string. That's another reason why I think FilePath is a misleading name.

6 Likes

The semantics of FilePath seem pretty clearly documented to me:

A null-terminated sequence of bytes that represents a location in the file system.

That seems to be exactly what you're looking for? The announcement even has a link to a pull request to add methods to manipulate file path components.

4 Likes

A type can have semantics beyond its physical representation. I have considered making a system string type (akin to Rust's OsString) to provide the backing storage for FilePath, especially if that becomes a useful concept in a world with Windows support. If we did have that, we wouldn't want all path operations (such as the proposed component view) to be on the system string; FilePath would provide a strongly typed wrapper around a system string.

8 Likes

I see, I think I missed the part in the announcement where it was said that FilePath was planned to evolve into a more complete type on which we'd plan to add more instance methods dedicated to file path manipulations in the future (like in your PR #2).
My remark might have been biased by my original discovery of the type when it was introduced back in WWDC and just saw the documentation saying it was "a null-terminated sequence of bytes"

So, in that case if FilePath is indeed destined to do more great things that will be specific to only manipulating file paths, then I'm ok with the name bearing semantics :+1:. I also like the potential idea that you introduce of having an OsString (or whatever we'd call it) dedicated to null-terminated strings so that we can use that in other contexts, and then have FilePath's internal storage use this OsString type internally, and adding semantics and dedicated instance methods to it.


That's exactly what I don't like with how NSString and NSURL are used in Foundation, as those types were supposed to be agnostic of any usage semantics – and only get instance methods that would make sense for any string and URL in any text or URL contexts – but instead they started to get APIs that were only tied to specific usage and contexts like path manipulation. I'd have preferred those instance methods to manipulate paths to be on a dedicated type (whose backing storage would probably be String or URL underneath) rather than on the general-purpose String or URL types themselves. And that's exactly what I want to avoid happening with SwiftSystem.

if fact, if that FilePath new type evolves in the long run to indeed get richer and specialised in path manipulation, I'm all for it, and would even love to see in the long term APIs in Foundation (like FileManager and friends) to start using it instead of using String and URL for paths arguments

4 Likes

That would be very cool!

Currently System supports Linux and Darwin, which have large swaths of overlapping API surface area. Windows (and it seems like Android according to your comment) may have largely different system call APIs, though some of the higher level types such as FilePath and concepts built on them would be shared.

System is only in its infancy—it currently includes a small number of system calls, currency types, and convenience functionality. As part of the effort to increase the API coverage, we’ll be working to adopt System in the Swift Package Manager. This will include enhancements to FilePath and adding support for the recently announced Swift on Windows.

We're focused on fleshing out the library for Swift's supported platforms, which includes adding Windows support. That will require us to figure out how to organize support for non-POSIXy platforms, what commonality can be expressed vs what has to be different, etc. Considering Android as another example of a non-POSIXy platform could help inform that design. Do you have an examples on-hand of what uses of the imported C API looks like?

Just one nitpick then that I think would make it a bit more clear in intent: I think we should update the documentation from saying "A null-terminated sequence of bytes that represents a location in the file system" to "A type that represents a location in the file system" (focus on the type's intent and semantics definition first) and only add in a "Notes / Remarks" subsection that "the storage for this type uses a null-terminated sequence of bytes internally".
Because from what you just said, that last part is an implementation details, not really the type's defining character after all.

Might seems like a small nitpick, but I think it can make quite a difference in how people understand and approach this new type, In fact, I think that this current wording of putting the "null-terminated" part in that documentation first is part of what made me misunderstand the definition and intent of FilePath at first, especially if that's supposed to only be an implementation detail (and not the semantically-defining part of it).

19 Likes

I agree with this. The goal of the type is to have the right representation for interacting with the target system; null-termination might not even be part of that on every system (although I think it is on all the major systems).

7 Likes

I'm very happy to see this, and I'm especially happy that it appears open to community evolution ideas. I just think that's a really important thing for an open-source project.

Just having a brief look at the path component proposal, I already have a few thoughts:

  • OS-dependent behaviour (like path separators) should be considered carefully. Does this mean that FilePaths created from literals will behave differently on different systems? Is this library meant to abstract over those differences in any way, or expose them as they are?

    Even the example code includes a path literal:

    let path: FilePath = "/tmp/log"
    

    What does this path mean on Windows? I'm not entirely sure what "Multi-platform not Cross-platform" means in this context.

  • I would consider splitting relative and absolute paths in to separate types. Code which treats relative paths and absolute paths equally will eventually need to decide what their base path is, which forces the introduction of a "working directory" as a piece of process-wide, mutable state (check out that big, red warning box).

EDIT: Oh, and one more thing: It's been a while since I used Windows, but I think the concept of a "root path" is a bit awkward on that platform. Windows kind of has several roots, one for each drive - i.e. C:\ in C:\Windows\. I may be mistaken, but I don't think there is any Windows filesystem API which allows you to treat some root-path string (like \ perhaps) as a parent folder which contains all drive roots.

But that is exactly why I'm so glad this project welcomes community feedback.

4 Likes

I don't think paths for different platforms should be unified, they are platform-specific and should be better left so. I guess the same would be applied to your second question: "root" in terms of Windows paths is what begins with "drive:\", because it's what makes a path absolute.

I wouldn't mind an abstraction over it. We can finally get rid of the / vs \\ non-sense.

1 Like

This library is not meant to abstract over platform differences in any way. That's what "multi-platform not cross-platform" means. Platforms can share API, but only when it doesn't take away from faithfully representing the semantics of those platform's operations.

Nothing. See above.

7 Likes

This is great news. While you’re in an Open Source-ing mood, how about Combine?...

5 Likes

Android doesn’t have much of a C API to wrap for, for example, file system operations. At least not ones that can be used in an Android app (as opposed to a low-level script, which I suspect 99.99+% of developers on the platform will not be making).

Everything has to go through the Java APIs which l suspect may put it strictly at odds with the goals of this package. There would need to be extra API on Android to deal with this (specifically, setting the context JVM object provided by the Android runtime in order to interact with the system).

Or what about open sourcing SwiftUI? :heart_eyes:

1 Like

We already have a unifying cross-platform abstraction of file paths: Foundation.URL. Introducing a new cross-platform abstraction without an Apple-endorsed deprecation path for Foundation semantics doesn’t seem desirable.

@Michael_Ilseman When building for Apple platforms, can you re-export the System APIs?

#if os(macOS) || os(iOS) || os(tvOS) || os(watchOS)

@_exported import System

#else

// SystemPackage APIs for Linux, etc.

#endif

This would be similar to how the Swift Crypto package is implemented.