SwiftOS, a proposal

Community Showcase
, , ,
1

SwiftOS

In support of an open source, opinionated, entirely Swift based OS

The community should build an open source SwiftOS inspired by the XNU kernel and Darwin OS open source Apple projects GitHub - apple-oss-distributions/xnu

The Problem

Independent device makers trying to do something novel generally use Android for ease of use and its open nature, however Android has a number of disadvantages in performance and memory management compared to Apple technologies present in Darwin and Swift, such as ARC, among others. The world should benefit from the efficiencies of Apple’s OS and Swift related technologies.

A company like Daylight should build their innovations on SwiftOS, not Android Daylight DC1: a new tablet with a better screen and no blue backlight - The Verge

The Solution

SwiftOS would have the following main goal and differentiated from other open source OS attempts in both the FreeBSD (PureDarwin, DarwinBSD, etc.) and Linux community (elementaryOS etc.) which are not XNU based, and other attempts try to have AppKit compatibility or reproduce macOS from 20 years ago (helloSystem, ravynOS https://ravynos.com). Instead, the goal is to go where the hockey puck is going, not where it has been.

As such, we should embrace SwiftUI and have a new target, SwiftOS, alongside Apple commercial platforms. SwiftOS should be a common standard layer that can run on small microcontrollers or large multi-core ARM servers. It should support ARM and RISC-V, but not bother with x86/amd64. It should run on Apple Silicon and Snapdragon X SoCs. It should embrace the modern, decentralized web and webAssembly. Of course it should be designed with local LLM / AI processing in its bones as is appropriate.

The Dream

The idea would be someone can build an app for Apple devices in SwiftUI, and have it also run on SwiftOS on an ARM device seamlessly. These non-Apple SwiftOS devices could be smartphones, tablets, computers, micro controllers, XR headsets, cars, robots, whatever. It should focus on a compelling OS alternative for the future, not try to replicate past desktop experiences.

This should be the next generation NEXTstep OS, and if done right, Apple should incorporate it into their own commercial products. (e.g. the community is helping them port Darwin to Swift) Of course they can have proprietary code and services embedded on their devices, so a Mac is still a Mac and different from someone’s SwiftOS pc, but the SwiftOS PC could still run all SwiftUI built software.

It should enforce the latest Swift 6 concurrency requirements. Things like Swift Charts and Swift Data should work as well (mapped to a different db and ORM than CoreData as necessary). The community would have to build the UI layer in lieu of UIKit / AppKit that SwiftUI hooks into. Of course, SwiftUI should also be open sourced. All of this should emphasize the SwiftUI maxim, learn once, apply anywhere.

Conclusion

I am not an OS engineer, I’m sure there’s a million technical problems with this idea, not to mention business problems. However, I still want this to exist, I think it would be better for the world. The world should not run on Linux, Dart and Flutter, JavaScript and React, it should run on Darwin, Swift and SwiftUI.

Tell me why it won’t work, why I don’t know what I’m talking about, I want to understand this problem better.

19 Likes
2

I can't help you but I love the ambition!

5 Likes
3

Reminded me of a "small" OS written in Rust;

2 Likes
4

Very cool, Rust should be our favorite cousin. Maybe Rust Swift interop will improve in the future, which could be relevant for this

1 Like
5

Creating a new OS is a huge endeavor. And even if some community or company manages to make it, nobody will use it unless it has thousands (millions?) of widely used client-side apps and server-side services that cover any use case scenario for everyone. This is why we see just a few well-established operating systems.

3 Likes
6

It might be an interesting start to build a kernel in Swift as a starting point; with embedded Swift that is definitely possible. In some of the bring-up and testing for it I did write some very rudimentary prototypes of a start to a kernel that runs on RPi4 so I am sure it is possible. There currently will be a few places you may need some assembly (even C/C++ need that to get a kernel bootstrapped).

The user land is definitely achievable - iOS and macOS have a considerable amount of their libraries written in Swift, and with the latest features one could even write stuff down to libc in Swift! This also would be useful since that could be reused by Swift on server projects.

There are a number of really good tutorials on YouTube & Github that step through the C implementations of what a kernel bringup would be like and Swift isn't that much different.

If you have the time to devote to it - this seems like a really cool idea/project that parts of it could have distinct community benefit; and even if you don't immediately get to that lofty dream you have it might be enticing enough for folks to contribute to it (provided they have time and the ability to contribute).

10 Likes
7

Definitely possible. If such an effort existed, it would definitely gather a community of devoted followers. But let me predict: It will always be a niche OS, because adoption depends on usability, and to achieve an acceptable level of usability, the OS not only needs to have apps for every use case, but also it absolutely needs to have GUI. Hoping that a new command-line OS will achieve noticeable level of adoption in 21st century is laughable. Well, GUI is also achievable. Nothing is impossible. What kind of devices this new OS should support in regard of GUI?

Mobile devices? The right choice for the contemporary world. Which manufacturer? All of them? Let's be realistic.

8

I agree with Philippe - writing your own OS is an incredible way to discover so many parts of software engineering that are often abstracted miles away from us by modern machines you can buy off the shelf. I'll note that it's taking the documentation, blog, and textbook industry a long time to catch up to the reality that the world is becoming increasingly powered by embedded arm chips and not x86. But that just means you'll have to pull out your trusty copy of the ARM ARM and get cracking.

This is also an excellent opportunity to push pure Swift to its absolute limits, and even its breaking points. I can think of one simple example: managing interrupts and interrupt safety. This is an absolutely foundational problem to solve, and one that often involves dropping to inline assembly and papering over the conventions with macros in C-like languages. What does this look like in Swift? Are there sufficient language features to express what you're after, and if not you've got some proposals to write!

Another area I can definitely say we're lacking is custom allocator support. In theory, one would have a general-purpose kernel heap allocator backing allocations the Swift runtime and standard library make. In practice, many parts of the compiler and runtime assume any reasonable system ought to have malloc or a malloc-ish system. Custom allocator support has made its way into a number of different languages, maybe it's Swift's time.

I'd also like to offer a separate viewpoint here on OS lineage: consider thinking outside of the box that XNU and the BSDs have built for themselves over the intervening decades. OS development isn't exactly radical rocket science these days - it's been said that the most complicated data structure in XNU is the linked list - but there are still other OSes we can learn a great deal from. For example, for file systems, I fell in love with beOS and its BFS file system that elevates file metadata to the level of file data and is designed to hold live "queries" open against the file system. (Be demoed their "address book" app which was quite literally a live file system query for files with contact info associated with them). Also consider whether or not you'd like to shoot for POSIX compatibility. Therein lies an enormous amount of baggage and... well let's be frank, technical decisions that have not aged well. How much can you get away with by e.g. writing POSIX compatibility as a library as Windows does? What do you want your portability story to be, and not be? Also consider whether you want to replicate the Mach VM. The overhead of virtual memory systems can be substantial, and is arguably chiefly for a world where memory unsafe languages can't be trusted to play nicely with the resources they're given. Does Swift at the core change that calculus?

I'll also say that the Solaris and Sun folks that make up Oxide's leadership also have an admirable philosophy on OS subsystem development where they build tools and debuggers alongside the components they bring up. Not surprising to hear that the dtrace folks are running the show there when you hear that. But it's an excellent way to build robust, introspectable, and even tunable systems when you engineer an OS with that philosophy in mind.

I'll offer some references that I found particularly great on my journey through this end of computer science:

32 Likes
9

Tell me why it won’t work, why I don’t know what I’m talking about, I want to understand this problem better.

It’s a big dream. Perhaps too big for a single person. But all of us together? We can do it.

2 Likes
10

Whether the OS would be "useful" or popular is definitely not guaranteed, but frankly I think that doing things for sh*ts and giggles is a vastly undervalued and underappreciated motivation; I'd much rather do something fun than strictly useful any day of the week :grinning:

Terrific idea in any case, and I'm all for it. Would give me the perfect excuse to learn OS design, too – I'm embarrassingly badly clued in to eg. assembler considering I've been programming in some form or another for over 30 years now.

4 Likes
11

I can add https://illumos.org/ . It was born out of Solaris when Oracle bought Sun Microsystems in 2010. You can find a list of illumos-derived systems there, most notably OpenIndiana. This thing (OpenIndiana) is successor to OpenSolaris, and reality is that it is slowly dying due to lack of developers behind it. What you will not find in the list of illumos-derived systems there, is Illumian distribution, because this thing is already dead. It was meant to be a new operating system by the people behind illumos.org .

What @MadeByDouglas proposes, is just a dream.

Edit: You may never heard of Illumos and Illumian, but both were created by ex-Sun Microsystems engineers: founder Garrett D'Amore and other developers such as Bryan Cantrill and Adam Leventhal.

4 Likes
12

:+1: but enthusiasm of doing things for fun cannot last long. definitely not the number of lines of code comprising a feasible OS.

3 Likes
13

I'd say that depends on the person(s) involved. Multiple hobby OSs have been built that were done purely for the heck of it, not to mention any number of other projects people have worked on for years just because they enjoyed it, even though what they were building wasn't necessarily "useful" in some more absolute sense.

Sure, building a "proper" OS that could actually be used by a wider audience would be a massive undertaking and take years, but I guess my question is "so what?" If someone wants to give it a go, or even just build a hobby OS in Swift that they dream might some day "make it big", I don't see much harm in it. Worst case scenario, the project withers away and dies at some point but the people involved will still probably have learned new things and had fun in the process. Maybe someone ends up with a niche hobby OS that exactly 37 peope have even heard of, but which lives on for years with folks working on it occasionall. And there's still a nonzero chance that it'd lead into something more interesting or even actually useful in some way – even if it's just eg. stdlib or framework development around some feature that'll be useful for embedded code.

2 Likes
14

Tell me why it won’t work, why I don’t know what I’m talking about, I want to understand this problem better.

That's an interesting idea, but this is in fact 2 different projects IMHO.

An OS is really composed of 2 parts: the kernel, and the user land, and honestly, users and most developers don't care what your kernel is. iOS user land may be running on Linux kernel, and Android user land may run on Fuchsia, it would not make any difference for mobile app developpers.

Your "dream" of OS for the futur, being able to write a SwiftUI app on Apple device and run it on any other ARM device does not require that all of them run with the same kernel. They just need Swift Frameworks ports.

This is already the case with Server Side Swift, running "seamlessly" on Darwin base OS, Linux, and Windows.

Now, writing a Swift kernel may be a great project.

It should be possible to write a Swift kernel, but keep it compatible with Linux/Posix user land. This is the what redox OS do with a rust microkernel.

One advantage is that you can start testing your kernel without having to develop all the user land components.

4 Likes
15

Agree with most of your points (although I must admit I’m particularly fond of ZFS if we’re picking file systems rather than alternatives - still think it’s a bummer apples port never shipped in production…).

Just one additional reflection, it would be interesting to consider a kernel interface where all interaction is asynchronous in nature - basically analogous to io_uring, but for everything - and have that asynchronous nature with completion events be the model.

2 Likes
16

That's what redox OS tried at first, but they had to abandon it due to ZFS design issues.

And by the way, the OpenZFS macOS port works quite well if you want to use ZFS on macOS.

17

Great list. If you’ll allow me, I’d like to suggest a few additions. All three are classics in the “systems design” genre and are now available for free:

14 Likes
18

While I think your massively underestimating the time and effort such an endevor would take. This is the only part that I think is impossible:

Apple will not be doing this no matter how good your OS is. This isn't 1998 and Apple does not have a Copeland situation. Darwin is good enough, and portable enough, that it will continue to be the basis for MacOS[1] indefinitely.

As long as you aren't counting on Apple replacing their OS with your own, then this should work out fine. Even if it takes you a couple of decades longer than you thought it would.

  1. No matter what I just can't bring my self to lowercase the M ↩︎

5 Likes
19

Exactly, that's why I posted it here, to share the dream and work together

21

Totally, which is why easy adoption / installation of the growing catalog of SwiftUI apps would be crucial

And to your second post, also completely agreed, GUI would be needed. As to devices, yep, just pick one to start, I don't know what that one should be yet.

I'm very much a believer of doing something small first that works, i.e. build a skateboard first, then a bike, then a car. Not a tire, then a suspension, which never is completed. (This is general startup advice and Apple are masters at this even as a large company.) So this post is to share the vision of building a car with the community, but asking how to start with a skateboard.