Seeking Advice: Building an Robotics and Edge-Compute Platform Entirely in Swift

[zamderax]

Hi everyone,

As robotics continues to gain traction due to AI, many developers are entering the space only to discover a barren landscape when it comes to developer productivity tools and infrastructure software. One recurring issue I've noticed is that nearly all robotics companies prototype in Python, only to later rewrite everything in C++ for production to meet performance requirements. Coming from a mobile development background, this approach seems incredibly inefficient.

The ecosystem fragmentation further exacerbates the problem. Tutorials, libraries, and APIs are scattered across languages like Rust, Python, and C99, making it difficult for developers to find cohesive resources or build a robust ecosystem.

From my perspective, the challenges of developing for robotics and other edge-compute applications are strikingly similar to those of mobile development. Issues like tooling, deployment, debugging, and language interoperability are universal.

This brings me to Swift. Its flexibility, strong C/C++ interoperability, and suitability for everything from embedded systems to UI applications make it a compelling choice for building a modern edge-compute stack. I really enjoyed the fact that Apple was rather strict with promoting Swift because it made a cohesive community around the language. Today's Edge landscape is wrought with developers bringing in a different platform and coding in all sorts of different languages adding to an incredible amount of confusion.

I’m planning to create an open-source Apache 2.0 Edge Platform entirely in Swift. The goal is to provide a seamless developer experience where you can:

1. Plug in devices like Raspberry Pi or Nvidia Jetson via USB.
2. Automatically recognize them as remote targets using SSH.
3. Use an IDE (e.g., Xcode or VSCode) to press CMD+R and deploy/debug applications effortlessly, akin to building iOS apps.

I also intend to introduce cross-platform APIs for:

• Networking, such as service discovery and distributed actors (inspired by the node-based architecture of frameworks like ROS2).
• Data encoding, using formats like CBOR for efficient serialization.
• Disruption Tolerant Networking (DTN), inspired by DTN7, as a high-level library similar to Apple’s Multi-Peer Connectivity framework, enabling robust communication even in networks with intermittent connectivity.

However, I’ve run into a key concern:

Unlike iOS or macOS development, Swift Package Manager doesn’t provide the same seamless experience for wrapping C-headed libraries using .binaryTargets. For example, if a developer wants to deploy SDL2 to a Linux device, they technically can, but the program won’t run because Swift on Linux requires manual steps like running apt-get install libsdl2-dev.

This creates a subpar experience for developers, especially when targeting platforms like Linux. Does anyone have ideas on how to address this, even if it involves building an extension to Swift Package Manager? For instance, could we create a solution that focuses on Debian-based systems to simplify library installation and integration?

Before diving in, I’d also love to get feedback on:

1. Do you see Swift as a viable candidate for an edge-compute platform at this scale?
2. What challenges or limitations should I anticipate, particularly in targeting diverse edge-compute devices?
3. Are there existing tools, libraries, or projects I should be aware of to complement this vision?

I’d greatly appreciate your insights!

P.S.

I've already considered Rust. I started a large startup that was written completely in Rust, and while it was a beautiful language and very well designed, it was incredibly difficult to work.

Zig is a great choice, but it's not 1.0 and stable yet

Go's interop story was too difficult to consider.

[Joseph_Heck]

@zamderax

Unlike iOS or macOS development, Swift Package Manager doesn’t provide the same seamless experience for wrapping C-headed libraries using .binaryTargets. For example, if a developer wants to deploy SDL2 to a Linux device, they technically can, but the program won’t run because Swift on Linux requires manual steps like running apt-get install libsdl2-dev.

This creates a subpar experience for developers, especially when targeting platforms like Linux.

That's definitely a space where there's a challenge - and I'd broaden it out to assert the same challenge exists for targeting Windows and Android platforms. There's been talk here and there, but so far no explicit proposals or pitches on how to wrangle improving this situation. A few hacks and workarounds have been proposed, and declined, loosely expanding an XCFramework-like concept. The scenario is complicated because XCFramework was assembled with an assumption of ABI stability, which doesn't exist in the Linux, Android, and Windows space with Swift (currently, and I'd rather expect for quite a while to come).

Artifact Bundles (SE-0305) extended some SwiftPM support for platform-specific binaries. However, that proposal explicitly didn't try to accommodate this use case, focusing instead of supporting "getting" binaries to help with development support when utilizing SwiftPM Build and Command Plugins. Future Directions in that proposal explicitly calls out that it could be a good jumping off point for supporting this specific need, citing:

In order to make this usable, however, a future proposal would need to define exactly how to provide libraries on Windows and Linux, and this would encounter even more ABI compatibility issues than executables, since the workaround of statically linking troublesome dependencies would not be available to libraries that themselves need to be linked into the client.

For what it's worth, I've seen almost identical needs expressed from a number of the folks who've been exploring the SwiftGodot bindings that @migueldeicaza and friends have been assembling, and periodically from people bridging other C, Rust, etc. (more generally FFI-enabled) libraries bridged over to Swift.

@zamderax

Does anyone have ideas on how to address this, even if it involves building an extension to Swift Package Manager? For instance, could we create a solution that focuses on Debian-based systems to simplify library installation and integration?

I would expect the path forward would be a pitch and possibly proof-of-concept proposal that does the work to lay out extending SE-0305 (artifactbundles) to support libraries, detailing at least one path to allow for linking to libraries, with the proof of concept wiring up the updated .artifactBundle into SwiftPM so that any relevant libraries are made available to the linker or referenced appropriately in it's build process.

@zamderax

2. What challenges or limitations should I anticipate, particularly in targeting diverse edge-compute devices?

I'd note that the embedded Swift capabilities, while amazing and effectively already, are still very nascent - and I would expect at least some amount of effort will need to be expended to create and assemble relevant SDKs for the platforms in order to support cross-compilation. I would very much recommend searching around these forums for other references to Embedded Swift, as there's a lot of really interesting activity across both microcontrollers and higher level compute systems (RaspberryPi), that all have overlapping needs and interests. You made find some of what you want to accomplish actively being worked on through those projects.

[zamderax]

@Joseph_Heck

I would expect the path forward would be a pitch and possibly proof-of-concept proposal that does the work to lay out extending SE-0305 (artifactbundles) to support libraries, detailing at least one path to allow for linking to libraries, with the proof of concept wiring up the updated .artifactBundle into SwiftPM so that any relevant libraries are made available to the linker or referenced appropriately in it's build process.

I wish I felt more capable or knowledgable about this space to even write a proposal. Do you know who I could reach out to spear head this? I think it's a big problem!

@Joseph_Heck

I'd note that the embedded Swift capabilities, while amazing and effectively already, are still very nascent - and I would expect at least some amount of effort will need to be expended to create and assemble relevant SDKs for the platforms in order to support cross-compilation. I would very much recommend searching around these forums for other references to Embedded Swift, as there's a lot of really interesting activity across both microcontrollers and higher level compute systems (RaspberryPi), that all have overlapping needs and interests. You made find some of what you want to accomplish actively being worked on through those projects.

Thanks for this! Currently, I'm little demand for Edge-Compute in smaller embedded devices at the very moment. Because of the extreme performance needs of running vision models, I only see a lot of demand around very beefy Linux devices with GPUs with 64-128GB of RAM and 12-24TB of storage. The market demands are closer in line with a Mac Mini M4 chip than a Raspberry Pi 5. This means that we don't really have to focus on Embedded Swift, but more Swift on Linux.