With the increase in GPU capabilities and expectations, I started to think about the possibility of system level programming with Swift, specifically for the GPU. Something Similar to that of Mojo?
I started to work on my own mini-compiler that takes Swift and translates it to Metal, but was wondering if there had been other methods of tackling this problem.
I had a somewhat working pass at this a few weeks ago. I don’t think it actually solves many of the problems we commonly encounter with Metal. But it was an interesting experiment.
While making a Transpiler to a GPU compilable language is possible, you might consider making more robust and consistent language support by exposing LLVM level support. To make a more robust compiler you might want to find a GPU compatible LLVM SDK or toolchain and see if it is compatible with Swift, potentially with amendments from the Swift LLVM fork.
NVIDIA provides LLVM level support for GPU execution with information provided below:
You might want to discuss this with the Embedded Swift Group. GPU Swift would likely be a subproject of for Embedded Swift as it has some similar constraints.
Notes:
I don’t know if Swift’s own SDK functionality supports LLVM SDK Bundling; If not then an installer or additional install instructions might be necessary; potentially dependent on SDK/Toolchain vendor distribution restrictions
You might want to setup custom optimizations as a batch processor is a significant deviation in processor type than Swift has previously supported
You might want to provide a Package and/or library to provide management functions for bulk processing operations as they need differing expectations
Musings on an API for GPU programming
These will likely need to be revised for an actual API.
However, I do seek to address technical issues for adaptation of Swift to a significantly deviant processor type to Swift’s normal processing and operating environments.
These do introduce an annotation for GPU work, @branchesYield which requires all branching operations have the await keyword as any sub executors that do not meet requirements need to be suspended until all branches complete while being taken sequentially[1]. And there are call
/// Create a bulk processing task cluster for working on provided data.
///
/// Use this function to schedule bulk processing operations on a GPU if available
///
/// > Warning:
/// > Any Branch points in your code incur their entire processing time penalties
/// > on all executors regardless of if they take short paths or long paths.
/// > To mitigate this, do not pass inputs which finish significantly before or after other executors are finished.
///
/// > Warning:
/// > Loops with dynamic repetitions should be avoided as they can result in unstable run durations.
/// > They should be able to be simplified to a non-branching expression that is GPU compatible.
@__callRestriction(disabled:GPU)
func withBulkTask<Input,Output>(
_ data: [Input],
body: @__branchesYield @__callRestriction(enabled:GPU) (_ view: Input) async -> Output
) async -> [Output] { ... }
let adjusted: [UInt8] = await withBulkTask([0b1011010,0b01010101,…]) { input in
// GPUs do not often support branches as they have a shared process counter per bulk operation block.
// Using any branching execution(ie. `if`, `switch`, `guard`, early `return` calls, `break`, `continue`, `func -> Never`, short circuiting operators/functions, `throw`)
// will suspend execution of all threads if multiple branches are reachable.
await if 0b001 & input == 1 {
return 1
} await else if 0b01001 & input > 0 {
return 0b01001 & input
} await else {
return input as UInt8
}
}
The compiler may have leniency to reorder these ↩︎
That's super cool! I was thinking of something similar. What type of problems do you usually face using Metal? Mine is I am not great at math lol. It's very difficult for me to translate ideas into linear algebra equations. I am imagining a world we could just describe what we want in each pixel and instead of doing the math by hand. do you think that's possible?
The problems with Metal, from a 3D graphics perspective, are mostly around the framework rather than the shader language. It’s complex, and there’s a lot of boilerplate and inside knowledge needed before you can render your first triangle: render passes and pipelines, vertex descriptors, moving data between the CPU and GPU, fragile shader bindings, and so on.
A new shader language doesn’t really help with much of that. I’ve been building MetalSprockets as a way to build Metal applications more like SwiftUI: composing smaller elements and using type safety where possible to smooth over some of the rough edges.
Part of the problem is that relatively few people use Metal directly. It’s often a target for ported third-party code, with shaders translated to Metal, or it sits underneath a larger framework such as SceneKit, RealityKit, or Godot.
So the audience for people writing Metal directly is relatively small. Apple seems to focus mostly on engine builders and ML, which makes sense—but perhaps also explains the C++ bindings and the fact that the Metal team still releases Objective-C sample code to this day.
The other side is ML and graphics research, where a lot of code uses CUDA. I end up porting some of that code to Metal, and it’s often awkward. Mojo may eventually help here, but a Swift extension that works similarly would likely need Apple’s full support. I have trouble seeing it as a viable small third-party project.
I don’t think there’s much point in this; you don’t need a convoluted, "deadly" syntax for the GPU. C++11 provides all the primitives necessary to fully leverage a graphics card’s capabilities, and these primitives are quite simple to learn. It would be far more promising to configure a package manager so that it could import C modules directly into shaders—something I wrote about here.
This will allow Swift—while remaining the "brain" of the application on the CPU—to dispatch tasks to the GPU and manage the responses in the most efficient way.
It would be even better if a package manager with such functionality could run on Linux and Swift were able to dispatch commands to Nvidia chips—that would be absolutely overpowered. Because, in essence, Сuda, Vulkan, and Metal are triplets.
I have worked around SPM's issues with Metal targets here:
It's a pretty horrible workaround to deal with SPM's severe limitations. I have filed issues in the past but progress has been invisible.
It's really hard to use SPM and Metal well together and I know a lot of people have just given up. My plugin works pretty well for me though. But first class support for Metal in SPM would be amazing.
It’s nothing too complicated—you just have to duplicate C structures in both the SPM target and the .metal file. Still, native integration between Metal/Cuda/Vulkan and SPM would be absolutely fantastic—on the same level as async/await. That would truly take multithreading to an entirely new, stellar level.