Expanding the power of build tool plugins to build something fun

As a tools developer, it's critical I try to empathize with developers who use the tools I help create. I am probably most intrigued by game developers. My favourite influencers are game developers. They solve some of the hardest problems in our industry and I have a lot of respect for their expertise and their pragmatism that allows them to deliver great software that demands great architectures and performance.

So I guess it wasn't surprising when a video dropped into my stream called "How to Make a 2D Game in C++ and SDL3." It is two and a half hours long and takes you from scratch through animating characters to handling shot collisions. And it's just a cute little game that you can create by hand yourself.

Now, of course, as I work on tools for the Swift package ecosystem, my first question was: where is the "How to Make a 2D Game in Swift and SDL3" video? So I set out to try and replicate this game in Swift and try to get it running on my desktop. And I wanted something that I could try out the Swift SDK for Android. Swift's powerful interoperability with C should make it a great experience.

Well, predictably, at this point in time at least, I quickly ran into hurdles with our tools.

  • I could brew install SDL3 and use a system target to hook it up to my package, or I could grab the release binaries from GitHub and create my own artifact bundle for a binary target. But I really want the experience to be much simpler. You should be able to run swift build on your package and everything just works.
  • Once I did get SDL3 hooked up and started using the API, I quickly noticed that SDL3 uses a lot of opaque structure pointers that turn into OpaquePointers in Swift and you lose the type safety you had in C. I had a great idea of adding a code generator to parse the Swift and parse the C API and generate bindings that replace them. But I'd like to do that all in one multi-language target so I can generate the module map for the C API and layer on the Swift bindings with a simple user experience.
  • Building that generator reminded me that Swift Syntax prebuilts still only support macros. Build times for code generators that parse Swift code could really benefit if that restriction was relaxed and supported all targets that built for the host.
  • And finally, there is the problem of how to generate the APK file for the Android version of the game. I need to take the shared library with the game logic and process it with Android's command line tools to produce the APK. If only build tool plugins had the power to create their own build products for a target...

This use case was inspirational and drove me over the summer and into September to build a proof of concept that solves these issues. That POC is here and I'm really happy how it is turning out: [POC] Custom Targets, External Targets, and Swift Syntax Prebuilts by dschaefer2 · Pull Request #10374 · swiftlang/swift-package-manager · GitHub .

It's a pretty huge change that I will break up into smaller proposals over the next few weeks. The SDL example is in the Examples/swift-sdl folder. It needs SDL checked out next to it at the 3.4.x release. I'll list here the main features and will work on adding details to the design document in the PR so I can start creating evolution proposals.

  • The main feature expands build tool plugins to allow them to contribute build products for the targets they are attached to. This is a powerful addition that leverages the work that was done to support plugin tools with Swift Build's CustomTasks and adds a mechanism to specify some or all of the output files as product files that get copied to the Products directory so other plugins can discover and consume them and produce their own artifacts.
  • Building on the recent support added for multi-language targets, we add the ability to have targets without any language. I call these CustomTarget in the code but they essentially allow build tool plugins to define all the build products for Swift Build's aggregate targets. This is used by the ApkBuilder in the example.
  • To simplify integrating C libraries like SDL, a specialization of CustomTarget is added called ExternalTarget that allows the source for that target to be external to the package. It allows that source to be at a local path, or a remote source archive SwiftPM fetches for you, or a git repo that SwiftPM will check out for you. You would then add build tool plugins that manage the build of that source. In SDL's case that's a plugin that runs the CMake configure and uses Ninja to build a static version of the SDL library. For Android, it also produces a jar file with its Android Java native bridge, both of which get added to the build products for the target.
  • It occurred to me that these mechanisms would also really help with Swift Syntax prebuilts. The prebuilts are just a static library that bundles all products in Swift Syntax. These could be modelled as CustomTargets as well and starts us down the road of figuring out how to support prebuilt binaries more generally.
  • And it reminded me I need to graduate the experimentalCGen feature and expand it to let plugins generate mixed language source for mixed language targets. This is used by the SwiftSDL target that produces the module map for the SDL3 library as well as an API notes file that facilitates the Swift bridge to wrap some of the C functions to use safer types but use the same function names.

I encourage you to take a look at the example and get a feel for the ergonomics. I've done a number of iterations on the ideas to make them feel easy to learn and simple to understand, but I could really use more eyes and fingers to ensure we're making something developers will love to use. It was one of those "woo hoo" moments when the app first built and ran with a Package manifest that felt good. As simple as it is, it just paints the window the Swift orange, it was a great feeling to see that orange window open on my desktop and then launch the same app on Android. Now all I need is to get swift run to just work everywhere too :slight_smile: .

34 Likes

Woo hoo! I have the SDL example now working on Windows.

14 Likes

Hi,
This looks awesome ! Thank you for building this kind of capabilities !
I'm working on embedded systems and I'd love something like this to handle some task like : dependencies, build firmware, build boot loaders, merging them and more. I was very exited to try your PR sadly it is only available to the new SwiftBuild and I'm stuck with --build-system native for now (see SPM issue). I'll keep an eye on your branch and can't wait for your proposals

1 Like

Thanks! We are making a push to get embedded working with SwiftBuild. There are a couple of issues we know about.

But yes, having a custom target that builds firmware from executable targets is one of the use cases I had in mind.

1 Like

Awesome don’t hesitate to ping me if I can help with detailing my issue, testing fixes or working on fixes but I have no previous experience with the compiler nor SPM. I’d be happy to contribute in any way that could help embedded :)

This is really exciting, Doug. Thanks for putting this together! It opens up a lot of possibilities for embedded workflows that currently need a separate build script around SwiftPM.

I’ve been experimenting with your branch in CPicoSDK, splitting configuration resolution, native Pico SDK compilation, and final firmware linking into separate custom targets. A small Swift target supplies board and stdio configuration, allowing the native SDK build and Swift application compilation to be scheduled independently. The final step consumes both archives and produces the ELF and UF2 files. That puts the work into one swift build graph, with dependency tracking and incremental builds: application-only edits reuse the SDK, while native stdio changes don’t require recompiling the Swift application.

This also exposed an interesting publication issue: release copy phases inherit COPY_PHASE_STRIP=YES, which caused the macOS strip tool to process the ARM SDK archive and corrupt its archive index. Locally, I’ve disabled copy-phase stripping for custom targets. My thinking is that publication should preserve the producer’s output, leaving any stripping to an explicit step using the appropriate toolchain.

The experiment also relies on two open PRs:

  • #10591: keep bare-metal destination settings from leaking into host plugin-tool builds.
  • #10622: avoid linking nonexistent archives for Clang targets without compilation inputs, while preserving their headers, module maps, and build dependencies.

There’s another local change I still need to submit: materializing explicitly static library products from dependency packages as actual .a files, rather than only materializing root-package products. That lets a custom target consume the application archive from another package.

With those changes, the full firmware build and incremental rebuilds are working locally. It’s encouraging to see this much of the embedded pipeline fit into SwiftPM’s build graph.

3 Likes

Thanks Gonzalo! That's an exciting use case. I'll keep an eye on your changes. I have a bunch of Pi Picos lying around that I can try things out on. It is really my hope that the package ecosystem will grow with packages that enable integrations like that.

I'm starting to do the official version of this and take it through the Evolution process. Once that's rolling we can make sure your example works as well.

3 Likes

This is cool! I'd love to have a link to the original video that inspired it as well if possible.

:slight_smile: I want to avoid directly linking it to keep spammers at bay. If you search for the title on YouTube, you should be able to find it. I'm also seeing lots of similar videos for Raylib. That would also be a good candidate.

I have a pet project that currently targets Windows and uses SDL. I’m very glad to see improvements in this area.

1 Like