Creating a new Platform, SDK, and toolchain, resources?

Development Compiler
,
1

I've started the process of learning how to add a new platform and create a custom toolchain for it. But Swift source code is a lot more overwhelming than I anticipated.

I'm thinking of starting with a simple device and I chose the GameBoy Advance.
The device has an armv4 (arm7tdmi) cpu which llvm appears to support.
There is no OS, but there are on device statically linkable functions along with lots of constants that I'd like to put in the Swift SDK for the platform.

The GBA cartage address space is only 32MB, so Swift is less than ideal for this platform but I think some cool content can still be made and others would enjoy it.

Creating a toolchain would mostly involve disabling things, so it seems like a good learning project to help me get used to navigating Swift source.

I've been looking through commits in swift-embedded and SwiftWasm to try and figure out some stuff, but there is way too much going on for me to just trial and error my way through it.

Can anyone point me to some helpful resources?

3 Likes
2

You may fall into a trap thinking that adding support for a "simple" platform is going to be a simple task. In my experience of working with SwiftWasm, that's not the case. If you're interested in learning, you should either look into how current platform support works, digging into the existing build system, infrastructure, runtime, etc, or look at a platform that's as close as possible to already supported platforms.

32-bit platforms are quite far from what upstream Swift supports. Good news is that you aren't targeting a 16-bit platform. :sweat_smile: As far as I know, none of the PRs upstream are tested on 32-bit platforms before merging, and looking at community CI, 32-bit Android had last successful build more than a month ago. I also frequently see Raspberry Pi people trying to keep the 32-bit build maintained saying it breaks with almost every release. SwiftWasm is (partially) 32-bit, but we had to make and still maintain plenty of changes from upstream to seemingly make it work.

Unfortunately, Swift wasn't designed (at least initially) with embedded platforms or bare metal support in mind. This doesn't mean it won't ever support such platforms, it's obviously up to the core team to decide when/if that's going to happen. In the meantime you'll have to resolve or work around a few major roadblocks that don't have good solutions in the upstream codebase yet.

This is exacerbated by the fact that GBA has (looking at Wikipedia :eyes:) 32 KB internal, 256 KB external RAM. It's going to be quite hard to get a binary produced by Swift that fits into memory, never mind making the rest of the things work.

The best place to start is probably existing C/C++ toolchains for GBA, especially if there are any based on LLVM. In the end, if LLVM, clang, and related tools don't fully support your target platform, I don't think there's much point to start looking at Swift until all the dependencies work well.

When you're ready to switch from C/C++ toolchains to Swift, here's an incomplete list of things I'd start looking at:

  1. The series of articles by @jrose about Swift runtime and metadata: Archive for “Swift runtime” // -dealloc. This other article about porting Swift to other platform is also a must read: Swift on Mac OS 9 // -dealloc
  2. The issue of relative/absolute pointers in the runtime. You need to check if upstream implementation of relative pointers works on GBA IRGen & Runtime: Add a flag to disable relative pointer for some object formats by kateinoigakukun · Pull Request #39259 · apple/swift · GitHub and Wasm Support - #16 by kateinoigakukun
  3. Turning multi-threading off, especially if you're targeting bare metal. ARC and probably the whole runtime assume the presence of atomics and thread-local storage. You may need some shims for that, or rely on LLVM's -femulated-tls Runtime: emulate TLS with a map for WASI target by MaxDesiatov · Pull Request #31694 · apple/swift · GitHub
  4. What libc are you going to use? Will you provide your own shims, or does some barebones libc port exist already for your platform? That's why testing existing C/C++ toolchains for GBA is crucial. The presence of WASI libc simplified a lot of things for SwiftWasm, but has some significant amount of overhead. There's also an undocumented "freestanding" configuration that was recently introduced to upstream Swift, which may be relevant. We plan to investigate its feasibility for Wasm builds, but haven't done that yet. Here are a few sparsely documented test cases to start digging: swift/check_freestanding_dependencies.py at 83484f85163bdfbe743f0a840451982268644d3c · apple/swift · GitHub and swift/check_freestanding_size.py at 0bf1389863193cf4a397234dbf15d65dafd7c4b7 · apple/swift · GitHub
  5. The last link above to check_freestanding_size.py may also be a good pointer for reducing the binary size to make it fit into the minuscule amount of memory you have. Looks like 640 KiB on x86_64 Darwin is the smallest you can get right now, and I'd assume that doesn't even link libc statically. But in the end this may be the biggest issue overall. Even though ICU dependency was removed from the toolchain, it didn't reduce the final binary size that much, since Unicode tables still need to be embedded. You'll probably have to disable all Unicode stuff with the String type in stdlib altogether for things to work. For example, StringUTF8.swift test is completely disabled in the freestanding configuration.

The other problem is that cross-compilation isn't supported very well (if at all) outside of Apple's platforms. You'll have to gain some good knowledge of the current toolchain build system to make it fully work. The fact that the build system is spread across shell scripts, Python, and CMake, makes cross-compilation particularly hard.

Having at least some basic knowledge of CMake is highly recommended. This article is a good place to start It’s Time To Do CMake Right | Pablo Arias, but may not be sufficient. Have a look at awesome-cmake list, and maybe Professional CMake book, or any other widely recommended book (like "Effective CMake" or "Modern CMake") if you want to dig deeper. Unfortunately, the official CMake reference does not seem to be a good intro, at least it didn't work that way for me.

I don't want to discourage you from working on GBA support, but you may also consider porting Swift to musl/Alpine Linux as a better place to start. This isn't too different from Linux distributions supported already, but is something that would be quite useful to support in upstream Swift, and as a learning project that may help you with GBA support in the future. For example, musl libc needs to be statically linked, which is probably going to be the case for GBA as well. I started working on that here, but got stuck on trying to make LLVM/clang work with musl: GitHub - MaxDesiatov/swift-alpine: Swift on Alpine Linux using musl instead of glibc (work in progress). Any help with that would be highly appreciated.

I also encourage you to have a CI setup as early as possible. This will help you get reproducible builds and will get you something to come back to later. GitHub Actions + Docker with hardcoded image versions may be a good option, especially as it's free for open-source projects.

I hope this helps!

15 Likes
3

Maybe https://www.swiftforarduino.com/microSwift.php is a better starting point than the full language?

4

Thanks for this very detailed intro Max.

I wonder why?

I remember briefly working on a computer with similar runtime limitations: 64K of memory per process. Simple apps were 5-20K, a simplified C++ compiler (no templates) was probably the most complicated app in there (about 200K), to fit it in memory overlay system was used. All in all, with enough motivation the task is doable, especially if to remove some complex features from swift (generics? existentials? escaping closures?)

1 Like
5

Because we want "𝐀".count to evaluate to 1 on all platforms. Or for capitalized property to work. The String type implies Unicode support. A proper alternative is to make this type unavailable on platforms that can't support it. Otherwise people will start porting their code assuming it will work, while it won't.

What I'm trying to say, if you don't need Unicode support, you shouldn't use String in the first place. [UInt8] should be enough. Whether stdlib should allow build configurations without String and Unicode tables aren't included is a separate question.

Quite possibly, that mysterious "freestanding" configuration already does that, I didn't look into it yet. And if it doesn't, I personally think that such build configuration should exist, specifically for platforms that can't afford handling large binaries.

1 Like
6

I see. It would be reasonable to not fully support unicode or support only a subset of it (e.g. UTF8). Capitalize will work properly only for ascii "a" .. "z" range, that's reasonable.

Does "count" require tables? A UTF-8 sequence like this:

F09F91A8 E2808D F09F91A9 E2808D F09F91A6 E2808D F09F91A6

(family of four) has count = 1, and that's easy to determine without massive tables: implementation needs to know how to break UTF8 sequence by code points, what "zero with joiner" is, etc, but that seems doable without tables.

7

The other problem is that cross-compilation isn't supported very well (if at all) outside of Apple's platforms. You'll have to gain some good knowledge of the current toolchain build system to make it fully work. The fact that the build system is spread across shell scripts, Python, and CMake, makes cross-compilation particularly hard.

Good post, but I think you exaggerate the difficulty of cross-compilation. LLVM and Swift have excellent support, particularly since the most popular use of the Apple toolchain is to cross-compile from macOS x86_64 to iOS arm64. I'm now able to cross-compile a Swift SDK for Android from trunk with only a single Foundation patch, by using the official prebuilt toolchain from other platforms. Only a new language like Zig, that focused on cross-compilation much more, is probably better.

Obviously, adding an entirely new platform to any compiler is not easy, but I don't think Swift's toolchain build adds much to that. Leaving out tests, I count around 20 CMake/bash/Python files in the compiler build config that mention Android. That's not much and would be a small fraction of the work required to add a new platform.

Finally, rather than reading all that stuff you linked, I recommend diving in with that easier Musl port you mentioned instead, falling back to the references only when needed.

2 Likes
8

Thanks for the detailed info @Max_Desiatov!
This is a lot to go through and I appreciate your insights.
The -dealloc blogs looks really interesting. I'll read them and cmake resources tonight.

There are plenty of open source toolchains for GBA using C, and at least 1 person used llvm to compile a simple ROM coded in Rust.

This had not occurred to me. Getting Swift to fit in there would require a lot of cuts and then what would even remain for the software...
It would be really fun and interesting to be able to turn a swift package into a ROM with swift build but I think waiting for official baremetal support before doing this makes more sense.
Assuming Apple ever wants to allow Swift for Metal shaders, AirPods, or Magic Device firmware. (Swift for Metal shaders :drooling_face:).

My ultimate goal is to add support for Nintendo Switch. I'd like to be able to deploy my games (All written in Swift) to it. I was deterred because it has a completely custom OS, but maybe I should just start with that. It's 64 bit ARM with plenty of ram.
Someone got a little swift code running on Nintendo Switch using C interop with swiftc and makefiles, so it should probably be easier than GBA. If I skip the OS stuff to start, maybe it won't be so overwhelming.

Yes, I was going to look into a CI once I get an unmodified clone building. I'll probably make another post when I get there asking for opinions and options for that.

But I'm struggling to get release/5.5 to build on macOS. I guess Xcode 13.2.1 is too different. Gotta go fishing for an old Xcode version and spend a few hours decompressing it :grimacing:

1 Like
9

UTF-8 vs UTF-16 isn't important - those are just different ways to encode the same Unicode code-points (UnicodeScalars in Swift terminology).

For grapheme breaking (determining how to combine scalars to characters), the tables are relatively small, so it's not too bad.

But then you need all kinds of other tables - comparison, for example, needs normalisation data, which is pretty big, and then you need more, even bigger tables for capitalisation and scalar properties (which are probably the easiest to live without, if you had to).

All things considered, I think it would be better for embedded platforms to only support ASCII. If you only choose to include some of the functionality, you start getting in to weird situations where Equatable, Hashable, and Comparable don't work as expected, or strings in switch statements behave strangely with Unicode text. But if you just redefine String as ASCII and ban all Unicode, you at least don't need to deal with these inconsistencies.

1 Like
10

I don't think you should build release/5.5 when trying to modify the toolchain and/or SDK, unless you're interested in fixing a bug exclusive to 5.5. The toolchain and LLVM move very fast in terms of API breakage and build system changes, you're better off experimenting and submitting patches against main directly.

11

If all you need is ASCII, why would you use String instead of [UInt8]?

In my ideal world StaticString would conform to ExpressibleByArrayLiteral where Self.ArrayLiteralElement == UInt8 to be able to write let x: [UInt8] = "hello". Assuming DCE works well and you don't use String type at all in your code targeting embedded hardware, all that hefty String implementation details would be stripped away altogether, leading to even smaller binaries.

This would help WebAssembly too, where we already have JSString bridged to JavaScript strings, together with init(_: [UInt8]) initializers on them.

12

It's still nice to have string literals and a dedicated type for text-related APIs, even if you don't have Unicode support.

Of course, an ASCII-only String type could be backed by a [UInt8]. There's no reason it needs to keep the current storage implementation.

For a platform/toolchain, my assumption is that you would want to be able to build as much existing Swift code as possible. String is the currency type for text, so you'd want some version of it, even if it doesn't exactly match the full API or feature-set on other platforms.

13

That was my original thinking too, but I couldn't get the build-script to finish successfully from main.
But, I see the CI dashboard lets you see the last successful builds commit. I'll try building a clone from that.

14

What kind of error messages are you seeing? Are you on Xcode 13.2.1? Everything builds with all tests passing off main successfully for me, at least in the last couple of days.

15

I didn't keep the messages, but it was unable to locate .o output at one point and couldn't find things in the Xcode.app toolchains at another point.

Yeah I'm using Xcode 13.2.1 with Command Line Tools installed too.

Just using the getting started guides build-script flags for macOS (ninja).
Using x86 and M1 at various times.

16

Well, I do recommend sharing such error messages with repro steps either here on forums or on bugs.swift.org. Feel free to tag me in these reports. It's either there's some misconfiguration, or we need to update the build instructions, or whatever else needs to be investigated to make this work.

2 Likes
17

Okay cool :sunglasses:
I'll try a fresh clone of main tonight and report on the output.

2 Likes
18

Worked this time. build-script finished with no errors from main with Xcode 13.2.1.
I must have got unlucky with my timing of the last clone.

So, next I'll try to figure out how to generate an xctoolchain.

19

Not for those of us that uses languages with additional characters. Like åäö (ÅÄÖ capitalized) - and those should be sorted in after Z, not anywhere else ;-)

3 Likes
20

Tried building the toolchain with the utils/build-toolchain script, but it fails.
I only used the bundle prefix flag.

FAILED: SwiftCompilerSources/SIL.o ...
FAILED: SwiftCompilerSources/ExperimentalRegex.o ...
.../build/buildbot_osx/swift-macosx-arm64/bin/swiftc: No such file or directory

Since build-script built the compiler fine, I'm guessing the errors are config related.

Building and installing the xctoolchain seems like something everyone needs to do frequently.
Is build-toolchain what everyone uses, or is there a simpler method?

Edit: Checked out WASM's build script. That's pretty helpful since my device will have the host and target libraries too.
But before I start making a build script I think I should be able to generate an unmodified macOS xctoolchain.