Hello everyone! As a personal project and as an excuse to (finally) learn Swift (I got started in my programming career as an Objective-C developer), I've been writing a game engine lately in the language. My objective is to write a fast, user friendly, and multiplatform Swift game engine, with rendering backends for Metal, DirectX, and Vulkan.
Those of you who are familiar with game engine design will be aware that modern game engines tend to opt for Data Oriented Design approaches. These days, Entity Component Systems are also all the rage, which tend to lay out game Components in contiguous buffers and then crunch through them, asynchronously and concurrently, via Systems that ensure there's no dependency collisions or other multithreading issues.
Therefore, I've been designing my Swift game engine with these sort of approaches in mind! For multiplatform reasons and to ensure the various different rendering backends and whatnot can have easy, contiguous access to the game data, I've written some really basic allocation backend stuff in C. I need to manually allocate and manage my memory with C and get Swift to place nice with this.
I've been fairly successful with this. Using a combination of Wes Wickwire's Runtime, Jordan Rose's Swift-PPC work, and Robert Widmann's work on Swift Gestalt, I've been learning about how the Swift runtime works and how it allocates and lays out memory and so on. Using Value Witness Tables, I have fairly granular control over struct data and can pass this to and from my buffers and still play nice with ARC.
I've also figured out how to do this with object instances, basically placement new but in Swift. Same deal, I'm basically able to manually allocate/deallocate instances into/from whichever pointer I want. However, so far the way I'm handling instances is not quite as efficient as it could be since this part of the Swift runtime is a tad less documented. So here are my questions:
1. I don't know how to run my instance's (required) initialiser on my manually allocated instances.
My current workaround is to allocate a throwaway Swift instance the usual way, iterate over and copy all the properties over to my manual allocated instance, and then let ARC reclaim the throwaway instance. The copy properties stage is so any instances, contained either through value types or just ref themselves, aren't deallocated when the throwaway gets dealloc'd.
This is suboptimal because we're paying for a malloc->free with every instantiation, and depending on the platform might even cause memory fragmentation. I know that Swift's class metadata contains info about all the methods in a class and I could probably get at the function pointer for the initialiser through that, but it's unfortunately not well documented enough (Swift Gestalt gets close but it's still a tad unclear). I was wondering if anyone here is familiar enough about how instance initialisation works in Swift at the really granular level, and would be willing to explain it to me? I'm also unsure if the initialiser has code for the property initialisation bit of instance initialisation, or if that's something else done by the compiler.
2. I know how to run an instance's destructor, but the compiler also seems to do some work here?
An instance's destructor is much more easily accessible and that part of the metadata is much better documented (thanks Swift-PPC!), and so I can call an instance's destructor fairly easily. However, calling that only really runs the destructor, and any properties are not dealloc'd and are leaked, which hints to me that ARC is the one doing this work. What happens when the ref to an instance is decreased, or how does this work? Does ARC also trawl down all the properties of an instance and decrease the count of any refs 'owned' by this instance? Or, alternatively, does ARC only do this trawling once we know the parent's instance is about to be dealloc'd? Is recursive trawling how this whole thing happens, or how does it work? I'm basically trying to learn how and through what method is it that ARC does ref retains/releases through any refs 'owned' by anything to guarantee there's no leaking.
My current workaround is to just always keep my manually allocated instances at ref 1 and just call an Unmanaged.release when I need the destructor and release of any children to happen. It works well enough and ARC doesn't seem to mind my custom allocated buffers. I haven't yet done any experiments to see if ARC trashes my memory or somehow free()'s my buffer (unlikely?), but that would be easily fixed by doing another one of my inefficient throwaway duplication thingies I'm doing to construct instances. :P
3. Dangling refs/pointers and the refcount sidetables.
When one manually allocates stuff like this one has to be prepared to deal with dangling pointers, so that bit's expected and doesn't scare me too much, but through my investigation I noticed that when you create a weak retain to an instance you create a 'sidetable', which I imagine is some sort of cross-referenced pointer thing to ensure safety when the instance gets dealloc'd, to nil the weak ref.
Another optimisation that I'll need to do is to move instances around all over the place, so unfortunately the pointer to an instance can't be guaranteed to always be at the same memory address. I'm using the standard approach of just wrapping pointers in handles that deal with this, but I'm wondering if those sidetables mean that any accidental weak ref or something along those lines could cause any unexpected undefined behaviour later on? How does this aspect of ARC work?
Just use structs and protocols?
I'm well aware that all of this talk of contiguous buffers and whatnot would be best served by ditching instances entirely, and just using structs and protocols to achieve what I need to achieve (and would ironically mirror the designs of some of the more performance oriented C++ ECS systems out there). However, one of the requirements of my game engine is to be user friendly, and unfortunately the big player engines out there like Unity, Unreal, and what have you don't make their end users use structs, they use instances and inheritance and all that, and a struct design would make my engine harder to quickly take up for the sort of people I want to write it for (beginners, indies, those sorts of people). Therefore I'm really keen to get instances working just as nicely as structs with my system, and would appreciate any advice that helps me achieve that.
Thank you everyone for the help!
