Hi folks, I need help learning something new. With the help of an LLM agent I’ve got something very rough that sort of mostly works, but now I’m at the point where I need to understand the options properly myself to make it robust enough to build on.
I’m integrating some Swift with some Rust. (For reasons. Not necessarily good reasons, but for the purposes of this question please assume they are.) At some point my Swift struct needs to get sent over to Rust, and I’m using the C FFI from both sides, and that works nicely for a toy example with just a couple of primitive-type properties. But I have some worries.
Strings. I need to get to something like a C string: the only ways I know to do this use a closure API which quickly gets out of hand as the number of properties grows.
Struct layout: I think for small structs the memory layout is just the properties one after another (modulo alignment I guess) but (a) is that actually true? and (b) does it stop being true at some point if I add more and more properties?
Unknown unknowns: I don’t know enough to know what questions to ask.
If anyone has suggestions, either specific or for good sources to read up on, I would greatly appreciate it.
Here are my LLM-fueled efforts so far:
public func withFFI<Result>(_ body: (UnsafePointer<AppStateFFI>) -> Result) -> Result {
return self.title.utf8CString.withUnsafeBufferPointer { titleBuffer in
return self.inputText.utf8CString.withUnsafeBufferPointer { inputTextBuffer in
let titlePtr = UnsafeRawPointer(titleBuffer.baseAddress!).assumingMemoryBound(to: UInt8.self)
let inputTextPtr = UnsafeRawPointer(inputTextBuffer.baseAddress!).assumingMemoryBound(to: UInt8.self)
let ffiState = AppStateFFI(
titlePtr: titlePtr,
titleLen: titleBuffer.count - 1, // Exclude null terminator
inputTextPtr: inputTextPtr,
inputTextLen: inputTextBuffer.count - 1 // Exclude null terminator
)
return withUnsafePointer(to: ffiState, body)
}
}
}
A String is not guaranteed to have a contiguous byte buffer at a fixed address in memory, so the only way around this is to copy out into a C string, which of course has performance implications.
Instead of using structs and dealing with alignment (plus byte order difference, in general, although that's not a worry in your usecase) another approach is to serialise / deserialise into some efficient structure like msgpack / protobuf / binary json / or similar. Could be as simple as <size> + <bytes> sequence. If there's a reasonable limit how large the maximum content could be you could used a fixed allocated once buffer for exchange, in a single thread case at least.
Depending upon how fast you want this to be / how often do you exchange strings you might want to invest into some infrastructure where you have two copies of the state, one in Swift and another in Rust, and exchange "delta messages" to sync those states. When you know that the string in question is already known on the "remote" end you may use its identifier / index to quickly pass it across instead of passing the string itself.
As for the struct alignment, I believe it is fixed for frozen structs de jure and it is also fixed for non frozen structs de facto (there's no guarantee that the latter won't change in some future compiler version).
Or maybe use something like zmq to communicate between the two worlds. I once used that approach in an iPad app where the frontend would talk to the (Python) backend over zmq (in-process).
Thanks all, that’s a lot to consider. Because this is mostly a learning project, I’m looking for the right balance between a genuinely good implementation (but too ambitious for me to code up in my evenings after putting the kids to bed) and something I can easily wrap my head around (but so simplistic it won’t even support the fairly limited performance demands I have in mind).
I think my options now are:
accept the callback pyramid to get pointers that minimise unnecessary copying out of the original strings
copy c-strings into a struct and pass the struct (I think to understand I would need to learn a bit about Swift and Rust memory layout expectations, and might get nasty surprises), profiling diligently because this is definitely the “don’t expect performance” option
do it smarter with delta messages (which, however, gives me all the same challenges just in representing the messages!)
do it even smarter by using something else for the transport (the zmq suggestion), which sounds awesome but is adding a couple of new technical challenges before I’ve handled the first one
I’ll see how far along the scale I get. If you have any good learning resources to throw my way, I’d be grateful!
At least this simplifies passing strings from Swift to C.
As for the other way around, if I am not mistaken there was "zero copy" way to make NSString from existing data (still is?), then treat it "as String" – easy to write, but I think here the bridging will not be free (I may be wrong).
I didn’t know you get free bridging there, that’s cool! (I guess it must be able to consolidate as needed, as String doesn’t guarantee contiguous storage.) This is another point to add somewhere in the middle of my sliding scale of “how much new stuff are you prepared to learn in order to get this right?” Thanks for the suggestions!
As for the other way around, if I am not mistaken there was "zero copy" way to make NSStringfrom existing data (still is?), then treat it "as String" – easy to write, but I think here the bridging will not be free (I may be wrong).
This is correct. I find new ways to make it cheaper every year, but I don’t expect to ever reach “free”.