Avoiding stack overflow when nesting String.withCString { ... } (how to handle an arbitrary number of temp values in general)

Hi @tera!

In my particular case, the optimization should be avoided if it creates a risk. The user has written valid code, so the library ought to run it correctly. I'm currently writing a fix, and don't recurse if there are more than 20 (chosen arbitrarily) arguments.

You can check remaining stack size and act accordingly, although this is rarely done.

Right, but I believe it will be overkill in my particular case. Is it worth the complexity/knowledge? What about future maintenance?

Do you see these stack overflows in practice? What's the relevant stack frame size of your functions? How much stack size is used per one level? For example if available stack size is 200K and one parameter eats 1K it'll take some 200 parameters before overflowing stack.

Those are excellent questions. On the SQLite version that ships on my Mac, statements can accept up to 500000 arguments. So GRDB users can provide up to 500000 arguments. Very few will do so. But they can. And they can complain if this does not work. With unbounded String.withCString { ... } nesting, that's up to 500000 nested function calls. I actually don't know if this could create a stack overflow (and I don't know if local variables in the recursive function are stored on the stack or in registers), but I also feel that the stack trace, in case of thrown errors from the top of the stack, or in case or hard crash, would look pathological :sweat_smile:

Eventually, providing a hard-coded and "sensible" limit is maybe not a bad option. Frequent use cases will profit from the optimization (we're talking about ~4% faster). Thanks again for your questions :-)

Here's a different approach that sidesteps recursion altogether

Avoiding copies of C strings is the whole point of the optimization, so unless I'm mistaken only Swift.withCString { cString in ... } guarantees the absolute minimum amount of work when I start from a Swift String: there is one and only one copy of the C string, the temp one produced by this method.


It all started from a challenge that I found funny and interesting :-) You never know where you end up!