How to prevent crash due to Swift Runtime initializing `os_log` on macOS?

Using Swift
41

Yes, sorry, that wasn't clear. Call bt all in the parent when it calls fork(), perhaps.

42

You’re mentioning this for the second time now. What gives you the impression there is a callback involved?

The flow is as follows:

  1. A client registers the function as available to Postgres. At its core this involves running SQL akin to „load function X from libMyExtension.dylib“. This adds the function as a possibility when future SQL is executed.
  2. A client queries arbitrary SQL invoking that function „SELECT X(…args) FROM some_data;“
  3. Go to 2.

The dylib in question is loaded after fork (unless specified otherwise) and the function in question is invoked.

Maybe we each have a different understanding of the word callback but I don’t see any invocation of callbacks there?

I did open a GitHub issue by the way: I also linked to it towards the start of the thread.

43

Probably not, honestly. To be clear, there’s no specifically sanctioned way of doing this (RPC with a separate worker process) that I’m aware of – I’m just aware it’s a thing.

This would be unproductive, from my understanding. The Swift function is helping to build a result in postgres‘ forked worker process. If I exec, I no longer “am” the worker and have no way to construct, let alone return, the result.

I’m not doing arbitrary work in Swift, I’m exposing a function called by postgres to do some very specific processing of the data postgres provides and then returning it to postgres so it can return it to the DB client, insert it into another database table, etc.

bt all
* thread #1, queue = 'com.apple.main-thread', stop reason = signal SIGSTOP
  * frame #0: 0x0000000196579f30 libsystem_kernel.dylib`kevent + 8
    frame #1: 0x0000000100c9dfbc postgres`WaitEventSetWait + 408
    frame #2: 0x0000000100c42278 postgres`ServerLoop + 392
    frame #3: 0x0000000100c41680 postgres`PostmasterMain + 3528
    frame #4: 0x0000000100b8faac postgres`main + 728
    frame #5: 0x00000001961f9924 dyld`start + 6400

From my understanding there’s no external library loaded at time of fork. AFAIK it’s loaded dynamically after fork, as outlined in my previous post.

I’m not trying to be antagonistic by continuing this thread, I’m just:

a) Trying to understand the issue deeply, and
b) Trying to understand what I can realistically do about it.

Not writing the Postgres extension is not an option for us. Nor is not writing the extension in Swift (it’s directly sharing logic with our application).

  • I’m fine to move to Linux – but the conversation continued at that point due to the assertion that doing this on Linux should be equally problematic. If both are equally bad, I’d rather fix it on Mac for better DX.
  • I’m fine to keep trying to Embedded Swift, but I heard the same assertion – at least initially – that this is equally a non-starter. This is still an experiment I’d like to try, but it’s not easy to prevent even Embedded Swift from importing random frameworks like Darwin when importing C modules.
    • I will need to basically re-write chunks of the postgres headers to achieve this. I’ll need define their types but without importing stdio etc., as postgres does. Importing these libc headers seems to be the cause for Swift auto-importing Darwin (?) – I’ve ran into this before experimenting with Embedded Swift and I wish I could prevent it somehow without re-writing my C dependencies.
  • I’m fine to work around the behaviour in the macOS Swift Runtime that appears to be the direct cause of the issue I reported. But for some reason nobody appears open to actually discussing possibilities surrounding this, as was my original intention for posting in the first place.
44

Thanks. I overlooked the link.

It's byzantine but yes, this is effectively a callback—and once again, it's doing something unsupported on macOS. Loading a library at runtime is (perhaps unsurprisingly) not async-signal-safe.

2 Likes
45

Because it's simply very unlikely to be worth your time or effort. The symptom in this case is that you're dying in os_log because Swift triggered it directly, but even if Swift were to avoid doing so, any transitive dependency from anything Swift links could call os_log just the same, and you'd be hosed in exactly the same way — and pretty much all logging on the system goes through os_log (and hoo-boy do frameworks log stuff). And this is just with os_log; any call anywhere that's not async-signal-safe would land you in the same spot, so even if you manage to work around this today, you're not guaranteed that it'll work tomorrow.

Hence, instead of leading you toward playing whack-a-mole with this, discussion has tried to define away the problem altogether.

You could investigate what it would take to disable swift::runtime::trace::setupLogs in a local build of Swift to see if you get any further, but beyond that, it may not be worth your while.

3 Likes
46

FWIW, I implemented this with Swift Embedded today and it works like a charm.

I stress-tested it the best I’m able: it successfully processes millions of rows with correct outputs – zero crashes or instability. I still don’t see any evidence of this implementation being problematic in practice. Of course, if postgres changes its internals that’d be even better, for the many good reasons presented in this thread. Thank you for working me through the ups(?) and downs.

The Swift Embedded version also about 5% faster, and the deployment bundle is about 100x smaller (even including the unicode data tables).

I plan to open source the underlying Swift Package as soon as I’m able. When I do, I might well put a link to this thread as a warning/disclaimer.

10 Likes
47

Excellent, thanks for reporting back!

1 Like