The "recent" trend in the Apple API's world (for quite some number of years already!) is to fix the problem "at source" on the C side by using blocks. A couple of examples:
When this not an option in some peculiar case – I tend to find a simple solution that does not involve lifetime management of the passed "refCon" - e.g. I know (and ensure) that my class instance will outlive all callback invocations, and thus I can pass my class instance reference at +0 (and cast it to void*), similarly on the callback side I can treat it as +0 passed reference and just cast it back to my class instance reference). Then there's no problem whether the callback is called 0 or 1 or N times. It's not a general approach but it served me well in all cases I've encountered so far, and the number of such cases steadily decreases every year given the trend outlined above.
I would really appreciate something like this. AudioToolbox doesn't seem to have implemented the "new way" mentioned above for functions like AudioQueueAddPropertyListener and AudioConverterFillComplexBuffer.
Do we have any idea how Apple's block-based replacements work? Are they hand-crafted wrappers, or are there some kind of annotation/macros that autogenerate them?
I suspect most of Apple’s block-based APIs have the real implementation, with the function-based APIs calling them, rather than the other way around. But also (a) blocks are easier to stuff in a pointer than Swift closures, and (b) when you’re wrapping an individual API you don’t have to worry about generalizing your ownership model.
So this is one of those peculiar cases I mentioned above; in this case I have an instance of MyAudioConverter class handy, this instance is responsible for audio conversion (obviously it has to stay alive until audio conversion is finished):
let userData = unsafeBitCast(myAudioConverter, to: UnsafeMutableRawPointer.self)
let err = AudioConverterFillComplexBuffer(converter, {
converter, count, io, outPacketDescription, context in
let myAudioConverter = unsafeBitCast(context, to: MyAudioConverter.self)
// do something here using myAudioConverter
return noErr
}, userData, &count, &io, &desc)
I'd recommend Unmanaged.passUnretained(myAudioConverter).toOpaque() over unsafeBitCast(myAudioConverter, to: UnsafeMutableRawPointer.self). It's semantically equivalent, but makes it a bit clearer that there's no retain going on.
Though in your case, your passing is essentially equivalent to unsafe unowned. If you're certain your callback is always called once, and only once, you can passRetained and make this safe without needing a "obviously it has to stay alive until audio conversion is finished" caveat
It is an audio conversion callback that's called many times, and by "obviously it has to stay alive until audio conversion is finished" I meant that it would be a logical error if it wasn't... The class that is responsible of doing audio conversion going away without waiting for audio being fully converted?! Kind of absurd. If I do not want all audio being converted (e.g. I want to stop conversion mid way) - then I'd make sure I am cancelling system audio converter properly (e.g. releasing it, so it won't call my callback any more) and then safely kill myAudioConverter instance without a fear.
the need to allocate a block of memory (and then memory manage it!) to pass closure to a pointer sized quantity; and this quote from a different thread:
Makes me wonder if the following can happen:
a closure functionPointer can be odd †
a closure functionPointer can point to some illegal instruction (unless of course you badly want the closure to die in a specific way)
If either of these can not (realistically) happen this could open up an opportunity to use a single word closures:
struct NewClosure {
var pointer: UnsafeRawPointer
}
The benefits would be a better interop between closures <-> blocks and closures <-> C API's that accept function pointer + userData pointers.
Option 1 ("closure function pointers must be even"):
pointer is odd †, ††:
yes? then pointer - 1 actually points to a heap allocated object with:
var function pointer
var closure captured variables
...
no? then it points to a function code directly and there is no captured state
† - a variation to this method could be using some unusual / unmapped memory address. As an example, imagine that all valid function and heap addresses must not have their most significant bit "on". If so happens we know that this is not a normal function address, so we invert the bits to get the heap allocated object address.
†† - a second variation would be to invert the cases: make function pointer odd-ball case (and adjust the pointer accordingly before calling through it) - if this plays better with ARC rules or some such.
Option 2 ("a specific illegal instruction must not happen as the very beginning of the closure function"):
pointer points to the chosen illegal instruction?
yes? then it is actually pointing to a heap allocated object with:
var illegalInstruction: UInt64
var functionPointer: UnsafeRawPointer
var closure captured variables
...
no? then it points to a function code directly and there is no captured state
Option 3 ("magic spell"):
pointer points to a specific sequence of magic words?
yes? then it is actually pointing at a heap allocated object with:
var magicSpell: (UInt64, UInt64) // like deadbeef feedface etc
var functionPointer: UnsafeRawPointer
var closure captured variables
...
no? then it points to a function code directly and there is no captured state
Looks interesting, but are we talking some new C compiler feature? Otherwise I fail to see how this could be used with existing C binary following standard calling conventions.
It's on Swift side only, C side remains the same. Put simply, if you want to stuff a two-pointer quantity (of the current closure) into a single pointer storage (to squeeze it into a C API's userinfo field) you'd need to allocate another intermediate block of memory:
struct Closure {
var functionPointer: UnsafeRawPointer
var closureContext: AnyObject?
}
new memory Block: [8 bytes for functionPointer, 8 bytes for closureContext]
This is in addition to the (optional) memory block that's already allocated for "closureContext".
The proposed method suggests a mechanism to only have a single memory block to worry about. And as with current closures when closure context is not needed there's no memory block at all, in this case the resulting closure is in an effect a function pointer.
This issue is surfacing almost any time we discuss adding something to the C interop in general.
I remember, that in the past, Swift (clang-importer?) was able to infer swift throwing API from Objective-C method returning (BOOL) and having (NSError **) argument and recently the ability to infer swift async method from Objective-C method having specfic completionHandler argument.
The ability to annotate C API with _Nullable, _Nonnull or various __attribute__ like swiftName (or in this case - using Clang Blocks) brings us no benefit outside of somewhat narrow world of Mac-focused C libraries.
I always thought the obvious solution is APINotesBlogpost, Clang Doc but I was never able to get it work and I suspect it does not work on non-Apple platforms at all.
In my view, the ability to provide additional context without modifying the header files would solve a lot of these issues (for example with C strings), the issue at hand included.
If you relax the problem from “interoperate with existing C code” to “augment the C code to make Swift do the right thing”, it’s a lot more straightforward to wrap the original C function in a function that takes an ObjC block.
Sure, but only if I am in control of the codebase.
I use Swift-C interop mainly on Linux with libraries like SDL, GTK, AdPlug, ...
Even if I somehow had the time and knowledge of the inner workings of those libraries, I don't think the maintainers would accept a PR that adds a lot of complexity for sake of 1 language that is kind of niche in the Linux world (and in case of Clang Blocks possibly major ABI breaking change).
And even if - it would be only useful if you interact with the most up-to-date libraries, which is also not a standard.
Writing your own custom API notes or module map is still introducing an intermediary between Swift and C that did not previously exist. Given the large universe of possible memory management semantics, it makes much more sense to write a custom C function in your project’s bridging header than to invent an API notes syntax that will only ever cover a subset of possible callback APIs.
My post is not about Clang Blocks in particular, but about interoperability with C more broadly.
AFAIK APINotes were created for this specific purpose: in order to provide additional information for C headers without modyfying them. Let's look at the example of SwiftGTK. It's the biggest codebase I can think of - that has no connection to Apple.
The GTK and it's dependencies are well documented (or at least well enough in most cases) via a machine readable format (in XML). In theory, we know a lot of more information we could provide to the clang-importer to create a nicer Swift API. Currently, the only option is to generate proactively Swift wrapper for all the code in the header files - which in this case produces 10s of MBs of useless binary (aside from 100Ks lines of code).
I admit that in the case of GTK, just APINotes won't be enough to get us rid of the code generator, but we could think how to reduce the amount of generated code.
My conclusion is, that APINotes won't make C apis more usable on it's own - but it might have the power to greatly reduce the amount of effort needed to create a nice Swift Package for a given library - and since most maintainers do it in their free time, I think making such things easier will benefit the Swift community at large.