Blocks are "non-standard" Apple extensions to C, a feature that was ported back from Objective-C.
From my past experience, even if you don't do audio completely in Swift, the interop required is basically Swift <-> C (perhaps with blocks), and obviously C has no problem talking to C++. I'd still put that C++ into .mm to make life easier, but otherwise you don't have to use Obj-C "@interface" or "@implementation" or call objects via Obj-C bracket syntax, etc if that's what you wanted to avoid.
For that you may find this thread and this WWDC video (37:50 ... 42:25) interesting.