Tangentially [perhaps], many tree-based algorithms benefit a lot from manual memory management, in Swift. Irrespective of whether you use structs or classes or other such architectural choices, letting Swift do the memory management is expensive.
See for example this thread.
A facet of this that might apply more directly is to use Unmanaged to manually eliminate some ARC traffic, which might help with erroneous "is not uniquely referenced" results.