How to run mixed Objective-C/Swift code with LLVM JIT?

Development Compiler
1

Hello.

This question is a Swift counterpart of another long thread on llvm-dev:

[llvm-dev] Is it possible to execute Objective-C code via LLVM JIT?

Here is a TL;DR:

I can run Objective-C code with LLVM JIT. To make it possible I register
Objective-C classes with Objective-C Runtime. My question is:

what do I have to do with Swift code to run it with LLVM JIT as well?

Detailed version follows below.

We are developing Mull, mutation testing system based on LLVM [1]. Currently, we
are trying to make Mull work with Objective-C and Swift projects using XCTest
framework. To make this possible we need to run Objective-C and Swift code with
LLVM JIT. The problem is that LLVM JIT does not register Objective-C runtime,
the job that is normally done by dyld and libobjc, so we have to find a way to
do it manually.

In that llvm-dev thread two different approaches are proposed:

I) Hook into JIT's memory manager where it allocates memory for object files,
parse Objective-C sections from Mach-O objects as they get loaded into memory
by JIT, and then use Public API of Objective-C runtime to register all necessary
classes.

II) Work on codegen level and make Clang emit Objective-C registration calls
so that bitcode contains initializer functions that register Objective-C classes.

I have tried the first approach and it seems to be working for Objective-C. My
code works as follows:

  1. I collect pointers to objc-related sections for which the memory is
    allocated. Before SectionMemoryManager::finalizeMemory() method is
    called, I register the ObjC runtime classes.
  2. I iterate over __objc_selrefs sections and "fix up" selectors. This
    does fix the original crash of the linked ObjC thread.
  3. I iterate over __objc_classlist sections and register the new
    classes using objc_allocateClassPair function, register the methods, properties,
    and ivars to these new classes, run objc_registerClassPair to complete
    the registration. (I still have to implement the protocol registration)
  4. I iterate over __objc_classrefs and __objc_superrefs and fix up the
    class pointers with the new classes created at step 3.

The very basic Objective-C code seems to work without any issues,
however when I switch to mixed Objective-C/Swift code I start getting
some crashes which I don't fully understand. This is what I do to handle
specifics of Swift code:

  1. Since I only use public API of Objective-C Runtime I cannot do what is called
    remapClass private function in libobjc, this is why I have to patch
    swift_rt_swift_getInitializedObjCClass with my own function that returns the
    classes that I register. This prevents Swift from running through the
    original unregistered classes which are located in Mach-O in-memory object I am
    working with.

  2. objc_allocateClassPair does not have any flags to create Swift-based
    classes in the runtime and its code decides on what to allocate based on a
    superclass: if the superclass is Swift class, then it allocates a pair of
    "Swift-enhanced" classes. This is why I subclass my Swift classes
    from a class that is known to be Swift-based, my very own Swift class:

open class SwiftTestCase: XCTestCase {}

I subclass my TestCase classes from SwiftTestCase. This ensures that
Objective-C Runtime treats my classes correctly as Swift-based, not Objective-C,
classes.

  1. I use my CustomXCTestRunner to drive the run of XCTests [3]. In my
    code I do:
void *runnerPtr =
  sys::DynamicLibrary::SearchForAddressOfSymbol("CustomXCTestRunnerRun");
auto runnerFPtr = ((int (*)(void))runnerPtr);
int result = runnerFPtr();

In particular, I am trying to run a simple XCTestCase test class written in
Swift and my code crashes when I access the property of this Swift's ObjC-based
class: it crashes when it tries to access its own property.

I have collected the screenshots. In particular, from the stack traces and
symbolic breakpoints I know that Swift tries to malloc a garbage-large number
for this property (it is an [Int] array).

My questions are almost the same as what they are in the linked Objective-C
thread:

  1. Should I do anything else to properly register Objective-C besides
    what I am doing at steps 1-5?

  2. Do I have to do anything special to make run Swift code with
    LLVM JIT?

  3. Anything that I am missing / should know about if I want to run
    mixed Objective-C/Swift code via LLVM JIT?

My very hacky code is located here [4]. Too many details are floating there so
I am happy to provide any additional details here or to make my code runnable
on any machine, if anyone would ask for that.

Any insights about the Objective-C / Swift Runtimes are very much appreciated.

Thanks.

Stanislav

[1] GitHub - mull-project/mull: Practical mutation testing and fault injection for C and C++
[2] [llvm-dev] Is it possible to execute Objective-C code via LLVM JIT?
[3] GitHub - mull-project/CustomXCTestRunner
[4] GitHub - mull-project/mull-jit-lab at objc-and-swift-is-not-working

Screenshots
Screenshots2000×5330 2.14 MB

2 Likes