Are swift_reflectionMirror runtime methods safe to use in forward compatible code?

Context

SwiftCrossUI (based off SwiftUI) needs to discover dynamic properties, such as those annotated with @State, at run time. We do this a lot so it needs to be fast. I originally constructed a Mirror each time that we needed to locate the dynamic properties of a new View-conforming struct instance.

This was taking up 50% of the time spent processing view updates, so I looked into alternative approaches and came up with a hacky technique to compute the offsets of such dynamic properties once they have been located using a Mirror. My approach relies on the in-memory representation of each dynamic property being almost guaranteed to be unique due to containing internal storage pointers, and compared the property value returned by Mirror to the base value at various different offsets until the correct offset was discovered. This significantly improved performance, doesn’t rely on platform specific runtime/ABI details, and gracefully falls back to constructing a new Mirror on every update in the case that we don’t find exactly 1 candidate offset for the property.

A contributor, Kaleb, recently discovered that the standard library has a _forEachField(of:options:body:) function that does exactly what we need. This method is public, but is guarded behind @_spi(Reflection). @_spi(Reflection) import Swift results in
”@_spi import of ‘Swift’ will not include any SPI symbols; Swift was built from the public interface at …”, so it appears that we can’t use the method directly. Kaleb has reimplemented the method’s functionality by importing the relevant runtime methods via @_silgen_name and all seems to be working well.

Question(s)

Is it safe to use swift_reflectionMirror_* runtime functions imported via @_silgen_name from a runtime evolution standpoint? Is it possible that these functions would get pulled out from under us in a future Swift version?

I realise that the runtime is only ABI stable on macOS, but on non-Apple platforms we only need declaration stability. Are runtime functions allowed to be deleted or have their signatures modified on platforms without Swift ABI stability? It’s of course possible for these runtime methods to be changed on non-ABI stable platforms from a technical standpoint; I’m asking from a runtime development policy standpoint.

Avoiding the XY problem

What we’re really trying to do is get the offsets of a struct instance’s properties at runtime, so any forward-compatible platform-agnostic solution to that problem would resolve our issue.

If you know a way to access the stdlib’s public @_spi(Reflection) _forEachField(of:options:body:) method safely, then we wouldn’t have to use these runtime methods directly in the first place, and we’d be happy.

That said, I’d still like to know the answer to the original question about runtime methods for future reference.

Code

@_silgen_name("swift_reflectionMirror_recursiveCount")
private func _getRecursiveChildCount(_: Any.Type) -> Int

@_silgen_name("swift_reflectionMirror_recursiveChildOffset")
private func _getChildOffset(_: Any.Type, index: Int) -> Int

private typealias NameFreeFunc = @convention(c) (UnsafePointer<CChar>?) -> Void

@_silgen_name("swift_reflectionMirror_subscript")
private func _getChild<T>(
    of: T,
    type: Any.Type,
    index: Int,
    outName: UnsafeMutablePointer<UnsafePointer<CChar>?>,
    outFreeFunc: UnsafeMutablePointer<NameFreeFunc?>
) -> Any

/// Calls the given closure on every field of the specified value.
///
/// - Parameters:
///   - value: The value to inspect.
///   - body: A closure to call with information about each field in `value`.
///     The parameters to `body` are the name of the field, the offset of the
///     field, and the value of the field.
func _forEachField<Value>(of value: Value, body: (String?, Int, Any) -> Void) {
    let childCount = _getRecursiveChildCount(Value.self)
    for index in 0..<childCount {
        let offset = _getChildOffset(Value.self, index: index)

        var nameC: UnsafePointer<CChar>? = nil
        var freeFunc: NameFreeFunc? = nil
        defer { freeFunc?(nameC) }

        let childValue = _getChild(
            of: value,
            type: Value.self,
            index: index,
            outName: &nameC,
            outFreeFunc: &freeFunc
        )
        let childName = nameC.flatMap(String.init(validatingCString:))

        body(childName, offset, childValue)
    }
}
4 Likes

I also work on a project that needs to find properties and methods at runtime. Unfortunately Mirror can only find properties, provided that they are not lazy. It actually see lazy properties, but with mangled names. It cannot find methods at all. It would be nice if Mirror could find methods in the future.

It's definitely worth calling out that making this possible isn't free (compared to a baseline of what's available today).

Except in specific cases (protocol witnesses, non-final class methods, and the like), Mirror can't find methods because the data simply isn't there. The metadata for a basic struct with regular methods doesn't track any of that because there's no need to—it's all known statically when code is generated so there's nothing to look up, and it just generates direct references to the symbols.

There may be some future design where it makes sense to offer things like method lookup on even simple value types, but it should be explicitly something that types have to opt into so everyone doesn't pay the cost of those extra tables.

3 Likes

Yes, it could be a special attribute, e.g. @findable or something like that.

Edit:
BTW exactly what I need is existing @IBAction attribute. Does it do anything on non-Apple platforms? Along with @IBOutlet ?

Edit 2:
Just asked Google, and got this answer from its AI:

On non-Apple platforms (like Linux, Windows, Android, WebAssembly), the @IBAction attribute has no special meaning or built-in functionality because it's deeply tied to Apple's Interface Builder (Xcode's visual UI editor) for connecting UI events (like button taps) to Swift code via the Objective-C runtime. When you use Swift outside of Apple's frameworks (UIKit, AppKit), @IBAction is essentially a no-op or ignored attribute, as there's no Interface Builder or underlying Objective-C bridging to interpret it.

Luckily SwiftCrossUI only needs to find stored properties, so Mirror does everything we need, just really slowly (when used as intended).

Thanks for sharing, quite interesting technique.
How do you use those offsets, is that for memcmp style comparisons?

Re the question itself about those SPIs. My humble opinion:

  • de jure – they could change
  • de facto – it is extremely unlikely they will.
  • The other (IMHO worth taking) risk is for Apple starting rejecting app store apps that use those private symbols.

I'd take the approach of only starting worrying about those things when (and if) they actually break, especially when you have a workaround strategy in mind (plan B your hacky implementation, plan C the original slow implementation, or we may have something better in standard library by then).

1 Like

How do you use those offsets, is that for memcmp style comparisons?

(context: I’m the contributor who initially discovered _forEachField(of:options:body:))

We’re essentially using them as incredibly cheap key paths:

func getProperty(of base: Base) -> Property {
    withUnsafeBytes(of: base) { buffer in
        buffer.baseAddress!.advanced(by: offset)
            .assumingMemoryBound(to: Property.self)
            .pointee
    }
}

After the view graph encounters a specific view type for the first time, we figure out the offsets of the type’s stored properties (either with my reimplemented _forEachField or stackotter’s hack) and cache those that refer to DynamicProperty-conforming values; this way property updates can be made essentially free after the initial update.

3 Likes

I thought this could be useful for model (ObservableObject) types and their @Published properties, or is it not applicable there?

Giving those calls a test – getting expected result:

@_silgen_name("swift_reflectionMirror_recursiveCount")
func childCount(_: Any.Type) -> Int

@_silgen_name("swift_reflectionMirror_recursiveChildOffset")
func childOffset(_: Any.Type, _ index: Int) -> Int

struct S {
    var a: Int8 = 0
    var b: Int8 = 0
    var c: Int64 = 0
}

print(childCount(S.self))        // 3
print(childOffset(S.self, 0))    // 0
print(childOffset(S.self, 1))    // 1
print(childOffset(S.self, 2))    // 8

Quite convenient that this works on types (not values as with Mirror).

Bonus point - also works with classes (unlike MemoryLayout.offset)

What's not to like. Hope we'll have this as a public API in some shape or form eventually.

1 Like

That’s how Combine does it, though at the moment our ObservableObject implementation uses Mirror to find @Published properties. There’s no reason we couldn’t use _forEachField, though.

(Another fun fact about that function: it bypasses CustomReflectable conformances entirely, meaning it’s a bit more robust than Mirror for things like this.)

1 Like

I was not aware of those undocumented calls, and they are life savers:

@_silgen_name("swift_reflectionMirror_recursiveChildOffset")
func getChildOffset(_: Any.Type, index: Int) -> Int
...

Given this test struct:

struct S {
    var a: Int8 = 42
    let b = "hello"
    var c = false
    let d: Int16 = 123
    var e: Double = .pi
    var f: E? = E.b
    var g = [1, 2, 3]
    var h: (Int8, Int8, Int8) = (1, 2, 3)
    weak var i: NSObject?
    let j: Set<Int> = []
    let k = S3(x: 1, y: 2, z: 3)
}

I was able recovering it one-to-one using those undocumented "not-quite-api" functions, along with the extra meta information shown in auto-gen comments:

struct S {
    var a: Int8 = 42                         // 0000, align: 01, size,stride: 01, kind: struct, display style: struct 
    let b: String = "hello"                  // 0008, align: 08, size,stride: 16, kind: struct, display style: struct 
    var c: Bool = false                      // 0024, align: 01, size,stride: 01, kind: struct, display style: struct 
    let d: Int16 = 123                       // 0026, align: 02, size,stride: 02, kind: struct, display style: struct 
    var e: Double = 3.141592653589793        // 0032, align: 08, size,stride: 08, kind: struct, display style: struct 
    var f: Optional<E> = Optional(App.E.b)   // 0040, align: 01, size,stride: 01, kind: optional, display style: enum 
    var g: Array<Int> = [1, 2, 3]            // 0048, align: 08, size,stride: 08, kind: struct, display style: struct 
    var h: (Int8, Int8, Int8) = (1, 2, 3)    // 0056, align: 01, size,stride: 03, kind: tuple, display style: tuple 
    weak var i: Optional<NSObject> = nil     // 0064, align: 08, size,stride: 08, kind: optional, display style: enum 
    let j: Set<Int> = []                     // 0072, align: 08, size,stride: 08, kind: struct, display style: struct 
    let k: S3 = S3(x: 1, y: 2, z: 3)         // 0080, align: 16, size: 33, stride: 48, kind: struct, display style: struct 
}   /* align: 16, size: 113, stride: 128 */

Bonuses compared to Mirror:

  • offsets support
  • setter support
  • var vs let info
  • strong vs weak info (technically it is strong vs not strong as it can't tell between weak / unowned, etc).
  • probably faster than Mirror (didn't compare the two myself yet).
  • ability to structure the code to follow the get/set(at: index) pattern instead of enumerate(callback) pattern – could be important in some projects (e.g. to be able to follow two or more structures at once (e.g. for comparison or merging purposes) and do this without allocating extra memory for an intermediate tree representation.
  • maybe something else I haven't encountered yet.

Thanks again for sharing.

4 Likes

If it can be of any help, I've used a library called Runtime to do exactly that (scan a struct for state variables) in my own SwiftUI (and Flutter bindings) shenanigans: GitHub - wickwirew/Runtime: A Swift Runtime library for viewing type info, and the dynamic getting and setting of properties.

It does the same thing as Mirror, except faster. It can also write to let constants (which you should not do, it obviously has some weird behaviours).

I think the library comes with its own runtime metadata section parser (the ELF section present in every Swift program that contains type metadata, generated by the compiler). It’s probably the same as what the SPI function you linked do, except that it’s a custom implementation.

3 Likes

Outstanding! Thank you for sharing and many thanks to its author @wickwirew

Wes, minor thing: most (all?) mutating func there could probably be non mutating.

Do you know how stable it is and do people use it in production?

I'm not sure, I've only used it in toy projects on macOS, iOS, Android, Linux and Windows. However the Swift ABI is stable (at least for Darwin platforms) so it shouldn't randomly break with language updates.

@tera its used by a few large companies. I know its in Facebook and others. I don't use it personally but its not really a moving target with a stable ABI and have updated it when it did break through swift versions.

But there are some wonky things in it's API that I've been wanting to change but just never got around to it. I wrote it a long time ago when I was very new to swift so it's a little clunky.

1 Like

BTW, why did we do observations via property wrappers (ObservableObject / @Published, etc) or macros (@Observable, etc) when we could have used this powerful approach?

(Answering myself - the equivalent of will/didSet is still needed for observation... Unless we could tolerate comparing the new "value tree" to the old with a timer to figure out the differences.)

Could a similar approach do even crazier things like those possible with objc exchange implementation?

struct S {
    var x: Int = 0 // no didSet
}

func f(_ oldValue: Int) {
    print("didSet called")
}

var s = S()
s.x = 1 // silence
// some voodoo runtime magic here
s.x = 2 // didSet called

I wouldn’t imagine so, at least not with the example you gave. AFAIK, if a property doesn’t have a willSet or didSet then the compiler doesn’t generate the code that would call those observers (because directly accessing the property is, naturally, a lot faster).

1 Like

Right, just because you can get the offset to a property doesn't mean you can observe arbitrary changes to it. This isn't Objective-C where every operation goes through a message send and the runtime provides hooks to override them. In Swift for a stored property without observers, assigning to s.x is just a store operation.

And I don't think we want that sort of voodoo magic. The benefit of having to explicitly denote in your code which properties have these sorts of behavior is that you can actually reason about your code and you don't have to worry about the runtime doing things behind your back.

5 Likes