Most efficient way to implement heterogenous, dictionary style storage?

Suppose that I'm trying to implement EnvironmentValues, from SwiftUI myself. I'm not, but for the purposes of this question, my needs are almost identical: given some sort of key, which always tries to store the same type of data, I want to be able to retrieve it with minimal overhead, ideally as close as possible to if it was a stored property of a struct. Let's assume that I can establish ahead of time exactly how much memory I might need in total, or that I don't care about allocating too much.

What I've experimented with so far

An obvious way to do this might be

protocol Keying {
    associatedtype Data
}

extension Keying {
     static var asKey: String {
         // idk how SwiftUI uses the type as a key, this is almost certainly 
        // the wrong approach. 
         String(reflecting: self)
     }
}

final class EnvironmentValues {

    private var storage: [String: Any] = [:]

     func value<Key>(for key: Key.Type) -> Key.Data? where Key: Keying  {
         storage[key.asKey] as? Key.Data
     }
    
    func update<Key>(
        _ key: Key.Type,
        to value: Key.Data?
    ) where Key: Keying {
        storage[key.asKey] = value
    }
}

This is fine, but it probably isn't very fast, between the dynamic cast and the boxing and unboxing of values. We "know" that the type is always the same, if it's present.

Unfortunately, switching to unsafeBitcast in a bid to increase performance fails, because the box is the wrong size.

We can get the unsafeBitcast if we switch to storing something that's always the same size:

class AnyBox {
    
}

final class Box<T>: AnyBox {
    
    init(wrapped: T) {
        self.wrapped = wrapped
    }
    
    var wrapped: T
}

// inside our impl: 

     func value<Key>(for key: Key.Type) -> Key.Data? where Key: Keying  {
         unsafeBitCast(
            storage[key.asKey],
            to: Box<Key.Data>?.self
         )?
         .wrapped
     }
    
    func update<Key>(
        _ key: Key.Type,
        to value: Key.Data?
    ) where Key: Keying {
        storage[key.asKey] = Box(wrapped: value)
    }


But now we're doing an allocation and reference counting. This is probably faster, but it seems like we can do better.

We could safely store AnyObjects by using Unmanaged and the like to correctly increment and decrement reference counts, but I'm not sure how we'd take ownership of a non-trivial value type like an Array.

In Array.swift in the standard library, it looks like we can do this using UnsafePointer.initialize:

(_buffer.mutableFirstElementAddress + oldCount).initialize(to: newElement)

But the documentation for that function states that " The destination memory must be uninitialized or the pointer’s Pointee must be a trivial type". So now I'm confused about how Array.append() is using the function, and why it's usage does not contradict the documentation.

That said, assuming this is the right function to call, it seems like I should be able to allocate a buffer that's the right size for my storage needs, and store the instances as they're set in the appropriate places. Presumably I would also need to figure out how to align them appropriately, or would need to settle for doing an individual allocation for each one.

Does this seem reasonable? Have I missed an even faster approach?

Array.append(_:) increases the count of the array, thus removing an uninitialized entry in the allocation and marking it initialized. This usage does not contradict anything in that documentation because it states "the destination memory must be uninitialized OR the pointer's Pointee must be a trivial type".

1 Like

Okay that is obvious in retrospect, for some reason it didn't occur to me that the author of Array would remember to call deinitialize when the count decreased. Thanks.

Any chance you could elaborate on what the exact meaning is of "initializing" a pointer in the context of Swift? I see that it seems to bottom out in a Builtin. I suppose I was expecting to see a call to retain or some equivalent thing to make sure that reference counts were handled, etc.

One thing you might try is to have some amount of inline storage allocated, but fall back to an external allocation if the type is too big. You could also include the actual type metadata just as a quick check when extracting the value, to make sure you implemented your heterogenous dictionary correctly.

Hey, this was facetious! I just described Any! Swift’s default representation for any types is the type metadata, some inline storage that might just hold a pointer to out-of-line storage, and any protocol implementations (witness tables) you need (0 for plain Any). The downcast is going to call into the runtime but it will exit out pretty fast given that the type will match exactly.

Now, okay, you could indeed build your own storage that doesn’t store the type, just to keep the memory use down and fast-path loading by not checking the type first. And that might be a good exercise to see how things fit together. But where I think it falls apart is when the environment is destroyed: you have to know how to deinitialize every value in it! And that means storing the type anyway (or storing a function per value to clean it up, which is basically the same thing with more steps).

Some other limitations of using Any:

  • You don’t get to pick the inline threshold; it’s part of the language ABI. (I believe it is three pointer-sized values.)
  • If the value is stored out-of-line, the allocation is always refcounted with rudimentary copy-on-write behavior. (I think this is technically not part of the ABI of the language; it’s a choice made by specific types, albeit automatically. But don’t quote me on that.)
  • Downcasts support a lot more than exact type matches, so you could still have mistakes. You could probably remedy this by checking for an exact match with type(of:).
  • You said you might know the total size of all values in the dictionary up front, so maybe you don’t want to store anything out-of-line.

But really, Any is surprisingly close to a good answer for this question, and you get it for free.

1 Like

I am pretty sure an array of UnsafeMutablePointer<T>? or even UnsafeMutableRawPointer? can work for you. You can use some of my work as an inspiration: GitHub - RussBaz/mini-alloc: Small container for retaining Swift object references - aka mini allocator

If they have to be heterogenous, you can store a raw pointer and its type in a struct in array cells instead of just a raw pointer.

Also, for sufficiently small arrays, it is often more efficient just to go through every element of the array than to perform a hash map based lookup.

As for the key, I believe ObjectIdentifier avoids the need for the expensive reflection

2 Likes