The problem with using a PropertyWrapper isn't that it's a struct. It's that it's initialized by the class or struct that uses it. In order to have a reference, the class or struct would have to pass self. However, it has to do so before it's fully initialized itself. To work, PropertyWrapper would have to be able to wait until it's caller was fully-initialized before being initialized.
This feels like a problem PropertyWrappers should be able to solve at first, but the more I've looked at it, the more I feel that this is really a different sort of issue. PropertyWrapper are really great for self-contained logic around a property. However, they really aren't designed to manage their caller. A PropertyWrapper solution for this feels like opening the door to reference cycles, etc.
I really tried to think of every way I could to make this as a PropertyWrapper, but it just seems like it's not the right tool for the job. I came to the conclusion that a better solution was to rely on property observers. This still doesn't quite work on reference type properties. However, I wonder if we could use a similar approach to Combine... so a dependent reference-type would have to implement ObservableObject and publish the needed properties to work with memoization. (This sort of revives the use of PropertyWrappers but on the other side of the equation).