Sure, I'm aware of that—but if the user wants to put all of their stored properties into heap-allocated space, I wonder if they're doing something where they would want finer control over that object and not have it hidden and inaccessible by the language. With enums, they don't have a choice unless they made their payload a class, but that would make the type itself harder to use because it would affect the pattern matching API, so I don't think the analogy quite holds.
However, thinking about it further, I will grant that adding something like @Indirect to the standard library could cause users to easily enter into situations that result in poor performance—someone annotating multiple properties with @Indirect would incur one allocation per property, whereas native language/compiler support could potentially gather all of the indirect properties together into a single combined allocation. So that would be a big advantage to deeper integration.
One critical issue that hasn't been addressed here yet though is how mutation of indirect properties would be handled. This isn't a problem yet for indirect enums or indirect cases because enums can't be modified in place, but indirect struct properties would definitely need to handle copy-on-write correctly for mutation.
Unfortunately, it turns out this is easy to get wrong even in a library-only property-wrapper-based version:
This version doesn't handle copy-on-write correctly, because copying an instance of the struct copies the reference to the class but nothing ever checks it for unique-reference or copies it, so mutating the property in one struct instance causes the other instance to reflect the change. The enum variant doesn't suffer from this problem because it does self-reassignment upon mutation:
Wrapper implementations copied from posts above
@propertyWrapper
enum IndirectE<T> {
indirect case wrapped(T)
init(wrappedValue initialValue: T) {
self = .wrapped(initialValue)
}
var wrappedValue: T {
get { switch self { case .wrapped(let x): return x } }
set { self = .wrapped(newValue) }
}
}
@propertyWrapper
class IndirectC<Value> {
var value: Value
init(wrappedValue initialValue: Value) {
value = initialValue
}
var wrappedValue: Value {
get { value }
set { value = newValue }
}
}
struct StructWithIndirectE {
@IndirectE var property: Int = 10
}
struct StructWithIndirectC {
@IndirectC var property: Int = 10
}
do {
let original = StructWithIndirectE()
var copy = original
copy.property = 20
print(copy.property) // 20
print(original.property) // 10 (👍)
}
do {
let original = StructWithIndirectC()
var copy = original
copy.property = 20
print(copy.property) // 20
print(original.property) // 20 (🔥🔥🔥)
}
You can't modify the associated values of a case in place when switching, but you can modify the enum value itself and the compiler already handles copy-on-write correctly. Consider this example:
indirect enum Example {
case value(Int)
}
var value = Example.value(42)
var copy = value
copy = Example.value(44)
print(value) // .value(42)
Is there something more you feel needs to be addressed?
Please refer to the initial post in the linked thread for the performance implications.
However, I'm not concerned specifically about mutation of indirect enums here; rather I was just using those to illustrate why mutation of indirect things hasn't been tackled at all yet, and circling back around to why it absolutely does need to be addressed for a proposal for indirect struct properties.
As you demonstrated, writing an @Indirect property wrapper is not trivial, considering these scenarios. Users doing this could encounter unexpected bugs at runtime.
Maybe, we should consider mutability of indirect properties and handling copy-on-write as benefits of having an indirect modifier for structs.
^ I'm confused by this. My understanding is that Optional is not indirect but Optional.Wrapped can be indirect with no restrictions. So the above quoted code should be legal because it doesn't require the case Optional.some itself to be indirect?
Can't you always "just" add an enum with an indirect member to a struct? Something almost identical to Optional but with the some case indirect.
For example some sort of 3D tree structure is sort of a pain as an enum because it could have any combination of xLow, xHigh, yLow, yHigh, zLow, zHigh children. So you would need an enum with 64 cases. A bit unwieldy. However you could make a struct with 6 IndirectOptional fields. That would work ok without "extending" the indirect keyword to support structs. (Also, this might not be the right way to make a 3D tree structure, I just remembered OctTrees have up to 8 child nodes, not six)
Having an indirect keyword you can use directly in a struct seems nicely symmetrical, but having IndirectOptional in the standard library and having the error message direct you to it would be functionally equivalent, and just as discoverable. Clearly not exactly the same as Optional gets a little extra syntactic sugar in various places that IndirectOptional doesn't (binding guard/if for example).
Maybe it would be better to have a "standard" propertyWrapper so you can use all the syntactic sugar Optional has.
Hi @xwu sorry for the late response, I've been a bit swamped these days.
But to me in this particular case some workarounds were better than others, as your example of using a final class, wouldn't be a good choice for me because(although it was not production code) I wanted to have full value semantics and define a set of API's that has mutating/non-mutating, explicitly specify an inout when I wanted a mutable parameter... and have the compiler helping when try violate those rules. So a final class wouldn't be a good choice. I didn't explore much of the other possibilities e.g. a @ Indirect property wrapper which would be just as ideal as the workaround I end up with. But in short, I just wanted to take full advantage of value semantics and have the compiler help me not do something I shouldn't like mutating a node in someplace I explicitly didn't tell is allowed. Also for readability when I wanted to see an & when I can have a node be modified inside a method. Just this little/minor things made me not go for classes.
At the end I went for something like @typesanitizer mentioned
struct BinaryTreeNode<E: Comparable> {
var element: E
var count: Int = 1
private var childs: [BinaryTreeNode<E>?] = [nil, nil]
var left: BinaryTreeNode<E>? {
get { childs[0] }
set { childs[0] = newValue }
}
var right: BinaryTreeNode<E>? {
get { childs[1] }
set { childs[1] = newValue }
}
init(element: E) {
self.element = element
}
}
It was really simple and serve its purpose well. That's one of the reasons I thought that although this would be nice to have, I think it wouldn't add significant improvements to the language since the cases where we would need something like this would be very specific and limited. And also, it is easy to find another solution when we stumble in one of those cases...
So those were my points @xwu hope it helps in some way :)
In this example I would go for case with six optional associated values, or a single associated OptionSet struct, while letting containing enum handle tree recursion.
What I don't like in struct serving as a tree or a list node is that it always has to have value property, regardless if you want it on every node or just on leaves.
Sure, but for the sake of argument lets say the value in my tree applies to all areas not covered by a specific child node. I have never used that in a 3D tree, but I have in a 2D tree.
That brings us back to a plausible struct that is forced to use an enum to get the indirect feature. It really isn't the end of the world, if the error message directed one to one of the existing solutions I would say this is more like a paper cut.
I can see a few reasons why we would prefer a language-supported indirect modifier over a property wrapper solution:
First, as you noted, indirect already exists for enums.
A wrapper-based solution would necessitate use of a class as the backing store, when it would be more efficient to use non-class heap allocations like we do for closures and indirect enum payloads.
A wrapper would necessitate a separate heap allocation for every indirect property. In the majority of cases, it would be better for there to be a single indirect buffer per value holding all its indirect properties.
Eventually, it may make sense for the compiler to do automatic layout optimizations, so that structs which are very large or which have multiple refcounted properties automatically indirect some or all of their fields. Having this optimization not rely on the existence of a library type would be nice.
Property wrappers interact with Codable and reflection in various ways, because the underlying storage is formally of the wrapper type. indirect enum cases, on the other hand, are mostly transparent to those APIs, and ideally, indirect struct fields would be as well.
Agreed—realizing this was the key thing that turned my opinion around on the property wrapper approach (I should have made my original post edit stand out more, now I have to live with the shame!). I wonder if projects like SwiftProtobuf could benefit here even in non-recursive cases if nested message structures could move large little-used submessages into indirect storage instead of being inlined into the parent's layout.
For my own curiosity, what are the differences in practice here? Is it just avoiding a few metadata accesses, or more substantial than that?
Metadata is certainly the biggest difference. Classes require a unique metadata record for every generic instantiation of every class, and on Darwin, every class needs to also be registered with the Objective-C runtime, which in turn writes to the class object and allocates some data structures, leading to dirty memory cost. Non-class boxes don't need as much metadata, and for boxes with known static layout, the metadata they do need can be completely statically generated by the compiler, needing only pointer fixup at load time.
This is super important, IMO. When writing entities I always strive to make them as pure as possible (not rely on property wrappers) for this exact reason.