The @_addressableForDependencies attribute is used in the stdlib, but it doesn’t seem to be documented anywhere. It’s not listed in the UnderscoredAttributes.md file. It seems to only be used on types that provide a span and are stored inline. Could someone explain what it is doing?
Here's the PR implementing the attribute:
1 Like
Thanks!
I read through that PR and the other related @_addressable… ones. I’m a bit confused about why these are needed. I was assuming borrowing already ensures that the value isn’t moved and the address is stable for the duration of a call. Do lifetime dependencies not also already guarantee this? If not, it seems like these attributes would be required for most uses for nonescapable types.
Most Span-vending types have their contents stored in a heap allocation that is already necessarily addressable, and therefore do not need any special attributes beyond those that define lifetimes.
However, there are some types that may have their contents inline in the object (like the SmallString form of String, which stores up to 15 UTF8 code units inline on most 64b systems). These small forms would be passed in registers under the normal Swift calling conventions, which are not addressable, and therefore would not be able to produce a Span.
"No problem," I hear you say, "just spill the registers to the stack." Ah, but a span property has to be able to return the span, which means it cannot be spilled to the property's stack, because that memory is reclaimed when we return from the property. So it actually needs to be spilled to the caller's stack (or another memory region with equal-or-greater lifetime). The addressable annotations force them to be addressable in memory in such a scope that satisfies the lifetime requirements of a Span property.
19 Likes
Thanks for the explanation!
I see this being useful in third-party libraries that define other inline data structures. Are there any plans to stabilize it or is it just intended for the stdlib?
Most third party code should not have to think about this, since InlineArray and any type that contains one will have the property intrinsically. @_addressableForDependencies is only necessary for these existing types which are for ABI reasons locked into layouts using tuples or other layout coincidences to simulate contiguous inline storage but which want to now be able to vend Spans pointing into that storage.
3 Likes
Unless I’m still completely misunderstanding something, I still see many common use cases where this would come up.
One place I personally would have found this useful (which is actually why I wanted to know about the attribute to begin with) is when using c interop. I’ve been building a safer wrapper over Vulkan in Swift and I wanted to ensure lifetime safety. I made a little example of something similar:
I have a UserInfoC struct that contains a pointer to an ImageInfoC struct.
(In Vulkan, this kind of layout is all over the place)
struct ImageInfoC {
int width;
int height;
// presumably there would be image data here
};
struct UserInfoC {
const struct ImageInfoC * __nonnull profileImage;
int age;
// etc.
};
In Swift, to make it a bit nicer to use, I have a very thin wrapper around the c structs:
struct ImageInfo: ~Copyable {
var raw: ImageInfoC
init(
width: Int32,
height: Int32
) {
self.raw = ImageInfoC(
width: width,
height: height
)
}
}
struct UserInfo: ~Escapable {
var raw: UserInfoC
@lifetime(borrow profileImage)
init(
profileImage: borrowing ImageInfo,
age: Int32
) {
self.raw = withUnsafePointer(to: profileImage.raw) { profileImagePtr in
UserInfoC(
profileImage: profileImagePtr,
age: age
)
}
}
}
If I then try to use it, I see that it only works if I add the @_addressableForDependencies attribute to the ImageInfo type.
static func someOperationThatUsesMemory() {
let hi: InlineArray<50, Int> = .init(repeating: 0)
print(hi)
}
static func main() {
let image = ImageInfo(
width: 100,
height: 100
)
let info = UserInfo(profileImage: image, age: 1)
print(info.raw.profileImage.pointee.width)
print(info.raw.profileImage.pointee.height)
someOperationThatUsesMemory()
print(info.raw.profileImage.pointee.width)
print(info.raw.profileImage.pointee.height)
}
// The output without the attribute
1860512288
1
InlineArray<50, Int>(_storage: (Unknown))
1860512288
1
// The output with the attribute
100
100
InlineArray<50, Int>(_storage: (Unknown))
100
100
Something else is clearly wrong with this code as well, because this doesn’t work at all in a release build and the values get messed up even before the inline array gets allocated, but the idea still stands that this is a case where something along those lines is needed. (Also, I’m sure some people will say “just use a reference type like a class”, but I’ve profiled this and it adds far too much overhead for the actual Vulkan example, and this can’t work in many similar embedded cases)
Even with that, I'm realizing that @_addressable and @_addressableForDependencies seem like a band-aid fix for a larger problem in Swift. There really is no way to use nonescapable types and lifetime dependencies without resorting to unsafe pointers. In rust, this problem is solved with references, but AFAIK there is no analogue in Swift.
It also seems like this would cause significant problems with some of the future directions for lifetime dependencies in general:
In the proposal, it talks about the idea of structural lifetime dependencies:
struct OwnedSpan<T>: ~Copyable { let owner: any ~Copyable @lifetime(borrow owner) let span: Span<T> @lifetime(span: borrow owner) init(owner: consuming any ~Copyable, span: Span<T>) { self.owner = owner self.span = span } }
AFAIK, this could not be implemented in swift currently because there is no way for the span to keep track of owner without using the address, and the address must now be able to change as it is now treated as a normal, escapable value.
I’m not sure if I’m just coming at all of this from the wrong angle, but it seems like something is missing here.
Nonescapable types don't by themselves change anything about withUnsafePointer and other unsafe APIs. It is still undefined behavior under any circumstance to escape the pointer outside of the withUnsafePointer block.
You are right; we are not yet finished providing the necessary primitives in the standard library and compiler implementation. InlineArray and Span provide the primitives for owning and referencing arrays of values, but we have yet to provide the same primitives for scalars akin to Rust's reference types. With a complete set of standard library reference types, there should be much less need to drop down to unsafe primitives. One complicating factor for Swift is that, since values do not generally have stable addresses, a stored borrow must sometimes be able to store an independent bitwise copy of the value representation, as you alluded to.
To get around the lack of scalar references, you might try using InlineArray<1, T> and Span as a stand-in:
struct ImageInfo: ~Copyable {
var raw: InlineArray<1, ImageInfoC>
init(
width: Int32,
height: Int32
) {
self.raw = [ImageInfoC(
width: width,
height: height
)]
}
}
struct UserInfo: ~Escapable {
var profileImage: Span<ImageInfoC>
var age: Int32
@lifetime(borrow profileImage)
init(
profileImage: borrowing ImageInfo,
age: Int32
) {
self.profileImage = profileImage.span
self.age = age
}
}
A fair question might be whether we should treat all imported C types as @_addressableForDependencies, since it is highly likely that code will want to form pointers to C types for interop purposes where a non-pointer Swift borrow representation would not be acceptable.
4 Likes