We've been informally discussing introducing a BufferView type in Swift for the past few years as a solution for safe, efficient access to memory buffers. For lack of a better term, we'll refer to the features that BufferView needs as support for "view types".
This roadmap discusses the language and compiler features required to implement view types in Swift. View types provide stronger guarantees about the ownership and lifetime of their values. They allow Swift library design to take advantage of safer and more efficient programming patterns. Non-copyable and non-escaping types provide the language foundation for view types.
Because of its length, I'll only link to the full roadmap:
Support for BufferView is being staged in with multiple evolution proposals. The roadmap for improving Swift performance predictability: ARC improvements and ownership control provided motivation for the non-copyable types proposals over the past year. This roadmap follows the same trajectory. During the evolution review process it will be a helpful reminder of the value of new features relative to other approaches. I intend to continue updating the roadmap as design discussions unfold.
[EDIT] I closed the evolution PR mentioned above. Future evolution proposals can refer back to this forum post. Here is a link to the latest version of the roadmap document.
I love where this is going. Having a standard library-native type for the "bag of bytes" will be great; a lot of folks have been needing that.
My first impression is that the name BufferView needs work. We frequently deal with different kinds of "views", whether SwiftUI views or database views. Having Yet-Another-Thing-Called-View will lead to confusion. (As a dumb example… if I have a descriptive list of BufferViews in a database, and I want to show a subset of them in my app's UI, would I do so in a BufferViewViewView? )
True, there are different kinds of "views," but this roadmap isn't the first use of the term with this meaning. We've long had String.UTF8View, String.UTF16View, and String.UnicodeScalarView—and unless I'm mistaken, these pre-date SwiftUI.
The fact that it's a view type is significant; it's not a standalone buffer.
Do you think this ~Escapable capability should co-exist with the @nonescaping attribute, or they are mutually exclusive? Because I think both approaches have their drawbacks.
struct FuncWrapper<Argument, Result> {
var function: (Argument) -> Result
init(function: @resultDependsOn (Argument) -> Result) { ... }
}
This type can be "escaping" when the underlying function is escaping, and "nonescaping" when it isn't.
So we can't always mark it with ~Escaping, and the @nonescaping attribute is a better option here. However, at the same time, we can't express a generic constraint for Argument to be escaping or non-escaping.
Besides escapability is not really binary for structural types. A function can escape one item of a tuple, but not the other one. There are three underscored attributes for expressing escapability, and they do excelent job in terms of granularity, but there is room for improvement semantically, syntactically and lexically here.
IMO, one of the goals should be a common ground and syntax for general escapability rules for function and non-function types. This way we can eliminate issues such as this one.
Right, it's the container/view relationship that's important. We can have various "bag-of-bytes" containers that have ownership of their buffers, like ManagedBuffer and eventually ManagedRawBuffer. The "view" aspect of a type allows us to see into those containers without knowing the container type.
I don't have a solution for unifying the concept escapability. I'll just outline some of the differences.
_effects annotations are completely unsafe compiler hints that make promises about a function implementation. (I'm sure a better syntax is possible. If we ever surface them as a language feature they'll at least need "unsafe" in the name). They simply have no affect on the programming model--they don't change where or how you can call the function, and don't even change the code you can write in the implementation. They just declare certain things the implementations might do to be undefined behavior.
@nonescaping/@escaping modifiers also make promises about a function implementation, but they are exposed to the API and ABI, and they change the programming model for closures. They exist to (1) allow more freedom in the implementation of escaping closures and (2) allow the compiler to use a more efficient ABI when passing nonescaping closures. They are restricted to function type arguments and do not take part in the generic type system. These argument type modifiers do not say anything about the capabilities of a type. All function types are escapable, which is why the withoutActuallyEscaping API can be used to reverse nonescaping-ness. So, "locally nonescaping" is really a special feature of function types. This feature could be extended to non-function types as well. But... the polarity would be reversed, causing confusion, and "locally nonescaping" values would be incompatible with all generic code. You could never pass them off a some P or any P. This just doesn't get us where we need to be for "view types".
~Escapable is a capability of the type. Unlike the previous two concepts, it is an intrinsic property of a type that is not context dependent. There is no way to add escapability to a type. withoutActuallyEscaping won't work. This is what allows compatibility with generic types. The arguments for why ~Escapable needs to be a negative capability rather than a local constraint are all the same as the arguments for making ~Copyable a negative capability rather than adding a moveOnly local constraint. Some discussion on that is here:
I expect more discussion soon, once generic support for NonCopyable is pitched.
I’m a bit surprised to see borrowing and mutating property declarations proposed in this document. They don’t mesh well with my recollection of the design debates around move-only types.
I’m very excited by the possibilities of ~Escaping. It rounds out what are currently some related ad-hoc language features.
Once we get to lifetime-dependent function results, why not just go all the way and adopt Rust’s named lifetimes? I think it’s far easier for the programmer to grasp dependency analysis when the nodes in the graph can be seen.
Would we be able to use this to (finally) get a safe Collection view of arbitrary-length homogeneous tuples? I think so, right? I mean, they are contiguously stored.
It can be kind of annoying right now when you have a tuple and you need to get the element at position x. You either need to write a big switch or drop down to pointers and navigate memory binding APIs.
And I think that should also work for homogeneous parameter packs, shouldn't it? So you could pass them in to generic algorithms which require a Collection (assuming packs don't gain the ability to conform to protocols directly).
Also, the document says that non-copyable types allow us to expose mutable buffer views (including stack allocations) safely. I'm not sure about this, because the mutable borrows that we have (inout bindings) allow reassignment, even for non-copyable types.
Very simple test:
struct MutableBufferView: ~Copyable {
var count: Int
}
func withStackAllocation(bytes: Int, _ body: (inout MutableBufferView) -> Void) {
var buffer = MutableBufferView(count: bytes)
body(&buffer)
print("Cleaning up \(bytes) bytes. Buffer has \(buffer.count) bytes")
}
func test() {
withStackAllocation(bytes: 42) { buf_one in
withStackAllocation(bytes: 99) { buf_two in
let old_buf_one = consume buf_one
buf_one = buf_two
buf_two = old_buf_one
}
// <HERE>
}
}
test()
This outputs:
Cleaning up 99 bytes. Buffer has 42 bytes
Cleaning up 42 bytes. Buffer has 99 bytes
In other words, in the location of the <HERE> comment, buf_one has a capacity of 99 bytes and references an allocation that has been freed.
So I think we might need to create a kind of mutable borrow that cannot be reassigned. However, that would make non-copyable types even more difficult to integrate with existing code, because we don't know whether existing mutating functions (or functions which take an inout parameter) are going to reassign.
This is one reason why they need to non-escapable in addition to being non-copyable. That would make it so that, while you can still replace an inout MutableBufferView with another value, you can only do so with one that has at least the same lifetime as the original.
Maybe. I'm not 100% sure that there's no way to break that. I need to think about it some more (and I'm really excited for whenever a prototype might be available -- although I appreciate that it will probably be a while yet).
In the mean time, I guess we could do something like:
func withStackAllocation(bytes: Int, _ body: (inout MutableBufferView) -> Void) {
let baseAddress = get_stack_pointer()
var buffer = MutableBufferView(baseAddress: baseAddress, count: bytes)
body(&buffer)
precondition(buffer.baseAddress == baseAddress, "Don't reassign the buffer")
}
So runtime enforcement rather than static enforcement, but I think it'll do the job until non-escaping types are possible.
)