What's not clear about the difference between them? A buffer is a pointer plus a length, which allows things like collection conformance and assertions in debug when going beyond that length.
I don't see a justification for including the convenience operator. This really isn't something that needs to be convenient.
Thanks Ben—I'll keep this feedback in mind when writing the formal proposal.
My examples were deliberately constructed so that all of the initial
Tacos were available from the start, or the initial count of the array
was 0 and the Tacos were appended. Maybe I have overlooked something,
but I do not see a need to default-initialize the FixedCapacityArray at
any index.
Dave
Just on this point: the distinction made in the past is that a buffer owns its storage and a pointer does not, hence "buffer pointer". In the case of this API, the API does own the storage, so "buffer" seems fine, but that's why the type isn't just called "UnsafeBuffer". Whether this distinction pulls its weight is also of course a matter of debate. :-)
Would also like to see this happening.
This would help to interface with C, C++ and Objective-C code a lot.
There's code that needs to deal with array of things, so or i had to use malloc/free, .allocate or allocate a temp array to use its inner buffer to pass things over, when you know its a lifetime within the inner scope, so calling malloc and free for things that could be stack allocated would be a huge win.
If someone also have some vision for passing buffers that is not so painful it would be nice.
If you have to pass more than one buffer of different types you end up cascading like this
.withUnsafeSomething { y in
.withUnsafeSomething { a in
.withUnsafeSomething { h in
.withUnsafeSomething { o in
.withUnsafeSomething { o in
doSomethingWith(y!.baseAddress, yCount, a!.baseAddress, aCount, h!.baseAddress, hCount, o!.baseAddress, oCount, o!.baseAddress, oCount)
}
}
}
}
}
It gives me all these bad flashbacks from the time i used to write things in Clojure.
Edit: I guess a arena allocator where the buffers could get intermixed in the same chunk of memory would be the best fit for this without any help from language constructions.
The problem is they are kind of painful to implement. Had to do one in C++ for a multi-column representation and its just awful what you have to do.
Hi Fabio! Thanks for the feedback. Indeed, I think these functions would make interacting with certain styles of C and C++ code much easier. They would also allow us to wrap those functions in Swift interfaces while maintaining the sort of efficiency possible in C.
As for the nested calls, I don't have a good solution for that in this pitch but I agree it's something the language, as a whole, could use better syntax for. You could start your own pitch thread proposing something!
If the compiler can already stack-promote Arrays, what value does this function add?
AFAICT, the value is that it uses a simpler heuristic than the optimiser. In that case, perhaps we should just fix the optimiser to more reliably stack promote, rather than adding things to the standard library to bypass the optimiser's own logic?
This is MHO, but I think the only justification for using unsafe constructs should be when there is no other way to do what you're doing - e.g. because the standard library doesn't provide a checked primitive for you to compose, or because you need to interoperate with C. It should be heavily discouraged, much more than it currently is (looking at you, Sequence.withContiguousStorageIfAvailable, and String.withUTF8).
I don't think "I need a temporary buffer" is a good enough justification to jump to unsafe code. It just doesn't pass the smell test to me.
That's a fair question. 
Warning: Incoming Wall of Text!
Array can escape its context, which means escape analysis must be performed and passed in order for it to be eligible for stack promotion. In the general case, escape analysis is undecidable. The compiler must often assume that, even for seemingly simple programs, values will escape their declaring contexts and so cannot be stack-promoted. This means that Array, while safe to use, does not always produce optimal codegen. The proposed FixedCapacityArray has the same problem: it can escape its calling context, so it must be subjected to escape analysis.
One common case where the compiler must assume an escape is possible is when a value is passed to a non-transparent function or closure: at that point, the compiler must assume that the value will escape. This limitation makes it very hard to use such a value in a complex project without sacrificing the performance benefits of stack promotion. The proposed unsafe function provides the ability to assert that the provided buffer cannot escape without violating the language's constraints (much like how the language trusts developers not to escape the pointer used in withUnsafeBytes(of:_:) or to call UnsafeContinuation.resume(...) more than once.)
Unsafe primitives are important in a language like Swift because they are ultimately needed in order to build higher-level, safer constructs. For example, there are a number of C APIs (think POSIX) that write to pointers of various types but don't allocate or deallocate them themselves. Currently, there are really only two ways to use such API in Swift:
UnsafeMutable[Buffer]Pointer or compatible type and pass it to the C API, orOptional to the API with &, which necessitates zero-initialization.Both come with performance costs that this function is intended to resolve. Ideally, such API would get customized Swift interfaces that hide the need for unsafe pointers. A developer, working on the Swift interfaces for these C APIs, can use the proposed function to produce an optimal Swift interface that is as fast as using the C interface directly.
This API is not for everyone. Like other APIs containing the word Unsafe, it's expected that most engineers will rarely if ever use it. There are some engineers who will use it because it is necessary for them to complete the tasks in front of them. It's okay to be in the former group instead of the latter group.
For what it's worth, someone could conceivably create an "Extra-Safe Swift" that didn't expose any API for any unsafe constructs, and I think that's probably an area worth exploring in general. I'd be happy to add my two cents to a discussion thread about such a project. 
Instead of adding more top-level functions, you could add a static method to each type:
extension UnsafeMutableRawBufferPointer {
public static func withEphemeral<R>(
byteCount: Int,
alignment: Int,
do body: (inout Self) throws -> R
) rethrows -> R
}
extension UnsafeMutableBufferPointer {
public static func withEphemeral<R>(
capacity: Int,
do body: (inout Self) throws -> R
) rethrows -> R
}
This may improve call sites, especially if the caller is using their own type aliases:
typealias Bytes = UnsafeMutableRawBufferPointer
Bytes.withEphemeral(byteCount: 42, alignment: 1) { bytes in /*...*/ }
I'll raise this question during the formal proposal!
I'm a big fan of not having unsafe APIs if it can be avoided. If Swift had fixed-size arrays with a stack storage guarantee, would we still feel that this unsafe mechanism was useful?
See my wall of text above. tl;dr it's not possible to implement such a type and make such guarantees and do so safely. 
Escape analysis is generally undecidable, but if a fixed-size array is a value type, then the law of exclusivity solves the problem. Very crudely using your original example, I get this:
func handleTacos<Tacos: Collection>(tacos: inout Tacos) where Tacos.Element == Taco {
for i in tacos.indices {
create(taco: &tacos[i])
}
eatTooMuch(tacos: &tacos)
for i in tacos.indices {
destroy(taco: &tacos[i])
}
}
if tacoCount < someLimit {
var stackOfTacos: (Taco * tacoCount)
handleTacos(&stackOfTacos)
} else {
var heapOfTacos: [Taco] = ...
handleTacos(&heapOfTacos)
}
Of course, (Taco * tacoCount) doesn't exist, but I think that it gives the right idea. (For a truly fixed-size array, you might have to replace tacoCount with an integer literal and have a count parameter in handleTacos, but neither seem like mortal sins to me.) What part of this is impossible, unsafe, or not guaranteed to use stack memory?
A fixed-size array type could certainly help with many use cases, but there are still places where I could see an uninitialized buffer being useful:
(n x ElementType) type from a local let constant n, but we're a ways from that.(n x ElementType) and then incrementally update it,I'll note that nothing in this proposal nor the draft implementation assumes that tacoCount is a compile-time constant.
Maybe I can demonstrate with some counterexamples? What happens if you say:
return stackOfTacos
// escape prevents stack promotion
Or:
struct X {
var tacos: (Taco * tacoCount)
// membership prevents stack promotion
}
Or:
let ptr = UnsafeMutablePointer<(Taco * tacoCount)>.allocate(capacity: 100)
// would we even allow this??
Or:
var stackOfTacos: (Taco * tacoCount)
print(stackOfTacos[1])
// assume tacoCount > 2
The last sample in particular should be a compile-time error because we are using the value before initializing it—if it is not an error, then this type is unsafe. One of the use cases for the proposed function is to avoid unnecessary premature allocation initialization.
The language already has an affordance for a fixed-size sequence of same-typed values whose count is known at compile-time: tuples. I'm sure you've seen a fixed-size C array imported as a tuple. That affordance is lacking in a lot of ways that have been noted previously, but it exists.
If we were to add fixed-size arrays to the language (and I do want to do that independently of this pitch) then it might make sense to implement them as syntactic sugar over a same-typed tuple. For example, (Taco * 2) might simply resolve to (Taco, Taco). I think we'd also want additional members (like a subscript) added to such tuples to make it work well, though.
Of course tacoCount is not a constant, but SOME_LIMIT in your original example hopefully is, given that variable-length arrays in C are problematic. You can have handleTacos take a count in addition to the collection to be modified. That's a small price to pay for safety, IMO.
I feel like these new examples don't go to the point of withUnsafeUninitializedMutableBufferPointer:
withUnsafeUninitializedMutableBufferPointer, so I think that the concern of returning stackOfTacos is moot (aside from many other problematic aspects; if you need to return your array, then a stack-allocated array was probably not the right choice in the first place!).withUnsafeUninitializedBufferPointer is inherently not using a struct member, so I don't know why the ability to use fixed-size arrays as struct members would invalidate this construct.Both of these seem to point towards fixed-size arrays being able to do more things, which is a good thing!
Towards the last point: I'm asking because I am deeply skeptical that the cost of zeroing stack memory is unbearable in high-performance applications.
There's been multiple real-world experiments all over the place out here that show that zero-init has a cost that falls within statistical noise. As a Security Guy™, and having some amount of experience adopting security features in large code bases, when people tell you: "we need to be able to opt out of these security features for performance reasons", they aren't telling you that there's a performance problem. They're telling you that they don't know if there is a performance problem.
Android has been shipping a whole userland with stack zero-init and Windows has it in the kernel. At this point, I feel that we're very close to showing that there isn't a performance problem, or that it's not big enough that there should be a standard library function to help you opt out of safety if there's a safe way to do the same thing. Significant effort is put towards closing this very issue in C code bases; I think that there should be careful consideration before it's reopened in Swift.
I agree that fixed-size arrays look like tuples, which is why I used a tuple-like syntax for them. However, last time this was brought up (several years ago), Joe mentioned that it would be problematic to have fixed-size arrays be too much like tuples because anonymous types can't have protocol conformances.
Either constant or known to the stdlib in some way—I have not attempted to define the heuristic here but in my POC branch it's a hard-coded 1KB.
That's my point: by adding a fixed-size array, you make these things possible, which forces the compiler to perform escape analysis, which is undecidable. The proposed function is, like all unsafe affordances in Swift, a way around that which is necessary in certain contexts (which have already been enumerated in this pitch thread.)
So, we're using initialization here to mean two things:
0xDEADBEEF or whatever), andinit(...) on every element in the sequence.I think we're conflating these two definitions. I know I have done so previously in this thread, which is entirely my fault and I should know better. My bad!
Zero-initialization is likely cheap in many many cases and has benefits and I'm not questioning that. Swift value initialization, when it's solely for the purpose of satisfying the compiler's requirement that all values be initialized, is a performance cost with measurable impact on real-world codebases. It's also one you can avoid today by using UnsafeMutableBufferPointer.allocate(capacity:), but that comes with the cost of heap allocation unless you're lucky and the optimizer decides to stack promote (hint: it usually won't.)
I think that's worth revisiting in a separate discussion, especially in light of a recent-ish pitch. It might be as "simple" as a new member function on tuples with same-typed values:
func withSequenceSemantics<R, S: Sequence>(_ body: (S) throws -> R) rethrows -> R where S.Element == Self.TypeOfAllValues
The syntax for that is all wrong but I'm sure you get the idea. In any case, I'd be happy to chat with both you and Joe about it in more detail.
I don't think this is a deal-maker or -breaker: for one thing, we do have non-escaping types in Swift (closures), and I'd love to see that generalised in some way. Another option would be if we had a withTemporaryArray function which checked that the buffer didn't actually escape; again, the precedent would be closures, which have the withoutActuallyEscaping function.
I don't know if this is directed at me specifically, but let me assure you that I use unsafe APIs rather frequently. I'm very much in the latter group. The thing is that I've become increasingly sceptical about how acceptable it is to use unsafe APIs at all.
The thing about memory safety is that it needs to be guaranteed even in the face of bugs. I've been thinking about that more and more - nobody writes bugs on purpose; they happen because some set of circumstances which you didn't think were possible turned out to, in fact, be possible (possibly due to changes elsewhere in the project, after you wrote the original code, so there's a temporal aspect, too). As an example, I've recently been trying to guard some code against malicious protocol conformances (e.g. a Collection which has a different number of elements every time you iterate it), because even in that extreme case, it's not acceptable to violate safety. Writing code which is resilient even to bugs is just super-difficult to do.
If you could completely trust engineers to get that right 100% of the time, there would never be any memory safety bugs (or bugs at all, for that matter). But the fact of the matter is that even the most knowledgable, most experienced engineers can't guarantee memory safety all of the time - if they could, we wouldn't be seeing this new generation of languages pitching safety as their top concern.
Just to make it clear - I'm not entirely opposed to this; I can even think of places in my projects where I'd like to use it, or have used similar constructs. But I'd really appreciate if we could thoroughly examine safe alternatives first. It's important.
This is tangential, but… there are deeper evils… 
All code is a layer cake of terribleness, and the top layer (which should be the most safe) is built upon lower layers of lesser safety. Foundation.Data ultimately calls malloc() and free() in order to manage its storage. When you pass an object to some generic function, it is boxed in an "existential container" which, for values more than 3 words long, also involves heap allocation and deallocation. When you throw an Error, it's allocated, caught, handled, and deallocated—again, possibly involving the heap depending on the size of the thrown error value.
For every layer you descend, you sacrifice a little safety, until you get down to a layer where there isn't really any memory safety at all. The proposed function (or, yes, some alternative) is a necessary part of one of these lower layers that is needed in order to build that safe topmost layer in an optimal way.
And that's really the point: we want Swift to be safe! But to make it safe at a high level, we need some unsafety underneath to support it all. The proposed function allows library developers to build higher-level APIs that are safe and performant instead of exposing C APIs or similar that carry with them all of C's laissez-faire memory management cruft.
I agree that it's important to examine alternatives, and I appreciate your interest in this pitch! I do hope my replies don't come off as dismissive, because that's not my intent.
I've spoken out of band with @grynspan and I feel that my opinion has been heard, but for the benefit of public discussion, here are my rough takeaways:
inout instead, but I think that none of the reasons that could cause a copy apply to this case (someone from the Core Team confirm that you basically never need to copy a value type passed as an inout parameter if it has automatic storage?)
withUnsafeUninitializedMutableBufferPointer in the standard library, there are no take-backsies, so it would be disappointing if we ended up with two things that do the same thing except that one is unsafe but sounds faster (when it probably is at best negligibly better)withUnsafeLocalStorage giving you an UnsafeMutableRawBufferPointer that always came from alloca, would make more sense as a "building block" to me, and it's easier to label with red crosses and skulls to tell people to stay away from it. You can then use that in code bases where it's this important to get stack-allocated buffers
chkstk calls sprinkled over this