SE-0245: Add an Array Initializer with Access to Uninitialized Storage

The review of SE-0245: Add an Array Initializer with Access to Uninitialized Storage begins now and runs through March 18, 2019.

The proposal is written by @nnnnnnnn .

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to me as the review manager via email or direct message on the forums. If you send me email, please put "SE-0245" somewhere in the subject line.

What goes into a review of a proposal?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift.

When reviewing a proposal, here are some questions to consider:

  • What is your evaluation of the proposal?

  • Is the problem being addressed significant enough to warrant a change to Swift?

  • Does this proposal fit well with the feel and direction of Swift?

  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

Thank you for contributing to Swift!

Ted Kremenek
Review Manager

15 Likes
  • What is your evaluation of the proposal?

+1.

  • Is the problem being addressed significant enough to warrant a change to Swift?

Certainly is. Array is a slightly useful type, but the current API requires that it be initialised in-order. That means anybody who can't guarantee that (e.g. because they want to initialise elements concurrently or via some C API) needs to make an intermediate buffer which doesn't have that requirement, then copy the results to an Array.

It's not the worst thing in the world, but it would be nice to have a way to avoid it. The more often we can avoid it, the faster Swift programs become overall.

  • Does this proposal fit well with the feel and direction of Swift?

Feel: Yes. We use closure-based APIs pretty heavily, and they seem like a natural fit for this kind of problem. We have a similar initialiser for Dictionary.

Direction: I hope so. I hope as many types as possible can add initialisers like this (where they allocate their own contiguous storage, you initialise it however you want, then return how much you managed to fill).

  • If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?

I don't think I have. Nothing that's really directly comparable.

  • How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

I've been asking for something like this for a looooooong time. Participated in previous discussion, review (was in favour of that one, too). Participated in most recent discussion. Still in favour of it.

4 Likes

Heck yeah!

Yes. This is a major obstacle for efficiency of low-level algorithms that should naturally use Array.

Yes. It preserves safe defaults, while allowing scoped escape of the model for performance reasons, without compromising safety of the rest of your code.

I have used other languages (C, C++, Fortran) where this construct is unnecessary because you always have access to uninitialized storage. This is a huge improvement on that model, while allowing programmers to achieve the same performance when necessary.

I experimented with prototypes of this feature in the context of reviewing shims for Accelerate. It feels like a great addition.

5 Likes

Update from Review Manager

The Core Team is considering shortening the review if no further feedback is forthcoming. As review manager I will be monitoring this thread and will infer whether running the complete review is needed.

5 Likes

Sorry for slow response on this. I followed and supported this the first time it came around.

I am still a big +1. This will help a lot with C and other low level interactions.

I am not familiar with other languages having a similar feature outside of full direct memory access.

I feel the naming and documentation make the meaning of this initializer very clear.

1 Like

I don't have much to add except to pile on more support. This is an important step in allowing Swift to efficiently interop with C and other languages, and just for generally optimizing algorithms that want to populate a fixed number of things without wasted initializations.

2 Likes

Why is the UnsafeMutableBufferPointer parameter to the initializer closure inout? The proposal doesn't discuss that and doesn't even mention the fact in its prose:

The new initializer takes a closure that operates on an UnsafeMutableBufferPointer and an inout count of initialized elements.

But the signature in the "detailed description" section shows it being inout:

public init(
    unsafeUninitializedCapacity: Int,
    initializingWith initializer: (
        _ buffer: inout UnsafeMutableBufferPointer<Element>,
        _ initializedCount: inout Int
    ) throws -> Void
) rethrows

And the "try this at home" code listing and implementation both match that. So, what gives? Can you assign a larger buffer? Do you need to deallocate the old buffer if you do?

Other than this one point of confusion, I think this is a great proposal.

1 Like

It's more convenience to account for throwing with partially initialized buffers if the count is inout rather than a return value, as you can update it as you go and then it just does the right thing without having to catch anything in many cases.

I believe he is asking about the UMBP, not the count.

edit: unless you mean UMBP's count, which is worth clarifying in the proposal

Oh, sorry, I misread the question!

I guess the same argument as .withUnsafeMutableBufferPointer. It's convenient to be able to call mutating methods on the buffer, which is a struct, without having to make a var copy. Though I agree the case is less compelling here as there probably won't be as much call to do so (though maybe consistency is also a justification). @nnnnnnnn can you confirm?

2 Likes

That's right — this was part of the discussion on the pitch thread, but the justification didn't make it into the proposal. The buffer is in-out so that you can call mutating collection methods while you're working. It's tempting to think it isn't needed because a constant buffer still has mutable elements, but calling something like sort() on the buffer doesn't work unless it's mutable.

I can't remember if this is in the implementation already, but if not we could/should add a check that the buffer doesn't get replaced, like in Array.withUnsafeMutableBufferPointer.

3 Likes

FYI. Stumbled upon this the other day: https://github.com/apple/swift/pull/13323#discussion_r156565955

The inout pointer argument dates back to primordial Swift. Notice
that withUnsafeMutableBytes , added later, correctly passes a
by-value pointer. This is one of a number of source-breaking API
cleanups that I never got around to because nothing was blocked on it.

Don't know if that still applies. @Andrew_Trick?

That exchange predates SE-0237, which established that the buffer argument is taken inout to facilitate calling mutating methods on it.

2 Likes

So I take it we will make this inout as well?

If it were proposed as a new API today, it probably should be. But for source and ABI stability reasons, we can't just change it. We could replace it in an ABI-stable way, but that would need a compelling reason, and there probably isn't one (consistency alone would be insufficient). Using mutating methods on typed buffers is common, but pretty rare on unsafe buffers which are really only collections as a convenience.

1 Like

p.s. if we were to revisit anything about that method, my opinion is that withUnsafeMutableBytes probably should not have been put on Array at all . Given you can trivially create a raw buffer from a typed buffer, it probably doesn't carry its weight as API on such a prominent type.

1 Like

Proposal Accepted

Based on the review feedback, SE-0245 has been accepted as written. Thank you to everyone who reviewed this proposal and provided feedback!

6 Likes