[Pitch] Piecewise consumption of noncopyable values

Evolution Pitches
1

We are proposing loosening a restriction on some noncopyable types to allow them to be consumed piecewise in certain cases. This enables writing code involving noncopyable values more naturally in many cases.

Motivation

In Swift today, it can be challenging to manipulate noncopyable fields of an aggregate.

For example, consider a Pair of noncopyable values:

struct Unique : ~Copyable {...}
struct Pair : ~Copyable {
  let first: Unique
  let second: Unique
}

It is currently not straightforward to write a function that forms a new Pair with the values reversed.
For example, the following code is not currently allowed:

extension Pair {
  consuming func swap() -> Pair {
    return Pair(
      first: first, // error: cannot partially consume 'self'
      second: second // error: cannot partially consume 'self'
    )
  }
}

There are various workarounds for this, but they are not ideal.

Proposed solution

We allow non-resilient, noncopyable aggregates without deinits to be consumed field-by-field.
That makes swap above legal as written.

This initial proposal is deliberately minimal:

  • We do not allow partial consumption of noncopyable aggregates that have deinits.
  • We do not support reinitializing fields after they are consumed.

Resilient aggregates can never be partially consumed.

The full proposal can be found here: [PartialConsumption] Pitch. by nate-chandler · Pull Request #2317 · apple/swift-evolution · GitHub

12 Likes
2

I strongly suggest limiting this to “within a module” or “within a package” (not counting @frozen), or you’re making adding a deinit a source-breaking change.

4 Likes
3

Beyond the source-breaking effect, it seems to me like it's a fundamental necessity to have visibility into a type's layout in order to support partial consumption of its value, since otherwise you don't know what's left after taking part of it away. So we definitely can't allow nonfrozen types in library-evolution-enabled modules to be partially consumed. I agree that we also shouldn't allow it for public types in general, regardless of library evolution mode, unless the type is explicitly @frozen. Note that SE-0390 already specifies that adding or removing a deinit from @frozen types is an API and ABI break, partially in anticipation of allowing for partial consumption.

5 Likes
4

i feel like this would eliminate what is (to me) one of the most motivating use cases of partial consumption, which is to yield access to some stored buffer (or whatever) as a different type.

_modify
{
    var encoder:SomeEncoder = .init(consume self.utf8)
    defer { self.utf8 = encoder.output }
    yield &encoder
}

this example sadly began crashing the compiler starting in 5.10, as swiftc has once again forgotten how to yield a local binding (more on that here). but in 5.9 it was (is?) possible to accomplish this more verbosely by reconstructing the entire struct from a memberwise init.

1 Like
5

A consumption followed by a reinitialization can be done explicitly through a separate inout binding, so
even if this isn't allowed:

consume(foo.x)
foo.x = newValue

you can still explicitly instead do:

func update(x: inout XType) {
  consume(x)
  x = newValue
}
update(&foo.x)

It will be nice to eventually allow the more direct partial consume/reinitialize to Just Work, but this is already possible today.

1 Like
6

no, that doesn’t work because yield can only appear in the original scope of the _modify/_read accessor. if i remember right, this is true even if you are using an immediately-executed closure in the accessor body.

1 Like
7

Having proper inout bindings would avoid that issue. Nonetheless, it's not pretty but you can still form an intermediate inout binding by putting an accessor in an extension method of the thing to modify. So:

extension UTF8 {
  var asEncoder: SomeEncoder {
    _modify {
      var encoder = .init(consume self)
      defer { self = encoder.output }
      yield &encoder
    }
  }
}

_modify {
  yield &self.utf8.asEncoder
}
2 Likes
8

right, but in a real code base i would probably just parrot the memberwise init instead extending Array<UInt8>. my point was not that there are no workarounds that exist today, just that i most often reach for partial consumption when i’m intending to reinitialize the value afterwards. leaving the value partially deinitialized is a less common occurrence.

9

The workaround is to use an intermediate inout. There is by contrast no way at all to consume part of a value while leaving the remainder behind, which is what the proposal adds. We will address direct partial reinitialization in the future, no doubt.

1 Like
10

that’s true, i’ll note this is specific to the choice to make Unique noncopyable.

when i look at all the instances where i am currently using ~Copyable, it is to wrap some copyable fields in a container that is only ~Copyable to enforce some typestate semantics.

then again when i thought about why i didn’t propogate the ~Copyable to the nested fields in the first place, i remembered i had tried that and gave up because i had no way of reorganizing the fields because they could not be partially consumed. definitely a failure to hill-climb on my part. +1 to this proposal

2 Likes
11

Would it be reasonable to extend the consume operator to work with property references (on objects that could otherwise be consumed) as part of this proposal? It feels like you've already done all of the work that would be necessary to support that, and it seems wrong to have implicit consuming operations be more powerful than the explicit consume operator.

7 Likes
12

It might also be worth considering relaxing the constraint preventing partial consumption of types that have deinits in those types' own deinits. It could be useful to forward ownership of parts of the value as it dies. One way to look at this would be that a deinit contains an implicit discard self, and the interaction would be similar to the future direction the proposal alludes to; however, keeping it confined to deinits does at least sidestep some of the bikeshed questions about what discard self means and how it ought to work in the general case.

4 Likes
13

The pitch has been updated with the following changes:

  • Legality of partial consumption is based on whether types are imported from a different module rather than on a compiler flag (-enable-library-evolution). Partial consumption of @frozen types remains legal.
  • Partial consumption within deinit is added.
  • Explicit consumption (via consume) of noncopyable fields is added.
  • A future direction for explicit consumption of copyable fields is added.
  • A future direction for partial consumption of copyable values is added.
6 Likes