[Pitch] Move Function + "Use After Move" Diagnostic

Hey everyone!

Attached inline below is a proposal to add a move function to the
swift stdlib that possesses a use after move diagnostic. The full
eternal URL is https://github.com/gottesmm/swift-evolution/blob/move-function-pitch-v1/proposals/000a-move-function.md

Thanks in advance for any feedback/thoughts! You can try it out
with the following toolchains:

or the next toolchain that is produced by SwiftCI after Thursday Dec, 9th, 2021.


Move Function + "Use After Move" Diagnostic

Introduction

In this document I proposing adding a new function called move to the swift
standard library that can be used to end the lifetime of a specific local let,
local var, or consuming parameter. In order to enforce this, the compiler will
emit a flow sensitive diagnostic upon any uses that are after the move
function. As an example:

// Ends lifetime of x, y's lifetime begins.
let y = move(x) // [1]
// Ends lifetime of y, since _ is no-op, we perform an actual release here.
let _ = move(y) // [2]
useX(x) // error, x's lifetime was ended at [1]
useY(y) // error, y's lifetime was ended at [2]

this allows the user to influence uniqueness and where the compiler inserts
retain releases manually that is future-proof against new changes due to the
diagnostic. Consider the following array/uniqueness example:

// Array/Uniqueness Example

// Get an array
func test() {
  var x: [Int] = getArray()
  
  // x is appended to. After this point, we know that x is unique. We want to
  // preserve that property.
  x.append(5)
  
  // We create a new variable y so we can write an algorithm where we may
  // change the value of x (causing a copy), but we might not.
  var y = x
  // ... long algorithm using y ...
  let _ = move(y) // end the lifetime of y. It is illegal to use y later and
                  // people can not add a new reference by mistake. This ensures that
                  // a release occurs at this point in the code as well since move
                  // consumes its parameter.

  // x is again unique so we know we can append without copying.
  x.append(7)
}

in the example above without the move, y's lifetime would go to end of scope
and there is a possibility that we or some later programmer may copy x again
later in the function. But luckily since we also have the diagnostic guarantees
that y's lifetime will end at the move meaning after the move, x can be
known to always be unique again. Importantly if someone later modifies the code
and tries to use y later, the diagnostic will always be emitted so one can
program with confidence against subsequent source code changes.

As a final dimension to this, once Swift has move only values (values that are
not copyable), move as we have defined it above automatically generalizes to a
utility that runs end of the lifetime code associated with the value (e.x.:
destructors for unique classes). This is a basic facility if one wants to be
able to express things like having a move only type that represents a file
descriptor and one wants to guarantee that the descriptor closes. That being
said, the authors think this facility is useful enough to bring forward and do
early in preparation for the addition of move semantics to the language. Since
this aspect is not directly related to this proposal, the author will not
mention it further.

Motivation: Allow binding lifetimes to be ended by use of the move function

In Swift today there is not a language guaranteed method to end the lifetime of
a specific variable binding preventing users from easily controlling properties
like uniqueness and ARC traffic. As an example, consider the following code:

func useX(_ x: SomeClassType) -> () {}
// __owned causes x to be released before consumeX returns rather than in f.
func consumeX(_ x: __owned SomeClassType) -> () {}

func f(_ x: SomeClassType) -> () {
  useX(x)
  consumeX(x)
  useX(x)
  consumeX(x)
}

notice how even though we are going to release x within consumeX (due to the
__owned attribute on its parameter), we are still allowed to pass x again to
useX, consumeX. This is because x is a copyable type and thus the compiler
will just insert an extra copy of x implicitly before calling the first
consumeX to lifetime extend x over consumeX, in pseudo-code:

func useX(_ x: SomeClassType) -> () {}
func consumeX(_ x: __owned SomeClassType) -> () {}

func f(_ x: SomeClassType) -> () {
  useX(x)
  let hiddenCopy = x
  consumeX(hiddenCopy)
  useX(x)
  consumeX(x)
}

for general programming use cases this is great since it enables the user to not
have to worry about the low level details and just to write code regardless of
the specific calling conventions of the code they are calling. But what if we
are in a context where we care about such low level details and want some way to
guarantee that a value will truly never be used again (e.x.: restoring
uniqueness to an Array or controlling where the compiler inserts ARC). In such a
case, we are up a creek and the compiler will not help us.

NOTE: One could write code using scopes but it is bug prone and one has no
guarantee that the property that all future programmers will respect the
invariant that one is attempting to maintain.

Proposed solution: Move Function + "Use After Move" Diagnostic

That is where the move function comes into play. The move function is a new
generic stdlib function that when given a local let, local var, or parameter
argument provides a compiler guarantee to the programmer that the binding will
be unable to be used again locally. If such a use occurs, the compiler will emit
an error diagnostic. Lets look at this in practice using the following code:

func useX(_ x: SomeClassType) -> () {}
func consumeX(_ x: __owned SomeClassType) -> () {}

func f() -> () {
  let x = ...     // Creation of x binding
  useX(x)
  let _ = move(x) // Lifetime of x is ended here.
  useX(x)         // !! But we use it here afterwards. Error?!
}

In this case, we get the following output from the compiler as expected:

test.swift:7:15: error: 'x' used after being moved
  let x = ...
          ^
test.swift:9:11: note: move here
  let _ = move(x)
          ^
test.swift:10:3: note: use here
  useX(x)
  ^
test.swift:11:3: note: use here
  consumeX(x)
  ^

Notice how the compiler gives us all of the information that we need to resolve
this: it tells us where the move was and gives tells us the later uses that
cause the problem. We can then resolve this by introducing a new binding ā€˜otherā€™
for those uses, e.x.:

func useX(_ x: SomeClassType) -> () {}
func consumeX(_ x: __owned SomeClassType) -> () {}

func f() -> () {
  let x = ...
  useX(x)
  let other = x   // other is a new binding used to extend the lifetime of x
  let _ = move(x) // x's lifetime ends
  useX(other)     // other is used here... no problem.
  consumeX(other) // other is used here... no problem.
}

which then successfully compiles. What is important to notice is that move ends
the lifetime of a specific local let binding. It is not tied to the lifetime of
the underlying class being references, just to the binding. That is why we could
just assign to other to get a value that we could successfully use after the
move of x. Of course, since other is a local let, we can also apply move to that
and would also get diagnostics, e.x.:

func useX(_ x: SomeClassType) -> () {}
func consumeX(_ x: __owned SomeClassType) -> () {}

func f() -> () {
  let x = ...
  useX(x)
  let other = x
  let _ = move(x)
  useX(move(other)) // other's lifetime ended here.
  consumeX(other)   // !! Error! other's lifetime ended on previous line!
}

yielding as expected:

test.swift:9:7: error: 'other' used after being moved
  let other = x
      ^
test.swift:11:8: note: move here
  useX(move(other))
       ^
test.swift:12:3: note: use here
  consumeX(other)
  ^

In fact, since each variable emits separable diagnostics, if we combine our code
examples as follows,

func useX(_ x: SomeClassType) -> () {}
func consumeX(_ x: __owned SomeClassType) -> () {}

func f() -> () {
  let x = ...
  useX(x)
  let other = x
  let _ = move(x)
  useX(move(other))
  consumeX(other)
  useX(x)
}

we get separable nice diagnostics:

test.swift:7:15: error: 'x' used after being moved
  let x = ...
          ^
test.swift:10:11: note: move here
  let _ = move(x)
          ^
test.swift:13:3: note: use here
  useX(x)
  ^
test.swift:9:7: error: 'other' used after being moved
  let other = x
      ^
test.swift:11:8: note: move here
  useX(move(other))
       ^
test.swift:12:3: note: use here
  consumeX(other)
  ^

If one applies move to a var, one gets the same semantics as let except that one
can begin using the var again after one re-assigns to the var, e.x.:

func f() {
  var x = getValue()
  let _ = move(x)
  // Can't use x here.
  x = getValue()
  // But I can use x here, since x now has a new value
  // within it.
}

This follows from move being applied to the binding (x), not the value in the
binding (the value returned from getValue()).

NOTE: In the future, we may add support for globals/ivars, but for now we have
restricted where you can use this to only the places where we have taught the
compiler how to emit diagnostics. If one attempts to use move on something we
donā€™t support, one will get an error diagnostic, e.x.:

var global = SomeClassType()
func f() {
  let _ = move(global)
}

yielding,

test.swift:9:11: error: move applied to value that the compiler does not support checking
  let _ = move(global)
          ^

Detailed design

We define move as follows:

/// This function ends the lifetime of the passed in binding.
///
/// For more information on semantics please see: $INSERT_SWIFT_EVOLUTION_URL.
@_transparent
@alwaysEmitIntoClient
func move<T>(_ t: __owned T) -> T {
  Builtin.move(t)
}

Builtin.move is a hook in the compiler to force emission of special SIL "move"
instructions. These move instructions trigger in the SILOptimizer two special
diagnostic passes that prove that the underlying binding does not have any uses
that are reachable from the move using a flow sensitive dataflow. Since it is
flow sensitive, one is able to end the lifetime of a value conditionally:

if (...) {
  let y = move(x)
  // I can't use x anymore here!
} else {
  // I can still use x here!
}
// But I can't use x here.

This works because the diagnostic passes are able to take advantage of
control-flow information already tracked by the optimizer to identify all places
where a variable use could possible following passing the variable to as an
argument to move().

In practice, the way to think about this dataflow is to think about paths
through the program. Consider our previous example with some annotations:

let x = ...
// [PATH1][PATH2]
if (...) {
  // [PATH1] (if true)
  let _ = move(x)
  // I can't use x anymore here!
} else {
  // [PATH2] (else)
  // I can still use x here!
}
// [PATH1][PATH2] (continuation)
// But I can't use x here.

in this example, there are only 2 program paths, the [PATH1] that goes through
the if true scope and into the continuation and [PATH2] through the else into
the continuation. Notice how the move only occurs along [PATH1] but that since
[PATH1] goes through the continuation that one can not use x again in the
continuation despite [PATH2] being safe.

If one works with vars, the analysis is exactly the same except that one can
conditionally re-initialize the var and thus be able to use it in the
continuation path. Consider the following example:

var x = ...
// [PATH1][PATH2]
if (...) {
  // [PATH1] (if true)
  let _ = move(x)
  // I can't use x anymore here!
  useX(x) // !! ERROR! Use after move.
  x = newValue
  // But now that I have re-assigned into x a new value, I can use the var
  // again.
} else {
  // [PATH2] (else)
  // I can still use x here!
}
// [PATH1][PATH2] (continuation)
// Since I reinitialized x along [PATH1] I can reuse the var here.

Notice how in the above, we are able to use x both in the true block AND the
continuation block since over all paths, x now has a valid value.

The value based analysis uses Ownership SSA to determine if values are used
after the move and handles non-address only lets. The address based analysis is
an SSA based analysis that determines if any uses of an address are reachable
from a move. All of these are already in tree and can be used today by invoking
the stdlib non-API function "_move" on a local let or move. NOTE This function
is always emit into client and transparent so there isn't an ABI impact so it is
safe to have it in front of a flag.

Source compatibility

This is additive. If a user already in their module has a function called
"move", they can call the Stdlib specific move by calling Swift.move.

Effect on ABI stability

None, move will always be a transparent always emit into client function.

Effect on API resilience

None, this is additive.

Alternatives considered

We could also introduce a separate drop function like languages like Rust does
that doesn't have a result like move does. We decided not to go with this
since in Swift the idiomatic way to throw away a value is to assign to _
implying that the idiomatic way to write drop would be:

let _ = move(x)

suggesting adding an additional API would not be idiomatic.

Acknowledgments

Thanks to Nate Chandler, Tim Kientzle, Joe Groff for their help with this!

45 Likes

Big +1. I'm also happy, but not surprised, that a move is Swift will be an actual move and not the hacked together typecast that C++ does.

Nice! Now I am wondering: What does this mean for the moved value itself? Is it move-only?

let x = ...
let y = move(x)
print(y)
print(y) // ok? or error: lifetime of y ended after first use

As described above, the moved value is treated normally. So, you'll simply print y twice. The pitch even has an example of this:

Sorry, but I have to disagree: In this example the moved value is thrown away, or with other words has no binding, because of the '_'

let _ = move(x) // x's lifetime ends

I interpret move(x) as a way to make x fall out of scope prematurely, before the scope actually ends. And optionally get its value from what the function returns.

While move() will be useful for move-only types (when they come), any type can be moved.


I think there's something confusing in that move(x) clearly affects x but looks like a regular function call that normally cannot affect x in any way. Perhaps move(&x) would be better; or a non-function-like syntax like move x or move &x.

19 Likes

I agree that &x would be more in line with what I'd expect syntactically, but & doesn't work on let bindings.

One thing that's not addressed here: what happens if I try to move from something that isn't a local binding? Can I move the result of a function call? Can I move out of things that have setters, like properties and subscripts? Can I move a value out of an async let binding, of a binding with property wrappers, of a global variable?

2 Likes

I think youā€™re misunderstanding the proposal. Lifetimes refer to bindings. Here, xā€™s lifetime ends, but other in the previous line is not affected in any way.

1 Like

You all seem to misunderstand me! I am NOT talking about the lifetime of 'x', I am talking about the lifetime of 'y'.

I am repeating my first example:

let x = ...
let y = move(x) // <-- what is the lifetime of y?
print(y)
print(y) // ok? or error: lifetime of y ended after first use

Iā€™d imagine that trying to move an r-value is a no-op (with a warning) or else an outright error.

I donā€™t see why async let bindings require any special treatment: it seems they should be movable as long as you await the result. When property wrappers can decorate local variables, I think these features would compose fine too.

At first blush, it may seem contrary to the stated purpose of this pitch to move a binding thatā€™s not declared in the local scope, but maybe not: We already allow users to put variables declared in outer scopes out of reach by shadowing them; seems sensible to allow users explicitly to move those same variables to disallow their use in the local scope without having to come up with an arbitrary shadowing value.

1 Like

Again, I think you misunderstand the pitch. No one else is talking about the lifetime of y because there is nothing going on at all with the lifetime of y; move(x) shortens the lifetime of x, and y and x are distinct bindings. The answer to your question is no different than as follows:

let x = ...
_ = move(x)
let y = 42
print(y)
print(y) // OK? Of course!
1 Like

Sigh, I am asking a question! How can that be "misunderstanding"?

Of course it is possible that this is out of scope of the pitch, and I respect that!

BUT because the pitch explicitly mentions ARC traffic prevention, I am curious, what if...

1 Like

You're misunderstanding the return value of move(). It's a copy of the input value. It allows creation of a new binding, which will be a copy of (the reference stored in) the parameter passed to move(). In your example, y is a new binding to a copy of x, and is not in any way affected by the call to move(x). That's what everyone has been trying to explain.

1 Like

I wonder if some of this reflects the name that's proposed; perhaps it'd be more intuitive if this were just named endLifetime (by analogy to withExtendedLifetime), with perhaps even @discardableResult.

8 Likes

While those names are more explicit, especially for people who haven't read the Ownership Manifesto, they are still missing the point which was misunderstood: that the function affects a particular binding, and not the underlying value itself.

Perhaps unbind(x)? But that's definitely open to confusion with binding in other frameworks (including SwiftUI).

2 Likes

This looks great! I had a similar reaction to @michelf:

It feels weird to me to think of this as ā€œpassing a bindingā€ as a function argument because thatā€™s not really something that exists in the language model today (unless Iā€™m misunderstanding). I agree that moving this to a keyword make sense. Of course, if the expectation is that (once Swift's support for move semantics is more complete) this would be describable with a 'normal' function (i.e., one that users could write themselves), I see the value in having a 'special' move function just for the sake of forward compatibility.

Yeah, I had the same thought. The pitch says

Which is all true, but the idiomatic way to define a function whose value, in the standard mode of use, may be validly discarded is to use @discardableResult. If this side-effect-only use of move(x) is considered too obscure to justify an @discardableResult annotation, then IMO that's justification for a separate drop function. It feels somewhat dissonant to say that _ = move(x) should be the idiomatic way to end a lifetime but @discardableResult would be inappropriate.

The relationship to withExtendedLifetime also stood out to me, and I agree that it provides a good precedent for having the language constructs refer to "lifetimes" directly. However, I don't think that

let y = endLifetime(x)

reads particularly well. It's not obvious that endLifetime(x) would return a copy of x, but let y = move(x) to me indicates well that x is being "moved to" y just based on the straightforward English interpretationā€”I think even without any knowledge of move semantics I would expect y to have the same value as x after the line.

If we want to keep the "lifetime" terminology, maybe transferLifetime would be more intuitive (and endLifetime could be the Void-returning counterpart)?

15 Likes

I don't know how drop is implemented in Rust. I had a short look at the implementation in Swift and it is done in SIL with Builtin.move, with other words at a much lower level than lexical analysis and pretty far from scopes.

For my limited knowledge of SIL I am pretty sure that for an implementor it would be possible to introduce a rule, that the result of Builtin.move cannot be used more than once.

That said, the name move leaks implementation details.

Just an FYI, I am updating the toolchain with some new stuff I came up with and will update the pitch as well. Specifically, I added support for new inout semantics and proper handling for let where you define the value later. So be aware that one may read again and see new stuff. I will add a change log to the top after I do this.

3 Likes

A quick response that I hope will help the discussion:

I wouldn't think of move being as being passed a binding. Instead the way to think about it is that you are moving the current value of the binding out of the binding and returning that value as a return value. move is not returning a copy of the value in the binding... it is returning the actual value that was in the binding. That is why we need the diagnostic to enforce that one can no longer use the binding afterwards since the binding value would no longer be valid. Remember that move is applying __owned for its argument (which I am going to be formalizing soon hopefully into the language) meaning it takes in the value at +1. Thus its argument comes in at +1 and is returned at +1.

We are on purpose not using discardable result since we want the user to explicitly have to show that they do not want to use the result. This is an expert feature where we want users to be explicit. So discardable result is inappropriate.

4 Likes

You have this reversed. Both the implementation and the syntax reflect the semantics. Read y = move(x) as: the value is moved out of x and placed into y.

Because the value was removed from x, the variable can no longer be used. Whether the value now in y is the original value of x or a copy of that value is truly an implementation detail that is hidden.

8 Likes