SE-0366: Move Function + "Use After Move" Diagnostic

The last line of that post says:

It would be perhaps useful to understand in what way the move function and other features from that post are necessary preconditions for move only types. Is the idea that move only types will somehow be built on top of the move function???

The dataflow analysis in SIL that detects uses after move is pretty much the same as what true move-only types will need to detect uses after being consumed. With move-only types, that consumption would be implicit when you pass off ownership of a value, but the underlying check will be the same.

2 Likes

I think it would strengthen the proposal if the motivation section explained how move(_:) contributes to the overall vision laid out by this roadmap.

3 Likes

The first paragraph of the roadmap says:

We're taking ideas from the ownership manifesto that are broadly applicable not only to move-only types but for providing finer control over high-level Swift code, and also using them as motivation to build the pieces we need to eventually support move-only types. The roadmap doesn't seek to supersede the ownership manifesto, but lay out the plan for how we intend to get there.

2 Likes

I’m generally in support of the proposal's scope. I too want to see __owned and __consuming formalized, but the roadmap indicates that’s in the offing, so I’m willing to wait for them. The same goes for move-only types and other such improvements. I want these things badly and I believe the authors when they tell me this is a good incremental step towards them.

I do feel like the move(_:) function is uncomfortable syntactically, but I don’t have a solid feeling about the right solution to that. So I’m going to ramble for a while and see where I end up.


Edit: You probably don’t actually need to read the rambly bit, so I’ve collapsed it

My first thought was that, since the argument to move(_:) is effectively modified (by ending its lifetime), it ought to be marked with &:

let y = move(&x)
consumeFinalY(move(&y))

This naïvely means that move(_:) should be <T>(inout T) -> T, not (__owned T) -> T, although that’s kind of an awkward fit since it leaves the argument uninitialized.

On the other hand, __owned actually has the same problem. This led me to think that perhaps the parameter to move(_:) should not be inout or __owned, but some new modifier—call it __moved as a strawman—that requires & and enforces the rules we want for arguments to move(_:). That would keep move(_:) from, say, being assigned into a variable for an <T>(__owned T) -> T function, and would perhaps allow other functions to use __moved to get the same behavior:

func consumeFinalY(_ y: __moved [Int]) { ... }
consumeFinalY(&y)

(This is basically the same as one of @ksluder’s suggestions above.)

But then I realized, do we actually want other functions using __moved? Or do we want functions to use __owned parameters and leave it to their callers to write move(_:), so they can choose whether they want moving or copying semantics on a per-call basis? On reflection, I think we want the latter—it’s more flexible and more explicit.


If so, I think I need to change my mental model of the situation: The move(_:) function itself is supposed to be the &-style marker of the unusual behavior being applied to the argument; the problem I’m having is that it looks so much like an ordinary function that my brain doesn’t flag it as marking unusual behavior.

I think that’s what @xwu is getting at when he suggests using a prefix-operator-style marker, like <-, to indicate the move. The problem I see with this is that, unlike &, <- is going to be very rare, so I’m worried that learning materials will skip over it and then people won’t know what it means when they see it in their code. Perhaps a different sigil would at least hint at the behavior more clearly (&&, perhaps?*), but something that involves a searchable word might be better.

* “You’re just copying && from C++!”, I hear you cry, but this is more a case of convergent evolution. In both C++ and this hypothetical Swift use, && basically indicates “kinda like &, but a little different and weirder”, but they would be used in very different places (call sites vs. parameter declarations).


That leads me to the keyword-style suggestions, like @move, #move(...), or just move.

I have to admit that I don’t like either of the sigiled forms. @ and # don’t suggest side effects to me; @ has never been applied to expressions and has always had an “adjective” feel that seems inappropriate here, while # generally has a "macro-like" connotation of being a shorthand for something you could have written out instead (although that’s not strictly true for #available). Neither one really feels appropriate for this.

That leaves us with plain move:

let y = move x
consumeFinalY(move y)

I like that contextual-keyword move is clearly not a normal function, but some kind of special language feature, much like try. It looks like something you haven’t seen before and should probably look up, and since it’s a word, you have a reasonable chance of looking it up successfully. But I am sensitive to the parsing concerns. I wonder if they could be addressed by also requiring an & on the operand:

let y = move &x
consumeFinalY(move &y)

The parser would then know that a move not followed by & is just the identifier move, so things like move(x) would not need to be explicitly banned.

But that does still leave the issue that it feels like overkill to design totally custom expression syntax for a niche feature. It doesn’t feel great to dedicate so much effort to something that won’t be used very often. (And realistically, to dedicate similar effort to similar things later in the roadmap, like the explicit copy operator we’ll need for @noImplicitCopy.) But maybe it’s the right thing to do anyway.


So I suppose where I end up is here: I think there are good ergonomic arguments for using a contextual keyword (perhaps with &), but I’m sympathetic to the notion that a contextual keyword would be too much effort for a feature so niche and we should just use a function (perhaps with &) and let the ergonomics suffer a little. I think the other ideas I’ve considered here fall into awkward spots where they’re too much effort, or too little ergonomic gain, or both.

9 Likes

One benefit to a move/__moved parameter modifier is that it provides an obvious spelling for initializers for move-only wrapper types:

/// A stdlib-provided replacement for all the adhoc @unchecked Sendable wrappers people are writing
struct Envelope<T> : MoveOnly, Sendable {
  var wrappedValue: T
  init(wrapping value: move T) {
    wrappedValue = move(value)
  }
}

(certain other details of this implementation assumed/omitted)

1 Like

I’m unclear on how __owned and __consuming relate to move(_:). Do they all deal with the same kind of ownership? Or do __owned and __consuming refer to ARC ownership, while move(_:) refers to a new kind of ownership that’s tracked by flow analysis? If they’re the same thing, then I guess my move parameter modifier suggestion above is just a formalization of __consuming?


My brain keeps coming back to how move(_:) interacts with captures. My understanding at this point is that this is disallowed:

func f() {
  let x: Int = 0
  DispatchQueue.main.async {
    print(x) // error: cannot capture 'x' because it is moved from later
  }
  if Bool.random() {
    let _ = move(x)
  }
}

which is safer and easier to understand than C++. But I can also imagine a situation in which being able to capture a moved value might be useful: Task cancellation.

func doSomething(with arg: Arg) async {
  withTaskCancellationHandler {
    return algorithmGuts(move(&arg))
  } onCancel: {
    // Presumably the compiler prohibits this to avoid a race with `move(arg)` above?
    print("Cancelled request \(arg)")
  }
}

If we had a way to move a value but leave a valid marker behind, we could do this safely:

extension Optional {
  /// Returns the wrapped value, if any, replacing it with `nil`.
  ///
  /// Returns `nil` if this optional is already `nil`.
  mutating func moveOut() -> Wrapped?

  /// Like `moveOut()`, but the swap with `nil` is done atomically.
  ///
  /// This version is slower and only available on platforms with atomic swap instructions.
  mutating func atomicMoveOut() -> Wrapped?
}

func doSomething(with arg: Arg) async {
  var localCopy: Arg? = arg
  withTaskCancellationHandler {
    // The cancellation handler doesn’t move from `localCopy`, so we know it’s always non-nil.
    let interiorCopy = localCopy.atomicMoveOut().unsafelyUnwrapped
    return algorithmGuts(move(interiorCopy))
  } onCancel: {
    if let arg = localCopy {
      print("Cancelled request \(localCopy)")
    } else {
      print("Cancelled too late!")
    }
  }

To start:

  • __consuming just means that self is __owned, in much the same way that mutating means self is inout.
  • When I talk about “copying” below, think “retain” for an object; when I talk about “destroying”, think “release”.

__owned means that the callee will destroy the value, so the caller should consider it unusable once the callee returns. The caller can handle this by either copying the value before passing it (the default) or ending its lifetime so it can no longer be used (what happens if you use move).

The alternative is that the callee will not destroy the value, so the caller can keep using it once the callee returns. This means that, if there’s a copy, it will be in the callee, not the caller. The caller can still use move, but it won’t actually eliminate the copy; the callee will still copy the original, but the caller will destroy the original once the callee returns.

So basically, __owned means that if you do move the value in, that will truly eliminate a copy. But using __owned by itself doesn’t eliminate the copy; it just allows the caller to eliminate it.

4 Likes

Sorry to bump this again (link). Perhaps I'm just missing some killer example that shows the superiority of "goto-style approach" compared to more structural alternatives (like nesting).

Thanks, this is extremely helpful clarification.

Extrapolation to explicit-copy and move-only types

My argument is that by giving the power to a parameter modifier, we can have our cake and eat it too. move(_:) can be an honest-to-goodness function whose argument is tagged move.

And in that vein, I’ve started to think about how a copy parameter would also help. arg: copy T could do for copy(_:) what arg: move T does for move(_:). Both of them are new syntax to effectively support one function, but if we have to introduce highly specialized syntax somewhere, why not confine it to an attribute and have the top-level syntax fall out naturally?

Here’s where I’m at so far on move and copy:

Parameter attribute Effect
arg: T Argument is copied into parameter, unless T is move-only, in which case it is moved. If T is explicit-copy, the caller must call copy(_:) or move(_:).
arg: copy T Argument is copied into parameter, unless it is the result of calling a function with a move return type, in which case it is moved. If T is explicit-copy, it is copied via invoking its copy method. Cannot be used if T is move-only.
arg: move T Argument is moved into parameter. Argument must either be the result of calling a function with a move return value, or it must be a variable prefixed with &.

The move keyword could also decorate a function return type, in which case it means the function returns its value via a new placement-return ABI, in which the caller allocates storage for the return value. This is akin to returning via an inout parameter, and is intended for two situations: tight loops and immediately passing the returned value to another function.

The Standard Library would use these new attributes to implement canonical move(_:) and copy(_:) functions:

/// Explicitly copies a value, returning the copy.
///
/// If T is explicit-copy, this function calls T.copy() to create the copy. T cannot be move-only.
func copy<T>(_ value: copy T) -> move T {
  // The `copy` attribute on the parameter does all the real work, effectively doing the following:
  // let value = T.self is ExplicitCopy.Type ? (value as! ExplicitCopy).copy() : value
  return value
}

/// Explicitly moves a value.
func move<T>(_ value: move T) -> move T {
  // The `move` attribute on the parameter does all the real work.
  return value
}

The interaction between move in return position and move or copy in argument position is what leads to the above table of behaviors.

Type of func g() Type of func f(_ arg: T) Result of f(g())
() -> T (T) -> Void g returns value by normal ABI, then caller prepares value to be passed to f.
() -> T (move T) -> Void Not allowed.
() -> T (copy T) -> Void Not allowed.
() -> move T (T) -> Void If f’s argument is not passed in registers, caller first prepares storage for passing argument to f, then calls g via placement-return API. g places returns value in prepared storage. Caller immediately invokes f.
() -> move T (move T) -> Void Same as above.
() -> move T (copy T) -> Void Since the lifetime of the return value ends when the temporary is discarded, this effectively acts the same as the above. This is what allows g(copy(f())) to work without recursively demanding copy(f()) be wrapped in copy(_:).

I mentioned ExplicitCopy in the pseudo-implementation of copy(_:) above. I think ExplicitCopy and MoveOnly should be true marker protocols, not @attributes. They would be mutually exclusive—a type could be ExplicitCopy, MoveOnly, or neither. ExplicitCopy would carry one requirement which would be auto-synthesized by default:

/// Marker protocol for types that cannot be copied.
///
/// A type cannot conform to both MoveOnly and ExplicitCopy.
protocol MoveOnly { }

/// Marker protocol for types that can be copied, but must be copied explicitly.
protocol ExplicitCopy {
    /// Implement this method if your type has any MoveOnly properties or if you need to modify either the source or the result.
    ///
    /// Swift synthesizes the implementation of this method for you if your type has no properties that conform to MoveOnly.
    /// The synthesized implementation allocates a new instance, and then initializes its properties by copying this object’s properties.
    /// If any of the properties conform to ExplicitCopy, this will result in their `copy` methods being called.
    ///
    /// If your type has any properties that conform to MoveOnly, you must implement this method yourself to initialize a copy with suitable values.
    ///
    /// Your implementation of this method can mutate self. For exmaple, your type might store its value inline until its copied, at which point it moves the value to a location shared with the copy.
    func copy() -> move Self
}
1 Like

To be clear, in this example:

let other = do {
	let x = ...
	useX(x)
	return x
} // `x` lifetime ended
// `x`  is moved to `other`

You are proposing new behavior for return inside a do block, right? That’s a pretty big change to existing syntax.

This is more or less what I expected, and it’s not entirely satisfying to me—much of the motivation for move is that it aspires to ‘lock in’ lifetime behavior regardless of optimizer choices, but without making this transformation invalid, it seems like we’re still relying on the optimizer being ‘smart’.

I take your point that this seems like a silly optimization in isolation, but are we so sure that in the full complexity of a real-world program, such behavior wouldn’t emerge? Granted, we can’t guarantee that bugs won’t ever arise in the implementation of the compiler, but it sounds like this wouldn’t even be a bug in any formal sense. What would be the impact of a harder rule such as “a uniqueness check can’t be hoisted above deinitialization of a potentially aliasing reference which has been explicitly moved”? Do we think that would have too many false positives to be worth it?

What does move solve that we can't address with allowing explicit retain/release calls for performance critical code? Am I missing something here? The solution seems very obvious from reading the proposals's motivation:

... in performance sensitive code, developers want to be able to control the uniqueness of COW data structures and reduce retain/release calls...

Moving values around and requiring copies to be explicit is essentially as close as you can get to making retain/release calls explicit without completely giving up on memory safety.

14 Likes

The proposal uses explicit __owned arguments to illustrate the interactions of passing values by move with a calling convention that consumes its arguments, but __owned isn't essential to the proposal. It is still useful to be able to shorten the lifetime of local values independent of function arguments, and to be able to move out of inout arguments and reinitialize them.However, even though __owned is not an official language feature, Swift still uses the consuming calling convention automatically in various situations. The default convention for initializers and setters is to consume their arguments, on the expectation that they are likely to use their argument in order to form part of the result value. Also, an argument is not annotated __shared, __owned, or inout can have its convention manipulated by the optimizer, if it sees that consuming or borrowing the argument contrary to the default convention opens up further ARC optimization opportunities.

If it helps, we can amend the proposal to avoid jumping ahead and remove references to __owned, leaving the interaction with move to be discussed when we formally propose shared and owned as part of the language. I think it's also worth discussing whether the proposed constraint disallowing move of non-__owned-annotated arguments is a good one, given that the convention of unannotated arguments is usually indefinite.

7 Likes

I’m +1 on the behavior as proposed. Makes sense. Captures some subtle things in a way that’s relatively easier to get one’s head around than alternatives I’ve seen.

On the sticky question of naming and syntax, I found that @beccadax’s post, including what she called the “rambly bit,” matched my own thinking.

Spelling it as a function is troublesome, agreed. But I don’t find the operator spelling particularly better. And I don’t see other good alternatives.


A thought in favor of using the word “move,” as opposed to symbols: together with the words “create” and “copy,” it forms a consistent metaphor. In this metaphor:

  • Values are physical objects
  • Variables are fixed locations in space
  • Variables are are containers which can “hold” values (although we don’t say a value is “in” a variable, so there’s a limit to how far the metaphor works) Edit: OK, we don’t say “5 is in y,” but we do say “5 is stored in y,” which is pretty darned close.

I always like when terms of art keep metaphorical consistency: it aids learning, and forms a handhold of casual reasoning for those who don’t want to be neck-deep in implementation details.

(Note that in this metaphorical schema, “assignment” is the odd one out. If we could get a do-over on PL history, perhaps using the term “copy” instead of “assign” would have been better.)

3 Likes

I disagree. I've heard and read variations of "the value stored in x" or the "the number 5 is stored in y" for almost 30 years.

2 Likes

I suspect there's a cultural break here between functional and imperative traditions, where a variable is its value in the mathematical tradition, but more like a storage location you put things in in the imperative programming tradition.

4 Likes

Just to provide an FYI, we are looking into posting a new version of this where we do the contextual keyword with move. @Joe_Groff is doing some editing/etc of the proposal with this in mind. With that in mind, lets focus the review on the semantics/less on the move function syntax for now. Joe will post here when the update is up.

18 Likes

For me it feels a smaller and simpler change (both to understand and to implement) compared to the pitch proposal.

Besides there are a couple of options available today, that allow ending variables lifetime early.
// 1. not so nice, but available now:
let other: T
do {
	let x = ...
	useX(x)
	other = x
} // `x` lifetime ended
// `x`  is moved to `other`

// 2. quite ugly, but available now:
let x = ...
useX(x)
let other = x  // `x`  is moved to `other`
guard let x = () as Void? else { fatalError() }  // previous `x` lifetime ended

// 3. available today, not so bad:
var other = { () -> T in
    let x = ...
    useX(x)
    return x
}()   // `x` lifetime ended
// `x`  is moved to `other`

// 4. might be available in the future (e.g. https://github.com/apple/swift/blob/main/userdocs/diagnostics/complex-closure-inference.md):

var other = {
    let x = ...
    useX(x)
    return x
}()  // `x` lifetime ended
// `x`  is moved to `other`

// 5. future ideal, listing for completeness:
var other = do {
	let x = ...
	useX(x)
	return x
} // `x` lifetime ended
// `x`  is moved to `other`


By a way of analogy, it feels like having a programming language that only has a higher level "forEach" statement, and as we sometimes need a lower level alternative we are now having a discussion on introducing a "goto" statement without considering "while" and "switch" statements first, which might be just enough for a typical task at hand. (And remember, even if goto can do much more compared to what "while"/"switch" can - goto is still considered evil and we don't have it in modern programming languages.)


I'd be very happy to be proven wrong and see some killer use cases that show superiority of "move" approach compared to nesting, just what I've seen so far (e.g. in the pitch description) doesn't look like a killer use case example.

1 Like