[Pitch] Move Function + "Use After Move" Diagnostic

Great proposal! I'm sooo excited seeing Ownership progressing!

I'm curious as how this proposal fits with roadmap of Ownership / the plan laid out in the Ownership Manifesto?

Fairly early in the Ownership Manifesto, three "tentpoles" are presented:

Swift already has an ownership system, but it's "under the covers": it's an implementation detail that programmers have little ability to influence. What we are proposing here is easy to summarize:

  • We should add a core rule to the ownership system, called the Law of Exclusivity, which requires the implementation to prevent variables from being simultaneously accessed in conflicting ways. (For example, being passed inout to two different functions.) This will not be an opt-in change, but we believe that most programs will not be adversely affected.
  • We should add features to give programmers more control over the ownership system, chiefly by allowing the propagation of "shared" values. This will be an opt-in change; it will largely consist of annotations and language features which programmers can simply not use.
  • We should add features to allow programmers to express types with unique ownership, which is to say, types that cannot be implicitly copied. This will be an opt-in feature intended for experienced programmers who desire this level of control; we do not intend for ordinary Swift programming to require working with such types.

These three tentpoles together have the effect of raising the ownership system from an implementation detail to a more visible aspect of the language. They are also somewhat inseparable, for reasons we'll explain, although of course they can be prioritized differently. For these reasons, we will talk about them as a cohesive feature called "ownership".

AFAICT move (and "use after move" diagnostics is neither of those tentpoles?

So I wonder, is the Ownership Manifesto outdated and needs an update? move is mentioned rather late in the Manifesto, maybe it is just how my brain works, but I would have expected that the features to be implemented first in the journey towards Ownership, would be mentioned first/early in the manifesto.

Can the Core Team give a general update to the roadmap of Ownership?

Few us like to give estimates on when a feature is shipped, but I'm going to be annoying and ask anyway, is there a plan for Ownership to be implemented during Swift 6.X (not 6.0 or 6.1..), or is it rather Swift 7.X? Or is this completely unknown.

Also, what is the prio of Ownership compared to any other major language feature? Does it have higher prio than let us say Variadic Generics? Or is Variadic Generics """easy""" ("""small""" amount of work) compared to the vast feature(set) of Ownership? (tripple quotes to mark that I do not believe Variadic Generics to be easy to implement, but rather Ownership seeming to be a beast in comparison).

Cheers!

4 Likes

An excellent question. My impression could be mistaken, but to me this pitch comes off a bit like a piecemeal attempt to “do something about ownership” without filling in the complete picture. Swift didn't include non-copyable types in its original design for understandable reasons, but now that design gap is beginning to cost the language a lot (in particular, it is an obstacle to complete C++ interop), and because it is a fundamental piece of the object model. IMO it needs to be addressed holistically.

@Alvae and I have been working on a language design that could serve as the basis for a holistic approach to ownership for Swift, if anybody's interested in that.

12 Likes

We're still working on move-only values and types. The move operator happens to be one thing that's already implemented and ready to go to review. As long as a language has implicitly-copyable values, being able to explicitly end the lifetime of one is a useful operation independent of ownership, in order to allow for guaranteed forwarding without having to assume the full burden of strict move semantics.

13 Likes

Yeah, but there are other ways to address that problem. First forwarding could be guaranteed at last use, couldn't it? Then explicitly ending the lifetime is just gravy for those who want a diagnostic. Then you could have a way to turn off implicit copying (but to make that tolerable you also have to scrub the idea that pass-by-value implies a copy from the user model). I'd rather see one #noimplicitcopy at the top of a file or scope than what you get with C++, where you have to scatter move() throughout otherwise-straightforward code if you want it to be efficient. The problem with establishing that practice now in Swift is that even after there's a more complete design, it will probably be cargo-culted into the distant future.

4 Likes

We definitely ought to forward ownership wherever possible, and no-implicit-copy is also a useful tool, but a move operator would still be useful even with perfect forwarding-at-last-use and no-implicit-copy. It may not be evident to future maintainers when implicit ownership forwarding is significant to the behavior of the program and not "just" an optimization, and an explicit move makes that more evident and also allows the compiler to check when forwarding doesn't happen. It's like the data equivalent of tail call optimization; it's a good thing to do everywhere, but sometimes you need it and want to make sure you know when it doesn't happen. Marking a value as "no-implicit-copy" would indeed also ensure that no uses occur after the consuming use, but it would also put the entire burden of move-only-ness onto the value's entire lifetime. You have to scatter move across code at most once per variable, but with no-implicit-copy, you have to scatter multiple copys across every place you do want to copy a value during its expected lifetime.

2 Likes

i started using _move(_:) and that’s exactly what ended up happening. once you write _move(_:) once in a function, it’s hard to argue why one variable should get _moved and not another. i often find myself _moveing everything that’s backed by a heap allocation.

a side effect of knowing that _move exists is, psychologically, you start to assume that everything that isn’t explicitly _moved is copyable. this ends up being a real (and not imaginary) problem because of the lack of memory profiling tools for swift on linux.

2 Likes

Has anyone tried disabling implicit copies and seeing how much of the compatibility suite breaks?

I doubt the standard library would build.

FWIW, @Alvae and I believe that implicit copyability should be a scope-wide setting rather than apply to individual variables. When you care about the cost of hidden copies, you often care about it even when the variable is an Int.

with no-implicit-copy, you have to scatter multiple copy s across every place you do want to copy a value during its expected lifetime

Perhaps, but Swift makes way more copies than required in order to achieve its observable semantics. We discovered this while exploring the meaning of value semantics: you don't need to copy to get pass-by-value semantics, and you don't need to copy in order to create a let binding (though you may need a copy to do a subsequent mutation during that binding's lifetime). Where copies are needed, the compiler has all the information to know that the copy is needed. These facts drove our design. The result is that:

  1. The cost to the user of writing the needed .copy()s is very low, because they are rare.
  2. Where copies are implicitly allowed, the compiler can implicitly insert exactly the needed .copy()s, and no more.
  3. The compiler can warn about needless .copy()s.
  4. .copy()s that actually appear in code don't seem like noise.

Of course, the existence of hidden reference semantics can undermine the compiler's ability to reason about these things, and our design hasn't tried to account for that, since our design doesn't have reference semantics. But if Swift wanted to go in this general direction, I'm sure some very smart people on the team could figure out how to evolve the necessary information into the language.

2 Likes

Disabling implicit copies is mechanically fixable. I’m just idly curious how much churn that would actually cause, and whether any correctness issues would arise from code that didn’t need an explicit copy inserted.

1 Like

This is generally true in Swift as well—most immutable "copies" can share the same underlying value, and many of them do after optimization. What you describe is roughly also what Rust's borrow checker and Swift's OSSA copy forwarding pass do, but with different "error" behavior—Rust says "tough luck", Swift makes the copy for you, and you all are offering to insert the explicit .copy().

If you find that the number of explicit copies that remain after checking are rare, then it seems like one could also argue that the number of copies that would get implicitly inserted by an ideal ownership checker is also rare enough to be acceptable for typical application code, which has been our running theory with Swift so far.

3 Likes

Sure, but in Swift all these “truths” depend on the optimizer… and optimizers always “fall over” in unpredictable ways. We're advocating a model where the fundamental model, before optimization, includes only the copies that are needed.

Another brainstorm: y = give x

Our ultimate plan is to make the bounds of the optimizer more concrete in Swift too. We maybe can't be as aggressive as a clean-slate language because we're stuck with various forms of shared mutable state we have to coexist with, but that's the goal of discussing the various "lexical lifetimes" models we've been thinking through to try to balance the desire for guaranteed good performance in "pure" code while still making it possible to maintain referencey code without being a compiler expert.

5 Likes

Maybe not… but I'm far from sure about that. We've been thinking hard about how to coexist with C++'s rampant references, and so far it looks tractable even though we have value semantics at the fundamental level. Of course everything is easy when your language has no users, ABI, or backward compatibility concerns!

Still, IMO it's worth thinking about how the same approach look for Swift, and working your way towards it, even if you have to get there incrementally. You'd end up with a better language that way… and the library definitely :wink: wouldn't have a move() function (though it is trivial to write one as soon as you've got a “consuming” annotation for parameters). What explicit move does to the programming model is ungood.

A lot, I think. IIUC every place an lvalue (even an object reference) is passed by value or let-bound it would need to be explicitly copied, because today, these copies are in Swift's fundamental model and only ever removed by optimization, which comes after semantic analysis, where the “you need to copy” diagnostic is generated.

How difficult do you think it would be for someone to hack together a compiler that re-printed the source with explicit copies inserted where the optimizer couldn’t eliminate the implicit ones? It would be interesting to get some real numbers on how infeasible it would be for a major Swift version to flip the copy model on its head.

1 Like

It would also be interesting to see how much that changed if a let or var binding still implied a copy but passing as an argument did not under any optimisation settings (unless required by ABI). I think the copy-on-pass-as-argument is the more harmful of the two since it's not explicitly under the user's control, whereas a shared or inout keyword would resolve the issue of implicit let/var copies.

I'm super excited for move-only types. Micro piece of feedback:

Normal arguments are passed by borrow, meaning that the lifetime of the value is managed by the caller. Although we could allow move on these arguments, shortening the syntactic lifetime of the variable, doing so would have no practical effect on the value's lifetime at runtime, so we choose to leave this disallowed for now.

It's true that it has no runtime effect, but so do a lot of other patterns. I'd consider whether it's a rule that carries its own (admittedly small) weight. Would it be better as a warning, just like unmutated var bindings are a warning?

1 Like

Thanks everyone for the pitch discussion here! I've scheduled this for review starting on Monday, July 25th through Monday, August 8th.

Holly Borla
Review Manager

11 Likes