I've recently found myself explaining our move-only types work to a lot of different people. As a result of those conversations, I thought it would help to have an informal sketch to help outline why move-only types are interesting, clarify a few subtle points (like what "move" really means), and briefly explain some of the issues we'll need to tackle in order to bring this to Swift.
Note: This is deliberately light on details -- those will be covered in various pitch and proposal documents. Some of those have already appeared (the "take" operator and "take" and "borrow" argument modifiers proposals), others will continue to appear over the next month or so.
An Informal Introduction to Move-Only Types
Background: Why do we care?
When you look at where current programming languages spend their CPU cycles, memory management is consistently at or near the top of the list. This is true whether you use a garbage collector, reference counting, or custom logic to determine when a particular piece of memory is in use. This complexity in turn exists in large part because references (pointers) get copied. The existence of these copies forces us to come up with some system to identify at run time what pieces of memory are still accessible.
Working with pointers in general has another serious problem: There is really no way for a compiler to examine a particular bit of pointer arithmetic in a language like C and determine whether it is safe. So high-level languages will eventually have to stop providing unverified support for raw pointers, and new memory management idioms must be based on guarantees that can be verified by a compiler. Different languages are trying different approaches to this, and time will tell which of the many techniques being explored will prove most effective at balancing verifiable correctness with simple programming models that are accessible to a large population of developers.
These same problems can occur with non-pointer values that are used as “handles” for some resource that is separate from the value itself. For example, you might track the integer index of an object in an array or the key of some data stored in a database. Just as with pointers, it can cause a variety of problems if those values get copied to many locations and we lose track of which ones are still valid.
Moving vs. Copying
As mentioned above, “copying” is one of the key reasons for runtime memory-management complexity. So what if we could mark a certain piece of data as not copyable? Or similarly, what if there could only be one reference to a particular piece of data? Or we could guarantee that a certain database key was stored in one place and never duplicated?
Such a property is easy for the compiler to verify: It just needs to produce an error whenever you do something that would require copying the data or reference. It also allows the compiler to omit most runtime memory management: For example, if you can never have more than one reference, you don’t need to count the references, so you can avoid garbage collection or reference counting overhead for that object. In general, if you can declare that a particular type of data cannot be copied, you can potentially provide guaranteed correct behavior with no runtime cost.
Unfortunately, compilers “copy” data for a lot of mundane reasons. For example, when you pass a value as a function argument, that data is “copied” from memory into a register. That means a truly non-copyable piece of data can only be passed by reference, never by value. Truly non-copyable data is tricky to work with.
But not all copies cause problems: In many cases, the “copy” exists in order to move a piece of data to a more appropriate location. For example, when you compute “x = x + 1”, the result may naturally end up in a different location (machine register or memory address) than it started. This isn’t really a “copy” since the first location is no longer valid. Similarly, when you return a value from a function, the original value inside the function is no longer valid. This makes it useful to distinguish between "regular copies" which result in two or more valid instances of the data, and "moves" which result in a single instance of the data in a new location, with the old location no longer being considered valid. Note that in "x = x + 1" and other cases, moves are really just side-effects of some other operation.
Aside : There is such a thing as truly non-copyable (“immoveable”) data. Locks and hardware registers are two examples, but they’re different enough from the cases I'm interested in here that I won't discuss them any further.
Limited Lifecycles
A value that can be moved but not copied has a very well-defined lifecycle. It is created, passed into functions and back out again, moved into different variables, and finally stops being valid. Throughout this lifecycle, there is only ever a single valid copy of this value. Among other things, this gives it a well-defined end of life that can be determined at compile time. This is unlike a regular copyable value which requires run-time support to correctly determine when every copy has stopped being used.
Consider for example a file descriptor. A file descriptor is an integer that represents a currently-open file. It’s important that after you close a file descriptor, that particular value can no longer be used. If a file descriptor is a “move-only type”, then we can arrange for the close operation to end the lifecycle and be certain that there are no valid copies elsewhere that could be accidentally misused. Note that this isn’t a reference — a move-only file descriptor is still just an integer under the covers and will still be efficiently handled as such at runtime. But at compile time, it will be an opaque entity with strong lifetime guarantees.
A similar issue arises whenever you have a pointer (a “view”) into another data structure. It is important that the pointer does not outlive the data structure it points into. There are several ways to provide this guarantee: One way is to wrap the pointer in a move-only type and force its lifetime to end before the data structure does. This is sufficient to ensure that the pointer can never be used after the data structure itself is gone. And this can all be proved at compile time without any runtime overhead.
Move-only types are not a panacea, of course, and they are not appropriate for every kind of data. For example, you cannot cache a move-only value because caches must store copies of the data. Similarly, there are limitations on how you can use move-only values as elements of other structures and collections.
Move-Only Types vs. Swift
Today’s Swift really likes to copy things. A simple statement like let x = a.b implicitly makes a copy of the value in question. Similarly, when you write for x in collec tion , each value in the collection will be implicitly copied into the variable x . Likewise for if let x = optional . The optimizer can frequently remove such copies, but whether any particular copy actually gets removed depends on precisely how the optimizer works, which can vary from release to release. So in order to support move-only values and types in Swift that must never be copied, we’ll need some new syntax that allows values to be explicitly “borrowed” and “taken.”
“Borrowing” is a key tool for working with move-only types that allows the value to be temporarily used in another location. Depending on the precise context, a “borrow” might be implemented by the compiler as moving the value to some place and then back again, or it might be implemented by creating a temporary reference to the original value. Regardless of the implementation, the key notion is that a “borrow” provides temporary, limited use of a value that might not be copyable.
Borrowing turns out to be useful even for copyable values. Because borrow operations are intrinsically limited in time, they allow the compiler to optimize memory management. For example, borrowing a reference can avoid reference-counting operations that would be necessary if the same reference were copied and then later invalidated.
Unlike a “borrow”, a “take” explicitly ends the lifetime of the original. If you call a function by writing f(take x) , you are explicitly moving x into the function and invalidating the local variable in the process. If you write let x = take a.b , then you are explicitly invalidating the property a.b and also the entire structure a . (In contrast, let x = borrow a.b allows a to continue being valid.) As with borrow , take is helpful today as a tool for fine-tuning reference counting operations but will be essential for working with move-only values when they become available.
So the first step in bringing move-only support to Swift is to add operations with different lifetime-management behaviors. This will include constructs such as for borrow x in collection that let you iterate over the items in a collection without requiring an implicit copy and f(take x) that explicitly invalidates the local value as part of passing it into a function. We’re also exploring variations of these that would allow you to temporarily gain mutable access to a value. These would allow you to efficiently mutate an element “in place” in various scenarios, which is a useful optimization tool for copyable values and an essential prerequisite for move-only values.
Move-Only Types and Generics
One of the most difficult issues bringing move-only types to Swift is the fact that Any is copyable. Were we to allow Any to store move-only types, then we would need to make Any itself un-copyable. This would break a lot of code.
Instead, we expect to refine the definition of Any . By making Any a synonym for any Copyable , we can ensure that Any is itself always copyable at the cost of limiting it to only store copyable values. This redefinition would preserve the behavior of current code that uses Any . Of course, this means we need to introduce a new type that can hold any value whether it is copyable or not. We’re still trying to come up with a good name for this. (We’ve considered any Moveable and any ?Copyable as possible ways to describe all types that are at least moveable and may or may not be copyable.)
Introducing a new “top type” to Swift’s type system is going to require some deep surgery to many parts of the compiler and will no doubt have additional implications for the operation of our generics system that we have yet to fully understand.
Of course, the bulk of our standard library collections are generic, so we’ll need to carefully assess the impact on those and see how far those can be generalized to support move-only types.
Where We Are So Far
The description above is based in part on design discussions that go back many years, including the “Ownership Manifesto” that was published in 2017. Over the last year, a couple of our compiler engineers have been actively experimenting to see how this might all fit together. For example, these experiments pointed out to us that borrow and take concepts were essential before even the most basic move-only types could be made useful.
We still have a long ways to go, but we’re making steady progress:
- We’ve started writing up design proposals for the various kinds of
borrow,take, and related keywords and how they will work. The first of these are currently in Swift Evolution review and we will continue to share those with the community as we proceed. - We have some prototype implementations behind a feature flag in the compiler. This is of course veryexperimental, but we encourage people to try it and give us feedback on what works, what doesn’t work, and whether the direction we’re going looks like it will be generally useful.
- One of our goals is to efficiently support pointers into collection contents via a new
BufferViewtype. We’re just starting to understand how that might work in practice. One thing that has become clear: The fullest implementation of this will require more than just move-only types. We’ll also need some additional tools to ensure the lifetime of aBufferViewends before the collection it references. - We’re just beginning to explore the implications for generics and what will be needed to have a new top type.
- We’re also just beginning to understand the impact for the standard library collection types.
- We’ve also been talking with people working on C++ interop. C++ can express types that lack the ability to be copied, and importing such types will rely on the same internal mechanisms that are used to represent Swift non-copyable types.
How Other Languages Handle This
For completeness, I’ll summarize a few alternate viewpoints:
-
Lifetime variables: Rust’s “lifetime variables” express lifetime dependencies between different objects in a way that allows the compiler to verify correctness at compile time and completely avoid a lot of runtime lifetime management. This gives more flexibility than just move-only types, but many developers seem to find this system difficult to learn and difficult to work with in practice. Swift’s focus on ease-of-use that has led us to look instead for approaches that can easily handle the most important lifetime constraints without the complex generality of Rust’s approach.
-
Immutable/Functional: Functional languages rely on fully-immutable values, which makes copying largely irrelevant. This makes these languages very uniform with strong guarantees that allow a lot of sophisticated optimizations. But immutability has a number of drawbacks, especially for less-experienced developers who can find some of the constructs awkward. A lot of the current work in Swift (and other languages) involves ways to provide mutability in most cases, restricting it only as necessary to provide key safety guarantees.
-
Enhanced Pointers: C and C-family languages such as C++ and Objective-C allow you to use raw pointers to optimize reference management. In practice, this has proven error-prone and language designers are trying to find ways to eliminate the use of raw pointers, similar to how language designers in the 1960s began introducing structured programming constructs in order to eliminate the need for
goto. You can view some of these new constructs as being ways to "augment" or "enhance" pointers with additional safety properties. But I prefer to think of them as more abstract concepts -- "views", "ownership", and "references" -- that just happen to be implemented as pointers, just aswhileloops andifstatement are ultimately implemented in terms of machine-level branch instructions.
