Hi everyone. I'd like to revive a pitch from a while back to turn our __shared and __owned parameter ownership modifiers into an official language feature. During the recent review of SE-0366, for the move operator, the community raised the importance of considering these modifiers, and their naming, as part of the naming of the move operator itself. I like the idea of using the same name for the operator and the related parameter modifier, and I think take is a nice name for both, and borrow is also a good name both for the borrowing parameter modifier and for an operator analogous to move that forces a borrow-in-place without copying. So I'd like to start this new pitch thread there. Here is the draft PR, and I'll include the initial draft of the revised proposal below:
borrow and take parameter ownership modifiers
- Proposal: SE-NNNN
- Authors: Michael Gottesman, Joe Groff
- Review Manager: TBD
- Status: Implemented, using the internal names
__sharedand__owned - Pitch v1: https://github.com/gottesmm/swift-evolution/blob/consuming-nonconsuming-pitch-v1/proposals/000b-consuming-nonconsuming.md
Introduction
We propose new borrow and take parameter modifiers to allow developers to explicitly choose the ownership convention that a function uses to receive immutable parameters. This allows for fine-tuning of performance by reducing the number of ARC calls or copies needed to call a function, and provides a necessary prerequisite feature for move-only types to specify whether a function consumes a move-only value or not.
Pitch threads:
- First pitch thread: https://forums.swift.org/t/pitch-formally-defining-consuming-and-nonconsuming-argument-type-modifiers
Motivation
Swift uses automatic reference counting to manage the lifetimes of reference-counted objects. There are two broad conventions that the compiler uses to maintain memory safety when passing an object by value from a caller to a callee in a function call:
- The callee can borrow the parameter. The caller guarantees that its argument object will stay alive for the duration of the call, and the callee does not need to release it (except to balance any additional retains it performs itself).
- The callee can take the parameter. The callee becomes responsible for either releasing the parameter or passing ownership of it along somewhere else. If a caller doesn't want to give up its own ownership of its argument, it must retain the argument so that the callee can take the extra reference count.
These two conventions generalize to value types, where a "retain" becomes an independent copy of the value, and "release" the destruction and deallocation of the copy. By default Swift chooses which convention to use based on some rules informed by the typical behavior of Swift code: initializers and property setters are more likely to use their parameters to construct or update another value, so it is likely more efficient for them to take their parameters and forward ownership to the new value they construct. Other functions default to borrowing their parameters, since we have found this to be more efficient in most situations.
These choices typically work well, but aren't always optimal. Although the optimizer supports "function signature optimization" that can change the convention used by a function when it sees an opportunity to reduce overall ARC traffic, the circumstances in which we can automate this are limited. The ownership convention becomes part of the ABI for public API, so cannot be changed once established for ABI-stable libraries. The optimizer also does not try to optimize polymorphic interfaces, such as non-final class methods or protocol requirements. If a programmer wants behavior different from the default in these circumstances, there is currently no way to do so.
Looking to the future, as part of our ongoing project to add ownership to Swift, we will eventually have move-only values and types. Since move-only types do not have the ability to be copied, the distinction between the two conventions becomes an important part of the API contract: functions that borrow move-only values make temporary use of the value and leave it valid for further use, like reading from a file handle, whereas functions that take a move-only value consume it and prevent its further use, like closing a file handle. Relying on implicit selection of the parameter convention will not suffice for these types.
Proposed solution
We can give developers direct control over the ownership convention of parameters by introducing two new parameter modifiers borrow and take.
Detailed design
borrow and take become contextual keywords inside parameter type declarations. They can appear in the same places as the inout modifier, and are mutually exclusive with each other and with inout. In a func, subscript, or init declaration, they appear as follows:
func foo(_: borrow Foo)
func foo(_: take Foo)
func foo(_: inout Foo)
In a closure:
bar { (a: borrow Foo) in a.foo() }
bar { (a: take Foo) in a.foo() }
bar { (a: inout Foo) in a.foo() }
In a function type:
let f: (borrow Foo) -> Void = { a in a.foo() }
let f: (take Foo) -> Void = { a in a.foo() }
let f: (inout Foo) -> Void = { a in a.foo() }
TODO: How to apply these modifiers to self, the newValue of accessors, ...
take or borrow on a parameter do not affect the caller-side syntax for passing an argument to the affected declaration. For typical Swift code, adding, removing, or changing these modifiers does not have any source-breaking effects. (See "related directions" below for interactions with other language features being considered currently or in the near future which might interact with these modifiers in ways that cause them to break source.)
Protocol requirements can also use take and borrow, and the modifiers will affect the convention used by the generic interface to call the requirement. The requirement may still be satisfied by an implementation that uses different conventions for parameters of copyable types:
protocol P {
func foo(x: take Foo, y: borrow Foo)
}
// These are valid conformances:
struct A: P {
func foo(x: Foo, y: Foo)
}
struct B: P {
func foo(x: borrow Foo, y: take Foo)
}
struct C: P {
func foo(x: take Foo, y: borrow Foo)
}
Function values can also be implicitly converted to function types that change the convention of parameters of copyable types among unspecified, borrow, or take:
let f = { (a: Foo) in print(a) }
let g: (borrow Foo) -> Void = f
let h: (take Foo) -> Void = f
let f2: (Foo) -> Void = h
Source compatibility
Adding take or borrow to a parameter in the language today does not otherwise affect source compatibility. Callers can continue to call the function as normal, and the function body can use the parameter as it already does. A method with take or borrow modifiers on its parameters can still be used to satisfy a protocol requirement with different modifiers. The compiler will introduce implicit copies as needed to maintain the expected conventions. This allows for API authors to use take and borrow annotations to fine-tune the copying behavior of their implementations, without forcing clients to be aware of ownership to use the annotated APIs. Source-only packages can add, remove, or adjust these annotations on copyable types over time without breaking their clients.
This will change if we introduce features that limit the compiler's ability to implicitly copy values, such as move-only types, "no implicit copy" values or scopes, and take or borrow operators in expressions. Changing the parameter convention changes where copies may be necessary to perform the call. Passing an uncopyable value as a take argument ends its lifetime, and that value cannot be used again after it's taken.
Effect on ABI stability
take or borrow affects the ABI-level calling convention and cannot be changed without breaking ABI-stable libraries.
Effect on API resilience
take or borrow break ABI for ABI-stable libraries, but are intended to have minimal impact on source-level API. When using copyable types, adding or changing these annotations to an API should not affect its existing clients.
Alternatives considered
Naming
We have considered alternative naming schemes for these modifiers:
- The current implementation in the compiler uses
__sharedand__owned, and we could remove the underscores to make these simplysharedandowned. These names refer to the way a borrowed parameter receives a "shared" borrow (as opposed to the "exclusive" borrow on aninoutparameter), whereas a taken parameter becomes "owned" by the callee. We found that the "shared" versus "exclusive" language for discussing borrows, while technically correct, is unnecessarily confusing for explaining the model. - A previous pitch used the names
nonconsumingandconsuming.
The names take and borrow arose during the first review of SE-0366. These names also work well as names for operators that explicitly transfer ownership of a variable or borrow it in place, discussed below as the take and borrow operators under Related Directions. We think it is helpful to align the naming of those operators with the naming of these parameter modifiers, since it helps reinforce the relationship between the calling conventions and the expression operators: to explicitly transfer ownership of an argument in a call site to a parameter in a function, use foo(take x) at the call site, and use func foo(_: take T) in the function declaration.
Effect on call sites and uses of the parameter
This proposal designs the take and borrow modifiers to have minimal source impact when applied to parameters, on the expectation that, in typical Swift code that isn't using move-only types or other copy-controlling features, adjusting the convention is a useful optimization on its own without otherwise changing the programming model, letting the optimizer automatically minimize copies once the convention is manually optimized.
It could alternatively be argued that explicitly stating the convention for a value argument indicates that the developer is interested in guaranteeing that the optimization occurs, and having the annotation imply changed behavior at call sites or inside the function definition, such as disabling implicit copies of the parameter inside the function, or implicitly taking an argument to a take parameter and ending its lifetime inside the caller after the
call site. We believe that it is better to keep the behavior of the call in expressions independent of the declaration (to the degree possible with implicitly copyable values), and that explicit operators on the call site can be used in the important, but relatively rare, cases where the default optimizer behavior is insufficient to get optimal code.
Applying borrow and take modifiers to the self parameter of methods
This proposal does not yet specify how to control the calling convention of the self parameter for methods. As currently implemented, the __consuming modifier can be applied to the method declaration to make self be taken, similar to how the mutating method modifier makes self be inout. We could continue that scheme with new function-level modifiers:
struct Foo {
mutating func mutate() // self is inout
taking func take() // self is take
borrowing func borrow() // self is borrow
}
Alternatively, we could explore schemes to allow the self parameter to be declared explicitly, which would allow for the take and borrow modifiers as proposed for other parameters to also be applied to an explicit self parameter declaration.
Related directions
take operator
Currently under review as SE-0366, it is useful to have an operator that explicitly ends the lifetime of a variable before the end of its scope. This allows the compiler to reliably destroy the value of the variable, or transfer ownership, at the point of its last use, without depending on optimization and vague ARC optimizer rules. When the lifetime of the variable ends in an argument to a take parameter, then we can transfer ownership to the callee without any copies:
func consume(x: take Foo)
func produce() {
let x = Foo()
consume(x: take x)
doOtherStuffNotInvolvingX()
}
borrow operator
Relatedly, there are circumstances where the compiler defaults to copying when it is theoretically possible to borrow, particularly when working with shared mutable state such as global or static variables, escaped closure captures, and class stored properties. The compiler does this to avoid running afoul of the law of exclusivity with mutations. In the example below, if callUseFoo() passed global to useFoo by borrow instead of passing a copy, then the mutation of global inside of useFoo would trigger a dynamic exclusivity failure (or UB if exclusivity checks are disabled):
var global = Foo()
func useFoo(x: borrow Foo) {
// We need exclusive access to `global` here
global = Foo()
}
func callUseFoo() {
// callUseFoo doesn't know whether `useFoo` accesses global,
// so we want to avoid imposing shared access to it for longer
// than necessary, and we'll pass a copy of the value. This:
useFoo(x: global)
// will compile more like:
/*
let globalCopy = copy(global)
useFoo(x: globalCopy)
destroy(globalCopy)
*/
}
It is difficult for the compiler to conclusively prove that there aren't potential interfering writes to shared mutable state, so although it may in theory eliminate the defensive copy if it proves that useFoo, it is unlikely to do so in practice. The developer may know that the program will not attempt to modify the same object or global variable during a call, and want to suppress this copy. An explicit borrow operator could allow for this:
var global = Foo()
func useFooWithoutTouchingGlobal(x: borrow Foo) {
/* global not used here */
}
func callUseFoo() {
// The programmer knows that `useFooWithoutTouchingGlobal` won't
// touch `global`, so we'd like to pass it without copying
useFooWithoutTouchingGlobal(x: borrow global)
}
If useFooWithoutTouchingGlobal did in fact attempt to mutate global while the caller is borrowing it, an exclusivity failure would be raised.
Move-only types, uncopyable values, and related features
The take versus borrow distinction becomes much more important and prominent for values that cannot be implicitly copied. We have plans to introduce move-only types, whose values are never copyable, as well as attributes that suppress the compiler's implicit copying behavior selectively for particular variables or scopes. Operations that borrow a value allow the same value to continue being used, whereas operations that take a value destroy it and prevent its continued use. This makes the convention used for move-only parameters a much more important part of their API contract:
moveonly struct FileHandle { ... }
// Operations that open a file handle return new FileHandle values
func open(path: FilePath) throws -> FileHandle
// Operations that operate on an open file handle and leave it open
// borrow the FileHandle
func read(from: borrow FileHandle) throws -> Data
// Operations that close the file handle and make it unusable take
// the FileHandle
func close(file: take FileHandle)
func hackPasswords() throws -> HackedPasswords {
let fd = try open(path: "/etc/passwd")
// `read` borrows fd, so we can continue using it after
let contents = try read(from: fd)
// `close` takes fd from us, so we can't use it again
close(fd)
let moreContents = try read(from: fd) // compiler error: use after take
return hackPasswordData(contents)
}
Acknowledgments
Thanks to Robert Widmann for the original underscored implementation of __owned and __shared: https://forums.swift.org/t/ownership-annotations/11276.
