`exclusive` parameter ownership modifier

Evolution Pitches
,
1

Introduction

mutating methods require the caller to ensure that the location passed as self can only be accessed through self. This guarantee of exclusive access is a very strong property that can be used by carefully written types like MutableSpan to achieve higher-level safety guarantees. However, operations that want exclusive access are not always semantically mutations, and tying these concepts together creates some real usability problems for these types. This proposal adds a new exclusive parameter ownership modifier which has the same exclusivity properties as mutating but does not permit the value to be changed. (This is only narrowly useful, and most programmers will never need it, but it is very important in those cases.) It also adopts the modifier on some existing APIs.

Motivation

Swift currently supports three parameter/method ownership modifiers: borrowing, inout/mutating, and consuming.

These modifiers can be understood concretely as statements about the basic capabilities of the parameter. For example, an inout parameter can be modified, but a borrowing parameter cannot.

These modifiers can also be understood more abstractly as propositions in Swift's formal logic of ownership. Each modifier represents a different assertion about the exclusivity that the value or location is known to have within the function. For example, an inout parameter is known to be the only way to access that memory location as long as the function is running, while a borrowing parameter makes no such guarantee.

These two levels are naturally related. The abstract propositions about ownership create conditions that make the concrete capabilities safe and reasonable. Swift permits multiple borrowing parameters to be bound to the same value because they can only be used to read it. inout requires exclusive ownership because that allows mutation to still be subject to static local reasoning.

However, the abstract propositions enable a lot more than just that. The exclusive ownership of inout states that this function is the only code that can access the given memory location. A carefully-designed type can make use of that to get Swift to enforce its own safety properties. For example, an UnsafeMutablePointer essentially has reference semantics: two different copies of the same pointer can be used to access the same memory. This can lead to memory safety bugs. A safe mutable pointer type could use non-copyability to maintain an invariant that there's only one pointer value that referenes a particular piece of memory. It could then use exclusivity to enforce that only one operation can be done to that memory at a time.

That is exactly what's already being done by MutableSpan. The construction of a MutableSpan has a precondition that the memory is uniquely referenced and fully initialized. The operations on MutableSpan conspire to maintain those properties as invariants: the type is non-Copyable, and it has no methods that can be used as effective copies. The operations that merely read the underlying memory are borrowing because it is safe that have multiple such operations occuring at once. The operations that mutate the underlying memory are mutating, not because they actually mutate the MutableSpan value (which is just the pointer and its bound), but because mutating ownership prevents any other operations from happening on that MutableSpan value at the same time, statically ruling out several classes of memory safety bug.

Hidden in that last point is a very real usability problem. mutating operations on value types are usually mutations of the value. Swift therefore, quite reasonably, only allows mutating operations to be performed on mutable memory locations. But if an operation is only mutating because it wants exclusive ownership of the value, this is too strong! It disallows useful patterns like calling the operation on a function result (e.g. array.mutableSpan.operation()) or a local let (e.g. mySpan.operation()). What's needed is a weaker form of ownership that still asserts exclusive access to the value but does not claim to mutate it. That is what we are proposing here.

Proposed solution

A parameter can be declared to have exclusive ownership:

func partition(values: exclusive MutableSpan<Int>, by pivot: Int) {
  // ...
}

Within the function, the parameter cannot be mutated or consumed, but it can be borrowed or used exclusively.

func testUses(span: exclusive MutableSpan<Int>) {
  // valid: can call an exclusive operation on MutableSpan
  span[0] = 10

  // invalid: cannot mutate an exclusive value
  span = span.extracting(last: 10)
}

The argument passed to the parameter must be an expression that can be used exclusively. This includes most expressions except references to storage declarations that can only be borrowed; the exact rules are given in the detailed design.

func testLocalVariable() {
  var array = [10, 0, 20, 5]

  // valid: array.mutableSpan produces an r-value that can be used
  // exclusively
  partition(values: array.mutableSpan, by: 7)

  // valid: a local variable (even an immutable one) can be used
  // exclusively
  let span = array.mutableSpan
  partition(values: span, by: 12)
}

func testInoutParameter(span: inout MutableSpan<Int>) {
  // valid: an inout or exclusive parameter can be used exclusively
  partition(values: span)
}

func testBorrowingParameter(span: MutableSpan<Int>) {
  // invalid: a borrowing parameter cannot be used exclusively
  partition(values: span)
}

A non-static method of a struct, enum, or protocol, including an accessor of a non-static property or subscript, can be declared to have exclusive ownership:

extension MutableSpan {
  exclusive func partition(by pivot: Element) {
    // ...
  }

  subscript(index: Int) -> Element {
    borrow { /* ... */ }
    exclusive mutate { /* ... */ }
  }
}

This behaves exactly as if the implicit self parameter was declared to have exclusive ownership. No other function can be declared exclusive.

Detailed design

exclusive is an ordinary parameter/method ownership modifier. It participates in the function type system like other existing modifiers:

  • exclusive cannot be combined with any other parameter/method modifiers: mutating, nonmutating, inout, borrowing, or consuming.

  • Two functions cannot be overloaded solely by the ownership of a parameter.

  • Whether a parameter is exclusive is part of the type of a function. Two function types that differ only by parameter ownership are different function types.

  • Changing the ownership of a parameter changes the ABI of the function.

  • For the purposes of function subtype conversions, consuming T is a subtype of exclusive T, which is a subtype of borrowing T. Since function subtyping is contravariant over parameter types, this means that e.g. (exclusive T) -> Int converts to (consuming T) -> Int.

  • A consuming protocol requirement can be implemented by an exclusive method. An exclusive protocol requirement can be implemented by a borrowing method.

exclusive parameters of copyable type are not implicitly copyable, in the same way that consuming and borrowing parameters are not implicitly copyable.

Exclusive uses

An expression can be used exclusively if:

  • it is an expression that produces a temporary value, such as a function call;
  • it is an expression that can be implicitly copied to produce a temporary value; or
  • it is a storage reference expression and both:
    • the storage declaration is one of
      • a stored variable, constant, or property,
      • a variable, constant, property, or subscript with a getter, or
      • a variable, constant, property, or subscript with some kind of modify accessor; and
    • the base expression, if any, can satisfy the ownership requirement of the accessor selected (treating a stored property as if it had an exclusive accessor).

Using a stored variable or property exclusively may cause exclusivity conflicts, essentially as if it were mutated.

A global let constant, static let property, or class instance let property cannot be used exclusively. These storage locations would otherwise have to used dynamic exclusivity enforcement, even on simple reads. This would be a binary compatibility problem for existing code, because they currently do not require enforcement. More fundamentally, it would be a poor trade-off to force dynamic exclusivity checking on all uses of lets just for a vanishingly rare case where are used exclusively.

An escaping local let constant may require dynamic exclusivity enforcement if it is ever used exclusively.

Changes to MutableSpan

All existing mutating methods and accessors on MutableSpan are revised to be exclusive. It is meaningful to have a mutating operation on MutableSpan, e.g. if it changed the bounds of the span, but none of the existing operations happen to look like that, so they should all be weakened to only require exclusive access.

17 Likes
2

I haven’t fully decided whether I like this (not that my opinion should be very relevant these days!) but I’ll note one lonely extra API that would benefit from this: isKnownUniquelyReferenced.

6 Likes
3

I agree with the general direction of this pitch, but I'm personally wondering if the language designers have considered splitting mutating into interior / exterior affordances. The idea is to also move mutable / immutable reference semantics from types to bindings. It's something I've had a lot of fun experimenting with, even without proper language support. While I don't have any suggestions for how it would be implemented, I imagine you could evolve the language in the following alternative direction (± aesthetics):

1. mutating           MutableSpan<T> =  mutating MutableSpan<T>
2. mutating(interior) MutableSpan<T> = exclusive MutableSpan<T> // the modifier being pitched
3. mutating(exterior) MutableSpan<T> ≈  mutating        Span<T>
4. mutating(none)     MutableSpan<T> ≈ borrowing        Span<T>
1 Like
4

Are there any types other than MutableSpan that would benefit? For the MutableSpan operations that would benefit from exclusive ownership, I would argue that their natural ownership is actually consuming. For example, a function with exclusive ownership of a MutableSpan may eventually want to locally mutate it, such as by shrinking the bounds, in which case it would need to create a local "copy" using something like MutableSpan.extracting(...). With consuming ownership, it would have a locally mutable value already. Another benefit is more flexible lifetimes: for example, a consuming version of MutableSpan.extracting could return a value with a copied, not scoped, lifetime dependency on the original.

In Rust, most operations on mutable references, including mutable spans such as &mut [u8], have consuming ownership over the mutable reference itself. Mutating ownership would look like &mut &mut [u8], and is rare. To prevent a &mut value from becoming unusable after being used, Rust will automatically create a local "copy" of it if necessary. This is called "reborrowing", and is not true copying because the "copy" has a mutating lifetime dependency on the original. It happens even if the &mut value is an immutable local variable, and is the only case in the language where an immutable variable can be mutably accessed. It is a special case in the language that only applies to &mut types, not user-defined types,[1] but there is currently an initiative to extend it to user-defined types.[2] It is basically syntactic sugar, and doesn't fundamentally change the ownership model.

  1. For example, this is why Option::as_deref_mut and Pin::as_mut exist. ↩︎

  2. Reborrowing in Rust is actually more general, because it also includes the implicit conversion of a &mut reference to an immutable & reference. In that case, the & reference has a copied, not scoped, lifetime dependency on the &mut reference. The initiative calls this "true reborrowing". ↩︎

2 Likes
5

Wouldn’t that be an ABI breaking change ? How would that be handle on ABI stable platforms ?

6

The operations on MutableSpan are all implemented in a way (@_alwaysEmitIntoClient) that does not require OS support.

7

Any reference-like type is likely to benefit from this. If we had an InOut<T> type that could capture an exclusive access to a single value, for instance, it could have an operation analogous to UnsafeMutablePointer.pointer(to:) to project an access to a derived field, and that operation would benefit from exclusive access.

I don't think "reborrowing" would be as effective in Swift as it is in Rust, since accesses in Swift often cross opaque ABI boundaries, and they can involve non-trivial setup and teardown from getters/setters, accessor coroutines, exclusivity checks, etc., so it seems like a mutable access cannot be simply "copied" except in very limited situations where we can see through all abstractions.

3 Likes
8

Does this modifier line up with "exclusive borrowing" referred to in some other prior conversations, and could we spell it as such: borrowing(exclusive)?

9

We're likely to at least add an Inout<T> (name TBD) that represents a single mutable reference, analogous to &mut T in Rust the same way that MutableSpan is analogous to &mut [T] (which I would argue has to be understood as a separate type from the scalar &mut in some ways).

This is an interesting idea. I certainly wouldn't call it natural, though — it is completely unlike the normal behavior of consuming.

Mmm. This is tricky, though, because ultimately the caller does want its span back. In Rust, this re-borrowing is all done with a lot of special-casing of the reference types, and, again, I think it amounts to the reference types basically just not behaving like normal affine types except as a conservative approximation.

Personally, I would rather add a new kind of ownership than to have a set of exceptions to the non-copyable type rules that essentially amount to "whatever Inout and MutableSpan need". That seems like a recipe for under-specified behavior. Rust does not seem very clear about exactly what's allowed here, either.

Part of my thinking for this is that I think there's a plausible future direction of supporting "exclusive use" as a fully-general kind of access to storage. For example, you ought to be able to write a coroutine accessor for a property or subscript that gives clients exclusive use of the current value — it's something we can do for physical storage, so a computed storage declaration that's trying to exactly represent physical storage ought to be able to do it as well. I suppose the counter-argument is that we wouldn't need exclusive use at all if we just special-cased spans and references the right way; maybe I need to think about it more.

2 Likes
10

Yes, this is the same concept, and it could be spelled like that. I am not totally sold on the current spelling I've proposed; I did consider coining a term of art like controlling but decided that that was the wrong set of trade-offs for such a specialized feature.

2 Likes
11

I like borrowing(exclusive) as I think it makes it clearer what is exclusive about the operation, but I don't feel strongly.

1 Like