`Borrow` and `Inout` types for safe, first-class references

[tera]

@Slava_Pestov

… how about Pointer and MutablePointer, then? ;-)

Makes sense...

In other languages pointers are considered unsafe, but in Swift we use "Unsafe" in the name to indicate unsafeness; otherwise everything is safe by default, so why not Pointer.

[rayx (Huan Xiong)]

@Slava_Pestov

… how about Pointer and MutablePointer, then? ;-)

I think the term pointer is only used in Swift and C interop programming and isn't a native concept in swift language? In all languages I know that support reference (c++, rust, swift), it has a different behavior than pointer. So calling it pointer is a bit confusing to me.

[Nobody1707]

Pointers are definitely a native concept, but I wouldn't want to call a safe type with a tracked lifetime dependency a pointer. {Mutable}Reference is a better name for that idea.

[Joe_Groff]

@CrystDragon

How would the compiler detect this convincingly? If I write it like below, will it be diagnosed?

@_lifetime(&value)
func helper(from value: inout Int) -> Inout<Int> {
    Inout(&value)  // This must be OK
}

@_lifetime(&target)
func noisyCounterRef(from target: inout NoisyCounter) -> Inout<Int> {
    return helper(&target.value)  // seems OK? because it does not directly invoke Inout.init
}

It would still be diagnosed, since the parameter that gets passed to noisyCounterRef is still dependent on the access into target.value, which can't be extended out of the current function. (The dependency behavior here is not really specific to Borrow or Inout, but how @_lifetime dependencies generally work; I reiterate the behavior in this proposal because @_lifetime is not itself officially part of the language yet, and because I thought that the dependency behavior is core to how these types are used.)

@Slava_Pestov

… how about Pointer and MutablePointer, then? ;-)

For better or worse, we didn't end up calling Span BufferPointer. Representationally, Borrow isn't always just a pointer, so calling it one might be misleading.

Since it's likely we will want to pursue some sort of language integration between these explicit types and reference bindings, I think we want to keep some naming connection between the types and the words we use for bindings.

[benrimmington]

There was a future direction in SE-0377:

inout is also conspicuous now in not following the -ing convention we've settled on for consuming/borrowing/mutating modifiers. We could introduce mutating as a new parameter modifier spelling, with no-implicit-copy behavior.

Should the proposed Inout type be renamed?

[Dmitriy_Ignatyev]

@Slava_Pestov

how about Pointer and MutablePointer, then? ;-)

In my opinion, naming these types Pointer and MutablePointer may not fully capture the key concept of ownership, which is central to these types' design. While it's true that Borrow and Inout are safer alternatives to UnsafePointer and UnsafeMutablePointer, I believe that the names should reflect the ownership semantics in first place, rather than focusing solely on the idea of "safe pointers."

For example, as pointed out in the proposal, bitwise-borrowable types are not passed as pointers. If Borrow was named something like Pointer, it could be misleading, as it wouldn't reflect the fact that a Borrow is a shared reference to a value, rather than a direct memory address.

So at least Borrow can not be named pointer. Now I see the disconnect between naming Borrow and MutablePointer. While Borrow represents a shared reference with no ownership transfer, Inout represents a reference with exclusive mutable access. The names Borrow and MutablePointer don’t seem to align well with each other in terms of conveying these different ownership models and access patterns.

Additionally, these types don’t just offer "safe" pointers. They also provide lifetime checks and guarantees that pointers do not. With Borrow and Inout, we gain a higher level of safety that enforces proper access and ownership rules, ensuring that references remain valid. This is a more sophisticated concept than what you get with traditional pointers.

Also, Borrowed<T> and Mutable<T> suggested by John are felt more descriptive and intuitive compared to Borrow and Inout

[Joe_Groff]

@ellie20

If Swift ever gets "ownership generics" in the future, maybe there could be a viable path to retrofit OptionalBorrow and OptionalInout (and MutableSpan), and APIs using these types. Retrofitting could be more difficult with Optional<Borrow<T>>, because it's already an Optional, but with a different interface than we might want.

typealias OptionalBorrow<T> = Optional<borrowing, T>
typealias OptionalInout<T> = Optional<inout, T>
typealias MutableSpan<T> = Span<inout, T>

Maybe it's worth explaining why I think "ownership generics" are a compelling enough future direction to warrant consideration. Basically, ownership generics would decouple being generic over ownership from being generic over types.

It would definitely be good to have a way to relate mutable/immutable reference pairs like Span and MutableSpan. We had considered the possibility of treating them as instances of a common generic type, but their interfaces diverge quite significantly at every level due to exclusivity; the mutable variants must be non-Copyable, their projection operations require transitive exclusivity, they need specialized exclusive-splitting primitives, etc., and there would be very little useful commonality at the generic level, and most of the practical interface would be bifurcated into separate extension where ownership == borrow and where ownership == inout worlds.

However, the relation between property/subscript accessors is largely resolved statically and doesn't really depend on any parametricity or separate-compilability that generics provide, so it seems possible that ad-hoc relating Span and MutableSpan as a related pair of reference-like types, with perhaps a protocol to describe the necessary operations to go from one to the other, would be enough to enable a future extension of storage declarations that provide both under one name.

[asdf_bro]

What about Alias<T> and MutableAlias<T>?

And a single new keyword alias:

var x = 0

alias y = x
alias z = &x

[dmt (Dima Galimzianov)]

@Joe_Groff

exclusive ownership or reborrowing for Inout.value

Inout shares an ergonomic problem with MutableSpan: for safety, mutation of the target value through Inout requires exclusive access to the Inout value. In Swift today, this exclusive access can only be exercised through mutable bindings, which will force Inout values to be assigned to var bindings even if the Inout value itself is never mutated, and prevent expressions returning Inout values from being used directly in mutation expressions

@Joe_Groff

the ultimate limit to the bindings approach is generic type composition: it is useful to be able to have an Optional<borrow T> or SomeKindOfArray<inout U>

Don't you consider solving the "borrowing an Inout makes the referee immutable" problem a prerequisite for shipping Inout? I find myself struggling to understand how the type is better than the already partially implemented inout bindings without that fix. The example from the pitch would effectively just be:

func updateTotal(in dictionary: inout [String: Int], for key: String, with values: [Int]) {
  inout entry = &dictionary[key, default: 0]
  for value in values {
    entry += value
  }
}

The example regarding generic composition also seems limited by the requirement that the composite must be mutable to perform any work:

func incrementIfSome(_ value: inout Inout<Int>?) { // Can't borrow
  value?.value += 1
}

We cannot generalize function parameters over the binding type. If I have a borrowing parameter, I lose the ability to mutate the underlying data even if the type is an Inout:

func genericProxy<T: ~Copyable>(_ value: borrowing T, _ f: (borrowing T) -> Void) { f(value) }

func use() {
  var i = 42
  genericProxy(Inout(&i)) { iAsInout in
    iAsInout.value += 1 // error
  }
}

And semantically, I think it's quite important to differentiate the levels of indirection, similar to C++:

• inout Inout<T>: We may modify the reference itself or the referee (T**).
• borrowing Inout<T>: We may modify the referee, but not the reference (T* or T** const).
• inout Borrow<T>: We may modify the reference itself, but not the referee (T const**).
• borrowing Borrow<T>: Neither the reference nor the referee is mutable (T const* const).

I am also concerned that shipping Inout with:

var value: Value { 
  borrow
  mutating mutate 
}

invites source compatibility problems. If we later transition to nonmutating exclusive mutate, the compiler will begin diagnosing "Variable was never mutated". And it might also be an ABI problem to solve.

[Joe_Groff]

@dmt

Don't you consider solving the "borrowing an Inout makes the referee immutable" problem a prerequisite for shipping Inout? I find myself struggling to understand how the type is better than the already partially implemented inout bindings without that fix.

I'm not sure what problem you're describing here, since that's a fundamental part of the rules no matter how we express them: an exclusive access to a value requires also making an exclusive access to any references you traverse on the way to the value. If we have inout bindings in the future, then like inout parameters today, they also become immutable while being borrowed:

struct NC: ~Copyable {}

func borrow(_: borrowing NC, andMutate: inout NC) {}

func foo(x: inout NC) {
  // error: cannot mutate `x` while it is also being borrowed
  borrow(x, andMutate: &x)
}

[dmt (Dima Galimzianov)]

Apologies, I realize now that I was conflating the proposed "exclusive" ownership mode with a different concept. I also thought there was a goal to make borrowing Inout<T> a meaningful concept in its own right. I had imagined that "exclusivity" might be a property of the type itself. Basically a type that requires exclusive borrowing of its values (like inout) but without strictly requiring mutation of the reference itself.

To be clear, I thought the overlapping access would be impossible because the compiler would forbid more than one simultaneous borrow of an Inout instance:

func borrowTwo(_ x: borrowing Inout<T>, _ y: borrowing Inout<T>) {}

func use(_ value: inout T) {
  let valueInOut = Inout(&value)
  // Error: can't borrow a value of an exclusively borrowing type more than once.
  borrowTwo(valueInOut, valueInOut) 
}

If that is not the case, then it seems borrowing Inout<Value> is effectively just a Borrow<Value> that is forced to be represented as a pointer (bypassing the optimizations for small, bitwise-borrowable types).

While the "Future Directions" section mentions the need for a "borrowing reference type that is always represented as a pointer", using borrowing Inout<T> as a proxy for that seems confusing. It results in a type whose primary name implies mutation, yet is used in a context where mutation is disallowed (borrowing), simply to gain a specific ABI representation.