`Borrow` and `Inout` types for safe, first-class references

Evolution Pitches
21

My gut is that this proposal and the borrow/mutate accessor proposal are inside out with respect to each other:

If you start with these types, then then you don't need the accessors; you can just use get/mutating get accessors that return Borrow/Inout references. Obviously at that point they can't have value properties, but then the language can offer to transparently dereference these types, which will result in more ergonomic syntax.

Regardless of that, it feels really weird to me that the Borrow type uses a borrow accessor, but the Inout type uses the mutate accessor. The terminology has got all tangled up here. Surely, whatever terms the accessor proposal uses, these types should mirror those names & vice versa.

1 Like
22

This is not strictly a Borrow/Inout, but this is the first time the mechanics of borrow-by-copy are described, so I have another question in that space: is this legal?

struct Foo {
	var array: InlineArray<4, Int>
	
	func bar() -> Span<Int> {
		array.span
	}
}

The proposal says that forming a Span over a bitwise-borrowed value is a problem, so InlineArray has a special attribute so that it always borrows by address. We also clarified earlier that the attribute is not inherited by structs that contain a field that has it. It seems to mean this could create a span that outlives the temporary it points to?

23

I guess I meant to say the compiler does not randomly assume this behavior for types, but it does do the usual ā€œif this thing contains an addressable-for-dependencies type, then this type itself must be addressable-for-dependenciesā€ so the example given:

Is perfectly safe; it passes self of Foo by address.

24

While this is true, you will very quickly realize you run into ambiguity issues if you didnā€™t have the accessors:

struct Array<Element: ~Copyable>: ~Copyable {
  subscript(i: Int) -> Borrow<Element> {
    get { ... }
  }

  subscript(I: Int) -> Inout<Element> {
    mutating get { ... }
  }
}

let a = Array<Atomic<Int>>(...)
let b = a[0] // error: ambiguous use of subscript what type is 'B'?

Of course some languages have this problem solved, but our languageā€™s type checker does not work like that. The borrow/mutate accessors help alleviate this issue.

1 Like
SE-0519: Borrow and Inout types for safe, first-class references
25

It is quite nice that this can compose with generics. You could model an Optional<Inout<T>>, which could be a convenient way to allow in-place mutation of enums with associated values without allowing the value to be set to nil.

enum Foo {
    case intArray([Int])
    /// ... and many more
   
    var intArray: Inout<[Int]>? {
        mutating get {
            // syntax borrowed from this old pitch: https://forums.swift.org/t/pitch-borrow-and-inout-declaration-keywords/62366
            switch &self {
            case .intArray(inout intArray):
                return Inout(&intArray)
            default:
                return nil
            }
        }
    }
}

However, you would still want to allow simple read-only access, e.g.:

extension Foo {
    // or a simple [Int]? return value
    var intArray: Borrow<[Int]>? {
        get {
            switch self {
            case .intArray(let intArray):
                return Borrow(intArray)
            default:
                return nil
            }
        }
    }
}

Is there a way to disambiguate this other than giving them different names?

2 Likes
26

Isnā€™t this more of an indictment of reusing the let keyword for both borrows and copies?

27

Why wouldnā€™t `modify Value?`work in this case (with a non-escaping Optional type)?

28

I think this might be one reason why Hylo requires explicitly marking all mutable accesses. There can't be any ambiguity, because the mutating version is required to be spelled &a[0].

2 Likes
29

I think having borrow/inout bindings is the direction we ultimately want to go, and that in most cases they are what developers should use once they exist. But as I see it, the ultimate limit to the bindings approach is generic type composition: it is useful to be able to have an Optional<borrow T> or SomeKindOfArray<inout U>, but within the design confines of Swift we've already established, generic parameters would need to have a type, distinct from T or U themselves, to be parameterized by. I can think of some ways we could extend sugar even to those situations, but I can't think of a way to avoid having these types entirely. And, as you noted, it's also helpful to look at future borrow/inout bindings or properties as sugar over explicit reference formation using explicit reference types. Perfect being the enemy of good, I think it's worth introducing these types now, which unlocks a lot of expressivity, even if we don't have all the sugar that could sit on top of it lined up right now.

It should be generally possible to write this function generically:

@_lifetime(borrow target)
func refer<T>(to target: T) -> Borrow<T> {
  // This ought to be allowed
  Borrow(target)
}

and it should also work to write that function concretely for any T. The Borrow representation coordinates with the calling convention ABI to make this work for all types. (The intent of this, and the "addressable for dependencies" concept, is that you generally don't need to think about it, if you aren't directly looking at memory dumps or assembly to see how things are implemented, and you aren't doing something "weird" like implementing primitive inline storage, since InlineArray should subsume the need for most people to do that anymore.)

A type becomes addressable-for-dependencies if it contains any fields in its inline storage that are addressable-for-dependencies. InlineArray is primitively addressable-for-dependencies (as well as a few other library types like String and Data that did shenanigans to provide inline contiguous storage before InlineArray existed). Generic and resilient types are always handled as if they're potentially addressable-for-dependencies, so adding or removing InlineArray fields does not affect their ABI. The main consequence of being addressable-for-dependencies is that a parameter gets passed indirectly if the function produces any return values dependent on that parameter, so @_lifetime can change the calling convention (but it also changes the API in source-breaking ways, so that's fine).

A reasonable question. withUnsafePointer(borrow.value) should just work, but it might make sense to have direct API on the reference types as well. One reservation I have there is that, if the path forward for these types is some sort of auto-dereference or dynamicMemberLookup forwarding to the target type, that any API surface on the reference itself potentially gets in the way of being able to use the value transparently. (Personally, I find that an argument against ever making dereferencing implicit in those ways, but I also don't want to foreclose on the design space prematurely.)

4 Likes
30

An aggregate does inherit the addressable-for-dependencies property if any of its inline storage has the property. Your function should work fine.

31

I understand the usefulness of Borrow and Inout as types rather than declaration attributes, this just comes a little abruptly. We had 15 years of Inout being inexpressible in generics and a few years of borrowing being the same. I donā€™t know if I just wasnā€™t paying enough attention, but we had a while to talk about whether itā€™s future-proof to add these qualifiers and we just didnā€™t. Itā€™s hard for me to find the continuity.

What I would like to avoid above all is going into somebody elseā€™s code base to help them adopt safe Swift, tell them something like ā€œyeah itā€™s less nice but thatā€™s how you have to do it nowā€ and be unable to tell them when things actually become nice. If we are serious about inout becoming sugar for Inout and becoming usable in more places, we probably need more rules in this proposal, such as ā€œthe compiler does not support overloading on inout versus Inout" and a few other descendants of that.

If the path forward is auto-dereferencing, withUnsafePointer(borrow) will give you a UnsafePointer<Borrow<T>>, and there wonā€™t be a borrow.withUnsafePointer.

32

If you were to ask me, I think we'll always have both inout and Inout, as distinct things, since there will always be times you'll need to work with a reference as a value in its own right. An imperfect analogy might be to consider T & and T * in C++; references tend to be preferred in API design, but you still need pointers for when you want a reference that's rebindable in its own right. (And although it's ancient history now, there was a time when C++ had only pointers before references were added.)

2 Likes
33

I may have missed it in the proposal but is there a proposed a subtyping relation between Borrow and Inout?

34 
  mutating func at(index: Int) -> Inout<Double> {
    switch index {
    case 0: return Inout(&x)
    ...

Is there a scope for syntax sugar for this feature, like so:

  mutating func at(index: Int) -> inout Double {
    switch index {
    case 0: return &x
    ...

which might well use the proposed Inout behind the scenes (i.e. treat the two fragments equivalent)?

4 Likes
35

Maybe it all comes down to a bike shed color in the end: borrow and inout are bindings, but Borrow and Inout are respectively UnsafePointer and UnsafeMutablePointer except safe. Pitching them as safe pointers instead of type-system versions of bindings is more coherent to me.

36

Have you considered Borrowed<T> and Mutable<T> as the type names? We intentionally don't use adjectives as type names very often, but the existing precedents ā€” Optional and Unmanaged ā€” are very similar to this one: a standard library type introducing a pretty fundamental kind of modality over a single value.

26 Likes
37

Admittedly, I hadn't considered optionals before. But yeah, optional borrow/inout references are a particularly compelling case, particularly for a noncopyable dictionary type. Optionals are convenient because of their status as a fundamental currency type with lots of syntactic sugar; it'd be unfortunate to lose that convenience the moment one needs a different kind of ownership.

However, I think Optional<Borrow<T>> (and Optional<Inout<T>>) would still be inconvenient, contrary to the general design goals of optionals. The main issue being that Optional<Borrow<T>> would have a very different interface from borrowing Optional<T>, even though they're just different representations of the same high-level value.

For example, if we get borrowing/mutating optional bindings, like if borrow/if inout, it'd be nice for optional borrows to use the same syntax as borrowed optionals. But with Optional<Borrow<T>>, one would use a normal consuming optional binding, with the wrapped value behind a layer of indirection. Pattern matching would have a similar issue, with the extra difficulty of pattern matching through a computed property.

These inconsistencies could hurt usability, and could make it more difficult to switch between different representations during refactoring.

I think it would be better to have dedicated OptionalBorrow and OptionalInout types instead. The main benefit is that OptionalBorrow<T> would have a consistent interface with borrowing Optional<T>. It could have the same syntactic sugar, including optional binding and optional chaining. It could have type sugar, such as (borrowing T)?.

Ideally, it would be easy to convert between different representations, such as converting a borrowed optional to an optional borrow, converting an optional inout to an optional borrow, or (for copyable types) converting an optional borrow to an owned optional. In Rust, the Option type has different helper methods for this purpose, such as as_ref, as_mut, as_deref, and cloned. The helper methods are better than explicit pattern matching, but I think they still create an undesirable amount of cognitive load.

I think implicit conversions would avoid that cognitive load. Implicit conversions would probably be less confusing if the different representations have a consistent interface. In other words, it'd be less confusing to have an implicit conversion from borrowing Optional<T> to OptionalBorrow<T>, than to have an implicit conversion from borrowing Optional<T> to Optional<Borrow<T>>.

(Rust also has implicit conversions for different representations of the same high-level value, such as mutable-to-immutable reference coercion, Deref coercion, and unsized coercion. In both Rust and Swift, implicit conversions are reserved for special types and concepts; but unlike Rust, optionals are special in Swift.)

If Swift ever gets "ownership generics" in the future, maybe there could be a viable path to retrofit OptionalBorrow and OptionalInout (and MutableSpan), and APIs using these types. Retrofitting could be more difficult with Optional<Borrow<T>>, because it's already an Optional, but with a different interface than we might want.

typealias OptionalBorrow<T> = Optional<borrowing, T>
typealias OptionalInout<T> = Optional<inout, T>
typealias MutableSpan<T> = Span<inout, T>

Maybe it's worth explaining why I think "ownership generics" are a compelling enough future direction to warrant consideration. Basically, ownership generics would decouple being generic over ownership from being generic over types.

To make an analogy, Swift currently has two effects: throws and async. Technically, effects don't need to be built in to the language, because any effect could be expressed in the return type instead; for example, a throwing function could just be a function returning Result<T>.[1] A problem is that, by coupling effects to types, we lose the ability to be generic over an effect without also being generic over the return type. In contrast, the rethrows keyword (and a hypothetical reasync keyword) is a limited way for higher-order functions to be generic over effects. Rust doesn't have a similar feature, because it uses Result; a workaround is for higher-order functions (that don't just forward the return value) to have both throwing and non-throwing variants, such as reduce and try_reduce.[2]

Similarly, sometimes it's useful to be generic over ownership without also being generic over the return type. The most prominent example I can think of is properties and subscripts. Stored properties are automatically generic over ownership, and accessors are a limited way for computed properties and subscripts to be generic over ownership, where the ownership of the return value depends on the ownership of self. Rust doesn't have a similar feature; a workaround is for an "accessor function" to have a different variant for each kind of ownership, such as get_ref and get_mut.[3]

Generalized ownership generics could remove the need for collections to have both span and mutableSpan properties, remove the need for a noncopyable dictionary to have both "lookup and borrow" and "lookup and mutate" operations, and remove the need for optionals and collections to have both "borrowing map" and "consuming map" operations.

  1. In functional programming jargon, an unbound generic type that represents an effect, such as Result (or <T> => Result<T>), is called a "monad". ā†©ļøŽ

  2. Another solution, used in functional programming languages like Haskell, is "higher-rank polymorphism", where code can be generic over unbound generic types, so code that would otherwise be generic over effects is instead generic over monads. ā†©ļøŽ

  3. Maybe it's also possible to represent an ownership kind as an unbound generic type, similar to how an effect can be represented as a monad: borrowing is <T> => Borrow<T>, mutating/inout is <T> => Inout<T>, and consuming is <T> => T. ā†©ļøŽ

4 Likes
38

ā€¦ how about Pointer and MutablePointer, then? ;-)

12 Likes
39

Yes, please! :slight_smile:

They sound and feel more natural than even these:

1 Like
40

I have a question about this part in the proposal:

@_lifetime(&target)
func noisyCounterRef(from target: inout NoisyCounter) -> Inout<Int> {
  // ERROR, would extend the lifetime of `Inout` outside of the formal access
  return Inout(&target.value)
}

How would the compiler detect this convincingly? If I write it like below, will it be diagnosed?

@_lifetime(&value)
func helper(from value: inout Int) -> Inout<Int> {
    Inout(&value)  // This must be OK
}

@_lifetime(&target)
func noisyCounterRef(from target: inout NoisyCounter) -> Inout<Int> {
    return helper(&target.value)  // seems OK? because it does not directly invoke Inout.init
}
2 Likes