`borrow` and `take` parameter ownership modifiers

[Joe_Groff]

@dabrahams

The overall direction I see in this pitch and SE-0366 seems to reflect an effort to give the beginning programmer an illusion that the difference between borrow and take is not fundamental (despite the very real need for @escaping). IMO that will result in a more complex language overall, reflected in part by the spurious distinctions take vs. @escaping and borrow vs. ordinary pass-by-value, and in the use of call-site annotations in “expert” code.

For copyable types, there's really nothing about borrow vs. take that affects whether you can escape the value. That requires an additional constraint on either the value or the type to take the ability to copy a borrow away. @escaping as we use it on closures is kind of a misnomer, by the time you've got something of @escaping () -> () type, it's already escaped, and you're generally just borrowing a reference to the escapee.

@dabrahams

Question for Swift designers: is it useful for us to discuss Val's approach in these forums? I think we're onto something that could benefit Swift greatly, but we don't want to just be a distraction.

It might be good to discuss holistically in a new thread. It sounds like you have a lot of ideas and it's hard to get a big picture when they're dropped incrementally in various other threads.

[dabrahams (Dave Abrahams)]

@Joe_Groff

For copyable types, there's really nothing about borrow vs. take that affects whether you can escape the value. That requires an additional constraint on either the value or the type to take the ability to copy a borrow away. @escaping as we use it on closures is kind of a misnomer, by the time you've got something of @escaping () -> () type, it's already escaped, and you're generally just borrowing a reference to the escapee.

That's the point: it's functionally just like take. IIUC, you're encoding a distinction in the language that doesn't make any difference to the programmer.

[John_McCall]

It is true that non-escaping function values can be thought of as borrows of a non-copyable function type. Because Swift type-erases functions, however, we need to distinguish between copyable and non-copyable erased function types: merely passing a function a borrow of a copyable function type would not express what we need to express with non-escaping functions, and we cannot make all function types non-copyable. So @escaping does not go away in a world with explicit take.

[John_McCall]

The type-erasure isn't instrumental here, of course. If Swift didn't erase function types, you'd have to be generic over a function type, and you'd have to choose whether you constrained that to be copyable or not, which would be mostly orthogonal from whether it was passed with ownership.

[dabrahams (Dave Abrahams)]

@John_McCall

Because Swift type-erases functions, however, we need to distinguish between copyable and non-copyable erased function types

Expressing copyability directly in the language makes sense, but is that what @escaping means?

I must be misunderstanding something, so please correct me where I've gone wrong.

I'm not even sure I understand what kind of function types would be non-copyable in Swift. Even with non-copyable captures, function types have reference semantics, so why would they ever be inherently non-copyable? You can always copy a reference to a non-copyable type.

I can see how you might say we are eliminating copyability today, as an optimization, when the function is never used in an escaping context—except that today there are non non-copyable captures, so it's always possible to type-erase the function's copy semantics.

The only explanation I can imagine for the idea of non-copyable function types is that you want to avoid the allocation for non-escaping closures that capture non-copyable types. IMO because non-copyable types don't exist today it's reasonable to say that you only get to avoid that allocation when the closure doesn't actually escape (i.e. when it is never a take parameter).

[Joe_Groff]

Yeah, the interaction of borrowing and functions becomes pretty complex in full generality. In Rust, they don't erase closure types, but there's still a whole matrix of function traits based on how the invoke operation borrows or consumes the function, and that's still orthogonal to how the function value itself is passed to a callee, and how the captures of the closure are owned by the implicit closure type.

[John_McCall]

@dabrahams

I can see how you might say we are eliminating copyability today, as an optimization, when the function is never used in an escaping context—except that today there are non non-copyable captures, so it's always possible to type-erase the function's copy semantics.

The only explanation I can imagine for the idea of non-copyable function types is that you want to avoid the allocation for non-escaping closures that capture non-copyable types. IMO because non-copyable types don't exist today it's reasonable to say that you only get to avoid that allocation when the closure doesn't actually escape (i.e. when it is never a take parameter)

I'm a little confused by this post. Swift has had non-escaping function types since (I think) Swift 2, and one of the things they allow is that you can capture an inout parameter, which is not copyable because of the lifetime/exclusivity restriction.

@dabrahams

You can always copy a reference to a non-copyable type.

This sort of reference must be lifetime-restricted, limiting what you can do with them similarly to inout.

[dabrahams (Dave Abrahams)]

@Joe_Groff

Yeah, the interaction of borrowing and functions becomes pretty complex in full generality. In Rust, they don't erase closure types

Rust made some choices that make the landscape much more complicated than it needs to be, though. That complexity is not inherent to the problem.

@Joe_Groff

It might be good to discuss holistically in a new thread. It sounds like you have a lot of ideas and it's hard to get a big picture when they're dropped incrementally in various other threads.

Good idea; I think we could probably write something useful.

[Joe_Groff]

@dabrahams

Rust made some choices that make the landscape much more complicated than it needs to be, though. That complexity is not inherent to the problem.

I'd be interested to hear you elaborate on this.

[dabrahams (Dave Abrahams)]

@John_McCall

I'm a little confused by this post. Swift has had non-escaping function types since (I think) Swift 2, and one of the things they allow is that you can capture an inout parameter, which is not copyable because of the lifetime/exclusivity restriction.

Nothing to be confused about here; I just forgot that case existed in Swift.

I'm pretty sure there no reason you can't make inout parameters of copyable type copyable also, as long as you “fork” them (copy them back) when the call with the inout parameter ends. Isn't that what we did unconditionally before @escaping was added?

@dabrahams

You can always copy a reference to a non-copyable type.

But the result has to be lifetime-limited, and so the copies are restricted.

When I said "a reference" I meant an object reference backed by a dynamic allocation, which doesn't have the same lifetime limitations.

[John_McCall]

You definitely cannot construct that kind of object with a member that refers to a borrow. To make that safe, we would need to lifetime-restrict the whole object. There’s “no get out of jail” card here.

[dabrahams (Dave Abrahams)]

@John_McCall

You definitely cannot construct that kind of object with a member that refers to a borrow. To make that safe, we would need to lifetime-restrict the whole object. There’s “no get out of jail” card here.

Yes, of course you can't construct a reference backed by a dynamic allocation to a noncopyable borrowed instance, but those don't exist today. It's important to be clear about whethere we're talking about current swift or future swift.

[John_McCall]

Well, we have non-escaping function types, so we do actually have that restriction in the language for certain types. I agree that adding user-definable non-copyable types does significantly increase the impact, though.

[Joe_Groff]

It is also potentially useful to combine move/take/transfer of ownership with a non-escaping restriction, to be able to model values that can be owned and transferred independently but have references into a parent that must remain valid at least as long as the child value is alive. So taking such a child value lets you transfer ownership further "down" or "sideways" in the call chain, but not let it escape "up". Examples would be things like C++-style iterators that point directly into their parent collection, or the safe BufferView type we've sketched out as an alternative to UnsafeBufferPointer for low-overhead slicing of contiguous memory. Rust models such things using lifetime type variables, but another way to cut it might be to be able to mark owned values as nonescaping too.

[dabrahams (Dave Abrahams)]

@Joe_Groff

It is also potentially useful to combine move/take/transfer of ownership with a non-escaping restriction, to be able to model values that can be owned and transferred independently but have references into a parent that must remain valid at least as long as the child value is alive. So taking such a child value lets you transfer ownership further "down" or "sideways" in the call chain, but not let it escape "up".

That's exactly what Val's projections do; it allows us to express many of the same things as Rust's named lifetimes without Rust's
complexity.

[Joe_Groff]

@dabrahams

That's exactly what Val's projections do; it allows us to express many of the same things as Rust's named lifetimes without Rust's
complexity.

Is a projection a special kind of type, or another ownership-transfer modifier? It also seems like it could be handled as a nonescaping take, if you allow for "escapable" and "borrowed" to be applied independently.

[dabrahams (Dave Abrahams)]

It's neither a type nor an ownership transfer modifier. A projection is a subscript or property access; it's built upon the ideas @John_McCall came up with for _modify and _read accessors; we think we just completed the calculus. When you project out of an instance (other than via a sink projection, which consumes the source), the result is a borrow that is non-escaping (unsinkable) and lifetime-dominated by the source instance.

[Joe_Groff]

Borrowing only enforces nonescapingness as long as you also can't copy the borrowed value (including not having any operations on the borrowed value at hand that can indirectly copy it), or alternatively, if copies transitively pick up the same nonescaping constraint.

[dabrahams (Dave Abrahams)]

@Joe_Groff

Borrowing only enforces nonescapingness as long as you also can't copy the borrowed value (including not having any operations on the borrowed value at hand that can indirectly copy it), or alternatively, if copies transitively pick up the same nonescaping constraint.

Sorry, I'm not able to follow you here. To start with, I'm missing context: I can't tell whether you're making a general statement, a statement about Swift today, or one about Swift in the future. I also can't understand the parenthesized remark. Care to clarify?

[Joe_Groff]

I think it's a general property that, if you have a copy operation that produces an identical value with independent ownership, then that can break the nonescaping property of a borrow. You're right that borrowing an uncopyable value keeps the value being borrowed from outliving its alloted lifetime:

func withInteriorPointer(of: borrow Thingie, body: (borrow InteriorPointer) -> ()) {...}

var pointerEscapeHatch: InteriorPointer?
withInteriorPointer(of: someThingie) { @noCopy pointer in
  pointerEscapeHatch = pointer // can't do this without copying!
}

but if you have that copy operation available explicitly, then you can still escape the value:

withInteriorPointer(of: someThingie) { pointer in
  pointerEscapeHatch = pointer.copy()
}

If there are any functions anywhere that can copy the value then they can escape the value too:

func sneakyEscape(_ p: borrow InteriorPointer) {
  pointerEscapeHatch = p.copy()
}

withInteriorPointer(of: someThingie) { pointer in
  sneakyEscape(pointer)
}

For a specially-crafted type that's meant to be a view into another value, you can make it non-copyable, and be disciplined about providing no API that surfaces owned values to users that they may be able to move around freely, but it only takes one slip-up to break that constraint. And if you want to maintain the nonescaping constraint generically, there isn't a good way to do so, since copyable types can conform to protocols that don't require copyability, and use copying in their implementation:

func withGenericallyInteriorPointer<T: InteriorPointable>
  (of: borrow Thingie, body: (borrow T) -> ()) {...}

func doStuffToGenericallyInteriorPointer<T: InteriorPointable>
  (of thingy: borrow Thingie) {

  withGenericallyInteriorPointer(of: thingy) { pointer in
    pointer.doPointyStuff()
  }
}

struct SneakyEscapingPointer: Copyable, InteriorPointable {
  func doPointyStuff() {
    pointerEscapeHatch = self.copy()
  }
}

so having a transitive "nonescaping" constraint that applies not only to the original borrowed values, but any copies it may produce, seems valuable.