[Proposal] Add an API to Unmanaged to get the instance it holds "@guaranteed"


(Arnold Schwaighofer) #1

Hi,

I would like to propose adding an additional method to the standard library’s ‘Unmanaged’ wrapper to mark values as having a guaranteed lifetime by another reference to it.

Currently, there is no way to opt out of ARC even if we know the lifetime of a value is guaranteed by another reference to it. We can pass around values in the wrapper without incurring reference counting but as soon as we take the instance out of the wrapper we will incur reference counting.

func doSomething(u: Unmanaged<Owned>) {
    // Incurs refcount increment before the call and decrement after for self.
    u.takeUnretainedValue().doSomeWork()

The addition of this API will allow selectively disabling ARC for the scope of the lifetime of the value returned by this API call.

  func doSomething(u: Unmanaged<Owned>) {
    // Incurs refcount increment before the call and decrement after for self
    // that can be removed by the compiler based on the assertion made.
    u.takeGuaranteedValue().doSomeWork()

Add an API to Unmanaged to get the instance it holds "@guaranteed"

Proposal: SE-0000 Add an API to Unmanaged to get the instance it holds guaranteed <https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md>
Author(s): Arnold Schwaighofer <https://github.com/aschwaighofer>
Status: Awaiting Review
Review manager: TBD
<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#introduction>Introduction

The standard library Unmanged<Instance> struct provides an instance wrapper that does not participate in ARC; it allows the user to make manual retain/release calls and get at the contained instance at balanced/unbalanced retain counts.

This proposal suggests to add another method to Unmanaged that will allow making an assertion about the guaranteed lifetime of the instance returned.

This assertion will help the compiler remove ARC operations.

public struct Unmanaged<Instance : AnyObject> {

  // Get the value of the unmanaged referenced as a managed reference without
  // consuming an unbalanced retain of it. Asserts that some other owner
  // guarantees the lifetime of the value for the lifetime of the return managed
  // reference. You are responsible for making sure that this assertion holds
  // true.
  //
  // NOTE:
  // Be aware that storing the returned managed reference in an L-Value extends
  // the lifetime this assertion must hold true for by the lifetime of the
  // L-Value.
  // var owningReference = Instance()
  // var lValue : Instance
  // withFixedLifetime(owningReference) {
  // lValue = Unmanaged.passUnretained(owningReference).takeGuaranteedValue()
  // }
  // lValue.doSomething() // Bug: owningReference lifetime has ended earlier.

  public func takeGuaranteedValue() -> Instance {
    let result = _value
    return Builtin.unsafeGuaranteed(result)
  }
Prototype: link to a prototype implementation <https://github.com/aschwaighofer/swift/tree/unsafe_guaranteed_prototype>
<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#motivation>Motivation

A user can often make assertions that an instance is kept alive by another reference to it for the duration of some scope.

Consider the following example:

class Owned {
  func doSomeWork() {}
}

public class Owner {
  var ref: Owned

  init() { ref = Owned() }

  public func doWork() {
    doSomething(ref)
  }

  func doSomething(o: Owned) {
     o.doSomeWork()
  }
}
In this context the lifetime of Owner always exceeds o: Ownee. However, there is currently no way to tell the compiler about such an assertion. We can pass reference counted values in an Unmanged container without incurring reference count changes.

public class Owner {

  public func doWork() {
    // No reference count for passing the parameter.
    doSomething(Unmanaged.passUnretained(ref))
  }

  func doSomething(u: Unmanaged<Owned>) {
    // ...
  }
}
We can get at the contained instance by means of takeUnretainedValue() which will return a balanced retained value: the value is returned at +1 for release by the caller. However, when it comes to accessing the contained instance we incur reference counting.

  func doSomething(u: Unmanaged<Owned>) {
    // Incurs refcount increment before the call and decrement after for self.
    u.takeUnretainedValue().doSomeWork()
  }
With the proposed API call the user could make the assertion that u's contained instance's lifetime is guaranteed by another reference to it.

  func doSomething(u: Unmanaged<Owned>) {
    // Incurs refcount increment before the call and decrement after for self
    // that can be removed by the compiler based on the assertion made.
    u.takeGuaranteedValue().doSomeWork()
  }
The compiler can easily remove the reference counts marked by this method call with local reasoning.

<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#impact-on-existing-code>Impact on existing code

This is a new API that does not replace an existing method. No existing code should be affected.

<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#alternatives-considered>Alternatives considered

A somewhat related proposal would be to allow for class types to completely opt out of ARC forcing the programmer to perform manual reference counting. The scope of such a proposal would be much bigger making it questionable for Swift 3 inclusion. I believe that the two approaches are complementary and don't completely supplant each other. This proposal's approach allows for selectively opting out of ARC while providing a high notational burdon when wide spread over the application (wrapping and unwrapping of Unmanaged). Opting completely out of ARC is an all-in solution, which on the other hand does not have the high notational burden.


(John McCall) #2

Calling this method makes a semantic guarantee that the returned reference is being kept alive somehow for a certain period of time. So, the obvious questions here are:

1. What are the bounds of that period of time?
2. How can the user actually satisfy this guarantee?
3. What happens if they’re wrong?

#1 is totally unclear in your proposal. Usually we do this sort of scoping thing with a callback, rather than a property, for exactly the reason that it lets us naturally bound the extent of the guarantee. The library’s implementation of that function would then use some builtin functions to bound how long the guarantee lasts. If we decide that that’s too awkward to actually use, we could burn this into the language as a custom result convention and invent some language rules for the guaranteed extents of such a result based on how the expression is used; that would permit more APIs to opt in to a +0 guaranteed result, but the language rules would probably be pretty ad hoc.

#2 is also uncertain. In ObjC ARC, we have a concept of an “object with precise lifetime semantics”. (Here, “object” is used in the C/C++ sense of “variable”, not the reference-type-OO sense of “class instance”.) Globals always have precise lifetime semantics, locals can opt in with an attribute, and members depend on their container. So one possible answer is to say that the value must be precisely referenced somehow, perhaps by the system. That does require doing some leg work in the proposal (to import the concept of precise lifetime), the language (to allow locals to be made precise), and and the compiler/optimizer (to honor precise guarantees).

It sounds like the only reasonable answer for #3 is “it’s undefined behavior”. That argues for putting “unsafe” somewhere in the name, although perhaps that’s already covered by the proposal to rename Unmanaged to UnsafeReference.

John.

···

On Mar 14, 2016, at 8:00 AM, Arnold Schwaighofer <aschwaighofer@apple.com> wrote:
I would like to propose adding an additional method to the standard library’s ‘Unmanaged’ wrapper to mark values as having a guaranteed lifetime by another reference to it.

Currently, there is no way to opt out of ARC even if we know the lifetime of a value is guaranteed by another reference to it. We can pass around values in the wrapper without incurring reference counting but as soon as we take the instance out of the wrapper we will incur reference counting.

func doSomething(u: Unmanaged<Owned>) {
    // Incurs refcount increment before the call and decrement after for self.
    u.takeUnretainedValue().doSomeWork()

The addition of this API will allow selectively disabling ARC for the scope of the lifetime of the value returned by this API call.


(Arnold Schwaighofer) #3

I would like to propose adding an additional method to the standard library’s ‘Unmanaged’ wrapper to mark values as having a guaranteed lifetime by another reference to it.

Currently, there is no way to opt out of ARC even if we know the lifetime of a value is guaranteed by another reference to it. We can pass around values in the wrapper without incurring reference counting but as soon as we take the instance out of the wrapper we will incur reference counting.

func doSomething(u: Unmanaged<Owned>
) {
    
// Incurs refcount increment before the call and decrement after for self.

    u
.takeUnretainedValue().doSomeWork()

The addition of this API will allow selectively disabling ARC for the scope of the lifetime of the value returned by this API call.

Calling this method makes a semantic guarantee that the returned reference is being kept alive somehow for a certain period of time. So, the obvious questions here are:

1. What are the bounds of that period of time?

I mention in the API doc that this is for the duration of lifetime the returned value.

  var x = takeUnretainedValue()
  var y = x

In this example this would be for the lifetime for the value in x which extends to the lifetime of y.

I will try to address your comments on needing more precise language here (precise lifetime), i.e for locals we need to fix the lifetime, classes, globals.

2. How can the user actually satisfy this guarantee?

By “fixing the lifetime” of the value is one way or by relying on lifetime relation ships, see the class example I give. The fact that we are calling a method on "self: Owner" guarantees that the reference in “ref: Ownee” was not ultimately released (Admittedly, this is relying on self being passed @guaranteed). The user must know that nobody will store to “ref” in-between.

Your point is that this needs clear language.

"
// Get the value of the unmanaged referenced as a managed reference without
// consuming an unbalanced retain of it. Asserts that some other owner
// guarantees the lifetime of the value for the lifetime of the return managed
// reference. You are responsible for making sure that this assertion holds
// true.
//
// NOTE:
// Be aware that storing the returned managed reference in an L-Value extends
// the lifetime this assertion must hold true for by the lifetime of the
// L-Value.
// var owningReference = Instance()
// var lValue : Instance
// withFixedLifetime(owningReference) {
// lValue = Unmanaged.passUnretained(owningReference).takeGuaranteedValue()
// }
// lValue.doSomething() // Bug: owningReference lifetime has ended earlier.
"

3. What happens if they’re wrong?

Undefined behavior. This is an unsafe extension. Users can shoot themselves in the foot today using Unmanaged by calling release() on it.

#1 is totally unclear in your proposal. Usually we do this sort of scoping thing with a callback, rather than a property, for exactly the reason that it lets us naturally bound the extent of the guarantee. The library’s implementation of that function would then use some builtin functions to bound how long the guarantee lasts. If we decide that that’s too awkward to actually use, we could burn this into the language as a custom result convention and invent some language rules for the guaranteed extents of such a result based on how the expression is used; that would permit more APIs to opt in to a +0 guaranteed result, but the language rules would probably be pretty ad hoc.

I am not sure you can bound it this that way. I think we want this assertion to be propagated through l-values.

var outlive: Ownee
var owningReference: Ownee

withFixedLifetime(owningReference) {
  Unamanged.passUnretained(owningReference).withGuaranteedValue {
     escape($0, to: &outlive)
}

I think we should just define this as undefined behavior.
  

#2 is also uncertain. In ObjC ARC, we have a concept of an “object with precise lifetime semantics”. (Here, “object” is used in the C/C++ sense of “variable”, not the reference-type-OO sense of “class instance”.) Globals always have precise lifetime semantics, locals can opt in with an attribute, and members depend on their container. So one possible answer is to say that the value must be precisely referenced somehow, perhaps by the system. That does require doing some leg work in the proposal (to import the concept of precise lifetime), the language (to allow locals to be made precise), and and the compiler/optimizer (to honor precise guarantees).

We have the withFixedLifetime API in the standard library today.

withFixedLifetime(ownningReference) {
}

Users can rely on that a stored reference in a class is not ultimately released during a method call on that class instance if they can guaranteed that nobody stores to that reference.

The compiler honor withFixedLifetime today (if not that is a bug).

Your point is that this needs to be made clear.

It sounds like the only reasonable answer for #3 is “it’s undefined behavior”. That argues for putting “unsafe” somewhere in the name, although perhaps that’s already covered by the proposal to rename Unmanaged to UnsafeReference.

I believe, Unmanaged is already unsafe. I have put unsafe in the name of the SIL builtin, I am happy to propagate this to the API call.

···

On Mar 14, 2016, at 11:53 AM, John McCall <rjmccall@apple.com> wrote:

On Mar 14, 2016, at 8:00 AM, Arnold Schwaighofer <aschwaighofer@apple.com> wrote:

John.


(John McCall) #4

I would like to propose adding an additional method to the standard library’s ‘Unmanaged’ wrapper to mark values as having a guaranteed lifetime by another reference to it.

Currently, there is no way to opt out of ARC even if we know the lifetime of a value is guaranteed by another reference to it. We can pass around values in the wrapper without incurring reference counting but as soon as we take the instance out of the wrapper we will incur reference counting.

func doSomething(u: Unmanaged<Owned>
) {

// Incurs refcount increment before the call and decrement after for self.

   u
.takeUnretainedValue().doSomeWork()

The addition of this API will allow selectively disabling ARC for the scope of the lifetime of the value returned by this API call.

Calling this method makes a semantic guarantee that the returned reference is being kept alive somehow for a certain period of time. So, the obvious questions here are:

1. What are the bounds of that period of time?

I mention in the API doc that this is for the duration of lifetime the returned value.

var x = takeUnretainedValue()
var y = x

In this example this would be for the lifetime for the value in x which extends to the lifetime of y.

What if I return it or assign it to non-local memory?

I feel like you’re trying to define this by optimizer behavior. That’s not a workable language rule; programmers are not going to reason about SSA values.

I will try to address your comments on needing more precise language here (precise lifetime), i.e for locals we need to fix the lifetime, classes, globals.

2. How can the user actually satisfy this guarantee?

By “fixing the lifetime” of the value is one way or by relying on lifetime relation ships, see the class example I give. The fact that we are calling a method on "self: Owner" guarantees that the reference in “ref: Ownee” was not ultimately released (Admittedly, this is relying on self being passed @guaranteed). The user must know that nobody will store to “ref” in-between.

Your point is that this needs clear language.

"
// Get the value of the unmanaged referenced as a managed reference without
// consuming an unbalanced retain of it. Asserts that some other owner
// guarantees the lifetime of the value for the lifetime of the return managed
// reference. You are responsible for making sure that this assertion holds
// true.
//
// NOTE:
// Be aware that storing the returned managed reference in an L-Value extends
// the lifetime this assertion must hold true for by the lifetime of the
// L-Value.
// var owningReference = Instance()
// var lValue : Instance
// withFixedLifetime(owningReference) {
// lValue = Unmanaged.passUnretained(owningReference).takeGuaranteedValue()
// }
// lValue.doSomething() // Bug: owningReference lifetime has ended earlier.
"

3. What happens if they’re wrong?

Undefined behavior. This is an unsafe extension. Users can shoot themselves in the foot today using Unmanaged by calling release() on it.

#1 is totally unclear in your proposal. Usually we do this sort of scoping thing with a callback, rather than a property, for exactly the reason that it lets us naturally bound the extent of the guarantee. The library’s implementation of that function would then use some builtin functions to bound how long the guarantee lasts. If we decide that that’s too awkward to actually use, we could burn this into the language as a custom result convention and invent some language rules for the guaranteed extents of such a result based on how the expression is used; that would permit more APIs to opt in to a +0 guaranteed result, but the language rules would probably be pretty ad hoc.

I am not sure you can bound it this that way. I think we want this assertion to be propagated through l-values.

var outlive: Ownee
var owningReference: Ownee

withFixedLifetime(owningReference) {
Unamanged.passUnretained(owningReference).withGuaranteedValue {
    escape($0, to: &outlive)
}

I think we should just define this as undefined behavior.

Why? It doesn’t seem harmful. It’s just that the guarantee doesn’t apply.

…to be clear, I certainly hope that the intended optimization strategy here isn’t just to unconditionally remove any retains and releases visible within the guaranteed block. You can still at best remove them in pairs. The effect on the optimizer here is just that uses that are known to occur within a guaranteed block can be ignored for the purposes of eliminating imprecise retain/release pairs.

#2 is also uncertain. In ObjC ARC, we have a concept of an “object with precise lifetime semantics”. (Here, “object” is used in the C/C++ sense of “variable”, not the reference-type-OO sense of “class instance”.) Globals always have precise lifetime semantics, locals can opt in with an attribute, and members depend on their container. So one possible answer is to say that the value must be precisely referenced somehow, perhaps by the system. That does require doing some leg work in the proposal (to import the concept of precise lifetime), the language (to allow locals to be made precise), and and the compiler/optimizer (to honor precise guarantees).

We have the withFixedLifetime API in the standard library today.

withFixedLifetime(ownningReference) {
}

Users can rely on that a stored reference in a class is not ultimately released during a method call on that class instance if they can guaranteed that nobody stores to that reference.

The compiler honor withFixedLifetime today (if not that is a bug).

Your point is that this needs to be made clear.

Yeah, I think withFixedLifetime is probably not a good enough tool; at the very least, you need to reason about indirect references. (e.g. if the object is held by an array element or a class property, then the array/instance itself needs to be held precisely, etc.) It’s probably better for local values than a precise-lifetime attribute on local variables, though, assuming withFixedLifetime works for value types, and assuming it doesn’t actually cause reference-counting traffic itself.

It sounds like the only reasonable answer for #3 is “it’s undefined behavior”. That argues for putting “unsafe” somewhere in the name, although perhaps that’s already covered by the proposal to rename Unmanaged to UnsafeReference.

I believe, Unmanaged is already unsafe. I have put unsafe in the name of the SIL builtin, I am happy to propagate this to the API call.

Unmanaged is already unsafe. Like I said, there’s an extant proposal to rename it to something like UnsafeReference. I was raising this issue to see whether swift-evolution has a sense of whether that’s good enough or whether we should call attention to the additional unsafely of this API.

John.

···

On Mar 14, 2016, at 12:51 PM, Arnold Schwaighofer <aschwaighofer@apple.com> wrote:

On Mar 14, 2016, at 11:53 AM, John McCall <rjmccall@apple.com> wrote:

On Mar 14, 2016, at 8:00 AM, Arnold Schwaighofer <aschwaighofer@apple.com> wrote:


(Arnold Schwaighofer) #5

I would like to propose adding an additional method to the standard library’s ‘Unmanaged’ wrapper to mark values as having a guaranteed lifetime by another reference to it.

Currently, there is no way to opt out of ARC even if we know the lifetime of a value is guaranteed by another reference to it. We can pass around values in the wrapper without incurring reference counting but as soon as we take the instance out of the wrapper we will incur reference counting.

func doSomething(u: Unmanaged<Owned>
) {

// Incurs refcount increment before the call and decrement after for self.

  u
.takeUnretainedValue().doSomeWork()

The addition of this API will allow selectively disabling ARC for the scope of the lifetime of the value returned by this API call.

Calling this method makes a semantic guarantee that the returned reference is being kept alive somehow for a certain period of time. So, the obvious questions here are:

1. What are the bounds of that period of time?

I mention in the API doc that this is for the duration of lifetime the returned value.

var x = takeUnretainedValue()
var y = x

In this example this would be for the lifetime for the value in x which extends to the lifetime of y.

What if I return it or assign it to non-local memory?

See my comment about the l-value.

I feel like you’re trying to define this by optimizer behavior. That’s not a workable language rule; programmers are not going to reason about SSA values.

I did mentioned that this applies to storing the value in an l-value.

But I think you are saying to define this in terms of a “precise lifetime language”. I will try to talk in person to flesh out the language.

I will try to address your comments on needing more precise language here (precise lifetime), i.e for locals we need to fix the lifetime, classes, globals.

2. How can the user actually satisfy this guarantee?

By “fixing the lifetime” of the value is one way or by relying on lifetime relation ships, see the class example I give. The fact that we are calling a method on "self: Owner" guarantees that the reference in “ref: Ownee” was not ultimately released (Admittedly, this is relying on self being passed @guaranteed). The user must know that nobody will store to “ref” in-between.

Your point is that this needs clear language.

"
// Get the value of the unmanaged referenced as a managed reference without
// consuming an unbalanced retain of it. Asserts that some other owner
// guarantees the lifetime of the value for the lifetime of the return managed
// reference. You are responsible for making sure that this assertion holds
// true.
//

Comment about the l-value:

// NOTE:
// Be aware that storing the returned managed reference in an L-Value extends
// the lifetime this assertion must hold true for by the lifetime of the
// L-Value.
// var owningReference = Instance()
// var lValue : Instance
// withFixedLifetime(owningReference) {
// lValue = Unmanaged.passUnretained(owningReference).takeGuaranteedValue()
// }
// lValue.doSomething() // Bug: owningReference lifetime has ended earlier.
"

3. What happens if they’re wrong?

Undefined behavior. This is an unsafe extension. Users can shoot themselves in the foot today using Unmanaged by calling release() on it.

#1 is totally unclear in your proposal. Usually we do this sort of scoping thing with a callback, rather than a property, for exactly the reason that it lets us naturally bound the extent of the guarantee. The library’s implementation of that function would then use some builtin functions to bound how long the guarantee lasts. If we decide that that’s too awkward to actually use, we could burn this into the language as a custom result convention and invent some language rules for the guaranteed extents of such a result based on how the expression is used; that would permit more APIs to opt in to a +0 guaranteed result, but the language rules would probably be pretty ad hoc.

I am not sure you can bound it this that way. I think we want this assertion to be propagated through l-values.

var outlive: Ownee
var owningReference: Ownee

withFixedLifetime(owningReference) {
Unamanged.passUnretained(owningReference).withGuaranteedValue {
   escape($0, to: &outlive)
}

I think we should just define this as undefined behavior.

Why? It doesn’t seem harmful. It’s just that the guarantee doesn’t apply.

Yes the guarantee does not apply, the user has escaped “the returned value to an l-value that outlives the “owningReference”.

…to be clear, I certainly hope that the intended optimization strategy here isn’t just to unconditionally remove any retains and releases visible within the guaranteed block. You can still at best remove them in pairs. The effect on the optimizer here is just that uses that are known to occur within a guaranteed block can be ignored for the purposes of eliminating imprecise retain/release pairs.

Yes, only paired removal. Just like we can do with @guaranteed parameters today.

But imagine the “escape” function has been inlined. The optimizer will see:

Unamanged.passUnretained(owningReference).withGuaranteedValue {
   outlive = $0
}

This is not different to

Unamanged.passUnretained(owningReference).withGuaranteedValue {
   let x = $0
   x.doSomething()
}

And we want to optimize the latter.

The closure syntax does not add any guarantee.

#2 is also uncertain. In ObjC ARC, we have a concept of an “object with precise lifetime semantics”. (Here, “object” is used in the C/C++ sense of “variable”, not the reference-type-OO sense of “class instance”.) Globals always have precise lifetime semantics, locals can opt in with an attribute, and members depend on their container. So one possible answer is to say that the value must be precisely referenced somehow, perhaps by the system. That does require doing some leg work in the proposal (to import the concept of precise lifetime), the language (to allow locals to be made precise), and and the compiler/optimizer (to honor precise guarantees).

We have the withFixedLifetime API in the standard library today.

withFixedLifetime(ownningReference) {
}

Users can rely on that a stored reference in a class is not ultimately released during a method call on that class instance if they can guaranteed that nobody stores to that reference.

The compiler honor withFixedLifetime today (if not that is a bug).

Your point is that this needs to be made clear.

Yeah, I think withFixedLifetime is probably not a good enough tool; at the very least, you need to reason about indirect references. (e.g. if the object is held by an array element or a class property, then the array/instance itself needs to be held precisely, etc.) It’s probably better for local values than a precise-lifetime attribute on local variables, though, assuming withFixedLifetime works for value types, and assuming it doesn’t actually cause reference-counting traffic itself.

withFixedLifetime just inserts the SIL builtin fix_lifetime (assuming optimization happens).

fix_lifetime
````````````
                                                                   
Acts as a use of a value operand, or of the value in memory referenced by an
address operand. Optimizations may not move operations that would destroy the
value, such as ``release_value``, ``strong_release``, ``copy_addr [take]``, or
``destroy_addr``, past this instruction.

This guarantee is strong enough to guarantee the “owningReference” lifetime.

It sounds like the only reasonable answer for #3 is “it’s undefined behavior”. That argues for putting “unsafe” somewhere in the name, although perhaps that’s already covered by the proposal to rename Unmanaged to UnsafeReference.

I believe, Unmanaged is already unsafe. I have put unsafe in the name of the SIL builtin, I am happy to propagate this to the API call.

Unmanaged is already unsafe. Like I said, there’s an extant proposal to rename it to something like UnsafeReference. I was raising this issue to see whether swift-evolution has a sense of whether that’s good enough or whether we should call attention to the additional unsafely of this API.

Ok.

···

On Mar 14, 2016, at 1:29 PM, John McCall <rjmccall@apple.com> wrote:

On Mar 14, 2016, at 12:51 PM, Arnold Schwaighofer <aschwaighofer@apple.com> wrote:

On Mar 14, 2016, at 11:53 AM, John McCall <rjmccall@apple.com> wrote:

On Mar 14, 2016, at 8:00 AM, Arnold Schwaighofer <aschwaighofer@apple.com> wrote:


(Andrew Trick) #6

The compiler should not be allowed to extend any object lifetimes past a fixed-lifetime marker.

I think it is necessary for the user of the proposed unsafe API to provide a fixed lifetime scope on an owning object, which may have levels of indirection. It’s also necessary for the user to avoid mutating any references that participate in that ownership chain within the fixed lifetime scope.

I don’t think this is an API that we want to promote. In the long run, I imagine a safe/automatic approach (a stored property can be obtained @guaranteed if it’s parent is transitively @guaranteed and is immutable). But that requires some language support and meanwhile we need a backdoor for bypassing ARC in special circumstances.

Andy

···

On Mar 14, 2016, at 1:29 PM, John McCall via swift-evolution <swift-evolution@swift.org> wrote:

We have the withFixedLifetime API in the standard library today.

withFixedLifetime(ownningReference) {
}

Users can rely on that a stored reference in a class is not ultimately released during a method call on that class instance if they can guaranteed that nobody stores to that reference.

The compiler honor withFixedLifetime today (if not that is a bug).

Your point is that this needs to be made clear.

Yeah, I think withFixedLifetime is probably not a good enough tool; at the very least, you need to reason about indirect references. (e.g. if the object is held by an array element or a class property, then the array/instance itself needs to be held precisely, etc.) It’s probably better for local values than a precise-lifetime attribute on local variables, though, assuming withFixedLifetime works for value types, and assuming it doesn’t actually cause reference-counting traffic itself.


(Dave Abrahams) #7

Thank you, John. That is the problem I usually have with evaluating
most optimizer proposals.

···

on Mon Mar 14 2016, John McCall <rjmccall-AT-apple.com> wrote:

In this example this would be for the lifetime for the value in x which extends to the lifetime of y.

What if I return it or assign it to non-local memory?

I feel like you’re trying to define this by optimizer behavior.
That’s not a workable language rule; programmers are not going to
reason about SSA values.

--
-Dave


(Arnold Schwaighofer) #8

Let me retract this, I think I understand how you see the semantics of withGuaranteedValue:

  tmp = Builtin.unsafeGuaranteed(_unmanaged_ref) returns a managed reference
  closure(tmp) // consumes a managed reference
  Builtin.endUnsafeGuaranteed(tmp) // ends the lifetime for which the guarantee must hold.

I agree that would make the awkward language of transitively having to guarantee the lifetime through l-values unnecessary.

···

On Mar 14, 2016, at 3:08 PM, Arnold Schwaighofer via swift-evolution <swift-evolution@swift.org> wrote:

On Mar 14, 2016, at 1:29 PM, John McCall <rjmccall@apple.com> wrote:

On Mar 14, 2016, at 12:51 PM, Arnold Schwaighofer <aschwaighofer@apple.com> wrote:

On Mar 14, 2016, at 11:53 AM, John McCall <rjmccall@apple.com> wrote:

On Mar 14, 2016, at 8:00 AM, Arnold Schwaighofer <aschwaighofer@apple.com> wrote:
I would like to propose adding an additional method to the standard library’s ‘Unmanaged’ wrapper to mark values as having a guaranteed lifetime by another reference to it.

Currently, there is no way to opt out of ARC even if we know the lifetime of a value is guaranteed by another reference to it. We can pass around values in the wrapper without incurring reference counting but as soon as we take the instance out of the wrapper we will incur reference counting.

func doSomething(u: Unmanaged<Owned>
) {

// Incurs refcount increment before the call and decrement after for self.

u
.takeUnretainedValue().doSomeWork()

The addition of this API will allow selectively disabling ARC for the scope of the lifetime of the value returned by this API call.

Calling this method makes a semantic guarantee that the returned reference is being kept alive somehow for a certain period of time. So, the obvious questions here are:

1. What are the bounds of that period of time?

I mention in the API doc that this is for the duration of lifetime the returned value.

var x = takeUnretainedValue()
var y = x

In this example this would be for the lifetime for the value in x which extends to the lifetime of y.

What if I return it or assign it to non-local memory?

See my comment about the l-value.

I feel like you’re trying to define this by optimizer behavior. That’s not a workable language rule; programmers are not going to reason about SSA values.

I did mentioned that this applies to storing the value in an l-value.

But I think you are saying to define this in terms of a “precise lifetime language”. I will try to talk in person to flesh out the language.

I will try to address your comments on needing more precise language here (precise lifetime), i.e for locals we need to fix the lifetime, classes, globals.

2. How can the user actually satisfy this guarantee?

By “fixing the lifetime” of the value is one way or by relying on lifetime relation ships, see the class example I give. The fact that we are calling a method on "self: Owner" guarantees that the reference in “ref: Ownee” was not ultimately released (Admittedly, this is relying on self being passed @guaranteed). The user must know that nobody will store to “ref” in-between.

Your point is that this needs clear language.

"
// Get the value of the unmanaged referenced as a managed reference without
// consuming an unbalanced retain of it. Asserts that some other owner
// guarantees the lifetime of the value for the lifetime of the return managed
// reference. You are responsible for making sure that this assertion holds
// true.
//

Comment about the l-value:

// NOTE:
// Be aware that storing the returned managed reference in an L-Value extends
// the lifetime this assertion must hold true for by the lifetime of the
// L-Value.
// var owningReference = Instance()
// var lValue : Instance
// withFixedLifetime(owningReference) {
// lValue = Unmanaged.passUnretained(owningReference).takeGuaranteedValue()
// }
// lValue.doSomething() // Bug: owningReference lifetime has ended earlier.
"

3. What happens if they’re wrong?

Undefined behavior. This is an unsafe extension. Users can shoot themselves in the foot today using Unmanaged by calling release() on it.

#1 is totally unclear in your proposal. Usually we do this sort of scoping thing with a callback, rather than a property, for exactly the reason that it lets us naturally bound the extent of the guarantee. The library’s implementation of that function would then use some builtin functions to bound how long the guarantee lasts. If we decide that that’s too awkward to actually use, we could burn this into the language as a custom result convention and invent some language rules for the guaranteed extents of such a result based on how the expression is used; that would permit more APIs to opt in to a +0 guaranteed result, but the language rules would probably be pretty ad hoc.

I am not sure you can bound it this that way. I think we want this assertion to be propagated through l-values.

var outlive: Ownee
var owningReference: Ownee

withFixedLifetime(owningReference) {
Unamanged.passUnretained(owningReference).withGuaranteedValue {
  escape($0, to: &outlive)
}

I think we should just define this as undefined behavior.

Why? It doesn’t seem harmful. It’s just that the guarantee doesn’t apply.

Yes the guarantee does not apply, the user has escaped “the returned value to an l-value that outlives the “owningReference”.

…to be clear, I certainly hope that the intended optimization strategy here isn’t just to unconditionally remove any retains and releases visible within the guaranteed block. You can still at best remove them in pairs. The effect on the optimizer here is just that uses that are known to occur within a guaranteed block can be ignored for the purposes of eliminating imprecise retain/release pairs.

Yes, only paired removal. Just like we can do with @guaranteed parameters today.

But imagine the “escape” function has been inlined. The optimizer will see:

Unamanged.passUnretained(owningReference).withGuaranteedValue {
  outlive = $0
}

This is not different to

Unamanged.passUnretained(owningReference).withGuaranteedValue {
  let x = $0
  x.doSomething()
}

And we want to optimize the latter.

The closure syntax does not add any guarantee.


(Arnold Schwaighofer) #9

Updated proposal: use a method that accepts a closure to delineate the required guaranteed lifetime for which the assertion by the programmer holds.
And, hopefully, no optimizer language speak.

Add an API to Unmanaged to get the instance it holds @guaranteed

Proposal: SE-0000 Add an API to Unmanaged to get the instance it holds guaranteed <https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md>
Author(s): Arnold Schwaighofer <https://github.com/aschwaighofer>
Status: Draft
Review manager: TBD
<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#introduction>Introduction

The standard library Unmanged<Instance> struct provides an instance wrapper that does not participate in ARC; it allows the user to make manual retain/release calls and get at the contained instance at balanced/unbalanced retain counts.

This proposal suggests to add another method withUnsafeGuaranteedValue to Unmanaged that accepts a closure. Calling this method is akin to making an assertion about the guaranteed lifetime of the instance for the delinated scope of the method invocation. The closure is passed the unmanaged reference as a managed reference.

  func doSomething(u : Unmanged<Owned>) {
    // The programmer asserts that there exists another managed reference of the
    // unmanaged reference stored in 'u' and that the lifetime of the referenced
    // instance is guaranteed to extend beyond the 'withUnsafeGuaranteedValue'
    // invocation.
    u.withUnsafeGuaranteedValue {
      $0.doSomething()
    }
  }
This assertion will help the compiler remove ARC operations.

public struct Unmanaged<Instance : AnyObject> {

  // Get the value of the unmanaged referenced as a managed reference without
  // consuming an unbalanced retain of it and pass it to the closure. Asserts
  // that there is some other reference ('the owning reference') to the
  // unmanaged reference that guarantees the lifetime of the unmanaged reference
  // for the duration of the 'withUnsafeGuaranteedValue' call.
  //
  // NOTE: You are responsible for ensuring this by making the owning reference's
  // lifetime fixed for the duration of the 'withUnsafeGuaranteedValue' call.
  //
  // Violation of this will incur undefined behavior.
  //
  // A lifetime of a reference 'the instance' is fixed over a point in the
  // programm if:
  //
  // * There is another managed reference to the same instance 'the instance'
  // whose life time is fixed over the point in the program by means of
  // 'withExtendedLifetime' closing over this point.
  //
  // var owningReference = Instance()
  // ...
  // withExtendedLifetime(owningReference) {
  // point($0)
  // }
  //
  // Or if:
  //
  // * There is a class, or struct instance ('owner') whose lifetime is fixed at
  // the point and which has a stored property that references 'the instance'
  // for the duration of the fixed lifetime of the 'owner'.
  //
  // class Owned {
  // }
  //
  // class Owner {
  // final var owned : Owned
  //
  // func foo() {
  // withExtendedLifetime(self) {
  // doSomething(...)
  // } // Assuming: No stores to owned occur for the dynamic lifetime of
  // // the withExtendedLifetime invocation.
  // }
  //
  // func doSomething() {
  // // both 'self' and 'owned''s lifetime is fixed over this point.
  // point(self, owned)
  // }
  // }
  //
  // The last rule applies transitively through a chains of stored references
  // and nested structs.
  //
  // Examples:
  //
  // var owningReference = Instance()
  // ...
  // withExtendedLifetime(owningReference) {
  // let u = Unmanaged.passUnretained(owningReference)
  // for i in 0 ..< 100 {
  // u.withUnsafeGuaranteedValue {
  // $0.doSomething()
  // }
  // }
  // }
  //
  // class Owner {
  // final var owned : Owned
  //
  // func foo() {
  // withExtendedLifetime(self) {
  // doSomething(Unmanaged.passUnretained(owned))
  // }
  // }
  //
  // func doSomething(u : Unmanged<Owned>) {
  // u.withUnsafeGuaranteedValue {
  // $0.doSomething()
  // }
  // }
  // }
  public func withUnsafeGuaranteedValue<Result>(
    @noescape closure: (Instance) throws -> Result
  ) rethrows {
    let instance = _value
    let (guaranteedInstance, token) = Builtin.unsafeGuaranteed(instance)
    try closure(guaranteedInstance)
    Builtin.unsafeGuaranteedEnd(token)
  }
Prototype: link to a prototype implementation <https://github.com/aschwaighofer/swift/tree/unsafe_guaranteed_prototype>
<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#motivation>Motivation

A user can often make assertions that an instance is kept alive by another reference to it for the duration of some scope.

Consider the following example:

class Owned {
  func doSomeWork() {}
}

public class Owner {
  var ref: Owned

  init() { ref = Owned() }

  public func doWork() {
    withExtendedLifetime(self) {
      doSomething(ref)
    }
  }

  func doSomething(o: Owned) {
     o.doSomeWork()
  }
}
In this context the lifetime of Owner always exceeds o: Ownee. However, there is currently no way to tell the compiler about such an assertion. We can pass reference counted values in an Unmanged container without incurring reference count changes.

public class Owner {

  public func doWork() {
    // No reference count for passing the parameter.
    doSomething(Unmanaged.passUnretained(ref))
  }

  func doSomething(u: Unmanaged<Owned>) {
    // ...
  }
}
We can get at the contained instance by means of takeUnretainedValue() which will return a balanced retained value: the value is returned at +1 for release by the caller. However, when it comes to accessing the contained instance we incur reference counting.

  func doSomething(u: Unmanaged<Owned>) {
    // Incurs refcount increment before the call and decrement after for self.
    u.takeUnretainedValue().doSomeWork()
  }
With the proposed API call the user could make the assertion that u's contained instance's lifetime is guaranteed by another reference to it.

  func doSomething(u: Unmanaged<Owned>) {
    // Incurs refcount increment before the call and decrement after for self
    // that can be removed by the compiler based on the assertion made.
    u.withUnsafeGuaranteedValue {
      $0.doSomeWork()
    }
  }
The compiler can easily remove the reference counts marked by this method call with local reasoning.

<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#impact-on-existing-code>Impact on existing code

This is a new API that does not replace an existing method. No existing code should be affected.

<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#alternatives-considered>Alternatives considered

A somewhat related proposal would be to allow for class types to completely opt out of ARC forcing the programmer to perform manual reference counting. The scope of such a proposal would be much bigger making it questionable for Swift 3 inclusion. I believe that the two approaches are complementary and don't completely supplant each other. This proposal's approach allows for selectively opting out of ARC while providing a high notational burdon when wide spread over the application (wrapping and unwrapping of Unmanaged). Opting completely out of ARC is an all-in solution, which on the other hand does not have the high notational burden.

···

On Mar 15, 2016, at 4:06 PM, Dave Abrahams <dabrahams@apple.com> wrote:

on Mon Mar 14 2016, John McCall <rjmccall-AT-apple.com> wrote:

In this example this would be for the lifetime for the value in x which extends to the lifetime of y.

What if I return it or assign it to non-local memory?

I feel like you’re trying to define this by optimizer behavior.
That’s not a workable language rule; programmers are not going to
reason about SSA values.

Thank you, John. That is the problem I usually have with evaluating
most optimizer proposals.


(Alex Migicovsky) #10

Hi Arnold,

This seems like a great optimization tool. The only question I have is about the method name. With the name `withUnsafeGuaranteedValue` we’re moving the term “guaranteed” into API in addition to it being an implementation detail of the compiler as far as I could tell. Is “guaranteed" the best name for this?

- Alex

···

On Mar 15, 2016, at 6:01 PM, Arnold Schwaighofer via swift-evolution <swift-evolution@swift.org> wrote:

Updated proposal: use a method that accepts a closure to delineate the required guaranteed lifetime for which the assertion by the programmer holds.
And, hopefully, no optimizer language speak.

Add an API to Unmanaged to get the instance it holds @guaranteed

Proposal: SE-0000 Add an API to Unmanaged to get the instance it holds guaranteed <https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md>
Author(s): Arnold Schwaighofer <https://github.com/aschwaighofer>
Status: Draft
Review manager: TBD
<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#introduction>Introduction

The standard library Unmanged<Instance> struct provides an instance wrapper that does not participate in ARC; it allows the user to make manual retain/release calls and get at the contained instance at balanced/unbalanced retain counts.

This proposal suggests to add another method withUnsafeGuaranteedValue to Unmanaged that accepts a closure. Calling this method is akin to making an assertion about the guaranteed lifetime of the instance for the delinated scope of the method invocation. The closure is passed the unmanaged reference as a managed reference.

  func doSomething(u : Unmanged<Owned>) {
    // The programmer asserts that there exists another managed reference of the
    // unmanaged reference stored in 'u' and that the lifetime of the referenced
    // instance is guaranteed to extend beyond the 'withUnsafeGuaranteedValue'
    // invocation.
    u.withUnsafeGuaranteedValue {
      $0.doSomething()
    }
  }
This assertion will help the compiler remove ARC operations.

public struct Unmanaged<Instance : AnyObject> {

  // Get the value of the unmanaged referenced as a managed reference without
  // consuming an unbalanced retain of it and pass it to the closure. Asserts
  // that there is some other reference ('the owning reference') to the
  // unmanaged reference that guarantees the lifetime of the unmanaged reference
  // for the duration of the 'withUnsafeGuaranteedValue' call.
  //
  // NOTE: You are responsible for ensuring this by making the owning reference's
  // lifetime fixed for the duration of the 'withUnsafeGuaranteedValue' call.
  //
  // Violation of this will incur undefined behavior.
  //
  // A lifetime of a reference 'the instance' is fixed over a point in the
  // programm if:
  //
  // * There is another managed reference to the same instance 'the instance'
  // whose life time is fixed over the point in the program by means of
  // 'withExtendedLifetime' closing over this point.
  //
  // var owningReference = Instance()
  // ...
  // withExtendedLifetime(owningReference) {
  // point($0)
  // }
  //
  // Or if:
  //
  // * There is a class, or struct instance ('owner') whose lifetime is fixed at
  // the point and which has a stored property that references 'the instance'
  // for the duration of the fixed lifetime of the 'owner'.
  //
  // class Owned {
  // }
  //
  // class Owner {
  // final var owned : Owned
  //
  // func foo() {
  // withExtendedLifetime(self) {
  // doSomething(...)
  // } // Assuming: No stores to owned occur for the dynamic lifetime of
  // // the withExtendedLifetime invocation.
  // }
  //
  // func doSomething() {
  // // both 'self' and 'owned''s lifetime is fixed over this point.
  // point(self, owned)
  // }
  // }
  //
  // The last rule applies transitively through a chains of stored references
  // and nested structs.
  //
  // Examples:
  //
  // var owningReference = Instance()
  // ...
  // withExtendedLifetime(owningReference) {
  // let u = Unmanaged.passUnretained(owningReference)
  // for i in 0 ..< 100 {
  // u.withUnsafeGuaranteedValue {
  // $0.doSomething()
  // }
  // }
  // }
  //
  // class Owner {
  // final var owned : Owned
  //
  // func foo() {
  // withExtendedLifetime(self) {
  // doSomething(Unmanaged.passUnretained(owned))
  // }
  // }
  //
  // func doSomething(u : Unmanged<Owned>) {
  // u.withUnsafeGuaranteedValue {
  // $0.doSomething()
  // }
  // }
  // }
  public func withUnsafeGuaranteedValue<Result>(
    @noescape closure: (Instance) throws -> Result
  ) rethrows {
    let instance = _value
    let (guaranteedInstance, token) = Builtin.unsafeGuaranteed(instance)
    try closure(guaranteedInstance)
    Builtin.unsafeGuaranteedEnd(token)
  }
Prototype: link to a prototype implementation <https://github.com/aschwaighofer/swift/tree/unsafe_guaranteed_prototype>
<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#motivation>Motivation

A user can often make assertions that an instance is kept alive by another reference to it for the duration of some scope.

Consider the following example:

class Owned {
  func doSomeWork() {}
}

public class Owner {
  var ref: Owned

  init() { ref = Owned() }

  public func doWork() {
    withExtendedLifetime(self) {
      doSomething(ref)
    }
  }

  func doSomething(o: Owned) {
     o.doSomeWork()
  }
}
In this context the lifetime of Owner always exceeds o: Ownee. However, there is currently no way to tell the compiler about such an assertion. We can pass reference counted values in an Unmanged container without incurring reference count changes.

public class Owner {

  public func doWork() {
    // No reference count for passing the parameter.
    doSomething(Unmanaged.passUnretained(ref))
  }

  func doSomething(u: Unmanaged<Owned>) {
    // ...
  }
}
We can get at the contained instance by means of takeUnretainedValue() which will return a balanced retained value: the value is returned at +1 for release by the caller. However, when it comes to accessing the contained instance we incur reference counting.

  func doSomething(u: Unmanaged<Owned>) {
    // Incurs refcount increment before the call and decrement after for self.
    u.takeUnretainedValue().doSomeWork()
  }
With the proposed API call the user could make the assertion that u's contained instance's lifetime is guaranteed by another reference to it.

  func doSomething(u: Unmanaged<Owned>) {
    // Incurs refcount increment before the call and decrement after for self
    // that can be removed by the compiler based on the assertion made.
    u.withUnsafeGuaranteedValue {
      $0.doSomeWork()
    }
  }
The compiler can easily remove the reference counts marked by this method call with local reasoning.

<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#impact-on-existing-code>Impact on existing code

This is a new API that does not replace an existing method. No existing code should be affected.

<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#alternatives-considered>Alternatives considered

A somewhat related proposal would be to allow for class types to completely opt out of ARC forcing the programmer to perform manual reference counting. The scope of such a proposal would be much bigger making it questionable for Swift 3 inclusion. I believe that the two approaches are complementary and don't completely supplant each other. This proposal's approach allows for selectively opting out of ARC while providing a high notational burdon when wide spread over the application (wrapping and unwrapping of Unmanaged). Opting completely out of ARC is an all-in solution, which on the other hand does not have the high notational burden.

On Mar 15, 2016, at 4:06 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:

on Mon Mar 14 2016, John McCall <rjmccall-AT-apple.com <http://rjmccall-at-apple.com/>> wrote:

In this example this would be for the lifetime for the value in x which extends to the lifetime of y.

What if I return it or assign it to non-local memory?

I feel like you’re trying to define this by optimizer behavior.
That’s not a workable language rule; programmers are not going to
reason about SSA values.

Thank you, John. That is the problem I usually have with evaluating
most optimizer proposals.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Arnold Schwaighofer) #11

Hi Alex,

Probably not, The name is a strawman. I guess I could not resist the optimizer speak after all. The programmer is asserting he guarantees the lifetime by other means. I expect if this proposal moves forward the standard library team will decide on the right name for this method.

assertLifetimeGuarantee, withAssertedLifetime, unsafePassWithAssertedLifetime, ... ?

···

On Mar 15, 2016, at 6:14 PM, Alex Migicovsky <migi@apple.com> wrote:

Hi Arnold,

This seems like a great optimization tool. The only question I have is about the method name. With the name `withUnsafeGuaranteedValue` we’re moving the term “guaranteed” into API in addition to it being an implementation detail of the compiler as far as I could tell. Is “guaranteed" the best name for this?

- Alex

On Mar 15, 2016, at 6:01 PM, Arnold Schwaighofer via swift-evolution <swift-evolution@swift.org> wrote:

Updated proposal: use a method that accepts a closure to delineate the required guaranteed lifetime for which the assertion by the programmer holds.
And, hopefully, no optimizer language speak.

Add an API to Unmanaged to get the instance it holds @guaranteed
Proposal: SE-0000 Add an API to Unmanaged to get the instance it holds guaranteed
Author(s): Arnold Schwaighofer
Status: Draft
Review manager: TBD
Introduction

The standard library Unmanged<Instance> struct provides an instance wrapper that does not participate in ARC; it allows the user to make manual retain/release calls and get at the contained instance at balanced/unbalanced retain counts.

This proposal suggests to add another method withUnsafeGuaranteedValue to Unmanaged that accepts a closure. Calling this method is akin to making an assertion about the guaranteed lifetime of the instance for the delinated scope of the method invocation. The closure is passed the unmanaged reference as a managed reference.

  func doSomething(u : Unmanged<Owned>) {
    // The programmer asserts that there exists another managed reference of the
    // unmanaged reference stored in 'u' and that the lifetime of the referenced
    // instance is guaranteed to extend beyond the 'withUnsafeGuaranteedValue'
    // invocation.
    u.withUnsafeGuaranteedValue {
      $0.doSomething()
    }
  }
This assertion will help the compiler remove ARC operations.

public struct Unmanaged<Instance : AnyObject> {

  // Get the value of the unmanaged referenced as a managed reference without
  // consuming an unbalanced retain of it and pass it to the closure. Asserts
  // that there is some other reference ('the owning reference') to the
  // unmanaged reference that guarantees the lifetime of the unmanaged reference
  // for the duration of the 'withUnsafeGuaranteedValue' call.
  //
  // NOTE: You are responsible for ensuring this by making the owning reference's
  // lifetime fixed for the duration of the 'withUnsafeGuaranteedValue' call.
  //
  // Violation of this will incur undefined behavior.
  //
  // A lifetime of a reference 'the instance' is fixed over a point in the
  // programm if:
  //
  // * There is another managed reference to the same instance 'the instance'
  // whose life time is fixed over the point in the program by means of
  // 'withExtendedLifetime' closing over this point.
  //
  // var owningReference = Instance()
  // ...
  // withExtendedLifetime(owningReference) {
  // point($0)
  // }
  //
  // Or if:
  //
  // * There is a class, or struct instance ('owner') whose lifetime is fixed at
  // the point and which has a stored property that references 'the instance'
  // for the duration of the fixed lifetime of the 'owner'.
  //
  // class Owned {
  // }
  //
  // class Owner {
  // final var owned : Owned
  //
  // func foo() {
  // withExtendedLifetime(self) {
  // doSomething(...)
  // } // Assuming: No stores to owned occur for the dynamic lifetime of
  // // the withExtendedLifetime invocation.
  // }
  //
  // func doSomething() {
  // // both 'self' and 'owned''s lifetime is fixed over this point.
  // point(self, owned)
  // }
  // }
  //
  // The last rule applies transitively through a chains of stored references
  // and nested structs.
  //
  // Examples:
  //
  // var owningReference = Instance()
  // ...
  // withExtendedLifetime(owningReference) {
  // let u = Unmanaged.passUnretained(owningReference)
  // for i in 0 ..< 100 {
  // u.withUnsafeGuaranteedValue {
  // $0.doSomething()
  // }
  // }
  // }
  //
  // class Owner {
  // final var owned : Owned
  //
  // func foo() {
  // withExtendedLifetime(self) {
  // doSomething(Unmanaged.passUnretained(owned))
  // }
  // }
  //
  // func doSomething(u : Unmanged<Owned>) {
  // u.withUnsafeGuaranteedValue {
  // $0.doSomething()
  // }
  // }
  // }
  public func withUnsafeGuaranteedValue<Result>(
    @noescape closure: (Instance) throws -> Result
  ) rethrows {
    let instance = _value
    let (guaranteedInstance, token) = Builtin.unsafeGuaranteed(instance)
    try closure(guaranteedInstance)
    Builtin.unsafeGuaranteedEnd(token)
  }
Prototype: link to a prototype implementation

Motivation

A user can often make assertions that an instance is kept alive by another reference to it for the duration of some scope.

Consider the following example:

class Owned {
  func doSomeWork() {}
}

public class Owner {
  var ref: Owned

  init() { ref = Owned() }

  public func doWork() {
    withExtendedLifetime(self) {
      doSomething(ref)
    }
  }

  func doSomething(o: Owned) {
     o.doSomeWork()
  }
}
In this context the lifetime of Owner always exceeds o: Ownee. However, there is currently no way to tell the compiler about such an assertion. We can pass reference counted values in an Unmanged container without incurring reference count changes.

public class Owner {

  public func doWork() {
    // No reference count for passing the parameter.
    doSomething(Unmanaged.passUnretained(ref))
  }

  func doSomething(u: Unmanaged<Owned>) {
    // ...
  }
}
We can get at the contained instance by means of takeUnretainedValue() which will return a balanced retained value: the value is returned at +1 for release by the caller. However, when it comes to accessing the contained instance we incur reference counting.

  func doSomething(u: Unmanaged<Owned>) {
    // Incurs refcount increment before the call and decrement after for self.
    u.takeUnretainedValue().doSomeWork()
  }
With the proposed API call the user could make the assertion that u's contained instance's lifetime is guaranteed by another reference to it.

  func doSomething(u: Unmanaged<Owned>) {
    // Incurs refcount increment before the call and decrement after for self
    // that can be removed by the compiler based on the assertion made.
    u.withUnsafeGuaranteedValue {
      $0.doSomeWork()
    }
  }
The compiler can easily remove the reference counts marked by this method call with local reasoning.

Impact on existing code

This is a new API that does not replace an existing method. No existing code should be affected.

Alternatives considered

A somewhat related proposal would be to allow for class types to completely opt out of ARC forcing the programmer to perform manual reference counting. The scope of such a proposal would be much bigger making it questionable for Swift 3 inclusion. I believe that the two approaches are complementary and don't completely supplant each other. This proposal's approach allows for selectively opting out of ARC while providing a high notational burdon when wide spread over the application (wrapping and unwrapping of Unmanaged). Opting completely out of ARC is an all-in solution, which on the other hand does not have the high notational burden.

On Mar 15, 2016, at 4:06 PM, Dave Abrahams <dabrahams@apple.com> wrote:

on Mon Mar 14 2016, John McCall <rjmccall-AT-apple.com> wrote:

In this example this would be for the lifetime for the value in x which extends to the lifetime of y.

What if I return it or assign it to non-local memory?

I feel like you’re trying to define this by optimizer behavior.
That’s not a workable language rule; programmers are not going to
reason about SSA values.

Thank you, John. That is the problem I usually have with evaluating
most optimizer proposals.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org
https://lists.swift.org/mailman/listinfo/swift-evolution


(Alex Migicovsky) #12

Hehe :slight_smile:

I’ll hold off suggestions until we need to cross that road!

- Alex

···

On Mar 15, 2016, at 6:28 PM, Arnold <aschwaighofer@apple.com> wrote:

Hi Alex,

Probably not, The name is a strawman. I guess I could not resist the optimizer speak after all. The programmer is asserting he guarantees the lifetime by other means. I expect if this proposal moves forward the standard library team will decide on the right name for this method.

assertLifetimeGuarantee, withAssertedLifetime, unsafePassWithAssertedLifetime, ... ?

On Mar 15, 2016, at 6:14 PM, Alex Migicovsky <migi@apple.com <mailto:migi@apple.com>> wrote:

Hi Arnold,

This seems like a great optimization tool. The only question I have is about the method name. With the name `withUnsafeGuaranteedValue` we’re moving the term “guaranteed” into API in addition to it being an implementation detail of the compiler as far as I could tell. Is “guaranteed" the best name for this?

- Alex

On Mar 15, 2016, at 6:01 PM, Arnold Schwaighofer via swift-evolution <swift-evolution@swift.org <mailto:swift-evolution@swift.org>> wrote:

Updated proposal: use a method that accepts a closure to delineate the required guaranteed lifetime for which the assertion by the programmer holds.
And, hopefully, no optimizer language speak.

Add an API to Unmanaged to get the instance it holds @guaranteed

Proposal: SE-0000 Add an API to Unmanaged to get the instance it holds guaranteed <https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md>
Author(s): Arnold Schwaighofer <https://github.com/aschwaighofer>
Status: Draft
Review manager: TBD
<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#introduction>Introduction

The standard library Unmanged<Instance> struct provides an instance wrapper that does not participate in ARC; it allows the user to make manual retain/release calls and get at the contained instance at balanced/unbalanced retain counts.

This proposal suggests to add another method withUnsafeGuaranteedValue to Unmanaged that accepts a closure. Calling this method is akin to making an assertion about the guaranteed lifetime of the instance for the delinated scope of the method invocation. The closure is passed the unmanaged reference as a managed reference.

  func doSomething(u : Unmanged<Owned>) {
    // The programmer asserts that there exists another managed reference of the
    // unmanaged reference stored in 'u' and that the lifetime of the referenced
    // instance is guaranteed to extend beyond the 'withUnsafeGuaranteedValue'
    // invocation.
    u.withUnsafeGuaranteedValue {
      $0.doSomething()
    }
  }
This assertion will help the compiler remove ARC operations.

public struct Unmanaged<Instance : AnyObject> {

  // Get the value of the unmanaged referenced as a managed reference without
  // consuming an unbalanced retain of it and pass it to the closure. Asserts
  // that there is some other reference ('the owning reference') to the
  // unmanaged reference that guarantees the lifetime of the unmanaged reference
  // for the duration of the 'withUnsafeGuaranteedValue' call.
  //
  // NOTE: You are responsible for ensuring this by making the owning reference's
  // lifetime fixed for the duration of the 'withUnsafeGuaranteedValue' call.
  //
  // Violation of this will incur undefined behavior.
  //
  // A lifetime of a reference 'the instance' is fixed over a point in the
  // programm if:
  //
  // * There is another managed reference to the same instance 'the instance'
  // whose life time is fixed over the point in the program by means of
  // 'withExtendedLifetime' closing over this point.
  //
  // var owningReference = Instance()
  // ...
  // withExtendedLifetime(owningReference) {
  // point($0)
  // }
  //
  // Or if:
  //
  // * There is a class, or struct instance ('owner') whose lifetime is fixed at
  // the point and which has a stored property that references 'the instance'
  // for the duration of the fixed lifetime of the 'owner'.
  //
  // class Owned {
  // }
  //
  // class Owner {
  // final var owned : Owned
  //
  // func foo() {
  // withExtendedLifetime(self) {
  // doSomething(...)
  // } // Assuming: No stores to owned occur for the dynamic lifetime of
  // // the withExtendedLifetime invocation.
  // }
  //
  // func doSomething() {
  // // both 'self' and 'owned''s lifetime is fixed over this point.
  // point(self, owned)
  // }
  // }
  //
  // The last rule applies transitively through a chains of stored references
  // and nested structs.
  //
  // Examples:
  //
  // var owningReference = Instance()
  // ...
  // withExtendedLifetime(owningReference) {
  // let u = Unmanaged.passUnretained(owningReference)
  // for i in 0 ..< 100 {
  // u.withUnsafeGuaranteedValue {
  // $0.doSomething()
  // }
  // }
  // }
  //
  // class Owner {
  // final var owned : Owned
  //
  // func foo() {
  // withExtendedLifetime(self) {
  // doSomething(Unmanaged.passUnretained(owned))
  // }
  // }
  //
  // func doSomething(u : Unmanged<Owned>) {
  // u.withUnsafeGuaranteedValue {
  // $0.doSomething()
  // }
  // }
  // }
  public func withUnsafeGuaranteedValue<Result>(
    @noescape closure: (Instance) throws -> Result
  ) rethrows {
    let instance = _value
    let (guaranteedInstance, token) = Builtin.unsafeGuaranteed(instance)
    try closure(guaranteedInstance)
    Builtin.unsafeGuaranteedEnd(token)
  }
Prototype: link to a prototype implementation <https://github.com/aschwaighofer/swift/tree/unsafe_guaranteed_prototype>
<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#motivation>Motivation

A user can often make assertions that an instance is kept alive by another reference to it for the duration of some scope.

Consider the following example:

class Owned {
  func doSomeWork() {}
}

public class Owner {
  var ref: Owned

  init() { ref = Owned() }

  public func doWork() {
    withExtendedLifetime(self) {
      doSomething(ref)
    }
  }

  func doSomething(o: Owned) {
     o.doSomeWork()
  }
}
In this context the lifetime of Owner always exceeds o: Ownee. However, there is currently no way to tell the compiler about such an assertion. We can pass reference counted values in an Unmanged container without incurring reference count changes.

public class Owner {

  public func doWork() {
    // No reference count for passing the parameter.
    doSomething(Unmanaged.passUnretained(ref))
  }

  func doSomething(u: Unmanaged<Owned>) {
    // ...
  }
}
We can get at the contained instance by means of takeUnretainedValue() which will return a balanced retained value: the value is returned at +1 for release by the caller. However, when it comes to accessing the contained instance we incur reference counting.

  func doSomething(u: Unmanaged<Owned>) {
    // Incurs refcount increment before the call and decrement after for self.
    u.takeUnretainedValue().doSomeWork()
  }
With the proposed API call the user could make the assertion that u's contained instance's lifetime is guaranteed by another reference to it.

  func doSomething(u: Unmanaged<Owned>) {
    // Incurs refcount increment before the call and decrement after for self
    // that can be removed by the compiler based on the assertion made.
    u.withUnsafeGuaranteedValue {
      $0.doSomeWork()
    }
  }
The compiler can easily remove the reference counts marked by this method call with local reasoning.

<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#impact-on-existing-code>Impact on existing code

This is a new API that does not replace an existing method. No existing code should be affected.

<https://github.com/aschwaighofer/swift-evolution/blob/unmanaged_unsafe_guaranteed_proposal/proposals/0000-unsafe-guaranteed.md#alternatives-considered>Alternatives considered

A somewhat related proposal would be to allow for class types to completely opt out of ARC forcing the programmer to perform manual reference counting. The scope of such a proposal would be much bigger making it questionable for Swift 3 inclusion. I believe that the two approaches are complementary and don't completely supplant each other. This proposal's approach allows for selectively opting out of ARC while providing a high notational burdon when wide spread over the application (wrapping and unwrapping of Unmanaged). Opting completely out of ARC is an all-in solution, which on the other hand does not have the high notational burden.

On Mar 15, 2016, at 4:06 PM, Dave Abrahams <dabrahams@apple.com <mailto:dabrahams@apple.com>> wrote:

on Mon Mar 14 2016, John McCall <rjmccall-AT-apple.com <http://rjmccall-at-apple.com/>> wrote:

In this example this would be for the lifetime for the value in x which extends to the lifetime of y.

What if I return it or assign it to non-local memory?

I feel like you’re trying to define this by optimizer behavior.
That’s not a workable language rule; programmers are not going to
reason about SSA values.

Thank you, John. That is the problem I usually have with evaluating
most optimizer proposals.

_______________________________________________
swift-evolution mailing list
swift-evolution@swift.org <mailto:swift-evolution@swift.org>
https://lists.swift.org/mailman/listinfo/swift-evolution


(Brent Royal-Gordon) #13

Probably not, The name is a strawman. I guess I could not resist the optimizer speak after all. The programmer is asserting he guarantees the lifetime by other means. I expect if this proposal moves forward the standard library team will decide on the right name for this method.

assertLifetimeGuarantee, withAssertedLifetime, unsafePassWithAssertedLifetime, ... ?

withUnsafeUnownedValue? I think that conveys the danger—you've been passed a value which you don't own, so something else had better keep it alive—even if it's not technically accurate.

···

--
Brent Royal-Gordon
Architechies