I'm encountering a behavior around object lifetime and deinitialization
that is surprising to me. Here's an approximation of what I'm encountering:
I have a class Handle whose deinit must execute on a certain queue (let's
call it q--in my case it is a global queue). I have a second class
AsyncHandle which owns a Handle and needs to be able to deinit on any
queue. There's a tension to resolve between the deinitialization contracts
of Handle and AsyncHandle. To resolve it, in AsyncHandle's deinit, I am
binding the owned instance of Handle to a local variable, dispatching async
onto the q, and from the async closure, calling extendLifetime with the
Handle, where extendLifetime is the following:
This approach seems to work--the Handle gets deinit on q, the appropriate
queue. Most of the time. In the debugger, there is never a problem.
Occasionally and inconsistently, on some devices, I am, however, seeing a
behavior that _seems_ to be the synchronous deallocation of Handle from
AsyncHandle's deinit on the queue that AsyncHandle happens to be
deinitializing on (not q). If that is indeed, the behavior I'm seeing, I
do not understand why it is happening.
A few questions:
(1) Given an object B owned (exclusively--no other objects have references
to A) by an object A, is it legal to extend the lifetime of B in the deinit
of A? (It might conceivably not be legal if there is a rule such as the
following: when the runtime observes that an object R referenced by a field
of a deinitializing object O has a reference count of one, it assumes that
R must die when O dies and consequently puts R into some moribund state
from which there is no revivification.)
(2) If it is legal, is calling extendLifetime from a dispatch async the
appropriate way to do it?
(3) Is my "implementation" (such as it is) of extendLifetime correct? (I
can't use the stdlib's _fixLifetime, unfortunately, or even implement
extendLifetime in the same way.)
(4) Does optimization based on visibility enter into this? In my case the
AsyncHandle is fileprivate.
(5) Is there any entirely different approach to satisfy the following
requirements? (a) AsyncHandle be releasable/deinitializable on any thread.
(b) Handle be deinitialized only on some dispatch queue q. (c)
AsyncHandle has the only reference to Handle.
I'm encountering a behavior around object lifetime and deinitialization
that is surprising to me. Here's an approximation of what I'm encountering:
I have a class Handle whose deinit must execute on a certain queue
(let's call it q--in my case it is a global queue). I have a second
class AsyncHandle which owns a Handle and needs to be able to deinit on
any queue. There's a tension to resolve between the deinitialization
contracts of Handle and AsyncHandle. To resolve it, in AsyncHandle's
deinit, I am binding the owned instance of Handle to a local variable,
dispatching async onto the q, and from the async closure, calling
extendLifetime with the Handle, where extendLifetime is the following:
This approach seems to work--the Handle gets deinit on q, the
appropriate queue. Most of the time. In the debugger, there is never a
problem. Occasionally and inconsistently, on some devices, I am,
however, seeing a behavior that _seems_ to be the synchronous
deallocation of Handle from AsyncHandle's deinit on the queue that
AsyncHandle happens to be deinitializing on (not q). If that is indeed,
the behavior I'm seeing, I do not understand why it is happening.
A few questions:
(1) Given an object B owned (exclusively--no other objects have
references to A) by an object A, is it legal to extend the lifetime of B
in the deinit of A? (It might conceivably not be legal if there is a
rule such as the following: when the runtime observes that an object R
referenced by a field of a deinitializing object O has a reference count
of one, it assumes that R must die when O dies and consequently puts R
into some moribund state from which there is no revivification.)
(2) If it is legal, is calling extendLifetime from a dispatch async the
appropriate way to do it?
(3) Is my "implementation" (such as it is) of extendLifetime correct?
(I can't use the stdlib's _fixLifetime, unfortunately, or even
implement extendLifetime in the same way.)
Have you tried using the standard library's withExtendedLifetime(_:_:) function?
···
On 27/02/2017 15:17, Nate Chandler via swift-users wrote:
(4) Does optimization based on visibility enter into this? In my case
the AsyncHandle is fileprivate.
(5) Is there any entirely different approach to satisfy the following
requirements? (a) AsyncHandle be releasable/deinitializable on any
thread. (b) Handle be deinitialized only on some dispatch queue q. (c)
AsyncHandle has the only reference to Handle.
I don't believe withExtendedLifetime can help out here. The trouble is
that withExtendedLifetime only extends a lifetime until the end of a scope,
synchronously.
In my case, I want to extend the lifetime of Handle from AsyncHandle's
deinit until a closure enqueued asynchronously (from AsyncHandle's deinit)
is invoked. Using withExtendedLifetime, I could only extend the lifetime
of Handle until the end of AsyncHandle's deinit method.
If I tried to use withExtendedLifetime, I could only write
class AsyncHandle {
let handle: Handle
deinit {
let handle: Handle = self.handle
withExtendedLifetime(handle) { handle in
q.async {
// This is where I want handle to finally be deinitialized
}
}
// This is where handle is actually deinitialized, prior (probably)
to the invocation of the closure passed to async
}
}
Please let me know if I am misunderstanding your suggestion or the
withExtendedLifetime API.
Thank you very much,
Nate Chandler
···
On Mon, Feb 27, 2017 at 12:31 PM Ole Begemann <ole@oleb.net> wrote:
On 27/02/2017 15:17, Nate Chandler via swift-users wrote:
> Hello all,
>
> I'm encountering a behavior around object lifetime and deinitialization
> that is surprising to me. Here's an approximation of what I'm
encountering:
>
> I have a class Handle whose deinit must execute on a certain queue
> (let's call it q--in my case it is a global queue). I have a second
> class AsyncHandle which owns a Handle and needs to be able to deinit on
> any queue. There's a tension to resolve between the deinitialization
> contracts of Handle and AsyncHandle. To resolve it, in AsyncHandle's
> deinit, I am binding the owned instance of Handle to a local variable,
> dispatching async onto the q, and from the async closure, calling
> extendLifetime with the Handle, where extendLifetime is the following:
>
> @inline(never) func extendLifetime<T : AnyObject>() {}
>
> AsyncHandle's deinit looks like
>
> deinit {
> let handle = self.handle
> q.async {
> extendLifetime(handle)
> }
> }
>
> This approach seems to work--the Handle gets deinit on q, the
> appropriate queue. Most of the time. In the debugger, there is never a
> problem. Occasionally and inconsistently, on some devices, I am,
> however, seeing a behavior that _seems_ to be the synchronous
> deallocation of Handle from AsyncHandle's deinit on the queue that
> AsyncHandle happens to be deinitializing on (not q). If that is indeed,
> the behavior I'm seeing, I do not understand why it is happening.
>
> A few questions:
> (1) Given an object B owned (exclusively--no other objects have
> references to A) by an object A, is it legal to extend the lifetime of B
> in the deinit of A? (It might conceivably not be legal if there is a
> rule such as the following: when the runtime observes that an object R
> referenced by a field of a deinitializing object O has a reference count
> of one, it assumes that R must die when O dies and consequently puts R
> into some moribund state from which there is no revivification.)
> (2) If it is legal, is calling extendLifetime from a dispatch async the
> appropriate way to do it?
> (3) Is my "implementation" (such as it is) of extendLifetime correct?
> (I can't use the stdlib's _fixLifetime, unfortunately, or even
> implement extendLifetime in the same way.)
Have you tried using the standard library's withExtendedLifetime(_:_:)
function?
> (4) Does optimization based on visibility enter into this? In my case
> the AsyncHandle is fileprivate.
> (5) Is there any entirely different approach to satisfy the following
> requirements? (a) AsyncHandle be releasable/deinitializable on any
> thread. (b) Handle be deinitialized only on some dispatch queue q. (c)
> AsyncHandle has the only reference to Handle.
If so, it seems that that may be a final answer to (3). It does raise an
additional question, however:
(6) Would we expect calling withExtendedLifetime to have different behavior
from calling an @inline(never) function from a closure enqueued async from
deinit?
Thank you again,
Nate Chandler
···
On Mon, Feb 27, 2017 at 1:20 PM Nate Chandler <nathaniel.chandler@gmail.com> wrote:
Hi Ole,
Thank you very much for your response.
I don't believe withExtendedLifetime can help out here. The trouble is
that withExtendedLifetime only extends a lifetime until the end of a scope,
synchronously.
In my case, I want to extend the lifetime of Handle from AsyncHandle's
deinit until a closure enqueued asynchronously (from AsyncHandle's deinit)
is invoked. Using withExtendedLifetime, I could only extend the lifetime
of Handle until the end of AsyncHandle's deinit method.
If I tried to use withExtendedLifetime, I could only write
class AsyncHandle {
let handle: Handle
deinit {
let handle: Handle = self.handle
withExtendedLifetime(handle) { handle in
q.async {
// This is where I want handle to finally be deinitialized
}
}
// This is where handle is actually deinitialized, prior
(probably) to the invocation of the closure passed to async
}
}
Please let me know if I am misunderstanding your suggestion or the
withExtendedLifetime API.
Thank you very much,
Nate Chandler
On Mon, Feb 27, 2017 at 12:31 PM Ole Begemann <ole@oleb.net> wrote:
On 27/02/2017 15:17, Nate Chandler via swift-users wrote:
> Hello all,
>
> I'm encountering a behavior around object lifetime and deinitialization
> that is surprising to me. Here's an approximation of what I'm
encountering:
>
> I have a class Handle whose deinit must execute on a certain queue
> (let's call it q--in my case it is a global queue). I have a second
> class AsyncHandle which owns a Handle and needs to be able to deinit on
> any queue. There's a tension to resolve between the deinitialization
> contracts of Handle and AsyncHandle. To resolve it, in AsyncHandle's
> deinit, I am binding the owned instance of Handle to a local variable,
> dispatching async onto the q, and from the async closure, calling
> extendLifetime with the Handle, where extendLifetime is the following:
>
> @inline(never) func extendLifetime<T : AnyObject>() {}
>
> AsyncHandle's deinit looks like
>
> deinit {
> let handle = self.handle
> q.async {
> extendLifetime(handle)
> }
> }
>
> This approach seems to work--the Handle gets deinit on q, the
> appropriate queue. Most of the time. In the debugger, there is never a
> problem. Occasionally and inconsistently, on some devices, I am,
> however, seeing a behavior that _seems_ to be the synchronous
> deallocation of Handle from AsyncHandle's deinit on the queue that
> AsyncHandle happens to be deinitializing on (not q). If that is indeed,
> the behavior I'm seeing, I do not understand why it is happening.
>
> A few questions:
> (1) Given an object B owned (exclusively--no other objects have
> references to A) by an object A, is it legal to extend the lifetime of B
> in the deinit of A? (It might conceivably not be legal if there is a
> rule such as the following: when the runtime observes that an object R
> referenced by a field of a deinitializing object O has a reference count
> of one, it assumes that R must die when O dies and consequently puts R
> into some moribund state from which there is no revivification.)
> (2) If it is legal, is calling extendLifetime from a dispatch async the
> appropriate way to do it?
> (3) Is my "implementation" (such as it is) of extendLifetime correct?
> (I can't use the stdlib's _fixLifetime, unfortunately, or even
> implement extendLifetime in the same way.)
Have you tried using the standard library's withExtendedLifetime(_:_:)
function?
> (4) Does optimization based on visibility enter into this? In my case
> the AsyncHandle is fileprivate.
> (5) Is there any entirely different approach to satisfy the following
> requirements? (a) AsyncHandle be releasable/deinitializable on any
> thread. (b) Handle be deinitialized only on some dispatch queue q. (c)
> AsyncHandle has the only reference to Handle.
Yes, that's what I'm suggesting. Sorry I didn't make that clear before. Since you mentioned that you couldn't use the stdlib's _fixLifetime function, I just wanted to point out that there is a public function in the stdlib for this purpose.
If so, it seems that that may be a final answer to (3). It does raise
an additional question, however:
(6) Would we expect calling withExtendedLifetime to have different
behavior from calling an @inline(never) function from a closure enqueued
async from deinit?
Yeah, I don't know the answer, sorry.
···
On 27/02/2017 19:34, Nate Chandler via swift-users wrote:
I'm not an optimizer person, but expecting '@inline(never)' to mean anything other than "don't inline this" is probably set for failure. The optimizer is still allowed to look at the body of the function. withExtendedLifetime is implemented more carefully to prevent eliding the capture.
That said, if you're not on when the outer deinit gets called 'q', you still have a race—it's possible that 'q' will execute the block you give it before the deinitializer exits! I don't think there's a good way to avoid this other than to make 'handle' optional (or ImplicitlyUnwrappedOptional) and explicitly set it to nil before enqueuing the "deallocation block".
Jordan
···
On Feb 27, 2017, at 10:53, Ole Begemann via swift-users <swift-users@swift.org> wrote:
On 27/02/2017 19:34, Nate Chandler via swift-users wrote:
Hi Ole,
A quick follow-up--are you suggesting calling withExtendedLifetime
inside the closure passed to async, as below?
Yes, that's what I'm suggesting. Sorry I didn't make that clear before. Since you mentioned that you couldn't use the stdlib's _fixLifetime function, I just wanted to point out that there is a public function in the stdlib for this purpose.
If so, it seems that that may be a final answer to (3). It does raise
an additional question, however:
(6) Would we expect calling withExtendedLifetime to have different
behavior from calling an @inline(never) function from a closure enqueued
async from deinit?
Your explanation of what the optimizer is free to do in the face of @inline(never) was enlightening. It's clear that, as Ole was suggesting, I
need to make extendLifetime call through to the standard library's
withExtendedLifetime to prevent the optimizer from peeking:
And thank you (very much) for calling out that race. I went ahead with the
change that you suggested, making AsyncHandle's Handle an IUO var which
gets nil'd prior to the call to async. AsyncHandle now looks like
Yes, that's what I'm suggesting. Sorry I didn't make that clear before.
Since you mentioned that you couldn't use the stdlib's _fixLifetime
function, I just wanted to point out that there is a public function in the
stdlib for this purpose.
If so, it seems that that may be a final answer to (3). It does raise
an additional question, however:
(6) Would we expect calling withExtendedLifetime to have different
behavior from calling an @inline(never) function from a closure enqueued
async from deinit?
Yeah, I don't know the answer, sorry.
I'm not an optimizer person, but expecting '@inline(never)' to mean
anything other than "don't inline this" is probably set for failure. The
optimizer is still allowed to look at the body of the function.
withExtendedLifetime is implemented more carefully to prevent eliding the
capture.
That said, if you're *not* on when the outer deinit gets called 'q', you
still have a race—it's possible that 'q' will execute the block you give it
before the deinitializer exits! I don't think there's a good way to avoid
this other than to make 'handle' optional (or ImplicitlyUnwrappedOptional)
and explicitly set it to nil before enqueuing the "deallocation block".