Parametrized deinitialization with Non-copyable (move-only) types

With the introduction of Non-copyable (move-only) structs and enums, it is now safer to use structs and enums as reference types. This proposal introduces deinit for Non-copyable types, which is described as:

A noncopyable struct or enum may declare a deinit, which will run implicitly when the lifetime of the value ends unless explicitly suppressed.

But this ignores the fact there might be scenarios where lifetime must be explicitly ended. One example of this could be current continuation type. Currently there is no way to validate continuation misuse at compile time. Continuation API describes following misuses:

You must call a resume method exactly once on every execution path throughout the program. Resuming from a continuation more than once is undefined behavior.

This can be avoided by making continuation a Non-copyable type and the resume method as consuming method. Hence once resume is invoked on the continuation it's lifetime ends and can no longer be used.

Never resuming leaves the task in a suspended state indefinitely, and leaks any associated resources.

Currently there is no way to guard again such misuse unless we require explicit lifetime end for continuation.

Similar to Copyable new protocol Droppable can be used by types to specify whether type supports implicit deinitialization or not. By default, all non-copyable types confirm to Droppable unless specified explicitly using ~Droppable.

struct Continuation: ~Copyable, ~Droppable {
    // impl
}

For the types that doesn't confirm to Droppable, any consuming method provided by this type needs to be invoked to end lifetime explicitly.

What are your thoughts on this pitch? I think this would be valuable having this and will add compile-time validation on misuse of these kind of types.

4 Likes

I like this idea. It feels natural.

The name should probably have deinit in it somewhere. e.g. ImplicitDeinit (to be used with the ~ prefix).

I wonder if - in the long term - this could allow implicit deinitialisation to be delayed, at the compiler's discretion? Once all types which care about precise deinit timing switch to requiring explicit deinit. That could give the compiler more freedom in how it arranges code and generally orders things (e.g. postponing deinitialisation can allow the effects of a task to become visible earlier, unblocking other tasks / threads sooner).

Gankra looked into this for Rust several years ago:

Swift doesn’t have implicitly-unwinding panics (essentially C++-style exceptions if you’re more familiar with those), so we wouldn’t have to worry about consumption being skipped. (The flow control of try and throw is modeled and checked explicitly in the compiler.) But all of the issues she brings up in “Usage Examples” still apply. I’ll pull out one part:

But, collections have many subtle things that wouldn’t work anymore.

  • array[i] = val (old val not returned)
  • map.insert/remove (old K not returned)
  • truncate, resize, clear, dedup, retain, filter, …

So basically you’d end up in the same state we are now with non-Copyable types, where you can’t, in practice, use them with generics…but with no real assumption things will get better. You can add alternate APIs to generic types that preserve the must-not-drop property, but that’s a lot of work for what’s probably a fairly niche feature.

(This is somewhat true for adopting non-Copyability in the stdlib as well…but less so, because generic types generally already try to avoid unnecessary copies for performance reasons.)

I’m not saying we can/should never do this, and in fact it’s possible that “while working through Copyable” is a good time to design it. (I no longer work on the Swift compiler, so what I might want is mostly immaterial.) I’m saying a good chunk of the work is in library design, not just a compiler feature to enforce this new restriction on certain types. That part’s relatively easy.

(And I do recommend reading the rest of Gankra’s article. It’s always nice when someone has done the research for you.)

6 Likes

It's also worth noting that the problem raised with Continuation, that it must be invoked at some point in the future, is not solved by a feature like this. We can't tell a program that never invokes resume apart from a program that possibly invokes resume at some indefinite point in the future, without imposing additional expressivity constraints on the type that would also interfere with legitimate uses. Strict linear types would prevent a value from being implicitly dropped, but can't stop it from being held forever.

3 Likes

True, and that somewhat diminishes the value. But even in these cases, wouldn't it help somewhat that you'd now definitively still have the Continuation object in memory somewhere, where you might notice it? (e.g. in debugging, as a memory 'leak' if you have lots of them over time, etc)

That's already possible. We don't need additional language features to track whether a continuation has been created and not yet resumed.

Ah, I was going to say I'm pretty sure I've seen Continuations disappear [from memory] without raising any alarms, but now that I look into that more I suspect those were UnsafeContinuations where of course that's expected.

And I suspect this no-implicit-deinit functionality wouldn't eliminate the need for UnsafeContinuation in practice, because of those library interoperability concerns that @jrose pointed out.

I still like the idea in spirit - it seems intuitive and sensible - but that Rust article via Jordan does pretty neatly explain why implementing it is probably not easy. Even just the first example - that assignments via subscripting would have to move-return the old value - seems like a potential deal-breaker since I can't figure out any way to make that behaviour make sense conceptually. let foo = bar[3] = troz in no way says to me that foo will end up with the old value from bar[3] (as opposed to troz).

Even for an UnsafeContinuation, it should be possible to look at the set of Tasks and see whether any are suspended waiting for a continuation.

True.

Though today, as far as I can tell, Xcode provides no way to view the set of live Swift tasks, in its debugging mode…? I can browse the object memory graph, though. I can infer actively executing tasks via callstacks, but Xcode provides no conveniences here (e.g. it doesn't even show the current Task when I'm looking at the current PC - although since tasks are anonymous there's a fundamental difficulty here of identification).

(this is with the latest Xcode beta)

There are a couple of widgets in Instruments for Swift concurrency, but they're undocumented and I can't make much sense of what they show. e.g. for a trivial Task which just sits in an infinite loop sleeping for 100ms at a time, Instruments shows every pass through that loop as a separate "task", which by all accounts it isn't. It's possible Instruments provides a means to identify abandoned continuations, but in a quick test I couldn't figure out how to use it to do so, even knowing that I had one in my test app.

I don't mean to rant or criticise - I'm just noting that the tooling is currently lacking regarding Swift concurrency. So turning Task lifetime issues into object (memory) issues is a pragmatic way to actually make them identifiable and debuggable, for now.