Replace the reference types weak and unowned with nonstrong

Evolution Pitches
1

This preliminary discussion covers the points A - F raised in the Swift Evolution Process at swift-evolution/process.md at master · apple/swift-evolution · GitHub.

Swift Evolution Process:
A. Review Commonly Rejected Proposal list. Done. No similar issues found.

B. Review Swift Evolution Forums. Done. Forum search at Search results for '' - Swift Forums. I found two potentially related proposals (see below). On further review, neither was applicable to this proposal.

  1. High Level ARC Memory Operations: not applicable
  2. Optional binding for upgrading self weak to strong #171: implemented 2016 Optional binding for upgrading self weak to strong by emaloney · Pull Request #171 · apple/swift-evolution · GitHub. This does not specifically cover the same topic as the proposal below.

C. Consider the goals of the upcoming Swift release. Done. Reviewed "On the Road to Swift 6" at On the road to Swift 6. There are three areas the Core Team is focusing on. This proposal supports issue #2, " Create a fantastic development experience". Specially the directive that "developers should be both highly productive and experience joy when programming in Swift".

On the Road to Swift 6:
#1: Accelerate growth of the Swift software ecosystem
#2: Create a fantastic development experience
#3: Invest in user-empowering language directions

D. Socialize the idea. Done. That's what this post is.

E. Develop the proposal. In progress. This is a set of future actions based on the response to this post.

F. Request a review. Deferred. This will be done only if the proposal is developed and there is sufficient interest in going forward.

The proposal below follows the Swift Proposal template at swift-evolution/0000-swift-template.md at master · apple/swift-evolution · GitHub.

==========
Proposal: SE-NNNN

Review Manager: TBD

Status: Not yet submitted for review

Title: Replace the reference types weak and unowned with nonstrong

Author: Patrick Weigel

Introduction
In Swift it is possible to create strong reference cycles between class instances. Strong reference cycles are bad as they create potentially crashing memory leaks (unused memory that is not made available to the app). Swift provides two ways to resolve strong reference cycles: weak references and unowned references. Swift requires the coder to determine which type (weak or unowned) to use, when in many instances the coder simply wants the type to be not strong.

Selecting the wrong type (weak or unowned) may itself cause a crashing error. This proposal replaces both weak and unowned with a new reference type, nonstrong, eliminating the crashing error.

See The Swift Programming Language (Swift 5.2), Chapter: Automatic Reference Counting, in the subchapters "Strong Reference Cycles Between Class Instances" and "Resolving Strong Reference Cycles Between Class Instances".

Motivation
There are two motivations: 1) Eliminate the crashing error that can be caused by incorrectly selecting weak or unowned; and 2) Reduce the barrier to entry to using Swift for beginning and intermediate coders.

There are two common causes of the default strong reference requiring manual intervention to eliminate strong reference cycles:

  1. Class A with a property of class B, and class B with a property of class A;
  2. Capturing self in a completion block.

The solution is for the coder to manually identify one of the properties or the self with either weak or unowned. The decision to use weak versus unowned is determined as outlined in “The Swift Programming Language (Swift 5.2).” Apple Books. This decision appears to be related to either:

  1. “Because a weak reference does not keep a strong hold on the instance it refers to, it’s possible for that instance to be deallocated while the weak reference is still referring to it.'; or
  2. Whether or not the “the other instance has the same lifetime or a longer lifetime”

In practice, the rule of thumb is if the property is an optional, use weak. If the property is nonoptional, use unowned.

Proposed solution
Why force the coder to either decipher the description in “The Swift Programming Language", or to remember the rule of thumb? This decision should be made by the compiler once the coder indicates that they do not want a strong reference. Thus the reference type nonstrong would be interpreted by the compiler as weak when that is appropriate, and unowned when that is appropriate.

From The Swift Programming Language (Swift 5.2), Automatic Reference Counting, in the subchapters "Strong Reference Cycles Between Class Instances" and "Resolving Strong Reference Cycles Between Class Instances" the current solution is to:

class Apartment
weak var tennant: Person?

and

class CreditCard
unowned let customer: Customer

The current solution outlined above in this proposal will be replace with the new code below, using nonstrong:

class Apartment
nonstrong var tennant: Person?

and

class CreditCard
nonstrong let customer: Customer

This solution (nonstrong) is safer than the existing method (weak and unowned) as it results in the compiler implementing the weak or unowned reference type, and removes the possibility that the coder will select the wrong reference type.

The current solution (weak and unowned) creates the potential for crashing bugs, which is the situation that the solution was attempting to correct. The weak and unowned solution can create crashing bugs if the coder selects the wrong type. The strong reference cycle creates crashing bugs via memory leaks.

Detailed design
This proposed solution does not involve new syntax, nor is it a new API.

However, it is NOT an additive change, as both weak and unowned will be replaced by nonstrong.

The design is:

  1. Disallow "weak" and "unowned" as reference modifiers, these will become compiler errors;

  2. Allow "nonstrong" as a reference modifier;

  3. When "nonstrong" is encountered, use the logic in “Resolving Strong Reference Cycles Between Class Instances”, from Apple Inc. “The Swift Programming Language (Swift 5.2).” to determine whether to treat "nonstrong" as "weak" or "unowned".

Source compatibility
This change is a breaking change to existing Swift 5 code. The existing construct (weak, unowned) is harmful as it forces the coder to do work that rightfully should be done by the compiler.

The volume of affected Swift 5 code is not small, but we can (with significant work) provide an automated migration path.

Existing correct Swift 5 code will stop compiling due to this change. It is possible to continue to allow weak and unowned, see "Alternatives Considered" for why in this instance backwards compatibility is not optimal.

It is possible (with significant work) to automatically migrate from the old syntax to the new syntax.

Effect on ABI stability
I do not believe that this proposal changes the application binary interface (ABI) of existing language features. As the proposal author I'm not familiar enough with this issue to comment authoritatively, I welcome any additional input on this matter. If this proposal needs to be changed significantly or if there is significant work to investigate and document this ABI issue, such effort should result in co-authorship (if desired).

Effect on API resilience
I do not believe that this proposal changes the resilience of the application binary interface (ABI). Similar to the API stability section, I welcome any additional input on this matter.

Alternatives considered

  1. Leave current implementation as is.

  2. Wait until memory is zero-cost, so that memory leaks are irrelevant.

  3. Handle memory leaks due to strong reference cycles some other way.

  4. Leave weak and unowned as is, add nonstrong.

  5. Use "notstrong" rather than "nonstrong"

  6. Leave current implementation as is.
    This is the most obvious alternative, and will create the fewest problems short term. However, long term the existence of coders thinking "weak/unowned, WTF?" and "Well I'll just try weak and see what happens" violates the founding principles of Swift: "Swift makes it easy to write software that is incredibly fast and safe by design. Our goals for Swift are ambitious: we want to make programming simple things easy, and difficult things possible." (https://swift.org).

  7. Wait until memory is zero-cost, so that memory leaks are irrelevant.
    This is the the easiest alternative, but does not resolve this issue short term.

  8. Handle memory leaks due to strong reference cycles some other way.
    If this is the correct solution, it will require a different Swift Evolution proposal.

  9. Leave weak and unowned as is, add nonstrong.
    This is the most attractive of the alternatives considered, but results in the accumulation of unneeded cruft in Swift. The default assumption of backwards compatibility will hold back Swift by resulting in a large, unwieldy language and in an increase in the amount of mental overhead required by coders.

  10. Use "notstrong" rather than "nonstrong".
    Alternatives to the keyword "nonstrong" should be considered.

2

Note that the type of object is a consequence of choice between weak and unowned, not the other way around.

  • You want access to deallocated object to return invalid marker, therefore you use weak, therefore you use Optional, or
  • You want access to deallocated object to trap, therefore you use unowned, therefore you use the class itself.

Switching both to nonstrong likely won't fix the wrong-choice bug this proposal claimed to fix (much more that what weak/unowned currently does).

14 Likes
3

How would the actual semantics of nonstrong differ from weak or unowned? I don't believe the text you've pointed to in TSPL is precise enough to actually implement (and the text you're talking about should be included in the proposal).

Fundamentally, unless you have a way to actually implement this feature, it's unlikely it will go anywhere.

1 Like
4

I believe the idea is if the nonstrong property is an optional, nonstrong == weak, and if not, nonstrong == unowned.

In any case, @Lantua summarized my thoughts pretty well. Whether to crash or not should be a choice the user is forced to make explicitly, and not dependent on whatever type they happen to define for the given property.

3 Likes
5

@Jumhyn summarized the gist (jist?) of my pitch (if the nonstrong property is an optional, nonstrong == weak , and if not, nonstrong == unowned ).

I agree the implementation details need to be significantly clarified (obviously if I can't clarify them then the proposal is a non-starter). I'll work on the clarification.

6

Thanks to @Lantua and @Jumhyn for pointing out one of the issues with this proposal - that some situations want/need weak or unowned (so that they can get the correct return from the deallocated object). I'm still trying to feel my way to a solution that covers the case where the coder doesn't know the difference (which is where I am mostly) but wants to avoid a strong reference cycle.

7

Optional unowned properties are useful though: they can serve as a tree node's "parent" property. While weak wouldn't change the semantics of the program here, it would change the performance characteristics.

Maybe unowned should have a more forbidding name to indicate that "the program will abort if you access this reference after the object has been destroyed", but I don't think we automatically want weak semantics just because someone wrote an Optional type. Moreover, I don't think someone who forgets an Optional on a variable meant to be weak should automatically get unowned semantics; they should be told that they forget the Optional.

14 Likes
8

Just because weak properties must be optional doesn’t mean unowned properties must be non optional.

It’s quite possible that you want to have

class CreditCard
unowned let customer: Customer?

Say because your making an app for a credit card manufacturer and it’s not been sent to a customer or something.

Also how would capturing a variable in a block work?

3 Likes
9

A static (i.e. language-based/compile-time) approach isn’t really possible IMO. Strong reference cycles are perfectly fine, and even useful, as long as the cycle gets broken eventually.

A cycle which leads to a memory leak is a problem in the dynamic state of your program. I think the only way to really detect those problematic cycles is at runtime, using any of the available tools, and analysing each one to judge if it really is a problem.

3 Likes
10

You know, I do wonder about that. If the compiler could statically forbid strong reference cycles and force you to use unowned (specifically unowned), how much code / how many designs would actually have to change, and how hard would that change be?

(This is rather academic because I don't think the compiler can prove that sort of thing using reference counting at all, though it might be able to do it if you have unique ownership.)

11

Well, all of NIO has to change.

NIO is strong reference cycles all over the place. ChannelPipeline is a doubly-linked list (reference cycle) of ChannelHandlerContexts, many ChannelHandlers store a reference to their ChannelHandlerContext (reference cycle), EventLoops hold references to their Channels which hold references to their EventLoops (reference cycle), Channels hold references to ChannelPipelines which hold references to their Channel (reference cycle). Just about the only core type that is not part of a reference cycle in regular NIO is EventLoopGroup (because that’s just a fancy array).

All of those would need to be rewritten. You can see how you’d do it: the “back” pointers in ChannelPipeline become unowned, ChannelPipeline holds Channel unowned, users need to hold ChannelHandlerContext unowned in their ChannelHandlers, Channels hold their EventLoop unowned, and so on.

But frankly, that’s less elegant than the situation we have now. The strong reference cycles in NIO exist because the lifetime of these various objects is strictly managed. Channels have clear signals of when they are done and no longer in use because the file descriptor they relate to is dead, so it’s straightforward to tidy up all their strong references. And unowned isn’t really right: as long as any node in the ChannelPipeline exists, we don’t want nodes to get magically freed. The crashing bugs there would be just as bad as the risk of memory leaks today.

If unowned did not impose a performance cost, then we’d probably be ok living in the hypothetical world @jrose proposes. But in today’s Swift it does, so we’d rather take the cost of manually breaking reference cycles to keep a clear model of strong references to everything.

5 Likes
12

I'm fine with strong references, that's the default (correctly) for new properties and capture lists. If the strong reference leads to a memory leak I agree that's on the coder. My proposal is for the case where: 1) the coder realizes the default strong reference leads to a memory leak; and 2) wants a reference level that is not strong; and 3) doesn't know/care about the difference between weak and unowned.

So part of the reason for this proposal is to reduce the barrier to entry for Swift - by not requiring new coders to know memory management to the level of differentiating between weak and unowned. If there's a better way to do that than "nonstrong" I'm open to any ideas.

13

It could mean two things:

  • You never access dealloc'd objects. In which case, you should use unowned.
  • You don't know if you'd ever access dealloc'd object. In which case, you should use weak.

I'm not sure static analyser can help much with that. We already conflated the notions of don't know and don't care, which should actually have different behaviours.

2 Likes
14

Thanks, this is helpful. The above two lines should be in Swift 5.2 right at the start of the section "“Resolving Strong Reference Cycles Between Class Instances”. That would have reduced a lot of confusion on my part. I also agree that not knowing is significantly different from not caring, and Swift needs to treat those two cases differently.

To clarify for me, when you say "they should" is that for efficiency reasons? Is it even possible to change Swift to replace weak and unowned by nonstrong? I'm trying to figure out if the objection to nonstrong is: 1) it's not possible; or 2) it would result in inefficient code.

15

A full list would add another point:

  • You sometimes access dealloc'd object. In which case, you should also use weak.

Efficiency-wise there should be no impact since it'd just compile down to either unowned or weak under the hood.

Ignoring the performance aspect since I'm not an expert, there're still ergonomics reasons. Hypothetically, you can do away with only weak, but that means unnecessary Optional if you actually don't need it. At best, it'd just be annoying force unwrap, at worst, it'd means dead code that no one noticed. Something along this line:

if a { ... }
else if !a { ... } // Ok...
else { ... } // 😱

if let obj = weakButNeverNil { ... } // Ok...
else { ... } // 😱

It does look like a joke, and probably is, but I've seen my fair share of codes along this line, sometimes not an obvious one.

All in all, you want to use unowned if you provably won't access dealloc'd objects, and weak otherwise. And the provably is the tricky part to automate.

2 Likes
16

This is interesting, because unowned shouldn't really have more of a performance cost than normal (strong) references. It's weak that's noticeably slower. I can imagine that unowned hasn't quite been optimized as much, but was there an investigation about this?

(The rest of your post is still convincing, thanks for the thorough example!)

1 Like
17

No.

5 Likes
18

The second item in your list is often and commonly believed to cause problems, and many people capture self weakly almost on auto pilot without much reasoning.

However, while it is technically causing a retain cycle, that is almost never really a problem, because a completion handler is by its very nature coupled strongly to the idea of something being done. The completion handler is released after being called. Therefore, strongly capturing self, will cause its lifetime to be slightly extended, but not actually cause memory leaks.

This is true for completion handlers, animation blocks, one-off dispatch blocks, alert button handlers and other closures that are executed exactly once. It is almost always ok to use the default strong capture. In fact, blindly capturing weakly followed by a guard and early exit, very often leads to subtle bugs.

However, for event handlers, combine operations, signal handlers and other multi invocation closures, I think your point is much more valid.

In practice though, the programmer needs to reason about the lifetime and capture semantics of their object.

4 Likes
19

The usual explanation for weak capture of self in a completion block is the belief that this is more efficient. e.g., Why handle the network transaction's result if self has gone away?

20

Sure. There are many valid use cases for it.

But it has lead to people doing it almost on auto pilot, without reasoning. And it has lead people to believe that completion blocks should always capture self weakly. It has lead to numerous pitches to switch the default capture mode for closures. It has lead to an anti-pattern where subtle bugs often result in incorrect reasoning about the lifetime of objects, too eager return and dangling completion handlers that are never called at all.

1 Like