[Pitch] Weak let

Evolution Pitches
1

Hi! Here is a pitch based on the discussion about weak capture in sendable closures:

Introduction

Currently Swift requires weak stored variables to be mutable. This restriction is rather artificial, and causes friction with sendability checking.

Motivation

Currently swift classes with weak stored properties cannot be Sendable, because weak properties have to be mutable, and mutable properties are not allowed in Sendable classes.

Similarly, closures with weak captures cannot be @Sendable, because such captures are implicitly made mutable.

Usually developers are not aware of this implicit mutability and have no intention to modify the captured variable. Implicit mutability of weak captures is inconsistent with unowned or default captures.

Wrapping weak reference into a single-field struct, allows stored properties and captures to be immutable.

final class C: Sendable {}

struct WeakRef {
    weak var ref: C?
}

final class User: Sendable {
    weak let ref1: C? // error: 'weak' must be a mutable variable, because it may change at runtime
    let ref2: WeakRef // ok
}

func makeClosure() -> @Sendable () -> Void {
    let c = C()
    return { [weak c] in
        c?.foo() // error: reference to captured var 'c' in concurrently-executing code
        c = nil // nobody does this
    }
    return { [c = WeakRef(ref: c)] in 
        c.ref?.foo() // ok
    }
}

Existence of this workaround shows that ban on weak let variables is artificial, and can be lifted.

Note that resetting weak references on object destruction is different from regular variable modification. Resetting on destruction is implemented in a thread-safe manner, and can safely coexist with concurrent reads or writes. But regular writing to a variable requires exclusive access to that memory location.

Proposed solution

Allow weak let declarations for local variables and stored properties.

Proposal maintains status quo regarding use of weak on function arguments and computed properties:

  • there is no valid syntax to indicate that function argument is a weak reference;
  • weak on computed properties is allowed, but has no effect.

Weak captures are immutable under this proposal. If mutable capture is desired, mutable variable needs to be explicitly declared and captured.

func makeClosure() -> @Sendable () -> Void {
    let c = C()
    // Closure is @Sendable
    return { [weak c] in
        c?.foo()
        c = nil // error: cannot assign to value: 'c' is an immutable capture
    }

    weak var explicitlyMutable: C? = c
    // Closure cannot be @Sendable anymore
    return {
        explicitlyMutable?.foo()
        explicitlyMutable = nil // but assigned is ok
    }
}

Source compatibility

Allowing weak let bindings is a source-compatible change that makes previously invalid code valid.

Treating weak captures as immutable is a source-breaking change. Any code that attempts to write to the capture will stop compiling. The overall amount of such code is expected to be small.

ABI compatibility

This is an ABI-compatible change.

Implications on adoption

This feature can be freely adopted and un-adopted in source code with no deployment constraints and without affecting source or ABI compatibility.

Full proposal text - swift-evolution/proposals/NNNN-weak-let.md at mpokhylets/weak-let · nickolas-pohilets/swift-evolution · GitHub

See also - Thread safety of weak properties

Note that some of the errors related to capture of mutable weak variables may not appear or have different error messages because of a bug in RBI checking. See Let's debug: missing RBI data race diagnostics.

24 Likes
2

-1 from me.

At a high level, let indicates something doesn't change. It doesn't have a different value on two function calls, and it doesn't suddenly become nil. I feel like your proposal deviates from this.

final class User: Sendable {
   weak let ref1: C? // error: 'weak' must be a mutable variable, because it may change at runtime
   let ref2: WeakRef // ok
}

I don't like this idea.

if user.ref1 != nil {
    print(user.ref1 != nil) // Not necessarily "true" (assuming some multithreaded context)
}

let should mean it doesn't change. It's the reason why computed properties are vars. This proposed ideas feels like it runs counter to every other context of let in Swift.

func makeClosure() -> @Sendable () -> Void {
   let c = C()
   return { [weak c] in
       c?.foo() // error: reference to captured var 'c' in concurrently-executing code
       c = nil // nobody does this
   }
   return { [c = WeakRef(ref: c)] in 
       c.ref?.foo() // ok
   }
}

I think explicitly having to write WeakRef is in the same vein. c as the let it was originally declared as is gone. You have explicitly said "the c I now have is a WeakRef, which can change." That explicitness is a little more verbose than weak c, but it's clearer.
I'm more willing to be convinced on this part, but I'm definitely opposed to weak let.

2 Likes
3

let does not truly mean “unchanging” in Swift. In addition to the debatable case of class references, whose setters can be invoked even if they are stored in a let, Atomic values are mutable and in fact are required to be declared let.

3 Likes
4

Weak reference is a reference to a side table. It's a bit more complicated with ObjC weak references, but observable behaviour is the same.

So weak let is an immutable reference to a side table. Once assigned it will always point to the same side table. When object dies, reference to the side table does not change, changes the state of the side table. Changing state of the side table is thread safe. Changing variable to point to another side table (or even nil) is not.

8 Likes
5

I think in those two cases, let still maintains this property.
Classes are understood, conceptually, to be references. Thus, my reference never changes, although the values stored in that reference may. Atomics are similar. I have explicitly written that I have an Atomic reference to some value. Of course, the underlying value could change, in the same way as classes (conceptually, not literally), but that its understood based on the type I declared, which is still immutable.

1 Like
6

Atomics are not references. They are stored inline.

7

I feel like explicitly declaring the type as WeakRef communicates this, while weak let hides that explicitness. When I'm working with a WeakRef, it is clear that I need to check it is still valid before using it, regardless of whether it's a let or a var. weak let potentially becoming an invalid reference feels like mutation of what I understand to be the thing I have, IMO.

8

I'm talking about the semantics, not the actual implementations. Atomic<T> wraps something, so it is clear I need to abide by the semantics of that wrapper when accessing. More generally, if there is the possibility for the value to disappear/change in a let, that's expressed by the wrapper type.
weak let myVar = C() is a non-obvious "wrapper," since there is a possibility the reference to my class can disappear, which is different from the behavior of let myVar = C(). I don't feel weak let makes this clear enough.

1 Like
9

At a semantic level, you must understand a weak reference as its own kind of value. It is bound to a specific object, but when you try to read it, you cannot always retrieve the object anymore because the object might have gone away. That changeability is inherent to weak references. Nonetheless, it is still logically the same value and will remain so forever: a weak reference bound to the original object. That is not the same as being mutable because it cannot be replaced with a weak reference bound to a different object.

I understand why people are attracted to this idea that weak references are actually spontaneously mutating themselves. Mutating a class reference is a concept you’re already comfortable with, and if you think about the implementation, of course there must be some mutation there to allow the object’s memory to be reclaimed. But the idea doesn’t really work as a model of the language, among other reasons because it implies such a strange way of thinking about structs that contain weak references — you suddenly cannot ever talk about them as values again, because they are always variables that can be mutated. I have seen people go so far as to say that Swift shouldn’t allow weak references in structs because of this, all in service of trying to hold on to their mental model of spontaneous mutation.

19 Likes
10

I'm going to retract my -1. I believe I mentally mistook WeakRef as defined above with being a standard language feature. My apologies. I think the need for immutable weak references, in some form, is necessary for the language.

I will, however, point out the need for better explanations for beginners to understand weak references:

At a semantic level, you must understand a weak reference as its own kind of value.

You say "must," but I think many beginners get by with the mutable understanding, however flawed. Until they start proposing to remove them from structs, this understanding is satisfactory for resolving cyclical references and whatnot.
Incorporating weak let forces the deeper understanding. I'm not opposed to forcing a more fundamental understanding, but it is one more thing for newcomers to understand that may not be necessary.
Something like an official WeakRef<T> makes that difference explicit. weak let also makes it explicit, but doesn't alert new users as much about the potentially non-existent reference. It certainly is cleaner and more consistent with what we currently have, though.

11

This description resonated with me. Can I propose a pseudo-Swift translation of it to check my understanding of these semantics?

A type containing a weak property, e.g.

class C {}

struct Container {
    weak var c: C?
}

is roughly analogous to

struct Container {
    var c_storage: WeakReference<C>
    var c: C? {
        get { 
            c_storage.isDeallocated ? nil : c_storage.object
        }
        set {
            c_storage.object = newValue
        }
    }
}

where

// Hand-waving:
final class WeakReference<Object> {
    var object: Object // Special retain semantics; gibberish if `isDeallocated == true`
    var isDeallocated: Bool {
        goAskASideTableIfObjectIsRetainedElsewhere(object)
    }
}

And in the above translation, using weak let would be equivalent to removing the computed set on var c, i.e.

struct Container {
    weak let c: C?
}

// Becomes:
struct Container {
    var c_storage: WeakReference<C>
    var c: C? {
        c_storage.isDeallocated ? nil : c_storage.object
    }
}

i.e. once set, the underlying object cannot change.

Imprecision acknowledged, is this a reasonable mental model?

3 Likes
12

Yes, that's a pretty good model. I think it captures all the details relevant for this proposal.

EDIT: Not exactly, see important correction by @John_McCall below.

13

Property wrappers only work for var and not let. IIRC, that's because let carries the connotation that the variable value itself cannot change. If weak let is allowed, it would make sense for let to be allowed for property wrappers too.

14

I've always thought that property wrappers only work for var, because property wrappers make that property computed, and computed properties can only be declared with var. The reasoning for the latter I don't know.

1 Like
15

That's very close, yes! I would suggest a few very subtle refinements:

struct Container {
    // Note that this is optional: the weak property can be nil
    // because its referent has been destroyed, but it can also just
    // be nil because you stored nil into it.
    var c_storage: WeakReference<C>?
    var c: C? {
        get {
            guard let ref = c.storage else { return nil }
            return ref.isDeallocated ? nil : ref.object
        }
        set {
            if let newObject = newValue {
              // Note that we make a new reference instead of
              // modifying the existing one. `Container` still has
              // value semantics: properties of copies of the same
              // value have to be independent.
              c_storage = WeakReference(object: newObject)
            } else {
              c_storage = nil
            }
        }
    }
}
11 Likes
16

weak let might seem nonsensical at first, but I think a good part of it is that Swift conditioned us over the years into thinking it's nonsense by telling us: "'weak' must be a mutable variable, because it may change at runtime".

Unfortunately we were all in the wrong and the earth is round. If new Swift doesn't mind contradicting old Swift, then it's perfectly fine.

4 Likes
17

I think the problem here (As outlined by your post) is that Sendability requirements are too strict, or maybe the checks being performed by the compiler should be changed a bit.

If you have a private weak var the only place it can be updated is from within the class itself. Maybe we should be showing compile errors at the point of mutation, rather than the point of declaration.

For example:

if you have:

class Bar: Sendable {}

class Foo: Sendable {
      // this variable is private, AND Sendable, should we really be showing an error here?
      private weak var bar: Bar

      init(bar: Bar) {
          self.bar = bar
      }

      func updateVariable() {
           // Should we not instead be showing the error here?
           // error: updaing mutable property 'bar' on Sendable nonisolated class `Foo` can cause data races
           bar = Bar()
      }
}
18

Yeah, good point. But that brings up the question of why computed properties are declared with var instead of let. A computed property without a setter or with a nonmutating setter is guaranteed to never formally mutate whatever backing storage it relies on, but it still has to be declared with var because (I assume) it communicates that the returned value can still vary, even if the backing storage is immutable. All the ways that the semantics of weak are being spelled out in this thread require the use of a computed property, which in my opinion shows that weak has the semantics of a property wrapper.

I guess var has a bit of a double meaning. For stored properties, it communicates the possibility of formal (exclusive) mutation. For computed properties, it communicates the possibility of any kind of mutation at all, including shared mutation; only the presence of a mutating setter indicates the possibility of formal mutation. For example, Atomic has to be declared with let, despite the value being mutable, because it allows for shared mutation instead of just exclusive mutation. But a hypothetical Atomic property wrapper would have to be declared as @Atomic var a, because the returned value can change, even though the backing storage is never formally mutated.

1 Like