Is the implicit `self` passed to instance methods always strongly retained?

Using Swift
, ,
1

This is an almost “trivial” thing (and I think we all know the answer), but even after hours of digging, I’m surprised I found absolutely NO clearly stated guarantee in the Swift language official specification about the lifetime of self (as well as other pass-by-reference parameters) on an instance method.

In other words, for the following code:

class MyClass {

    func instanceMethod() {
        print("Method start")

        Thread.sleep(forTimeInterval: 2.0) // Imagine during this sleep, all strong refs of self disappear. 

        // Question: Is 'self' guaranteed to remain valid 
        // throughout this entire method execution?
        self.doSomeWork() // is self still here?

        print("Method end")
    }
    
    func doSomeWork() {
        // Additional work using self
    }
}

What happens in the following cases?

If instanceMethod indeed starts executing, how do we know doSomeWork will run? (I know the answer is yes, but who says so?)

// Case 1: weak reference
let closure = { [weak myClassInstance] in 
    myClassInstance?.instanceMethod() // does doSomeWork execute?
}

// Case 2: unowned reference
let closure = { [unowned myClassInstance] in 
    myClassInstance.instanceMethod() // does doSomeWork execute?
}

What I have pieced together:

  • This forum post suggests a strong reference is loaded on weak reference accesses – the SIL emits strong_retain (or similar). Joe Groff states they are released at or after “the last formal use of the strong reference” (what precisely does this mean?
  • This reply also seems to support the idea of a lifetime extension for the object for a method
  • The Swift Runtime correspondingly contains methods such as swift_weakLoadStrong and swift_unownedLoadStrong method. This suggests that unowned and weak references generate strong references for some duration
  • We know that since Swift 4.2, the calling convention for methods has shifted to Guaranteed +0 from +1. This means the callee method no longer has a retain for its parameters within its body.
  • Swift SIL – “A guaranteed direct parameter is like an unowned direct parameter value, except that it is guaranteed by the caller” – but this says nothing about the behavior of ARC in the caller, especially relating to weak or unowned references.

Can we add a guarantee about this in the ARC chapter of Swift language specs? Or is the (almost bedrock for me) assumption about ARC and Swift not actually something that we can take for granted?

2

Yeah, it's gotta be that way. Otherwise, you'd have to guard against self being nil or use question marks all over the place in every instance method.

@Test func test() {
  final class MyClass {
      func method(_ otherInstance: inout MyClass?) -> Bool {
        otherInstance = nil
        return MyClass.value(self)()
      }

      func value() -> Bool { true }
    }

  var instance: Optional = MyClass()
  let value = MyClass.method(instance!)(&instance)
  #expect(value)
  #expect(instance == nil)
}

I think that desugared syntax helps for understanding?

final class MyClass {
  func instanceMethod() {
    MyClass.doSomeWork(self)()
  }

  func doSomeWork() { }
}

let closure = { [weak myClassInstance] in
  guard let myClassInstance else { return }
  MyClass.instanceMethod(myClassInstance)()
}

let closure2 = { [unowned myClassInstance] in
  MyClass.instanceMethod(myClassInstance)()
}
3 Likes
3

For strong references by themselves, this is just basic semantics in a memory-safe language. Values are not allowed to be corrupt when you go to use them, and a reference to an object that’s been torn down and deallocated would be corrupt. Swift doesn’t generally guarantee exactly when and how it will perform retains to make sure references aren’t invalid, but unless you’re using unsafe features incorrectly, you should never be exposed to an invalid reference when you go to use a value of a strong reference type.

Weak and unowned references complicate this, not in that they permit corruption, but in that they allow you to observe whether objects have been deallocated. So if you’re using not a strong reference, but a weak or unowned reference, it’s material whether some strong reference that you’re not going to use again but is still formally in scope is guaranteed to keep the object alive. Generally, the answer is no, but we have come to an informal consensus that self is special here. @Andrew_Trick can speak to this.

So, the first thing is that it runs because there’s no control flow in this block that would stop it from running. If self were somehow an invalid reference here, the method call on it might crash, but it’s not going to fail to run in any kind of well-defined way.

The second thing is that the self value must be valid at this point because it is used by the call. By the basic definition of memory safety, Swift must ensure that values are valid when they are used.

At an implementation level, a particular scope may own the responsibility to destroy a value. For example, loading a weak reference performs a retain that the local context is responsible for balancing with a release. The safety rule means it must not actually do this until it can prove the value will not be used again within the scope. Swift generally does not make methods own this responsibility for parameters like self; instead, the caller is responsible for ensuring the reference stays valid during the call. In the absence of inter-procedural optimization, this prevents self from becoming invalid at any point in the function, even if it is not used.

If the function is inlined into its caller, and the caller owns the responsibility to destroy the value it passed as self, it could theoretically do so within the inlined body (but only after all uses there, of course). This could be observed by a weak reference, and it could also potentially invalidate values loaded from self, e.g. if it has a deinit that closes a file descriptor or frees an unsafe pointer. However, as I mentioned above, we have an informal consensus that self, at least in class methods, guarantees its validity for the entire method, as if it were used at every point. This is mostly because of the dangers of invalidating values loaded from self, although the fact that Swift naturally produces reliable lifetime semantics for self in the absence of optimization also just makes it hard to justify breaking them under the likely-to-be-rare conditions of the method being inlined and the optimizer deciding it’s profitable to move the release earlier.

7 Likes
4

John unsurprisingly said it best. I think it’s useful to note that this is an improvement over Objective-C, in which it’s entirely possible to drop the last strong reference to self before the end of an instance method, even with ARC enabled.

2 Likes
5

Calling a method requires a strong reference (like Danny shows above and John explained). Because self is a strong reference inside the method, it remains valid up to the last (strong) use within the method. The weak case may fail to run the method, and the unowned case may crash simply because the reference is already invalid before the method call. Whether the ARC convention is +0 or +1 does not change the programming model, and may change between compiler releases. Of course, if you are relying on unspecified behavior, the ARC convention may affect that.

The language allows values to be destroyed any time after their last use (not including weak/unowned references). Beyond those constraints, lifetimes are implementation defined. As the optimizer improved (namely due to SIL-OSSA form), it became important for the compiler implementation to follow more conservative rules to avoid widespread source breaks at -O. Since 5.7 (2022) the compiler introduced the concept of a deinitialization barrier within a variable's lexical scope. We refer to these rules under the potentially misleading name "lexical lifetimes":

FORUM: What is the current status of ‘lexical lifetimes’?

Shortening the lifetime of self within a method was one of the optimizer behaviors that developers found most surprising. But we saw similar confusion with other local variables. Now the optimizer treats self just like any local variable. If any weak or unsafe pointer access occurs within the variable's scope, and might refer to any object kept alive by the variable, then the optimizer conceptually extends the variable's lifetime to cover those accesses. Deinit barriers are irrelevant in OP's instanceMethod because there are no weak or unsafe accesses inside the method.

These aren't formal language rules because we may decide to refine what constitutes a deinit barrier, and there are some outright exceptions:

  • CoW types, which themselves aren't formalized, but cover Array, Set, Dictionary, and String, have "optimized lifetimes"
  • ~Copyable values that may have a custom deinitializer have "strict lifetimes"
  • ~Escapable values have "optimized lifetimes" (unless they are also ~Copyable + custom deinit)

Optimized lifetimes give you no guarantee beyond the language's last use rule. This can trip up programmers who build an Array of weak references expecting the array to keep those object alive even though they never use the array.

Strict lifetimes treat the value's deinitialization as a proper side-effect (fully ordered), similar to other non-GC languages that have "synchronous deinitialization". This is consistent with non-Copyable being a way to opt-out of all the ARC baggage.

If you're aware that your code has unsafe/weak references and is relying on some Copyable local variable to keep them alive, then I recommend following the formal language rules and making that explicit with extendLifetime(variable). This includes extendLifetime(self) if needed.

4 Likes
6

Hey folks, thanks for the detailed explanations given. Unfortunately, the depth of detail makes it a bit hard for me (even as an experienced programmer) to follow and get a clear-cut answer to what I was trying to ask.

In the spirit of Swift’s progressive disclosure, and for novices reading this in the future, I am hoping to clarify things.

I’ll start with what I think answers my question at the language level:

In my sample code’s closure , I am calling the method via an unowned reference. I clearly don’t have a strong reference –

  1. Are you implying that my unowned reference is upgraded to strong inside the body of the method?
  2. If so, is this formal, specified behavior in the Swift language? Please link and quote the line – the very reason I made this post was because I never found such a statement.

To double-check my understanding with the following example:

class MyClass {
    func someMethod(with otherClass: MyOtherClass) -> (() -> Void)  {
        // do stuff using self and otherClass -- are they both valid throughout this methods body?
    }

    func createEscapingClosure() -> (() -> Void)  {
        let myOtherClass = MyOtherClass()
        var someEscapingClosure = { [unowned self, unowned myOtherClass] in
            self.someMethod(with: myOtherClass)
        }
    }
}

You’re saying that if the closure returned from createEscapingClosure is invoked, and we don’t crash before we get into the body someMethod , the self and otherClass objects remain valid for the entire body of someMethod

What if, in the body someMethod , we created an escaping closure capturing the otherClass – is the lifetime of the otherClass object extended then?

I apologize, but I’m struggling to understand what this statement even means – What is this “basic definition”? Does the “Swift” in this sentence mean the Swift compiler, or the Swift runtime (i.e. crashes if not valid)?

I assume you meant to say instance methods here – AFAIK self is not a keyword in class methods. If so, why is this an informal consensus?

Right, I’ve read this many times, but I take it as an assumption ARC can make when compiling the callee code – it says nothing about what ARC must have the caller do. Is there something in the ARC spec that enforces retains on parameters in the caller code when it calls the callee?

7

Well, this is tricky, because you seem to want us to say “the compiler has to retain here and release here”, but that is not the actual language rule. The language rule really is this vague principle that values must be valid when used, and if the implementation can satisfy that while eliminating unnecessary retain/release pairs, that’s good.

Defining the rules as strictly requiring retains and releases at specific points would force a ton of overhead that’s unnecessary to achieve the basic safety goals of the language.[1]

  1. Except under artificial constraints caused by unreasonable uses of unsafe APIs. You can always invent scenarios like “I know there are formally 10 strong references to this object right now, so it should be okay to spontaneously release it 9 times and then balance that with 9 retains before continuing.” That would forbid any static optimization of reference-counting operations, but there’s no good reason to allow it instead of using the more efficient rules admitted by a straightforward system of ownership. ↩︎

3 Likes
8

At first I was missing the nuance of this question, but now I realize that I’m not totally sure I know the answer because I haven’t ever used unowned references much. I think the answer is that you are correct - that if you invoke a method on a not-yet-released unowned reference, or if you pass a not-yet-released unowned reference to a function, then in both cases the referenced instance is retained and guaranteed to exist at least for each usage inside the method/function. It sounds like there isn’t a guarantee that it will be retained for the full body of the function, only through the last usage.

2 Likes
9

I would be happy if you can say something like:

”The compiler ensures that if a method is called, then within the scope of the method body self and all direct parameters to that method are semantically strong references”

As @Danny said, “it’s gotta be that way”. Why not make this formal?

10

There's a lot of complexity in how you're thinking about this, and it might be good for you to find answers for all of it. I just think about it like this:

"Parameters can't be marked as weak or unowned like stuff in a capture list can be."
together with
"self is a parameter."

@Test func test() async {
  final class MyClass {
    func method(_ reference: inout MyClass?) {
      reference = nil
      _ = self
    }

    lazy var closure = { [unowned self] (reference: inout MyClass?) in
      reference = nil
      _ = self
    }
  }

  await #expect(processExitsWith: .success) {
    var instance: Optional = MyClass()
    instance!.method(&instance)
  }

  await #expect(processExitsWith: .failure) {
    var instance: Optional = MyClass()
    instance!.closure(&instance)
  }
}
1 Like
11

weak/unowned is not part of the type, it's a storage modifier on the variable. You can't do anything with a weak/unowned reference other than store it in a weak/unowned variable and unwrap it. Unwrapping gives you a strong reference. unowned references are just like weak references but implicitly unwrapped every time you use them. In both cases, the caller needs to create a strong reference to call a method because a method’s self parameter is a strong reference.

2 Likes
12

This would be significantly different from how ObjC works, where calling a method on an unowned reference simply loads the reference into x0/rdi and invokes objc_msgSend(). This is the cause of many real-world application crashes. I suspect this is where @winstondu’s question is coming from.

It sounds like you are explicitly saying that unowned let foo; foo.bar() must atomically fetch-and-retain foo before calling bar. And since retains must be balanced, it will release foo. The only reasonable point to do this is after bar() returns. And if bar() enqueued any asynchronous work, it would be helpful to have written in blackletter language law that self is a strong reference, because that implies the lifetime of self extends to that asynchronous work.

13

The question about this was the title of the post, and I’m glad we have an answer. Is this statement in the Swift spec? If not, can we put it there?

In addition, what about the other (non-value type) parameters passed to methods? Are they strong references too?

14

What document are you referring to as the “Swift spec”? The Swift Programming Language ‘book’? Swift doesn’t have a formal spec.

15

The only time self is special is within computed property bodies, which can't take any parameters, and subscripts, which can't be used via curried syntax. It's overly complicated to think about self as not-just-another-parameter. If parameters weren't strong, you couldn't, for example, return them as non-optionals. weak and unowned do not pass through the type system that way.

@Test func test() {
  final class MyClass {
    func implicit(reference: inout MyClass?) -> MyClass {
      reference = nil
      return self
    }

    static func explicit(_ self: MyClass) -> (_ reference: inout MyClass?) -> MyClass {
      { reference in
        reference = nil
        return self
      }
    }

    subscript(_ dereference: () -> Void) -> MyClass {
      dereference()
      return self
    }
  }

  do {
    var instance: Optional = MyClass()
    _ = MyClass.implicit(instance!)(reference: &instance)
    #expect(instance == nil)
  }

  do {
    var instance: Optional = MyClass()
    _ = MyClass.explicit(instance!)(&instance)
    #expect(instance == nil)
  }

  do {
    var instance: Optional = MyClass()
    _ = instance![] { instance = nil }
    #expect(instance == nil)
  }
}
16

All class types are strong reference types. If you have a value of a class type, you have a strong reference. self in a class instance method is a value of class type, so it is a strong reference. With a strong reference, Swift always acts to ensure that something is locally guaranteeing the validity of that reference. The only way to break that is with an unsafe feature like Unmanaged or unowned(unsafe).

weak and unowned variables do not store strong references. They store weak and unowned references, respectively. When you load one, you turn the weak or unowned reference into a strong reference, and Swift thereafter manages the strong reference safely like any other.

This is all definitely quite different from ObjC ARC, where values of ObjC pointer types are still basically unsafe, and the compiler is just enforcing the ownership transfer conventions around calls and the invariants of __strong variables. There are quite a few ways to break the safety of ObjC ARC.

4 Likes
17

Yes, that would be the preferred place to put this.