class C: DDelegate {
init(d: D) {
d.delegate = self
}
func wasCalled() {
print("blah")
}
}
protocol DDelegate: AnyObject {
func wasCalled()
}
class D {
weak var delegate: DDelegate?
func call() {
delegate?.wasCalled()
}
}
func test() {
let d = D()
let c = C(d: d)
d.call()
}
test()
Compiling and running this program produces the expected output:
% swiftc -O test.swift
% ./test
blah
%
In Swift today, the line let c = C(d: d) receives a warning: "Initialization of immutable value 'c' was never used; consider replacing with assignment to '_' or removing it". However, following the advice of the compiler, and changing this line to read let _ = C(d: d) produces different behavior!
% swiftc -O test2.swift
% ./test2
%
This feels like a bug to me, but before I file I was curious if anyone knows of a way to silence this error.
That's because C was deinited at different times. Compiler can remove C after the last use, which means immediately after it's constructed. The fact that it happens slightly later in the first example is a tiny missed optimization opportunity, but not a bug.
If you want to make sure C is alive for some time, use the withExtendedLifetime function.
Ahh, right. I learned about that "last use" rule (as opposed to "end of scope," which is the model I had before) a couple weeks ago, but missed the fact that it applied here. Seems like withExtendedLifetime should work nicely!
Nice. In light of @cukr's response, this line would also have to be placed after d.call() in order to guarantee that c stays alive.
Right, I just meant that if that strategy is used just to silence the warning, it has to be placed at the end of the function—I agree that withExtendedLifetime is the better approach!
No, it will silence the warning regardless of where it is placed.
Furthermore, it will not guarantee the lifetime of c, regardless of where it is placed. The compiler is free to rearrange the lines “_ = c” and “d.call()”, because semantically they do not affect one another.
Ah, I elided the rest of the thought which was "...in order to fix the underlying issue that my code was exhibiting".
Is this documented somewhere? That seems somewhat circular—clearly they do semantically affect one another, since if c is used after d.call(), the program will print "blah" and if c is not used after d.call() the program may print nothing at all. The order in which the loads and stores actually occur (or not) post-optimization is irrelevant to whether the lifetime of c outlasts d.call().
The closest you have is threads with Swift developers in which they said, in effect, that when an object is deinited is not considered to be observable behaviour from the perspective of the optimising compiler. That is, the compiler is not required to enforce that optimisations do not change the order in which things happen with respect to deinits being invoked.
The rules here may not be obvious but they are clear once you know them: you must not rely on deinit being called at a specific time. It may be called later than you expect, or earlier, or not at all. The only ordering rule that applies to deinit is that deinit of an object will always happen before deinit of an object it holds. That’s it.
Well, there’s also the “deinit will not be called before the last use,” rule, which makes the question here subtly different than just the ordering of deinit. Specifically, is the ordering of last use not considered observable behavior? I.e. is c not guaranteed to live at least as long as d if, in source order, c is used after d?
Not necessary. If the usage does not require that the object is actually alive, then it's acceptable for that object to die early. For example, if the "last use" of an object is to assign it into an array that is never read from. Arrays that are never read from can just as easily not exist (they don't have any effects that count from the "as-if" rule perspective), so they are free to be optimised out. If they are optimised out, then the store is now meaningless (again, the "as-if" rule indicates that we couldn't observe that the object was stored in the array: if we could, it wouldn't have been legal to optimise the array out of existence). Thus, from the perspective of when to call release, the store to the array is completely irrelevant: it can be treated as not existing.
I'm not convinced that this follows logically from the previous lines (though it may very well be the model Swift uses)—is this also based on authoritative statements you've seen here? It would also be a perfectly reasonable model for the store to be eliminated while leaving the retain/release ordering according to source order.
(Not intending to come off as overly contradictory here, I'm just interested in developing the most accurate model I can since this has bitten me a couple times now!)
a reference is guaranteed to be alive while at least one strong reference remains.
references are considered dead at the point the compiler can prove they are no longer accessed.
Thus in the original example, the local reference c is dead right after the line which declares it. It is provable by inspection that test() does not access the reference after that point.
We know that Array is a class-backed-struct. That is, it's defined approximately like this:
public struct Array {
private var backingStorage: BackingStorage
class BackingStorage { }
}
This means that creating an Array must allocate storage for that backing class. The backing class is also where the elements go.
It also means that passing an Array around incurs reference counting traffic. You can see this by considering this code:
class F {
var arr: [Int]?
}
As shown in Godbolt, the getter for arr emits a call to swift_retain. What is it retaining? The answer is, the backing storage for the array.
I point this out to indicate that creating an Array implicitly creates an object with managed lifetime. That backing object essentially has a deinit. This will be relevant.
Returning to the code from part (1), we notice that the optimiser has not just improved the math, but it has eliminated the array entirely. The array no longer exists in this code. That's meaningful, because it means the Array backing storage doesn't exist either.
For the Array backing storage to not exist, the compiler must have eliminated a release. This would have been the release that caused the array backing storage to vanish. Of course, as the array backing storage never came into existence, it never needs to vanish. Thus, we can conclude a new rule, derived logically from the above: malloc/free/retain/releasecounts are not part of the observable behaviour of a Swift program. It is acceptable to change them. If this were not true, we could not avoid constructing the Array in our sample program, and we'd like to avoid it because it makes the code way faster.
So what, you say? You didn't ask about whether the retain/releases could be removed, only if their order persisted! Well, there are two follow-on points.
The first is to consider a case where one object is eliminatable, but the other is not. Let's update our code:
func test() -> (Int, Int?) {
let x = [1, 2, 3, 4]
let y = [5, 6, 7, 8]
let second = y.randomElement()
let first = x.reduce(0, +)
return (first, second)
}
This new program has the original test program contained within it, but now creates a different array and calls randomElement on it. To call randomElement we must actually have an Array: this function is complex enough to require the real object.
That real object has an unambiguous last usage site: the line let second = y.randomElement(). That unambiguous last usage site unambiguously precedes the equally unambiguous last usage site of array x, which is the line after.
This now represents a problem. The rule we're investigating is "can an optimizing compiler change the ordering of releases". In this context, we need y to be released before x, as that's the source order of the last usage sites. However, we also want x to be eliminated entirely! It should have no releases. What to do?
If we want to have the rule "x must be released after y", that rule must require that x actually be allocated. It's the only way to satisfy the rule. We could have a more nuanced rule, "x must be released after y unless x can be proven not to be needed at all", but that's a weirdly specific rule, and very hard for a compiler to actually implement.
So, what happens? Godbolt to the rescue again: we see clearly that both a) array y is created, retained, and released; and b), array x still does not exist.
This is incompatible with the clear "retain/release ordering follows source order". It could be compatible with the less clear rule involving complete compiler elimination of the object, but that rule is, as noted above, not really plausible for a compiler to implement.
Where does this leave us?
Well, we can see that not only can Swift say that the "last usage site" of an object is well before its last usage site in the code, we can see that Swift can say that an object has no usage sites at all! It can even do this in cases where the object is clearly used: reduce is implemented on top of a bunch of Array machinery, but the Swift compiler can eliminate all of it.
We can also see that we want Swift to be able to do this. It makes our programs much faster! Not allocating an intermediate Array here is great: we can spell things clearly to signal intent, but Swift can give us the behaviour we want quickly.
But if Swift can say that objects have no usage sites, it follows that Swift can also say that objects that appear to exist in the source will never have their deinit called. They never existed at all. Thus, we can conclude that having deinit called cannot possibly be an observable behaviour from the perspective of the Swift compiler.
If having deinit called at all is not observable, it cannot be that deinits must be called in source order. If that was the rule, we'd defeat these optimisations that involve eliminating objects entirely. (As a side note, we'd also eliminate a whole class of cases where CoW's can be elided, forcing way more memory traffic than we currently see.)
Here is my mental model of deinit, which so far as I can tell is all that is promised:
deinit will never be called more times than init
Each object will have deinit called no more than once
deinit will not be called while necessary strong references still exist (that is, where the strong reference cannot be eliminated without violating a different language invariant).
If object A holds a strong reference to object B, the deinit of object B cannot be called before the deinit of object A. Note that if A has been eliminated entirely, the strong reference never existed and so this rule does not apply.
That's it. Note that these rules do not include:
deinit will be called. Swift does not promise this! deinit may never be called.
deinit will be called at the end of function scope. As we've seen before, deinit may be called well before that.
deinit will be called once per class object created. As we see above, Swift is free to eliminate the class instance altogether.
deinit will be called in a timely fashion. deinit may be arbitrarily delayed if Swift choses to do so.
deinit will be called after the last appearance of a strong reference in source. Swift is capable of shortening the lifetime of strong references if it can prove they are unneeded.
Thanks for the comprehensive write-up @lukasa. Super helpful!
It's not actually immediately apparent to me why this would be difficult to implement (despite the fact that it would maybe eliminate certain optimization opportunities). The machinery for eliminating allocation entirely is obviously already available, and "last use" ordering for local variables is plainly apparent from the source. What additional complexity am I missing?
Do you know how the reference count/swift_release fit in here? If swift_release is called and decrements the refcount to 0, is deinit guaranteed to be called at that point (i.e., are these rules more accurately about the placement of calls to swift_release)?
I'm not a compiler engineer, so I'm working with one hand tied behind my back here. But my understanding is that the algorithm required to implement a conditional rule like this is a lot more complex. In particular, it requires keeping track of a lot more state.
It can also inhibit dead-code elimination. To eliminate the array in current Swift it's sufficient for the compiler to observe that the array does not invoke any code that leads to an observable effect other than the math, and so it can simply hoist the math out and remove the array. If retain/release ordering were considered an observable effect, the dead-code elimination pass can't do this anymore: there exists a swift_release for this array, so the array cannot be eliminated.
This means we need a compiler pass that looks for things that could have been dead-code eliminated but weren't, and then remove their retain/releases so that dead-code elimination can be unblocked. This makes optimisation pipelines longer, compiles slower, and risks us failing to achieve dead-code elimination in some cases where there weren't enough passes.
I wouldn't dismiss the optimisation penalties either. Any time Swift calls out to code it cannot see (e.g. that comes from a library or framework) it cannot know what happens in that code. That means it has to assume that that code may do anything, including call swift_release. As our rule ("releases occur in source code order") does not exempt function calls to other code, we can never move a release earlier than the last opaque function call in a function body. That's a pretty serious barrier: in practice, it'll cause us to lose many optimisation opportunities.
Yes, that's correct. The rules are not actually about deinit but about calls to swift_release.
I can't promise that deinit is guaranteed to be called: Swift does not have a formal language specification, so that kind of promise is meaningless. However, in practice, I don't believe swiftc will ever diverge from that behaviour.
Ah, in my imagined model the dead-code pass would not be affected—it would just be the ARC pass's responsibility to properly order any swift_release calls for the objects that survive the dead-code pass.
The rule I'm imagining is closer to "swift_release will not be called on an object at least until the site of last use (according to source order)". So if foo(x: AnyObject) were an opaque function call in the following snippet.
class C {}
func bar() {
let c1 = C()
let c2 = C()
let c3 = C()
foo(c2)
foo(c1)
_ = c3
}
We would have that c3 cannot be released before c1, and c1 cannot be released before c2. However, the compiler is free to a) eliminate c3 entirely (since swift_release is never called, the rule is not violated), and b) release c2 before the call to foo(c1) (since any uses of any references inside of foo occur after the last use of c2). Anyway, my speculation about a hypothetical rule isn't particularly productive
Ultimately, I'm less concerned about late-release than I am about early-release, but I suppose it all comes back to "if you need to guarantee the lifetime then just use withExtendedLifetime." For some reason reaching out for a library function is just not that satisfying to me—I wish we had something like objc_precise_lifetime in Swift.
Thank you for taking all the time to write these extensive responses! Will be very helpful to refer back to when I inevitably hit an issue like this again in a few months
