Also note that promise to have stable identity of live proxy objects is not the same as to promise of stable ObjectIdentifier’s produced out of them.
The first one is nice to have and probably could be implemented without unbound memory growth using weak references. Simple implementation might use a dictionary with weak references as values, and prune dead keys from time to time. In this case you are promising clients that if they have a live instance, it’s identity is stable. And if clients don’t have a live instance, they should not care if it’s stable or not.
The second one is not needed, IMO. Even with non-actor classes, you can observe memory address being recycled for object of the same or different class. It is already not safe to let ObjectIdentifier escape, unless you are taking additional measures to ensure that instance stays alive.
There is also identity of weak references (address of the side table). There are some applications where this is needed, but currently it does not really work in Swift anyway (see Hashing Weak Variables). But if it would work, stability of weak references cannot be implemented exactly for the reasons you mentioned.