Classes, stored properties, pointer conversion and address stability

Using Swift
1

consider code such as this, which wraps an os_unfair_lock in a class:

final class MyLock {
    private var lock = os_unfair_lock_s()

    func lock() {
      os_unfair_lock_lock(&lock)
    }

    func unlock() {
      os_unfair_lock_unlock(&lock)
    }
}

this particular pattern is explicitly called out as problematic in many places [1]. as i understand things, the primary concern is that, since os_unfair_lock is a struct in Swift, the 'inout-to-pointer' conversion that occurs when passing the stored property via the & operator is not guaranteed to produce a pointer with a stable address [2].

now, i'm all for following the rules/guidance here – having locks not work as expected sounds terrifying. however, in my investigations into these sorts of patterns, i've yet to find a way to actually demonstrate the problematic behavior, at least in the circumstances in which i'm interested. in the examples i've tried so far, the 'inout-to-pointer' conversions of stored struct properties within classes appear to have stable addresses.

does anyone know (or have ideas about) how to construct an example that:

  1. uses a class with a stored struct property (without a custom setter)
  2. passes the struct somewhere as an unsafe pointer via an 'inout-to-pointer' conversion
  3. produces different pointers via this process over time

alternatively, can you provide rationale that this cannot happen (under whatever assumptions)? i assume part of the motivation for steering people away from this pattern is that the language simply does not promise anything about this behavior, and so one should conservatively avoid relying on it working in a particular manner.

any ideas and insights would be greatly appreciated – thanks in advance!

  1. see 'The Peril of the Ampersand', 'The Law', this prior thread, and the OSAllocatedUnfairLock documentation for example. ↩︎

  2. i am aware that there are other concerns around values being copied, and exclusive memory access overhead, but my primary question here regards address stability. ↩︎

2

My reading of the stuff in the links you provided (which was a fairly quick scan, not in depth study) is that none of them talk about the pattern where a value type is a member of a class instance.

IOW, none of them are "this particular pattern".

Let me try to restate the problem in the most informal terms…

In Swift, when you apply & to a value-typed value, Swift needs to "materialize" a memory location for that value, if the value isn't somehow known to be "materialized" already. The resulting memory location may be transient, so you have to be careful about anything that makes assumptions of its longevity.

In the case of a class member, the value-type value is essentially "pre-materialized" because the class instance is itself necessarily materialized on creation in order to construct a reference to the instance.

That's going to make it very likely that the value's memory location always ends up being the same thing, and to make it very difficult to construct an example where it isn't.

However, it still doesn't seem safe, and I don't see why anyone would want to take the risk of assuming stability here. For example, there may be a few optimization scenarios where the compiler works with a copy of the stored property value, and that might break the assumption.

3 Likes
3

Put another way: as a member of MyLock, lock is necessarily stored inline in the MyLock instance. Since MyLock is allocated at a stable address, lock already has a stable address.

The reason this is still theoretically unsafe is that there's technically nothing illegal about &lock copying the lock elsewhere and returning the address to that — it would just be inefficient and unnecessary. Someone who works on the compiler can likely confirm, but to the best of my knowledge, the language semantics say that this does happen, but that the optimizer is allowed to elide the copy; since this is such a basic optimization to perform, I can't imagine the optimizer not eliding the copy, but if some bug in the optimizer regressed the behavior, this would silently stop being correct without a good way for you to know.

2 Likes
4

Another deeper semantic problem is that taking lock inout with &lock asserts exclusive access to lock, but locks need to be simultaneously accessible from multiple threads to do their work. And even if you performed a shared access to the variable, the expectation for a value of a normal type is that they remain immutable during such an access, which is also not true of a lock or atomic. The standard Mutex type in Swift 6 avoids these problem by being a noncopyable type and exposing the lock acquisition operation as a borrowing operation.

12 Likes
5

thanks for all the insights – much appreciated.

here are a couple that prompted the question. from 'The Peril of the Ampersand' there is this:

'Another Gotcha' There is another gotcha associated with the ampersand syntax. Consider this code: 
class AtomicCounter {
    var count: Int32 = 0
    func increment() {
        OSAtomicAdd32(1, &count)
    }
}

This looks like it’ll implement an atomic counter but there’s no guarantee that the counter will be atomic. To understand why, apply the tmp transform from earlier:

class AtomicCounter {
    var count: Int32 = 0
    func increment() {
        var tmp = count
        OSAtomicAdd32(1, &tmp)
        count = tmp
    }
}

So each call to OSAtomicAdd32 could potentially be operating on a separate copy of the counter that’s then assign back to count. This undermines the whole notion of atomicity.

Again, this might work in some builds of your product and then fail in other builds.

which covers the risk of the value type being copied locally and the copy being passed 'inout-to-pointer' (i think – please correct me if i'm misunderstanding).

from 'The Law' there is this:

The Law: Atomics are hard 
// Incorrect! Do not use this!
final class UnfairLock {
    private var _lock = os_unfair_lock()

    func locked<ReturnValue>(_ f: () throws -> ReturnValue) rethrows -> ReturnValue {
        os_unfair_lock_lock(&_lock)
        defer { os_unfair_lock_unlock(&_lock) }
        return try f()
    }

    func assertOwned() {
        os_unfair_lock_assert_owner(&_lock)
    }

    func assertNotOwned() {
        os_unfair_lock_assert_not_owner(&_lock)
    }
}

which is effectively the same case from the initial motivation.

IIUC this is the primary concern/risk in the above cases? i.e. if you end up getting a pointer to a temporary value, you lose actual mutual exclusion/atomicity. so, it's not that the location of the property self.lock would ever change (it's inline within a heap-allocated type), but that this form of use could end up not using that address if passed in this manner?

can this semantic incongruity cause specific problems at runtime? i've seen that the generated code for these sorts of examples contains many more begin/end_access instructions in the SIL, so i assume there is additional performance overhead associated with those.

6

That's correct. inout-to-pointer conversion for value types is not technically guaranteed to return the same address, even if the value type otherwise has a stable location in memory.

Yes! Depending on your build settings (where, IIRC, full exclusivity checking is the default), the Swift runtime will trap on dynamic detection of exclusivity violations. See

i.e., you'll get a crash on any lock contention, because &lock asserts that you have exclusive access to lock, and attempting to take that lock elsewhere would be a violation.

2 Likes