Automatic pointer conversion / stable addresses for stored properties

I've seen conflicting reasons as to why using os_unfair_lock from Swift is unsafe. They are generally:

  1. inout can make a copy.
  2. No guarantee of stable addresses for stored properties.

I'd like to confirm my understandings.

The first one (inout can make a copy) doesn't apply because

var lock = os_unfair_lock()
os_unfair_lock_lock(&lock)

...isn't actually an inout, it's passing by pointer. (see function signature below)

public func os_unfair_lock_lock(_ lock: UnsafeMutablePointer<os_unfair_lock_s>)

Essentially, the example above should be functionally equivalent (ignoring the stable memory address nuance) to:

let lock: UnsafeMutablePointer<os_unfair_lock> = {
  let pointer = UnsafeMutablePointer<os_unfair_lock>.allocate(capacity: 1)
  pointer.initialize(to: os_unfair_lock())
  return pointer
}()

os_unfair_lock_lock(lock)

For the second concern (no guarantee of stable memory addresses). Does this mean that using a static property would be "safe" in Swift today?
Trivial example:

enum Foo { 
  private static var lock = os_unfair_lock()
  private static var cache: [String: Int] = [:]
  static func get(_ s: String) -> Int { 
    os_unfair_lock_lock(&lock)
    defer { os_unfair_lock_unlock(&lock) } 
    // read / write to `cache`
  }
}
1 Like

Looks totally safe to me: static variable for the dictionary and for the lock are effectively global variables with stable addresses. (I use a similar technique, although normally on reference types with a normal non static member variable of type NSLock / NSRecursiveLock).

1 Like

Is The Peril of the Ampersand | Apple Developer Forums relevant here?

3 Likes

That is not a real pointer. That is not valid for a long-lasting value such as a lock.

What you need in order to use a lock (or any construct that relies on the processor's atomic instructions) is to allocate on the heap, using UnsafeMutablePointer.allocate(). That will get you a pointer that you can rely on. You can wrap that allocated pointer in a (final) class for convenience (and deal with cleanup/deallocation). There are many correct implementations like that out there.

7 Likes

Including OSAllocatedUnfairLock which was introduced in the latest beta SDKs for Apple's platforms.

3 Likes

Thanks, that explanation is very helpful.

That is not a real pointer. That is not valid for a long-lasting value such as a lock.

If I understand correctly, it is a "real" pointer, but the compiler is allowed to move the value to a more convenient location before making a pointer, then move it back. Which is where the risk of locking a copy comes into play.

2 Likes

Exactly. After the call with a pointer conversion returns, the compiler could move your would-be atomic variable somewhere else for convenience.

The purpose of pointer conversion is to pass information between Swift and C code in the same thread; there is a sequential aspect to the logic. The purpose of atomics (and locks) is to pass information between threads, and the logic of inout-to-pointer conversion breaks down. Where is your atomic variable? If it's on the stack, not only is its location not guaranteed over time, but it is also by definition thread-local and it is (rightly) hard to reliably escape it to another thread. If it's in a closure capture you don't know at what moment it actually becomes shared, or where it is located. If it's a global variable, then why bother? A global lock is not very effective; if that's what you need, serialize your code with a queue.

The heap is where Swift data can keep a stable address, and the way to reliably put things in the heap is to use manual memory allocation with one of the Unsafe*Pointer.allocate() functions.

2 Likes

Aside from address stability, the other big concern when trying to store locks, atomics, volatile data, etc. in stored properties are the exclusivity semantics of accessing properties. Even if the property isn't moved, it is required that, during a read, there be no simultaneous writes, and that during a write, nobody else is accessing the property at all. When you use a pointer conversion operation like withUnsafePointer or the & argument conversion, that also counts as a read or write access for the duration of the pointer's validity, so imposes the exclusivity requirement on the property for the duration of the call. Since locks and atomics by their nature need simultaneous access from multiple threads, that exclusivity requirement flies entirely in the face of their functionality.

In the future, we will hopefully have move-only types, which will let us model the semantics of atomics and locks in a composable way that integrates with the rest of the language. Until then, your best bet is to keep things with unusual memory semantics stored in "raw" memory that doesn't come from a Swift stored property, such as the tail allocation of a ManagedBuffer, or memory you got from malloc, UnsafeMutablePointer.allocate, or some other allocation API.

8 Likes

Is it something compiler already does today, or do you mean that as the language spec allows it - the future compiler might do this, even if it doesn't do it today? I tried to replicate this behaviour with the following simple test where I pass the address of a global (or static) variable to C code: the address is getting remembered there, then I modify the contents of this variable from Swift and try to reproduce the case when I do not see the corresponding change when viewing that location in C through the remembered address. If the global variable address was unstable the test would fail, but it doesn't fail. :thinking: I also tried a few Xcode sanitiser options unsuccessfully ("Undefined behaviour sanitizer", etc).

the test
-------------
// test.swift
import Foundation

// test1
var value: UInt8 = 0

// test2
//struct S {
//    static var value: UInt8 = 0
//}
//
//var value: UInt8 {
//    get { S.value }
//    set { S.value = newValue }
//}

func test() {
    
    // test1
    c_setValueAddress(&value)
    
    // test2
    // c_setValueAddress(&S.value)
    
    Timer.scheduledTimer(withTimeInterval: 0.001, repeats: true) { _ in
        value = .random(in: 0 ... 255)
        precondition(value == c_getValue())
    }
    
    while true {
        RunLoop.main.run()
    }
}

test()

-------------
// Test.h
#import <Foundation/Foundation.h>

void c_setValueAddress(uint8_t*);
uint8_t c_getValue(void);
void c_setValue(uint8_t);

-------------
// Test.m
#import "Test.h"

static uint8_t* address;

void c_setValueAddress(uint8_t* addr) {
    address = addr;
}

uint8_t c_getValue(void) {
    return *address;
}

void c_setValue(uint8_t value) {
    *address = value;
}

-------------
// BridgingHeader.h
#import "Test.h"

I wonder what are drawbacks of using NSLock instead. Is it not simpler? I also wonder why we don't have the Lock Swift counterpart of NSLock already (as we did with LocaleNSLocale, StringNSString, etc).

It does work with global variables right now. While I'm not sure whether there is any reason it should stop working, it's not guaranteed to keep working either. This being said, a global variable is a very poor place to keep a lock or atomic variable for contention reasons in addition to it being incompatible with Swift's exclusive access rules.