Implementation details of new Mutex type

[Alejandro (Alejandro Alonso)]

I'd be glad to answer these questions

@ph1ps

1. What exactly is this type doing? I understand that os_unfair_lock_lock takes a pointer to an unfair lock structure but what is the difference of using _Cell (and I guess Builtin.addressOfRawLayout) instead of UnsafeMutablePointer<os_unfair_lock> which has been initialized like this:

let lock: UnsafeMutablePointer<os_unfair_lock>.allocate(capacity: 1)
lock.initialize(to: .init())

So Mutex stores its value inline instead of the usual out of line strategy. Swift really didn't give us the tools necessary to provide these sorts of types because it wanted to copy everything all the time (os_unfair_lock is just a UInt32, so passing a value of this to a function would just copy it). We could get pretty close with classes because a value is just the reference itself, but referencing ivars inside a class introduces runtime exclusivity checks that we have to avoid. So typically, the only safe way to do this in Swift was manually allocating the storage with a pointer like you have or using something like ManagedBuffer to give us some control of the tail allocated storage.

Mutex on the other hand uses an underscored attribute @_rawLayout that guarantees we pass a value of it by address when borrowing it. So calling a borrowing function or passing a borrowing Mutex<T> will always be by address (getting us a stable address). Builtin.addressOfRawLayout is just a way to get this address. _Cell is just the generalization of this where it stores a value T inline and has an API that lets you have a direct pointer to that storage (because passing a _Cell is by address, so anything that stores a _Cell is also by address).

@ph1ps

2. Why is _Cell also used to wrap Value in the Mutex struct? What is the difference between passing the pointee of the address passed and not just &value like this (simplified):

func withLock<R>(_ body: (inout Value) throws -> R) rethrows -> R {
    os_unfair_lock_lock(lock)
    defer { os_unfair_lock_unlock(lock) }
    return try body(&value)
}

_Cell is used to wrap the value of the mutex because the withLock API is a borrowing function, so you cannot access the inout operator & within this scope. If value was a regular stored property we wouldn't be able to say &value because the compiler knows you cannot do this operation in a borrowing scope. _Cell solves this problem because it lets us get a direct pointer to the value that we can convert into an inout. Note this is usually super unsafe, but we can guarantee runtime exclusivity due to enforcing that the mutex is locked while accessing this value.

@ph1ps

Could a not-Swift-stdlib implementation of _Cell (or a wrapper of os_unfair_lock) even be done correctly, since this implementation is using Builtin.addressOfRawLayout(self) which is not available to normal developers (as far as I know)?

Anyone can access the Builtin module and any underscored attributes, but generally they are unsafe, hard to use correctly, and may change at a moments notice. John already mentioned that we're looking to figure out how best to expose some of this functionality so that you can actually implement Mutex yourself (perhaps we just need to expose a public version of _Cell? but it's super unsafe with just giving you a raw pointer )

Of course, anyone can use ManagedBuffer or the pointer APIs to do your own out-of-line implementation which is what most of the community has been doing for over a decade now. (Those APIs are also pretty unsafe and hard to use)