Pitch: C++ interop support for UnsafePointer and friends

This pitch is a refinement of my CppReference pitch, making the relevant functionality both more integrated into Swift as well as clarifying the relationship to import:reference. For those of you who I've been discussing this with, there might be some rehashing here but I felt it would be useful to try to put the whole story in one place.

There has been much discussion of how Swift could import C++ types with customized copy behavior, such as move-only types. So far, the main tool for working with such type is the import:reference annotation, which tells the C++ importer that a particular type has Swift class semantics and should be imported as such. import:reference is tailor-made for connecting ARC with C++ types that have retain/release semantics (indeed it requires specifying a retain and release function for your type). However, early adopters of C++ interop are currently using it more generally to work around the importer's inability to import types with customized value behavior, such as move-only types (this is evidenced by the common usage of retain:immortal and release:immortal as a hack to opt-out of retain/release semantics).
Currently, the path forward has been advertised to me roughly as "Swift will get move-only types eventually, and at that point we can update C++ interop to handle such types". There is an opportunity to unlock many of these use cases prior to the implementation of move-only types in Swift, and to do so in a way that will cover use cases we probably never want to be valid in Swift. The core of this approach is to enable the use of UnsafePointer and friends to interop with C++ in much the same way we currently use that family of types to interop with C.

In my experience bringing up Swift support in MLIR (part of the LLVM project) and CIRCT (an LLVM incubator project), there are many different types that the current C++ importer fails to import. For most of these types, the fact that they are move-only (or have other customized value semantics) is not directly relevant to their API. By this I mean the value semantics don't affect how developers interact with these types. For most of them, the API consists of instantiating this type (likely on the heap) and then only ever interacting with it through a pointer. Even for those where eventually we may want to interact with the value itself (as opposed to a pointer), it isn't necessarily the case that the type matches Swift's class semantics (i.e. is retain/release) or that it will match the eventual move-only or borrow-based semantics we come up with for Swift. As evidenced by the existence of Unmanaged in Swift, even Swift types with retain/release semantics may require an escape hatch from ARC in certain special cases. I posit that the same will be true for imported C++ types.

I suggest a three-part solution:

  • Implement a .new static method on UnsafePointer<T> when T is an imported C++ type. This method will behave the same as calling new T(args) in C++, but from Swift (a delete method also makes sense). Note: it is likely we will need pointee to become unavailable in the case a type cannot be represented in Swift, since T in this case would likely have the same semantics as Never.
  • Import C++ methods directly on UnsafePointer<T> when that API can operate on an instance of T by reference.
  • Allow passing of UnsafePointer and friends as an argument to C++ methods when the argument is a reference (I believe something similar to this is already the case for C interop).

This way, we can directly interact with unimportable types in the short term, exactly how we would in C++ and with no need to annotate the source or add API notes. As types become importable source code can be updated on the developer's time frame to use a newly-importable T instead of UnsafePointer<T>. Going forward, as Swift finalizes semantics for borrows and move-only types, we can use UnsafePointer as an escape hatch where Swift's semantics don't support certain use cases (since Swift prioritizes progressive disclosure and safety more than C++, I think it is safe to expect Swift's version of these semantics to be more restrictive than C++).

Regarding import:reference, there will still be a need to mark certain types as import:reference when they match the semantics of Swift's class, but it will no longer be necessary to do this for types with customized value semantics that have pointer-based APIs.

I'd love to hear your thoughts!

1 Like

I'm not up to speed on C++ interop but I have some thoughts.

Implement a .new static method on UnsafePointer
a delete method also makes sense

UnsafeMutablePointer already has allocate, initialize, deinitialize, and deallocate functions. The compiler could generate an overload of initialize that takes the C++ type's constructor arguments. It would make sense to me that deinitialize would call the C++ destructor but I don't know if that is the plan/has been implemented. Assuming the above, new and delete functions could be nice but shouldn't actually be necessary.

Import C++ methods directly on UnsafePointer<T> when that API can operate on an instance of T by reference.

This seems semantically confusing. UnsafePointer and UnsafeMutablePointer already have apis of their own that explicitly deal with pointer things. Mixing in members of the underlying pointee type mixes up the semantics. Also how should naming conflicts between the Swift UnsafePointer and C++ members be resolved?

I see in the interop manifesto section dealing with move-only types:

The compiler would only allow UnsafePointers to them, and would allow calling methods on UnsafePointer.pointee

with this example:

func useOneFile(_ f: UnsafeMutablePointer<File>) {
  // var f2 = f.pointee // compile-time error: can only call a method on 'pointee'.
  print(f.pointee.ReadAll()) // OK

  // Move `f` to a different memory location.
  var f2 = UnsafeMutablePointer<File>.allocate(capacity: 1)
  f2.moveInitialize(from: f, count: 1)
  // `f` is left uninitialized now.

  print(f2.pointee.ReadAll()) // OK
  f2.deallocate() // OK
  // The file is closed now.

This approach looks reasonable to me, allowing C++ type member access without conflating the pointer and underling types. Has this just not been implemented yet?

1 Like

This is an aesthetic question, and I'm not strongly one way or the other. I thought using the C++ terms might be clearer, but accept arguments to the contrary.

Oh wow I completely missed this, thanks for the link. Yes, such an approach would work. @zoecarver @Alex_L is this still the plan-of-record?

No, we currently don't support working with non-copyable types via UnsafePointer types or accessing their methods via .pointee. I don't think we intended to support this kind of use case at the moment until Swift gets support for move only types and borrows, but @zoecarver can correct me if I'm wrong.

I guess my pitch can be rephrased then as “maybe we can make this work without fully supporting move only types?” I’m not sure how much of a hack it would be to have pointee be a special thing that can only be accessed directly on UnsafePointer (and how weird it would be that you can’t store that value).

I will say it’s starting to look like for my use case this is a blocker, and import:reference doesn’t really create a compelling enough alternative, especially without API notes support.