Pitch: CppReference wrapper type as a replacement for `import_as_ref`

There is a fundamental tension with Swift C++ interop due to C++'s ability to represent significantly more complex types (basically, anything with a custom or missing copy constructor). Currently, with the C++ interop team focusing on limited usage of LLVM, there is the import_as_ref workaround. This approach seems over-indexed on the use-an-LLVM-context use case (don't import constructors, manually specify retain/release functions) and doesn't generalize well to the very similar MLIRContext, or even using LLVM's context in a more complete manner (i.e. managing its lifetime). We also currently force a binary decision: import a type as a reference or try to use the default import behavior, which will be problematic as the default behavior improves (if a type that was import_as_ref is now supported, you need to decide when to flip the switch and possibly reevaluate/update all usages of its API). I've been thinking about these issues and think I have a simpler solution that may dovetail nicely with something @Alex_L mentioned @egor.zhdan might be working on.

The basic idea is to introduce a CppReference<WrappedType> type. This type will be somewhat analogous to UnsafePointer and friends, with a few key differences:

  1. the pointee will only be available if the imported type can be faithfully represented in Swift.
  2. C++ API on WrappedType will be available on a CppReference<WrappedType> (we may be able to do something clever with mutability/const).
  3. CppReferences can be created using the C++ constructor for WrappedType, which would be equivalent to calling new WrappedType(args) in C++ (as opposed to WrappedType(args), which will only be available if the type is directly bridgeable to Swift)
  4. C++ API would be able to accept CppReference<WrappedType> where appropriate (we can choose to allow some quality-of-life conversions from WrappedType, if available, and vice-versa)

Lets look at this approach in the context of a move-only type (heretofore MoveOnlyType):
Currently, Swift does not support move-only types, so they are not imported by the C++ importer. You currently could mark that type import_as_ref with immortal retain/release, which would make MoveOnlyType available from Swift, but would not allow you to use any constructors. If in a future release of Swift, move only types were implemented, you could remove import_as_ref, but then you may have to change all of your usages of MoveOnlyType (since the type would retain the same name), some of which may only care about using a reference to that type.
With CppReference, the importer would import MoveOnlyType in a way that you could never have a value of that type (because it is not representable in Swift). The importer would allow you to create a CppReference of that type by calling a constructor, or support API which returns a reference to a type which is not representable in Swift. Later, if Swift gains support for move only types and the import learns how to import them, MoveOnlyType would transparently become available as a Swift value outside of CppReference and CppReference<MoveOnlyType> would gain a pointee argument which respects MoveOnlyType's semantics. New API would now be able to work with MoveOnlyType directly, if that is what the user desires, or it can continue to use CppReference<MoveOnlyType> if that is what is required.

As an afterthought, I think this approach would also make some casting operations more straight forward. Want to cast a multiple inheritance type to one of its parent classes? use CppReference<MultipleInheritanceType>.asParentA().

3 Likes

It seems like your are describing two different problems here: 1) lack of support for move only types and 2) supporting a pattern where reference types have constructors that are only meant to be used with the new operator.

The latter is a much simpler problem to tackle. I don't see any reason we couldn't synthesize constructors on foreign reference types that expand out to new Type(args...).

The first issue is more complicated. We basically need to figure out what the story is for move-only types.

It seems like you are trying to import types that are not references as reference types, and that is where the issues are coming from. We don't support move only types. There may have been some miscommunication that "import_reference" allows you to use move only types, but this attribute should only be applied to types with reference semantics.

What do you mean by this? How would you manage the lifetime of an LLVM context? I don't think CppReference has a better story (unless I'm missing something).

We also currently force a binary decision: import a type as a reference or try to use the default import behavior, which will be problematic as the default behavior improves (if a type that was import_as_ref is now supported, you need to decide when to flip the switch and possibly reevaluate/update all usages of its API).

The decision to mark a type as "import_reference" should not come from our lack of support for that type (for example, move only types). A type is either a reference type or a value type. If it is a reference type, it should be expressed using Swift's language features for this (classes). Unless the fundamental semantics of the type change, the annotation should not go away.

I understand that C++ blurs the line here a bit, and there are some types which can be used as reference types or value types depending on the context. However, that does not seem to be the case you're talking about. Regardless, moving to Swift (a language with stronger conventions) may require a clear decision about the type's semantics, which I don't think this a bad thing.

Sorry if I made it more confusing, but I was using MoveOnlyType as an example of a type which cannot be represented in Swift because of things like overloaded copy constructors (but may eventually be representable); this is issue is not unique to move-only types. Another way phrase this is “how can we allow interop with C++ types which cannot be faithfully represented in Swift?”, and additionally how do we plan for some of these types eventually becoming expressable in Swift and provide a smooth migration path. Well, a reference to such a type is always representable in Swift, because it is just a pointer (i.e. even if a type is move-only you can create as many references to it as you please). So, for types not representable in Swift, we can add an API (CppReference<MyUnrepresentableType>) which would allow that type to be used from Swift even thought the type itself cannot be represented.

You can construct one, and destroy it once it is no longer needed.

There is quite a bit of gray area here in C++, since a “value type” can have arbitrary semantics (via overloaded methods) and labeling something a “value type” doesn’t really tell you much about it. I think the core of what I’m saying is that it may be better to think about using a type as a value or a reference, rather than a type being a value or reference type. CppReference provides the mechanism to do this, using CppReference<MyCppType>, indicates you are using that type as a reference type, whereas MyCppType would be using that type as a value type. This is doubly valuable in the case where MyCppType can’t be represented in Swift, since you can often still use much of an API as a reference. And if a subsequent generation introduces support for bridging MyCppType, you can disambiguate between treating it as a value or reference while you incrementally update a (possibly large) usage of that API.

Also, there may still be room for import_as_ref, which would essentially mean “CppReference<MyCppType> has the same semantics as a Swift class, so just make it a class

1 Like

I don't see any reason we couldn't synthesize constructors on foreign reference types that expand out to new Type(args...) .

Regardless, you should be able to use std.allocator, std.construct_at, and std.destroy_at to achieve this. It would probably be a good idea to add some helper APIs to the stdlib overlay, though. And maybe add a destory member if we synthesize a constructor.

an example of a type which cannot be represented in Swift because of things like overloaded copy constructors

I do not think we want a catch-all way to import "other" APIs. In any case, that is not what a foreign reference type is. I think we want clear mappings that allow imported APIs to map to a specific Swift convention that matches their semantics. Over time we can add support for more API patterns, so a higher percent of C++ codebases will be usable in Swift. Until then, you can use UnsafePointer.

I think rather than finding these catch-all solutions, we should focus on the specific cases you're talking about: move only types and non-copyable types that are not reference types. If we can identify patterns here, we can find clear mappings that fit well with the Swift programming model and feel more ergonomic.

This is interesting, because it's something that C++ and Swift sort of disagree on. Ignoring reference types for a second, Swift provides features like inout, borrow, and (soon, hopefully) no-implicit-copy that allow you to "reference" a value type. Swift decided to make all of these tools separate from the type system, though. This allows users to construct more composable APIs and allows the language to be a lot simpler. Having this information encoded in the type system introduces a lot of complexity and cognitive burden. C++ is an amazing example of how bad this can get. I think it would be a real shame to lose the Swift model here, even just for C++ types. I think there is a path where we can import most C++ APIs without introducing that complexity, and we should try to follow that path if at all possible.

What do I mean by "separate from the type system"? Swift has done something pretty cool that it doesn't get a lot of credit for, and that is pull out mutability and ownership from the type system. Mutability might be a bit easier to understand, but the same applies to ownership features like inout, borrow, etc.

Rather than thinking about "const" as a part of the type, like we would in C++, in Swift a value can have a constant scope. For example, here we first define a value with a mutable scope, then define a new scope, where the same value is constant:

func callee(_ x: T) { /* constant scope */ }

var x: T = ... // mutable scope
callee(x)

The type of x never changes. This is extremely valuable when you start composing types and APIs together, and is just generally simpler to reason about. C++ sometimes requires four overloads of the same function (prvalue, ref, const ref, xvalue). This would never be required in Swift, because the mutability and value categories are not encoded in the type. Swift separates these two "concerns" (types and mutability/value category) into two different language features. CppReference starts adding that back into the type system, which is a slippery slope.

1 Like

I didn't do a great job explaining this here, but it just occurs to me that this might be a useful thing (especially the difference in models here) to highlight in The Roadmap. Maybe I can try to explain it a bit better there.

3 Likes