C++ Interop workgroup meeting notes (August 10th 2022)

This post contains the summarized meeting notes from the Swift and C++ interoperability workgroup sync-up meeting from the 10th of August.

Attendees: compnerd, Alex_L, richard, zoecarver, tongjiew, George, Adrian_Prantl, augusto2112, cabmeurer , plotfi, ravikandhadai

Discussion

The discussion centered around goals for reverse interop (importing Swift into C++).

@Alex_L:
High-level overview of the key principles / high level goals:

The implementation of this feature should not impose specific design decisions or restrictions that might change the way that Swift framework authors or library API, designers, and authors write their Swift code. This is fairly vague, but extension could be a good example of that, as there’s no great way to map extensions for a type declared in another module into C++, but we should try to design some support for that in the most ergonomic manner possible, as that’s a common Swift pattern we don’t want to deprive our users of. As a goal we want to keep it mind when designing every feature for interop.

Next goal is type safety. We need to ensure that right things are passed to Swift APIs. Specifically when it comes to generics, if a generic function has specific constraints, we should respect them in C++ as well and figure out some way of type-checking them from C++.

Next goal is memory safety. We should follow Swift’s memory model for things like reference types (retain/release) them correctly, and in general when working with a Swift value, we wouldn’t expect it to suddenly get destroyed while we’re still using it in C++, and we don’t expect the user to have to manually destroy in C++ it as that would be bug-prone.

Next goal is respecting the Swift semantics. We should preserve the Swift semantics in C++. This is again fairly vague. For some concrete examples you can think of semantic differences between value and reference types, so we would want to reflect that difference into C++ as well. Also, copy-on-write semantics for value types like Array should still be preserved from C++.

Next goal is ergonomics. We should try to map things to C++ language features that make sense. Sometimes, there might be a feature like an enum that doesn't really have an ideal mapping to C++ because enums are much more capable. So in that case we can use another C++ feature like a class type to represent the Swift enum. Additionally, we it comes to library evolution, there should be no ergonomic difference for how you use a type regardless of whether it comes from a module that opts-in into library evolution, or from a module that does not. Except in specific required cases, like checking for an @unknown default case in a C++ switch for a Swift resilient enum.

Performance is a key goal as well. We should be as efficient as possible, and should strive to have minimal overhead. In certain cases, we might depend on boxing allocations of values on the heap in C++, for instance when working with resilient Swift types. In those cases we should still strive for minimal overhead by using some optimizations that help in common cases, e.g. having an inline buffer to store the value and use that buffer without heap allocations, if the value fits!

Another goal is Objective-C support. If you have Swift API that, for instance, takes or returns and objective C type we should still expose that to Objective-C++, but not necessarily C++ itself.

@ravikandhadai:
As for the objective C+ support, will we be exposing the APIs that are already exposed to Objective-C to Objective-C++? But also, in the same way, right?

@Alex_L: Yeah, right. I can add it to the Objective-C support goal.

@Alex_L: Now let me touch quickly on the additional goals. They’re more specific implementation goals:

  • Language standard support: we should support several recent C++ language standards, not just the latest one. Some features might require C++20 and above though.
  • Compiler support: We do require clang features to support swift calling convention correctly, so clang would be a requirement. However, we should not depend on clang-specific C++ language extensions. It’s ok to depend on some specific clang only features that enhance the developer experience, for instance clang attributes that inform the indexer that this a declaration from Swift.
  • The generated C++ binding header should be viewed as a temporary build product.

@zoecarver: Broad question - C++ and Swift are obviously very different in capacities. Swift has specific goals and strong idioms. It has specific goals like safety, which you mentioned. And for reverse interop, is there a set of goals like that, and do we adopt them? Specifically, for Swift, even when exposing C++ APIs, do we try to maintain the same goals that Swift itself has?

@Alex_L: We will have to stick to Swift’s principles as much as possible when we are generating bindings. For instance, when using a Swift value in C++ we would copy it using Swift copy semantics when it’s copied around in C++ code. However, we can’t prevent you from doing weird things. You can just always like reinterpret cast that value to something else, or get to the buffer where the value is stored yourself from C++, and do whatever you want with that storage. I think in general we should probably discourage that, so it might make sense to have a set of diagnostics in Clang that discouraged patterns like that. I think the question is, will people actually want to do something like that? Like maybe project the value as another type for direct property access, which is not really safe, but it might be easier to use. So, I think if we find that there is demand for that, or if people are inclined to do that, maybe we should be more strict and actually prioritize those kinds of warnings.

@zoecarver: Should it at least be stated explicitly as a goal? I wonder if swift semantics is that sort of bullet that's encapsulating this right now.

@Alex_L: I think t's kind of covered by both semantics and memory safety goals. It might be good to explicitly state that somewhere.

@zoecarver: A specific question since you were talking about casting. What about iterators? I think iterators are a great example that sort of force a decision here about where exactly to draw the line for specific language features. For forward interop, this was really helpful, as we we found this iterator pattern and decided to expose that pattern in a native interface. I don’t know if that’s the case for reverse interop. Do we want to expose a native C++ iterator interface, which would be a pair of begin and end date raters, which would violate exclusivity? 'm not saying that that's a bad thing, but it’s the type of thing we should get at with these discussions.

@Alex_L: I think it would be important to provide some support that people come to expect out of C++, so iterating over a Swift array in a for loop is something that we’ll have to support. In what way it is implemented can be argued, perhaps there’s a way to do it without violating exclusivity. Something like that could be covered by “ergonomics”, so I can mention that in that point.

@zoecarver: Something we’re exploring now with forward interop, is what’s the tradeoff here? Ergonomics and Swift semantics might be at odds with each other in the case of iterator, for example. But we don’t necessarily have to answer that in this document.

@ravikandhadai: Are these goals arranged in terms of priority? Could performance be ever be overriding these earlier goals for instance?

@Alex_L: Not necessarily prioritized. They could be equally important but at odds with each other.

@ravikandhadai: Could performance ever be a reason for us to give up on type safety or memory safety, when we have those goals conflict?

@Alex_L: We might need to prioritize performance in some cases, but we don’t know yet. Safety should be paramount though most likely.

@zoecarver: Does it make sense to state in goals that safety is the top priority? That might actually be like a a helpful thing to have established somewhere.

@compnerd: There’s the interesting aspect of Swift going the opposite way as well. This came up recently, for instance, that it might be a good idea to have the ability to project a tuple as a contiguous thing that you can iterate over. Right? So, sometimes you do want a projection of the data, even if that, violates some sort of safety once it's implemented.

@Alex_L: For the projection case, I think there are ways to make more ergonomic projections while still prioritizing the aspects of C++ safety as well. So we can still prioritize safety above ergonomics I think.

@Alex_L: I can make a change that explicitly states that safety would be prioritized over something like performance.

@zoecarver: I have another direction. We talked a lot about this exposed attribute and so I'm surprised to not see on here some goal about specific API mappings. This is something that we have in forward interop and, it might be good to touch on, either by saying we do want to have specific API mappings, or we don’t want to have specific API mappings.

@Alex_L: That's a good point. This is intentionally left to be vague in this document, because we have different valid approaches. We could try to generate binds for all public APIs as much as much as we can, from a swift library or generate bindings only for things that are annotated as expose. if we do the latter, then the attribute will definitely validate its declaration, if it can have bindings for the declaration. And I don't necessarily think we need to state that in the roadmap itself.

@zoecarver: I’m talking about something slightly different. I'm not saying which specific APIs are imported or exported. I’m talking about API patterns. Like the iterator pattern from forward interop. From a messaging point of view for a user experience, having a list that states iterators are supported and we have a clear mapping is maybe a good point to have.

@Alex_L: I think we should definitely have a list of that for reverse as well. I don't think it's necessarily part of the goals. It can be part of the roadmap.

@zoecarver: Right, but we can have a goal that C++ interop aims to provide a clear defined mapping for the APIs that can be imported, to draw a clear boundary for what’s supported and what isn’t.

@ravikandhadai: Can we make that point in ergonomics? That we could try to support as many C++ patterns as possible, for example operators.

@zoecarver: No, that’s almost the opposite suggestion. I think it’s important for us to make a distinction between what are things that we support and what are things that are not supported. There’s actually a valid argument that you can make that you can support everything as Swift is a simpler language and we don’t need to map specific patterns, but that’s a specific argument to make, and it’s not what we did with forward. So we should clarify it for reverse too.

@Alex_L: We will provide specific documentation for how language constructs from Swift are mapped, and which language constructs are not mapped, either now , or possibly never. I would state that we'll have all of the specific mappings documented e.g. how a specific type is mapped, how a specific method is mapped documented.

@zoecarver: Yes that’s it. So it is a goal to define the boundary between Swift language features that are supported and those ones that aren’t. I’m not sure this needs to be in the goal section, but I'll just bring it up anyway, how do you communicate that to users? Like how do you say when you use something that we can't import, why can't we import that? This is like something that we did a lot of work on this in forward interop and it turned out to be extremely helpful.

@Alex_L: Makes sense. We should definitely outline the fact that we need to establish that this boundary document, and provide diagnostics for things that are unsupported as much as we can. There might be specific cases where it might be impossible. But I think in the general case, we can try to do some best effort job.

@zoecarver: Is it a goal to support every swift API?

@Alex_L: Partially. The goal would be to support a rich set of Swift language constructs and types, but there will be specific API / library patterns that we won't support from C++. For example, if a library expects you to create a value type that conforms to protocol declared in that library and pass that library, we won’t support that from C++.

@zoecarver: That might be actually a really great thing to capture in this this goals.

@plotfi : I have a quick question about how type conversions work.I recall that we don’t intend to support (for C++) what we do now with Objective-C types between array and let’s say a String and NSString. My first question is, is that due to performance considerations? My second question is reverse interop going to be able to fill the need of not having to do so many manual type conversions?

@zoecarver: The goal is not to have hidden performance traps. And vector -> Array conversion is a huge hit in performance, that I don’t think gives us anything that a conformance to a protocol (for vector) gives us.

@plotfi: The feedback that I gather from folks on the app side, is that maybe they don't understand the performance traps, and that they would rather have some way to have more easy conversions. Maybe the solution is that the C++ containers do conform to specific protocols though, and that makes it easier to just do the conversions for them.

@zoecarver: This is actually implemented on main and and you don't even have to have a conversion. This is just a conformance to Collection or Sequence protocol, which we automatically conform to for an iteratable type.

@zoecarver: Additionally, the standard library overlay should definitely and will provide these conversions using a Swift way to do this which is an explicit like constructor.

@plotfi: Yeah that seems reasonable. My other question is basically about the idea of having Swift containers accessible from C++.

@Alex_L: Yes, reverse interop could help you to have an alternative solution to this problem. You could create the original collection in C++ using the Swift Array type.

@plotfi: And then you have zero overhead for passing it back and forth between Swift and C++.

@Alex_L: Yes, exactly.

@Alex_L: Any other feedback?

@richard: Quick question. I’ve been looking into possibility of using CryptoKit framework. It would be an interesting case if you could call this library from C++. The only way I found to do it right now is via Objective-C bridging layer.

@Alex_L: Great comment. I cannot comment on whether CryptoKit will be supported specifically, but reverse interop is meant to provide facilities to access Swift library APIs from C++ and CryptoKit’s APIs seem suitable for that purpose.

@zoecarver: Side note, will post draft of forward interop roadmap whose goals were discussed in last sync-up to Github for everyone to comment and contribute.

@Alex_L, @Adrian_Prantl, @compnerd concluded the discussion by discussing how to skip C++ thunk frames that call into Swift functions in the debugger backtraces when debugging C++ code that calls into Swift (Clang’s attribute nodebug and alwaysinline should suffice). Also they discussed potential ideas on debug info representation details for Swift types imported into C++.

1 Like