C++ Interop workgroup meeting notes (Mar 9th 2022)

Alex_L · March 15, 2022, 4:37pm

This post contains the summarized meeting notes from the Swift and C++ interoperability workgroup sync-up meeting from the 9th of March.

Attendees: compnerd, Alex_L, dabrahams, CaelanMacArthur, plotfi, zoecarver, cabmeurer, Adrian_Prant, adlere, bulbazord, drodriguez, degoo, mikepinkerton, atierian, NuriAmari

Action Items

zoecarver and Alex_L will isolate and reevaluate goals from the original interoperability manifesto, so that we can start distilling the project criteria into one high level vision document.

Discussion

Discussion started by going over the action items from the last two meetings.

Cabmeurer brought up the issue of the method naming bridging for getters and setters. Should we have a list of known abbreviations for things like “utf8” and “html”, to map such names to a Swifty property name? Also, could we remove some other prefixes, in addition to “get” and “set”, e.g. could “get_processed_utf8_string” become just “utf8String”?

dabrahams: The elimination of the word processed, might be appropriate for this 1 particular API, but the word “processed” is not necessarily redundant in general.
cabmeurer: agreed.
atierian: There’s also the question of capitalization, should “HTML” be capitalized?
zoecarver: we want to have utility that figures out how names are imported and transformed. Maybe we should have a capitalization list, for things like “html”?
dabrahams: Have you looked at the stuff that Doug built for doing this for Objective-C names?
zoecarver: We should look into that, it could be helpful.
dabrahams: There were lots of lists of words, and I'm sure there's a list of abbreviations that needed to be capitalized and things like that.
zoecarver: Yeah, that would be super helpful. Thank you for bringing that up.
zoecarver: Another point, we should try to finalize this as soon as we can, especially before making interop, non-experimental, to avoid introducing source breaking changes for our adopters.

Next we discussed the issue of calling member functions in base classes and value copies that are being produced by Swift , that prevent Swift from invoking the base class method correctly.

zoecarver: We have this very interesting problem here. I'll give a quick overview of how we import base classes today and then will describe the problem.
zoecarver: Today the derived class gets every member in all of its base classes (when imported into Swift). Those are synthesized as either computed properties or methods, which cast the self argument to the base class, and then invoke the method on that base class.
dabrahams: C++ works in the same manner.
zoecarver: Yes. However, in C++, you have a relationship between the derived and base classes. In Swift you have no relationship like that between the two types. It’s something that we might want to model long term, but not in the foreseeable future.
dabrahams: Swift has a very limited set of subtype relationships. The more of those there are in the compiler, the worse type deduction performance gets. So, if you start making base classes, super types, you’re going to have to expect the type-checking costs to go up.
zoecarver: Yeah, in the short term we will add something like a casting utility and beyond that, I'm not sure if we're gonna create a relationship there.
compnerd: I have a question about how that actually works in practice. How do you redefine this? When you're calling to C++ code, you need to pass it this. Where does that value come from? Is it from that instance or not? Because if it's from that instance, I'm worried about how this is going to play out with differing vtable layouts and class inheritance hierarchies.
dabrahams: Zoe said there’s a cast. If the cast is not a trivial address preserving cast, but if it's an actual C++ cast then you're fine.
zoecarver: Yes, we ask C++ to do a cast.
compnerd: So you're basically creating thunks that just use C++ to cast this and then dispatch.
zoecarver: Exactly. Right now the cast is done with some wrapper functions around static_cast.
dabrahams: You're really saying that classes are gonna work, like in C++, except for that there's no subtyping relationship between base and derived classes.
zoecarver: Exactly. There’s no problem with that. Here’s where the problem lies. Swift likes to make a lot of copies. And this is a broader problem. C++ structs are not necessarily aggregates. You can't just break them apart. We can't do objects slicing on them. And you can't copy those slices and then just shove it back together and hope that it works.
zoecarver: The specific problem is, sometimes Swift creates an object slice, takes the base class, and copies it, but what if base class assumes something about its derived class?
zoecarver: For example, a class like small vector base class could assume it has some storage buffer in a derived class. What if it wants to do something with that and that buffer does not exist because the derived class does not exist because this is a copy of the base class, which is an object slice. Does that make sense?
dabrahams: Yeah. All of the extra copying is a problem, so shouldn't we be addressing that?
zoecarver: Right. Could move only types help (could we move the base class out and back)?
dabrahams I think we have a basic problem that swift doesn't have a solid model for move only types. Also, if I move out the base class value of that bigger derived object, what’s the state of that bigger derived object?
dabrahams: I’ve been working on a research language called ‘Val’. The kind of rules that we're coming up with for Val could allow a move out of a type temporarily as long as you move something back there while it's being projected. And that's fine because you have value semantics, so you have unique access to it. So that's 1 possible strategy for dealing with move only types and getting access to them, without being destructive to the owning thing. So we need to project each of those instances into whatever's going to process them, but get them back at the end.
zoecarver: You need a way to borrow.
dabrahams: Yeah, you don't really want to move them out, right? That's just an extra expense. In inout, you pass something in and get it out to you, so you've logically moved it out and moved it back, but you don't actually have to pay for any movement in memory.
zoecarver: makes sense.
dabrahams: It seems like there's either a plan that I don't know about that's very complete for how move only types work in Swift or there's a whole design project needed just for Swift. That's what needs to be carried out, with background attention to how it's going to work with C++.
zoecarver: Does that really solve it in the context of this question? Really we could want immovable types that you can borrow.
… brief discussion about non-copyable / move only types …
zoecarver: On main, right now there's this sort of experimental no implicit copy attribute. I don’t want to use it in this context. Could we say no implicit move instead? I.e. it states that you can't implicitly copy or move this thing. It can’t be relocated.
compnerd: you can say borrowed.
dabrahams: What is wrong with the semantics of no implicit copy?
zoecarver: That could still imply a move. A move operation on a base object is going to have the same problem as a copy of the base object.
dabrahams: Those create totally different problems. If you move the base object, you’ve got a partially formed object.
dabrahams: And copying a base object is not necessarily a problem. There's no reason you shouldn't be able to do it if your C++ classes are designed to be used that way.
zoecarver: the problem is, when the base class assumes something.
dabrahams: Yeah, although those based classes should have been made non-publicly accessible right?
zoecarver: Not necessarily. You see this pattern a lot.
dabrahams: I mean there's a broader philosophical question about whether you're going to correct the defects in the underlying C++ code when importing it into Swift. You can say that this piece of C++ code is well encapsulated and, and it's written in the optimal way for C++. And that would not expose the inherited methods (and thus would avoid the copy after cast problem altogether).
zoecarver: We need the average C++ to work. We need the LLVM headers to import nicely because 1 of our adopters is the Swift compiler.
dabrahams: What you're talking about is an unsafe construct in C++ that was exposed to something safe and now we want to make it actually safe to handle. You're not going to make it safe because the C++ code is already unsafe.
zoecarver: Good point.
dabrahams: The thing that worries me about trying to do that is that it's biting off an incredibly large challenge. There may be some places where you can do mitigations like this, but doing it in general, there's no chance of that. My point of view is that C++ code imports into Swift, such that if you use it the way you would use it in C++ you you get good results in Swift without any annotations.
zoecarver: You brought up a lot of really good points there. My worry here is that Swift creates so many copies that I'm genuinely not sure that using something in Swift could be the same as using it from C++ because there’s a different default to what a copy is.
dabrahams: the creation of implicit copies has to stop because there are important move only types in C++ and they can’t be copied.
zoecarver: The solution to this problem can't be replace copies with moves. I think it has to be more explicit, like I should state if I actually want to make a copy here or a move. I think the bottom line we agree on is that we're gonna need some way to express types that are not copyable or movable.
compnerd: Maybe I'm missing something, but it felt to me that what you're really getting at is that we need a way of representing a reference type in Swift.
… discussion about reference types, and immutable borrows / value passing …
zoecarver brought it back to discuss the base classes problem, and the question of is it a violation of C++ and if it’s unsafe to expose that copy of the base class in Swift.
dabrahams: Everything in C++ is unsafe as far as you know, without looking at the implementation. So, it's not unsafe. if the type has been written to be safe for that. And there are plenty of types for which that is okay.
zoecarver: Yeah you’re right. Return types also could be a problem. If a return type is a reference type, it returns an unsafe pointer. Is there some way to do something with that that doesn't invoke a copy? Right now every time you do a dot, you get a copy, which is extremely frustrating.
dabrahams: That shouldn't be the case. It should be a read accessor that doesn’t do a copy. If you mutate it, it would need the mutate accessor.
zoecarver: I guess that pretty much solves it. Are there any safeguards you'd want around it to make sure people don’t copy out of a pointee? I guess that’s unsafe.
dabrahams: It might be unsafe. if it's a pointer to int, then you've got no problem.

Switching gears to a discussion related to goals & criteria:

dabrahams: The criteria that we could make here, is that we can make it work to use the C++ code (from Swift) as you would use it correctly from C++. If you believe in that, I think that should be written down as a goal. I think this project in general has a missing list of criteria, like what we're going to achieve.
zoecarver: Yeah that would be a good thing for us to have. I think we've talked about very similar things to this before. Having some more formalized design would be really good here. The broader point about having a big design document with goals is invaluable.
dabrahams: I think actually, in terms of guiding our work, the smaller, you can make it and more to the point, the better. We need top level questions that will drive all the other decisions. You need to decide what the working framework is, to give us a guide for our current work.
zoecarver: That's another good idea. I guess that was done with the interop manifesto, but we should isolate and re-evaluate those goals now that nobody who's worked on the manifesto is still working on this project.
Alex_L: We started working on high level documentation, and the status page is ready to post. We can also work on the vision documentation and start thinking about the criteria that you mentioned and write them out as part of that vision document.

benrimmington · March 15, 2022, 6:26pm

This is documented by @jrose at: