A short-term roadmap for C++ interop

Development Compiler
1

A short-term roadmap for C++ interop

This is a high-level roadmap of where I'm planning to take C++ interop over the next couple of months. If anyone would like to pitch in, or propose a change to this roadmap, feel free.

Primary goal

My primary goal over the next few months will be to fully import libc++ headers, LLVM headers, and Swift headers without crashing. This doesn't mean we'll necessarily be able to use any of these APIs, just that we'll be able to load the headers without crashing or having other errors. Ideally, we won't regress in terms of what C++ programs we can accept.

This should be mostly compiler bug fixes. However, there will be some larger projects, such as making all the headers we plan to use self-contained.

Other features and roadblocks

Here are the other two big features that I'd like to make some progress on in the near term:

  1. Improving template support, as discussed here. Currently, this is blocked on the ability to extend C++ types (patch here). This will allow us to more fully utilize the C++ APIs we import. For example, it will allow us to hack around inheritance for the time being (which will be a much more difficult feature to implement).
  2. Type conversions, mainly from Swift -> C++. I have a patch in the works (not yet finished) for importing Swift structs in C++; this will make templates much more useful in Swift. There is also a lot of potentials here for converting well-known types, such as std::string or Swift.String.

Getting involved

A lot of this work is actually pretty simple. The type conversion logic is a perfect example of this. There is a lot of low hanging fruit where the design and implementation is well known and relatively simple (there are still many things that aren't simple to design or implement, but we can worry about those later).

If anyone is interested in getting involved, I've marked a few good starter issues on Jira, but if there's something you're interested in, just do it! Additionally, please feel free to reach out to me. I'm happy to work with people in the community who want to get involved and push this forward.

Smaller items

Here are a few other things I have on the back burner related to C++ interop (that aren't necessarily in Jira, mainly because they're not yet well-defined):

  1. It would be great to find some source compatibility projects to ensure that we don't regress in terms of what we can import. Once we can import the standard library headers, this will become a lot easier.
  2. Some test suite improvements, such as running codegen tests on multiple targets locally.
  3. We need to figure out how to import things such as operator->. (Maybe as a .pointee member?)
  4. We should really, properly, add support for member functions. This includes fixing them on Windows and writing a bunch of tests. Here's a Jira bug in case anyone's interested.

Going forward

So, after we are able to import the standard library headers, Swift headers, and LLVM headers... what's next?

Even with the current, limited APIs we're able to import, I think we can start writing the compiler in Swift. This will be a slow process. But I think it is a good idea to write some part of the compiler in Swift as soon as possible, for the same reason we should have source-compatibility projects: it will ensure that we don't regress.

There are two compelling candidates for the first compiler features to be written in Swift:

  1. A simple utility pass that we can put into the SIL Optimizer. There are several passes that aren't required to compile Swift code, and there are even some which aren't required to compile optimized Swift code. These passes are ideal candidates because we won't have to do any bootstrapping, and they are relatively simple.
  2. The textual SIL Parser is another compelling candidate. While it is currently a dependency of the Swift compiler, we might be able to fix that, so it too wouldn't require bootstrapping. (After all, why should the Swift compiler need to know how to parse SIL?)

Luckily it will, at the earliest, still be a few months before we have to worry about these types of issues. But sometimes it's still fun to imagine :)

50 Likes
2

How about the following:

Is this solved or will be solved soon?

Also I have read that the original developers (from the Tensorflow project?) are not investing into the C++ interop any more (is this true?), so it is good to see that works continue here.

Thank you for your efforts!

3

I think this would actually be a really easy fix. It's just a matter of telling clang to use "Objective-C++" rather than pure C++. Feel free to submit a patch if you'd like. Otherwise, I will probably get to it after I make some headway compiler bug fixes. This is one of those things we only need once we want to start using C++ interop in "real" projects, so I'll probably start working on it once the compiler is more stable.

Here's the post they made. While it's true that Google developers were the first and primary developers (especially in the beginning), there has always been a community of other developers around them, helping with reviews, discussions, etc. In the last few months especially, the community has grown and diversified. Part of the goal of this forum post is to push that even further. I'd love to find more developers who are interested in C++ interop. As I said in the post, there are several issues that would be great for beginners who are trying to figure out how they want to contribute to the Swift compiler. If you're interested in contributing, I'm happy to work with you (the indefinite you) to make that happen.

4 Likes
4

Yes, I'd like to use it in a real project soon. (Sorry, I am not able to supply a patch. But at least part of the project will be an open source library.)

5

For what it's worth, this is a bigger problem than it sounds because some types in Apple's headers are defined differently in Objective-C vs. Objective-C++ modes, which can change the ABI of Swift functions that use those types. This is definitely something that will need to be solved, possibly with Apple's cooperation, before it gets turned on for everybody, but it's also something that needs to be experimented with sooner rather than later, so it shouldn't stop the change to enable Objective-C++ for testing!

1 Like
6

That project should run on Mac, Red Hat and Windows. So I hope the C++-interop is "cross-platform".

7

Thanks for letting me know. I think we'll need to have a longer discussion about C++ interop's ABI guarantees in general. Obviously, this is something we'd like to have, but there are certain cases where it might be difficult, such as the case you brought up where libraries have different function signatures for C/C++ (or ObjC/ObjC++). At some point someone (maybe me) should make a forum post where we can properly discuss this issue and problem solve.

1 Like
8

I wonder if this will give Swift the badge of being the first LLVM language to write its compiler in itself without losing full access of the LLVM C++ API?

I also wonder how we can make the story on the REPL (and why not JIT) side a little better.

I was hacking around REPL and it was actually LLDB binding to a executable main entry point over python scripting, and once i've tried the same code that worked in python but with the C++ LLDB api i didn't got the same result. Since i dont have that much time nowadays, i didn't go along investigating what was happening..

Anyway, this is all cool from a hackish point of view, i understand how it would help a IDE like XCode by being this way, but from the point of view of just wanting to at least interpret (not even JIT through MCJIT) its a super convoluted process.

So i cant help to think of how this natural interop with C++ can help in the REPL front, even before the swift compiler in swift endeavor, given this will at least could give us access to the Swift AST directly from Swift if we need.

(BTW: if anyone knows how i can get a REPL by binding to a DSO distributed with the Swift compiler, without having to compile the whole thing from source or having to expose some private api entries as public into the DSO, i thank you kind stranger)

5 Likes
9

Hello all.

I just wanted to give a quick (and exciting!) update on this short-term roadmap. We are now able to import the C++ standard library without crashing on main. And we are able to use many of the APIs right out of the box. For example, a std::string_view can be constructed from a Swift String and then used and converted back. Here is a small test repo demonstrating it.

Looking forward to more progress in the coming weeks!

68 Likes
10

This is amazing progress Zoe, great job!

2 Likes
11

Great work, thanks. May I ask again about C++-interop not "disturbing" the use of the Foundation library (using "Objective-C++ rather than pure C++", see above)? I would like to use the C++-interop very soon in a real project, and sorry, I am not able to do any fixes here myself.

12

Hey @sspringer. I'm sorry to deliver bad news, but I don't think C++ interop is going to be ready for a real-world project any time soon. We're still designing and implementing it, and it's mostly in a "proof of concept" stage. We will likely break things in the future, so I wouldn't want you to rely on it in any sort of "real project."

That being said, I'm happy that you (and others) are so excited about using it! If you do want to play around with it, you should be able to just import Foundation and (at least parts of) it will work. You can also add both -enable-cxx-interop and -enable-objc-interop to get clang to use Objective-C++ mode. I don't think we have much, if any, testing for this, however, so use it at your own risk!

6 Likes
13

Heya, just wanted to drop by to say that as a Windows user, I've been following along excitedly. Huge shout of support to all the folks who have worked on this.

From what I know, currently Swift (behind a flag) is the only language capable of importing and interop'ing with C++ without writing manual bindings/host-language mappings. (You have to write a modulemap, but that doesn't really count and is easy to automate)

The closest is D-lang, which has extern (C++) and supports C++ calls via convention, but you need to write the declarations yourself, IE:

extern (C++, `SomeNamespace`) {
  int foo() { return 1; }
}

Ada and Nim can do this too, though (personal opinion) not quite as ergonomically.

With the SourceKit LSP and the recent plugin support for Swift as a Jetbrains IDE addon, I'd be willing to bet the Windows crowd is going to start giving Swift a good look in the coming years =)

3 Likes
14

Is it possible to at least use C++ stacked allocated objects? and if so, its possible to move between C api's?

I ask this because even if you have to stick with C Shims function to C++, just being able to not allocate things on the heap having C++ RAII working on Swift would be a godsend.

Without this you often have to heap allocate wrapper objects bypassing the lifetime control over to Swift. And its worse because it forces you to use a Class (and therefore refcounts) on the Swift side just because you need the deinit {} to destroy the heap allocated object.

I don't know how the lifetime management works for C++ structs allocated on the Swift side, but nevermind the C shim functions, what bothers me the most is 'stupid' heap allocations you have to make on the C side to properly regulate the objects lifetime on the C++ side.

So is this already possible now?

2 Likes