A short-term roadmap for C++ interop

A short-term roadmap for C++ interop

This is a high-level roadmap of where I'm planning to take C++ interop over the next couple of months. If anyone would like to pitch in, or propose a change to this roadmap, feel free.

Primary goal

My primary goal over the next few months will be to fully import libc++ headers, LLVM headers, and Swift headers without crashing. This doesn't mean we'll necessarily be able to use any of these APIs, just that we'll be able to load the headers without crashing or having other errors. Ideally, we won't regress in terms of what C++ programs we can accept.

This should be mostly compiler bug fixes. However, there will be some larger projects, such as making all the headers we plan to use self-contained.

Other features and roadblocks

Here are the other two big features that I'd like to make some progress on in the near term:

  1. Improving template support, as discussed here. Currently, this is blocked on the ability to extend C++ types (patch here). This will allow us to more fully utilize the C++ APIs we import. For example, it will allow us to hack around inheritance for the time being (which will be a much more difficult feature to implement).
  2. Type conversions, mainly from Swift -> C++. I have a patch in the works (not yet finished) for importing Swift structs in C++; this will make templates much more useful in Swift. There is also a lot of potentials here for converting well-known types, such as std::string or Swift.String.

Getting involved

A lot of this work is actually pretty simple. The type conversion logic is a perfect example of this. There is a lot of low hanging fruit where the design and implementation is well known and relatively simple (there are still many things that aren't simple to design or implement, but we can worry about those later).

If anyone is interested in getting involved, I've marked a few good starter issues on Jira, but if there's something you're interested in, just do it! Additionally, please feel free to reach out to me. I'm happy to work with people in the community who want to get involved and push this forward.

Smaller items

Here are a few other things I have on the back burner related to C++ interop (that aren't necessarily in Jira, mainly because they're not yet well-defined):

  1. It would be great to find some source compatibility projects to ensure that we don't regress in terms of what we can import. Once we can import the standard library headers, this will become a lot easier.
  2. Some test suite improvements, such as running codegen tests on multiple targets locally.
  3. We need to figure out how to import things such as operator->. (Maybe as a .pointee member?)
  4. We should really, properly, add support for member functions. This includes fixing them on Windows and writing a bunch of tests. Here's a Jira bug in case anyone's interested.

Going forward

So, after we are able to import the standard library headers, Swift headers, and LLVM headers... what's next?

Even with the current, limited APIs we're able to import, I think we can start writing the compiler in Swift. This will be a slow process. But I think it is a good idea to write some part of the compiler in Swift as soon as possible, for the same reason we should have source-compatibility projects: it will ensure that we don't regress.

There are two compelling candidates for the first compiler features to be written in Swift:

  1. A simple utility pass that we can put into the SIL Optimizer. There are several passes that aren't required to compile Swift code, and there are even some which aren't required to compile optimized Swift code. These passes are ideal candidates because we won't have to do any bootstrapping, and they are relatively simple.
  2. The textual SIL Parser is another compelling candidate. While it is currently a dependency of the Swift compiler, we might be able to fix that, so it too wouldn't require bootstrapping. (After all, why should the Swift compiler need to know how to parse SIL?)

Luckily it will, at the earliest, still be a few months before we have to worry about these types of issues. But sometimes it's still fun to imagine :)

37 Likes

How about the following:

Is this solved or will be solved soon?

Also I have read that the original developers (from the Tensorflow project?) are not investing into the C++ interop any more (is this true?), so it is good to see that works continue here.

Thank you for your efforts!

I think this would actually be a really easy fix. It's just a matter of telling clang to use "Objective-C++" rather than pure C++. Feel free to submit a patch if you'd like. Otherwise, I will probably get to it after I make some headway compiler bug fixes. This is one of those things we only need once we want to start using C++ interop in "real" projects, so I'll probably start working on it once the compiler is more stable.

Here's the post they made. While it's true that Google developers were the first and primary developers (especially in the beginning), there has always been a community of other developers around them, helping with reviews, discussions, etc. In the last few months especially, the community has grown and diversified. Part of the goal of this forum post is to push that even further. I'd love to find more developers who are interested in C++ interop. As I said in the post, there are several issues that would be great for beginners who are trying to figure out how they want to contribute to the Swift compiler. If you're interested in contributing, I'm happy to work with you (the indefinite you) to make that happen.

4 Likes

Yes, I'd like to use it in a real project soon. (Sorry, I am not able to supply a patch. But at least part of the project will be an open source library.)

For what it's worth, this is a bigger problem than it sounds because some types in Apple's headers are defined differently in Objective-C vs. Objective-C++ modes, which can change the ABI of Swift functions that use those types. This is definitely something that will need to be solved, possibly with Apple's cooperation, before it gets turned on for everybody, but it's also something that needs to be experimented with sooner rather than later, so it shouldn't stop the change to enable Objective-C++ for testing!

1 Like

That project should run on Mac, Red Hat and Windows. So I hope the C++-interop is "cross-platform".

Thanks for letting me know. I think we'll need to have a longer discussion about C++ interop's ABI guarantees in general. Obviously, this is something we'd like to have, but there are certain cases where it might be difficult, such as the case you brought up where libraries have different function signatures for C/C++ (or ObjC/ObjC++). At some point someone (maybe me) should make a forum post where we can properly discuss this issue and problem solve.

1 Like

I wonder if this will give Swift the badge of being the first LLVM language to write its compiler in itself without losing full access of the LLVM C++ API?

I also wonder how we can make the story on the REPL (and why not JIT) side a little better.

I was hacking around REPL and it was actually LLDB binding to a executable main entry point over python scripting, and once i've tried the same code that worked in python but with the C++ LLDB api i didn't got the same result. Since i dont have that much time nowadays, i didn't go along investigating what was happening..

Anyway, this is all cool from a hackish point of view, i understand how it would help a IDE like XCode by being this way, but from the point of view of just wanting to at least interpret (not even JIT through MCJIT) its a super convoluted process.

So i cant help to think of how this natural interop with C++ can help in the REPL front, even before the swift compiler in swift endeavor, given this will at least could give us access to the Swift AST directly from Swift if we need.

(BTW: if anyone knows how i can get a REPL by binding to a DSO distributed with the Swift compiler, without having to compile the whole thing from source or having to expose some private api entries as public into the DSO, i thank you kind stranger)

4 Likes
Terms of Service

Privacy Policy

Cookie Policy