[Manifesto] Interoperability between Swift and C++

Swift provides ergonomic interoperability with C (How Swift Imports C APIs), Objective-C (Importing Objective-C into Swift, Importing Swift into Objective-C), and Python ([1], [2], [3]).

Interoperability between Swift and C++ has been previously discussed on the forums ([1], [2], [3]). The Swift for Tensorflow project has already started to build it out under the hidden -enable-cxx-interop flag. I have been trying to understand what interop will look like across all C++ features. I have summarized possible design options in a manifesto: Interoperability between Swift and C++.

It is not a fully finalized design: many sections list multiple alternative options that we should discuss and evaluate their pros and cons. Some sections are missing, notably, calling Swift from C++. Nevertheless, this manifesto covers enough ground to show what Swift/C++ interop could look like, what will work well, and where the rough edges are.

I would like to invite everyone to provide feedback on the manifesto, discuss the design, evaluate alternative options explained in the doc, and brainstorm further alternatives that we didn’t think of. If there is lots of discussion on a particular topic, we will branch that discussion into a separate forum thread.

Interoperability with C++ is a broad topic, and the manifesto is quite long (more than 100KB!). I would like the discussion to focus on the most difficult and risky parts of the design, although you should feel free to comment on any topic. In particular, I think these are the most critical topics to discuss:

  • Goals, use cases, and constraints.
  • Mapping for C++ const reference parameters.
  • Mapping for C++ namespaces.
  • Mapping for move-only classes.
  • Mapping for non-movable classes.
  • Mapping for function and class templates.
  • Mapping for C++ enums, which are most often not annotated with NS_ENUM.

We will continue implementing Swift/C++ interoperability under the experimental flag -enable-cxx-interop. We will invoke the Swift evolution process if we need changes to the Swift language itself (as a hypothetical example, if we seriously considered adding namespaces as a feature to Swift).

I look forward to your input!

63 Likes

Hi @gribozavr ! This is a great and very in-depth manifesto, I really enjoyed reading it.

I have a two suggestions to offer though, regarding approach to templates and to standard types.

1. It is possible imo to bring SFINAE into Swift for C++ templates, without any added syntax to both Swift and C++.

We can do this by reverse-importing Swift protocols into C++ as proxies for type substitution.

Consider this example:

// Swift code:
protocol CustomSwiftProtocol {
    func isLess(than other: Self) -> Bool
}

func maximumValue<T: CustomSwiftProtocol >(_ v1: T, _ v2: T) -> T {
    return std_max(v1, v2)
}

// C++ code:
template<T>
T max(T v1, T v2) {
    if v1 < v2 {
        return v2;
    } else {
        return v1;
    }
}

// Auto-generated proxy header
template<T>
struct Swift::Bridge {
    Swift::TypeExistential<T> __type;
}

// User-provided bridging header
template<>
struct Swift::Bridge<Swift::CustomSwiftProtocol> {
    friend operator<(Swift::CustomSwiftProtocol, Swift::CustomSwiftProtocol);
}

bool operator<(Swift::CustomSwiftProtocol t1, Swift::CustomSwiftProtocol t2) {
    return t1.isLessThan(__type, t2);
}

When compiling maximumValue, Swift compiler will use all constraints it has on generic type and put those into generated Swift::_MangledConstraintName_Proxy class. It will then pass Swift::Bridge< Swift::_MangledConstraintName_Proxy as template argument to the C++ template. Swift::_MangledConstraintName_Proxy will have exported definitions for all protocol methods as proxies to witness table entries.

Swift compiler can then provide implementations for some standard protocols like Equatable, and users can add their own when needed. If no such implementation is available, substitution will fail at C++ site with a fairly understandable error.

This should be possible if we when generating Swift definition out of C++ headers just mark those with the source header file location and let the compiler do a special pass when those are instantiated, generating temp header and a clang pass on the result to generate specialized definitions.

2. std:: types can be wrappers for Swift types instead of the reverse how it is proposed in the manifesto.

Consider the fact that most (imo) C++ code the users want to use as libraries is seldom vendored, since C++ compilers don't guarantee proper ABI when crossing boundaries between libs and especially between compilers.

So in fact the situation where we compile C++ sources as part of the same project may be more prevalent than when we have a vendored lib. This in turn means that we have the same toolchain for both our Swift codebase and C++ codebase. And this in turn means that we may build a substitute libc++ specially for Swift bridging. Such library would provide standards-conformant API, but behind the scenes use Swift types and/or specialized abstractions to ease Swift interop.

In such case, std::string can be implemented to operate on the same backing storage as a Swift String , as well as slightly deviate and require the string to be a valid UTF-8 string. If the library designer or user knows that the string may not be UTF-8 conformant, the user may just add a -DUSE_NONUTF8_STRING to compilation flags and then a) the Swift importer will map std::string to Array<Int8>, and b) std::string will be instead using Array backing storage.


What do you think about those approaches? :slight_smile:

1 Like

Thank you for the suggestions!

I don't think you are describing a way to deal with SFINAE (std::enable_if constraints). I think you are suggesting two things:

  • a way to call C++ templates from Swift generics.

  • a way to export Swift types to C++.

As far as calling C++ function templates from Swift generics, the primary difficulty with your suggestion is that specializations of C++ function templates will not be picked up. Consider what happens when we call maximumValue with T=SomeCppClass. ::max could have a user-defined specialization ::max<SomeCppClass>, but instead interop will call ::max< Swift::Bridge<Swift::CustomSwiftProtocol>>.

Another difficulty is that mapping Comparable to Swift::CustomSwiftProtocol as shown above is not fully safe. C++ does not have an equivalent of Swift's Self in protocol definition. Therefore, C++ code can pass mismatched arguments to operator< and it would still compile. It would be possible to detect the error at runtime, but it is important to keep in mind that the mapping is not as straightforward.

I think this is a very interesting direction, but I think first we should discuss how to export simpler Swift types to C++, like plain structs and classes, and how to map their protocol conformances. Once we have these building blocks, we will explore how they compose with templates.

I think that would imply that std::string would have to be backed by a copy-on-write buffer, which is prohibited by the C++ standard since C++11.

Also I think that a significant number of C++ users would be unhappy about atomic reference counting and copy-on-write semantics in value types like std::string and std::vector. COW and ARC maybe could have been very reasonable ways to implement data structures in C++, but since C++ implementations have not been doing that (or even actively stopped doing it since C++11), I am certain that changing eagerly-copying data types into COW and ARC based data types will significantly degrade performance in lots of C++ code.

I'm not sure what makes you say that: many compilers and standard libraries do guarantee ABI stability. libc++ on MacOS, iOS and Linux is ABI-stable, and so is libstdc++. GCC and Clang both implement the Itanium ABI and are link-compatible. Clang also implements the MSVC ABI and is link-compatible with code produced by the Visual Studio compiler. These are not just theoretical arguments, many real systems are built on top of them (e.g., most Linux distributions, as well as MacOS and iOS). You can also look at the amount of pain Linux distributions experienced when libstdc++ had to break the ABI for std::string due to C++11 changes.

I wish we could handwave the ABI issues and tell everyone to vendor their C++ standard library. While some people can do that, a significant number of users lives in a very different world that depends on a stable ABI in the C++ standard library.

The question actually goes beyond ABI stability -- it goes into being able to rebuild from source all C++ code that the user wishes to load into the process. There are significant difficulties with loading two C++ standard libraries into one process. People will run into these difficulties if they want to depend on some system library (or a library that is distributed as a compiled binary) that loads the C++ standard library from the system, and on Swift/C++ interop that will depend on the Swift-specific C++ standard library.

I think that making Swift and C++ standard libraries cooperate is a very interesting and possibly very rewarding direction, but we must ensure that stuff that works today continues to work the same way when Swift/C++ interop is enabled. Sure, if enough users of Swift/C++ interop can vendor a C++ standard library, and breaking the ABI of the C++ standard library helps significantly enough to warrant maintenance costs (maintaining a custom C++ standard library is a huge effort), it would be an important optimization, but it won't work for every user of Swift. Therefore, we have to implement the general solution first.

4 Likes
Terms of Service

Privacy Policy

Cookie Policy