C++ interoperability is broken (Windows platform)

dima_kozhinov · June 14, 2024, 2:19am

The -cxx-interoperability-mode command line option for the swiftc compiler used to work, but then (seemingly after an update to MS Visual Studio 2022) it stopped. How to reproduce:

Create a dynamicLibrary.swift file:

public func foo() -> Int {
    return 42
}

Try to compile it with the following command:
swiftc dynamicLibrary.swift -emit-library -cxx-interoperability-mode=default

Expected result: dynamic library is created.
Actual result: The following console output:

e[1m<module-includes>:1:10: e[0me[0;1;30mnote: e[0me[1min file included from <module-includes>:1:
e[0m#include "ccomplex"
e[0;1;32m         ^
e[0me[1mC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\include/ccomplex:8:10: e[0me[0;1;30mnote: e[0me[1min file included from C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\include/ccomplex:8:
e[0m#include <yvals_core.h>
e[0;1;32m         ^
e[0me[1mC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\include\yvals_core.h:898:1: e[0me[0;1;31merror: e[0me[1mstatic assertion failed: error STL1000: Unexpected compiler version, expected Clang 17.0.0 or newer.
e[0m_EMIT_STL_ERROR(STL1000, "Unexpected compiler version, expected Clang 17.0.0 or newer.");
e[0;1;32m^
e[0me[1mC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.40.33807\include\yvals_core.h:519:44: e[0me[0;1;30mnote: e[0me[1mexpanded from macro '_EMIT_STL_ERROR'
e[0m#define _EMIT_STL_ERROR(NUMBER, MESSAGE)   static_assert(false, "error " #NUMBER ": " MESSAGE)
e[0;1;32m                                           ^
e[0me[1m<unknown>:0: e[0me[0;1;31merror: e[0me[1mcould not build C module 'std'
e[0merror: fatalError

etcwilde · June 14, 2024, 4:43am

Yes, Microsoft updated the minimum required version of Clang to Clang 17. The clang that is embedded in the Swift compiler for importing C and C++ APIs is aligned with the Clang 16 release. For the time being, you will need to use an older Windows SDK that is compatible with Clang 16 (or older).

Edit: At least according to the WinGet installation documentation, Microsoft.VisualStudio.Component.Windows11SDK.22000 should work.

dima_kozhinov · June 15, 2024, 9:57am

Nope. Just did a clean install of Microsoft.VisualStudio.Component.Windows11SDK.22000 (no later SDKs installed). Same error message.

tiff · June 15, 2024, 11:48am

I believe this can be avoided by adding the following compiler flags to the command

-Xcc -D_ALLOW_COMPILER_AND_STL_VERSION_MISMATCH

Alternatively, you can add them to the cxxSettings of Package.swift, like this

cxxSettings: [
  .define("_ALLOW_COMPILER_AND_STL_VERSION_MISMATCH", .when(platforms: [.windows]))
]

dima_kozhinov · June 15, 2024, 3:32pm

It works, thank you. What are the implications of ignoring the version mismatch? Is generated binary technically correct?

stevapple · June 16, 2024, 2:13am

Visual C++ STL is packaged along with MSVC tools, and has nothing to do with the Windows SDK. This is a breaking change introduced within VS 17.x release cycle, and the valid workaround should be to manually install an older MSVC toolchain and specify it manually with -visualc-tools-root.

tiff · June 16, 2024, 2:14am

It means that the Microsoft STL is the version that has been tested with Clang. (this article).
It is expected to typically work unless you are using the latest features (perhaps C++23 or 26?).

If you encounter problems, try installing a different version of the STL (download), or you can find information about it here.

stevapple · June 16, 2024, 2:17am

I think we need a table to clarify the version of Clang packed within the Swift toolchain, and match it with STL requirements (if such version bump is still going to happen).

dima_kozhinov · June 16, 2024, 4:30am

I use C++ interoperability to just emit a C/C++ header file with the -emit-clang-header-path option of the swiftc compiler. No C++ code actually involved, just Swift.

dima_kozhinov · June 16, 2024, 4:34am

Shouldn't Swift compiler rely on Windows SDK only, and not a closed-source commercial product from Microsoft? Next time Microsoft may break the entire compiler, not just one feature.

stevapple · June 16, 2024, 4:42am

It’s not Swift depending on VC++, it is Clang that really depends on it (by default, on Windows).

In theory you can use another C ABI with Swift, but in that way you’re no longer able to build directly against MSVC ABI, eg. with a hypothetical Swift SDK for Cygwin or msys2.

furby-tm · August 2, 2024, 7:38pm

Will Swift 6 on Windows embed a clang aligned with the Clang 17 release? There's alot of friction here for Windows users, since none of this is documented and most users will simply follow these docs, which at the time of writing - correlates to every C/C++ target failing to compile for users, which is not quite the out-of-the-box experience Swift provides on all other platforms.

As an additional note, Release (swift build -c release) builds, as well as Optimized (swift build -Xcc -Os) builds of C/C++ targets also break C++ interop on Windows.

Example output (slightly redacted to avoid printing too much noise in this comment) when compiling C/C++ targets with one of the aforementioned commands:

Function has token parameter but isn't an intrinsic
  call void @_GLOBAL__sub_I_tag.cpp.cold.1(token %0) #22, !dbg !4333
in function _GLOBAL__sub_I_tag.cpp
fatal error: error in backend: Broken function found, compilation aborted!
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace, preprocessed source, and associated run script.

...

1.      <eof> parser at end of file
2.      Code generation
3.      Running pass 'Function Pass Manager' on module 'C:\Users\furby\Wabi\MetaverseKit\Sources\Yaml\tag.cpp'.
4.      Running pass 'Module Verifier' on function '@_GLOBAL__sub_I_tag.cpp'
Exception Code: 0xE0000046
 #0 0x00007ffd7f2fc95a (C:\WINDOWS\System32\KERNELBASE.dll+0xcc95a)
 #1 0x00007ff78960316a (C:\Users\furby\AppData\Local\Programs\Swift\Toolchains\5.10.1+Asserts\usr\bin\clang.exe+0x16e316a)

...

clang: error: clang frontend command failed with exit code 70 (use -v to see invocation)
clang version 16.0.0
Target: x86_64-unknown-windows-msvc
Thread model: posix
InstalledDir: C:\Users\furby\AppData\Local\Programs\Swift\Toolchains\5.10.1+Asserts\usr\bin

...

As a final observation, with the following define present

cxxSettings: [
  .define("_ALLOW_COMPILER_AND_STL_VERSION_MISMATCH", .when(platforms: [.windows]))
]

with a single C++ target's header file including the following

// SomeCXXTarget.h
#include <version> // from stdlib

it errors with the following when attempting to import it into Swift:

// SomeApp.swift
import SomeCXXTarget
       ^~~~~~~~~~~~~
vcruntime.modulemap:696:10: error: module 'std.version' requires feature 'cplusplus20'

Note: I am using the following C/C++ language standards in my Package.swift manifest.

cLanguageStandard: .gnu17,
cxxLanguageStandard: .cxx17

Thanks!

compnerd · August 2, 2024, 8:31pm

The IR Verification error regarding "Function has token parameter but isn't an intrinsic" is related to Hot/Cold Splitting (HCS). This was recently fixed upstream and I've backported it for the 6.0 release. You can disable cold code splitting to avoid this error.

The error with std.version is related to the C++ standard you are using (C++17). Bumping that to C++20 should alleviate that. The Microsoft STL aggressively adopts new C++ features requiring that you update the versions pretty quickly.

furby-tm · August 2, 2024, 8:42pm

The error with std.version is related to the C++ standard you are using (C++17). Bumping that to C++20 should alleviate that. The Microsoft STL aggressively adopts new C++ features requiring that you update the versions pretty quickly.

I'm afraid that bumping the C++ standard to C++20 is an astronomically breaking change.

I use Swift/C++ interop to build much of the ASWF projects and all if its dependencies, changing the C++ standard has drastic breaking changes across nearly every library.

The best way forward here was ensuring that <version> does not exist in any of the Swift imports on the Windows platform, for me it was coming from the oneTBB dependency, simply ensuring the header including that file was nowhere visible in the umbrella header was enough to workaround the issue, allowing me to keep the standard strictly to C++17.

Which leads me to the question, why error here?

Is it possible to have Swift ignore incompatible modulemaps and simply not import their APIs into Swift if this is the case? This issue is a Swift one, not a C++ one (the error only occurs during the compilation of the Swift file that imports an offending target that imports incompatible modulemaps).

The IR Verification error regarding "Function has token parameter but isn't an intrinsic" is related to Hot/Cold Splitting (HCS). This was recently fixed upstream and I've backported it for the 6.0 release. You can disable cold code splitting to avoid this error.

Great work! Is that available from this download here?

compnerd · August 2, 2024, 10:24pm

Not AFAIK; the modulemaps will be loaded. In the case of the umbrella header, all of those were re-exported and thus caused the module to be serialised and loaded.

No, it requires a newer snapshot that will hopefully be made available there soon (CC: @mishal_shah).

processeus · August 8, 2024, 8:05am

Has anybody managed to download the older version of Visual Studio / standard library? Assuming I need MSVC 17, I would need to install Visual Studio 2019 Community, which I couldn't find anywhere (it's not available on winget and on the official older versions download page)

furby-tm · August 28, 2024, 8:38pm

I started experiencing a super weird issue with the MSVC C++ Standard Library in github actions, something changed with windows-latest and now you cannot #include <string> with this error on v14.41.34120, I do not however experience this issue on v14.40.33807.

The error is the following:

C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.41.34120\include\tuple:957:9:
error: static assertion failed: get<T>(tuple<Types...>&) requires T to occur exactly once in Types.
(N4971 [tuple.elem]/5)

Did Microsoft break their own C++ Standard Library and ship that error?

@compnerd I'm not sure if you know what might be going on here, but it's quite strange, thanks!

dima_kozhinov · August 28, 2024, 11:00pm

After one more Microsoft update things are broken again The

-Xcc -D_ALLOW_COMPILER_AND_STL_VERSION_MISMATCH

does not help anymore, apparently the problem is different this time. Again, how to reproduce:

Create a minimal dynamicLibrary.swift file, e.g.

public func foo() -> Int {
    return 42
}

Try to compile it with the following command:

swiftc dynamicLibrary.swift -emit-library -cxx-interoperability-mode=default -Xcc -D_ALLOW_COMPILER_AND_STL_VERSION_MISMATCH

Compiler output

<module-includes>:1:10: note: in file included from <module-includes>:1:
#include "ccomplex"
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include/ccomplex:11:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include/ccomplex:11:
#include <complex>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\complex:13:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\complex:13:
#include <sstream>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\sstream:10:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\sstream:10:
#include <istream>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\istream:10:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\istream:10:
#include <ostream>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\ostream:10:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\ostream:10:
#include <ios>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\ios:10:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\ios:10:
#include <xlocnum>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\xlocnum:15:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\xlocnum:15:
#include <iterator>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\iterator:11:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\iterator:11:
#include <xutility>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\xutility:11:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\xutility:11:
#include <__msvc_iter_core.hpp>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\__msvc_iter_core.hpp:10:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\__msvc_iter_core.hpp:10:
#include <utility>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\utility:11:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\utility:11:
#include <type_traits>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\type_traits:131:5: error: static assertion failed: Calling declval is ill-formed, see N4950 [declval]/2.
    static_assert(false, "Calling declval is ill-formed, see N4950 [declval]/2.");
    ^
<module-includes>:33:10: note: in file included from <module-includes>:33:
#include "condition_variable"
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include/condition_variable:17:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include/condition_variable:17:
#include <mutex>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\mutex:18:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\mutex:18:
#include <thread>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\thread:18:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\thread:18:
#include <tuple>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\tuple:957:9: error: static assertion failed: get<T>(tuple<Types...>&) requires T to occur exactly once in Types. (N4971 [tuple.elem]/5)
        static_assert(false, "get<T>(tuple<Types...>&) "
        ^
<module-includes>:33:10: note: in file included from <module-includes>:33:
#include "condition_variable"
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include/condition_variable:17:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include/condition_variable:17:
#include <mutex>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\mutex:18:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\mutex:18:
#include <thread>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\thread:18:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\thread:18:
#include <tuple>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\tuple:969:9: error: static assertion failed: get<T>(const tuple<Types...>&) requires T to occur exactly once in Types. (N4971 [tuple.elem]/5)
        static_assert(false, "get<T>(const tuple<Types...>&) "
        ^
<module-includes>:33:10: note: in file included from <module-includes>:33:
#include "condition_variable"
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include/condition_variable:17:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include/condition_variable:17:
#include <mutex>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\mutex:18:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\mutex:18:
#include <thread>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\thread:18:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\thread:18:
#include <tuple>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\tuple:981:9: error: static assertion failed: get<T>(tuple<Types...>&&) requires T to occur exactly once in Types. (N4971 [tuple.elem]/5)
        static_assert(false, "get<T>(tuple<Types...>&&) "
        ^
<module-includes>:33:10: note: in file included from <module-includes>:33:
#include "condition_variable"
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include/condition_variable:17:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include/condition_variable:17:
#include <mutex>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\mutex:18:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\mutex:18:
#include <thread>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\thread:18:10: note: in file included from C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\thread:18:
#include <tuple>
         ^
C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include\tuple:993:9: error: static assertion failed: get<T>(const tuple<Types...>&&) requires T to occur exactly once in Types. (N4971 [tuple.elem]/5)
        static_assert(false, "get<T>(const tuple<Types...>&&) "
        ^
<unknown>:0: error: could not build C module 'std'
error: fatalError

furby-tm · August 29, 2024, 7:01am

Yep same error as mine above, MSVC 14.41 is the culprit

As far as I know this is a unfixable bug in this specific version of MSVC, I don’t think there’s anything Swift is doing causing this error, but also it would seem very odd if this specific version of MSVC is totally broken, so I’m not quite sure how to explain this

furby-tm · August 29, 2024, 7:29am

Out of curiosity, if you set the C++ Language Standard to 20 does that “fix” this error? (I know it’s not a real fix, I’m just curious if that gets things rolling again as a pretty dramatic workaround, this version mismatch is less than ideal).