The swift/bridging header defines C/C++ annotations that are recognized and used by Swift's ClangImporter for interop. This header file currently ships with the toolchain in the <toolchain>/usr/include/ subdirectory, which is part of Clang's default search paths for Apple-built toolchains, but not for OSS toolchains distributed via Swiftly/swift.org.
To make this behavior consistent, I'd like to propose shipping this header file in a separate "Swift resource directory", which would consist of the following changes:
(swiftlang/swift) move the installation destionation for the swift/bridging header from <toolchain>/usr/include/ to <toolchain>/usr/lib/swift/include, which is the location of the Swift resource directory.
(swiftlang/llvm-project) add <toolchain>/usr/lib/swift/include to Clang's default search paths
(swiftlang/swift-installer-scripts) adjust package manifests to include the new resource directory.
The addition of this default search path to the OSS Swift toolchain's Clang will allow that toolchain to resolve #<swift/bridging> without explicitly adding that search path to the build invocation.
Note, however, that this will not affect C/C++ compilers outside of the Swift toolchain. In particular, upstream Clang will still not be able to resolve #include <swift/bridging> without adding an explicit search path (which is the status quo). Those toolchains will still be able to derive that directory relative to Swift-Clang’s own resource directory, i.e., $(swift/toolchain/clang -print-resource-dir)/../../swift/include .
The motivation to behind moving the directory from <toolchain>/usr/include to <toolchain>/usr/lib/swift/include is to ship it alongside the rest of the toolchain-specific files. We already ship a number of header files in other subdirectories of <toolchain>/usr/lib/swift to support the toolchain, and as well as Clang’s own resource directory under <toolchain>/usr/lib/clang/<version>/ .
I'd love to hear your thoughts and suggestions about this. Thank you!
I would rather that we do not do this; the resource directory is already stuffed with a lot of content that we should be working to remove from there. Windows and macOS already do this to a certain extent and it causes some confusion.
This would be reasonable - but we could also change it to add ../include to the default search path as an alternative.
The content in the resource directory in clang is pretty different. The (clang) compiler resources are generally for exposing the intrinsics. I don't think that the macros are exposing intrinsics, they are just providing convenience macros for the attributes.
I’ve encountered the exact same problem when using C++ interop in one of my cross-platform Swift packages, and even reported it separately. From my experience, this looks like a bug in SwiftPM’s current handling of the swift/bridging header search path.
Apologies for not following up on this for so long. I was exploring the alternative of keeping the bridging header to the <toolchain>/usr/include, and adding that directory to Clang's default search paths, but I got caught up trying to resolve some Apple-only search path behavior that complicates this path forward. I'm still working on that, but would like to revive this discussion in the meantime.
I personally think that this subdirectory being over-crowded shouldn't prevent us from adding things there altogether, but I agree that it's over-crowded and think there should be a better plan for how to organize it. I was hoping that putting the bridging headers in there could be a start, while also opening a pathway for relocating the contents of swiftToCxx.
Those macros aren't compiler intrinsics, but they are similar in the sense that they are closely tied to the version/implementation of the compiler itself. The closest analogue to swift/bridging that already ships in Clang's resource directory is ptrcheck.h, which consists entirely of annotation macros that expand to attributes.
@compnerd just to clarify, your preference is that we should keep the swift/bridging header in <toolchain>/usr/include, and add that to the Clang default search path? I'm wondering if you can elaborate on your reasoning, or if there are any precedents you are following?
Side note: another concern I have about the current way we’re shipping the headers, in <toolchain>/usr/include, is that there’s no language/interop version associated with its location. There's <toolchain>/usr/lib/swift/swiftToCxx/experimental-interoperability-version.json, but it should probably be renamed + the file in my currently installed toolchain just contains "16" and I'm not entirely sure what that means.
It is less about being over-stuffed and more about the fact that we are in this transitory phase where the content is split across locations based on the platform. Relocating the contents of swiftToCxx seems very reasonable to me.
I think that that I would prefer <SDK>/usr/include. But, if they are just convenience macros, <toolchain>/usr/include or <toolchain>/usr/lib/swift/include would make some sense. That would be similar to the SwiftShims. This has the precedence of clang resource headers, swift shims, etc.
This seems like a separate issue, unrelated to the where the content goes. The question of what the content should be is still valid, but I think that is best handled as a separate conversation.
I'm not sure what <toolchain> is referring to here. Is it the compilers, the SDK, or the blob mixing both of them that we currently have on Linux? I'm going to assume you mean the location where the compilers are, though that gets confounded on Linux.
To go completely opposite to @compnerd, if it’s truly a header for the clang-import and only the clang-importer, this might be a case of an actual compiler resource going in the compiler resource directory. Unfortunately, we have a whole lot of SDK content shoved in this subdirectory that will complicate this. SDK content that includes dynamic libraries can't really move all that well without breaking binary compatibility. On the flip side, compiler-resources are tied to a specific build of the compiler, so we could potentially move the actual compiler resources to a separate directory. Since the contents should be compiled into any outputs and search paths should be handled by the compiler automatically, we shouldn't have to worried about binary compatibility issues in this case.
I don’t like this idea though. If it’s a clang-importer thing specifically for Swift importing C/C++ annotations, I don’t think that normal clang invocations should look here implicitly. The clang-importer invocation should probably be responsible for adding the search path relative to the -resource-dir flag, just as clang does for its own resource headers.
Heh, technically the clang-importer should be shipping its own copy of the resource headers that match the embedded clang compiler.
There, I think I'm positioned almost exactly opposite to what @compnerd said.
Now, for swiftToCxx since that can has been opened and I have a bone to pick with the _SwiftCxxInteroperability.h header. I don't know that any of the default search locations are the right thing for that header. It contains content that is specific to the header generated by a specific compiler. If I have a library that vends a C/C++ API and ships a generated header, we need to ensure that the semantics are maintained. If I use a Swift 0 toolchain to generate the header, then send the pre-compiled library and bridging header to my friend who has a Swift 1 toolchain installed, the semantics of the bridging header that ships in both toolchains could have changed. Really, I think that header needs to be embedded in the generated header itself, or put next to the generated header to ensure that the intended semantics of those macros are maintained and you're not accidentally getting "swift 1" semantics on a header that was intended to have "swift 0" semantics. Then we don't have to ship a copy of every bridging header under the sun, the intended semantics are kept with the appropriate file.
I think that a layout of <toolchain>/lib/swift/<major>.<minor>/include would be reasonable for the C++ interop headers. This would be similar to the clang (builtin) headers as compiler resources. The shims are ideally something which would be in the SDK.
Yeah, the stdlib shims should go along-side the Swift.swiftmodule as part of the SDK.
<toolchain>/lib/swift/<major>.<minor>/include is good. Still have my thoughts about Swift -> C++ headers for the reasons I put above, but for C++ -> Swift, seems fine for that to be part of the toolchain and implicitly pulled in by the clang importer.
I'm referring to where the compiler is shipped, i.e., the output of swiftly use -p for OSS toolchains and realpath $(xcrun -f swift)/../../../ for Apple toolchains.
I see, thanks for clarifying!
The header file only consists of interop annotations (i.e., convenience macros that expand to swift_attrs) used by ClangImporter, and are closely tied to the Swift compiler version, so I don't think it should be considered part of the SDK. The closest analogue I can think of is Clang's ptrcheck.h, which is shipped in Clang's resource dir.
I'm also in favor of <toolchain>/lib/swift/<major>.<minor>/include.
If Clang doesn't look here, then that would mean the clang binary that ships with the Swift toolchain won't accept header files written for both Swift and C/C++. On one hand, it's odd to me that two compilers that are part of the same toolchain would need this additional configuration. And there is some precedent here, since the Apple-vended clangdoes automatically find the swift/bridging header.
On the other hand, I do see the value in keeping the Swift toolchain's clang's search path behavior consistent with non-Swift toolchain C/C++ compilers, which would be good to support. Someone elsewhere suggested exposing the location of the swift/bridging header in the output of swiftc -print-target-info, so that it can be queried by a build system and passed along to any C/C++ compiler.
Yeah, I actually really like this idea. Last I looked, the reverse interop header we generate uses some gnarly relative paths to get at the location of _SwiftCxxInteroperability.h; simply dumping the contents _SwiftCxxInteroperability.h in there would remove reliance on any header searching altogether. I think this is also a separate conversation but certainly something I'd like to do.
This is so that the people can write the macros in their header and it has a definition? That is unfortunate. It's entirely possible that they will need to do the undefined macro dance for their public headers anyway,
#ifndef BLAH
#define BLAH
#endif
An arbitrary C/C++ library will likely need to be consumed in environments that don't have a Swift toolchain installed (or available at all), or clang even. In those cases, we certainly won't have the header around, regardless of where we put it, so the public headers will still need to declare all of the macros that they use for making it nicer to import into Swift.
