I work on an open source project in which we package a C++ library with SPM. I was testing a new version of the C++ lib the other day and ran into the following issue which I would like to understand better:
One of the .cpp files in the project uses std::unique from <algorithm> but does not import <algorithm> explicitly. It does import <vector>. When I build on macOS everything is fine, but when I build in a Swift Docker container, I get error: no member named 'unique' in namespace 'std'.
Looking at the depfile, I see that the <vector> being used on macOS is from LLVM's libcxx, which imports <algorithm>. In the Docker container, however, <vector> is from GNU's libstdc++ which apparently does not import <algorithm>.
Our specific problem is easily fixed by adding #import <algorithm>, but I'm now questioning whether I understand the behavior of the cxxLanguageStandard option in the Swift package definition. Our Package.swift specifies cxxLanguageStandard: .cxx11, and I previously thought that meant we'd get LLVM's libcxx on both macOS and Linux. The behavior described above leads me to believe that my understanding is flawed. Can anyone help explain what's going on?
FWIW, I tried changing the cxxLanguageStandard to .gnucxx11. The depfiles were nearly identical on macOS and were exactly identical on Linux.
SPM ends up invoking the equivalent of clang++ --std=c++11/gnuc++11 in order to effect the compile (change clang++ to g++ for Linux, unless you change the compiler). SPM builds the command line and executes it.
I've always found that, no matter what system I've developed C++ programs (MacOS, Linux, FreeBSD, SGI IRIX, VAX C++, SunOS C++, HPOS), the best bet when using is to explicitly include all the standard library headers you need in a code unit. In your case, I would include both <vector> and <algorithm> explicitly in your code file. All the standard library headers have include guards that guard against multiple includes of the same header file. It's just too difficult over the various standard library implementations, versions of the standard libraries, etc,, to manage all the ways library maintainers have included other headers in each header file.
I think there is a way to tell SPM what compiler to use. Maybe through an environment variable (export CXX=clang++)? Others more experienced in SPM should be able to point you in the right direction. May also need to specify which standard library you want to use (at least on MacOS, you can use libstdc++, even with the clang compiler)
Thanks @jonprescott. This is helpful. I've been conflating the C++ language standard with the C++ standard library. Inspecting the output from $ swift build -v confirms that cxxLanguageStandard: .cxx11 yields -std=c++11 and cxxLanguageStandard: .gnucxx11 yields -std=gnu++11. Both my macOS environment and my Swift Docker container environment are using clang as the compiler for both settings. Here are some more details about my environment:
macOS: Apple Swift version 5.3.1 (swiftlang-1200.0.41 clang-1200.0.32.8); Target: x86_64-apple-darwin19.6.0
Swift Docker: Swift version 5.3.1 (swift-5.3.1-RELEASE); Target: x86_64-unknown-linux-gnu
My next task will be to learn how to select the C++ standard library in my Package.swift.
Note that you can use .gnucxx11 on MacOS, if you are taking advantage of GNU extensions to the language. And, you can use .cxx11 on Linux if you want the strict language definition. Both can be accommodated by clang++ and g++.
There should be a way of specifying linker flags in the manifest.
I see that I can add .unsafeFlags(["-stdlib=libc++"]) to my target's cxxSettings, however, the docs state that:
the use of unsafe flags makes the products containing this target ineligible for use by other packages
so that's not really an option for me. (FWIW, I tried this and was able to confirm via the depfile that it has the desired effect (at least for the step where SPM invokes clang).
Even if this approach were an option, would I also need to add some corresponding option to linkerSettings?
Moreover, is this even a good idea? My impression is that it's probably better if we can just make sure that the library works with either standard library.
If you specify the stdlib, clang will automatically add the right linker settings at the right time (at least based what Xcode does) if the standard libraries are where they are supposed to be. Otherwise you will need to add something to the linker flags (-L instances for example).
Then, maybe all of this can be limited to your library's test targets, where you need to build an executable which would require actually linking of the standard library to build the executable.
The original issue was happening during compile though (no member named 'unique' in namespace 'std'), so I don't think that a link-time-only solution will work. At this point, I'm still persuaded that the best thing to do is to add those <algorithm> imports. Thanks to this discussion though, I'm also more clear on what cxxLanguageStandard is and is not. Thanks again for your help!
You absolutely should! Most of <algorithm> is header-only, as well as <vector>, and the code will generated at compile time. However, there are some implementations that use bits and pieces that get put into the runtime library (the -lxxx). The problem is there is no standardization of the implementation, only the library user interface standards. You may still need to link in the runtime to actually build an executable using your library. The question then becomes "how do you specify the appropriate linkage in SPM to use the library you want, for at least your test targets?"
You will also have to specify -stdlib to put a path to the include directory that includes you standard library headers. That's also part of what the -stdlib sets up. And, the header files need to match up with the run-time library (you can't mix libc++ headers with a libstdc++ run-time binary).
you can't mix libc++ headers with a libstdc++ run-time binary
is one of things I was worried about. I failed to mention that this particular package is configured to produce a dynamic library, which means that the linker is invoked even when just doing a $ swift build of the package itself. It invokes swiftc with the -emit-library flag which, if you run the same command adding -v appears to pass through to ld. The flag -lc++ is included automatically on macOS and -lstdc++ in the Swift Docker container. I wonder whether it's even possible to override those later in the command. My guess is not since the linker is probably just accumulating a list of libraries to link and wouldn't have any awareness of 2 libraries being competing implementations.
The -stdlib flag sets up clang for the appropriate environment. For example, in Xcode, I can use either libc++ or libstdc++ by changing a build setting. That translates into -stdlib=xxx for the library you specify.
However, the the library on MacOS, unless you build it yourself, is only there for backward compatibility to C++98 it seems, doesn't support C++11 and forward. Have you compiled libc++ in your Docker image? Do you want to use libc++ on your Linux image? Do your customers want to support libc++ as an option with your library?
The other issue you have is that SPM is really the Swift Package Manager. There are assumptions that are built in, as far as I can tell, to ensure that the environment provided can support C/Objective-C/C++/Objective-C++ components within the Swift system itself (like Foundation, Dispatch, etc.). These components are built using the native C++ library for each platform (MacOS-libc++, Linux-libstdc++, Windows support libraries on Windows, etc.). So, even if everything is C++ and you're trying to simply use SPM as a build system and packaging system, I think you are going to run into these edge cases when you can't follow the conventions. SPM may evolve over time to address these edge cases, but, it doesn't appear to support your particular use case right now.