Proper Support for MSVCRT linkage

Development Compiler
1

As the Windows port gets more stable, one thing which starts to stand out is the lack of support for proper linkage to MSVCRT. I would like to solicit some advice on how to best expose this in the swift driver.

The first issue is that there is a confusing landscape to cover. When I say MSVCRT (Microsoft Visual C Runtime) I really just mean the C runtime. There is the "deprecated" MSVCRT and the new more modern "ucrt". However, there is really no point in differentiating this as the ucrt is available in all partitions of Windows and in both the WIn32 mode as well as the Metro mode. So for the sake of this discussion, msvcrt and ucrt are synonymous as we will always use ucrt.

Now, the problem is that the runtime is available in 4 variants, each of which is ABI incompatible. So you must select one variant and be consistent throughout for it (the Swift runtime/standard library, libdispatch, Foundation, and application code). Failure to do so will result in runtime failures which are difficult to diagnose (as expected). Note that when performing a static link, ALL libraries must be statically linked into the final binary, including the standard library, any dependencies, and the C runtime as you may only have a single instance of the runtime otherwise you will have cross-domain frees and you will have mismatched resource tables (e.g. file descriptors).

The selection of the runtime in the C/C++ land is driven by a set of flags:

  • /MTd
  • /MT
  • /MDd
  • /MD

This is due to the programming model compatibility with the pre-multithreaded model. /MT enables the multithreaded model with static linking, and /MTd enables the multithreaded model with static linking with additional debugging features. /MD is similar just with shared linkage (DLL mode). Note that the debug mode builds are not meant for distribution (there is no debug variant redistributable of the runtime). These flags enable a different set of C preprocessors.

   Library    | Associated DLL | Option | Preprocessor Directives | Distributable
--------------+----------------+--------+-------------------------+--------------
 libucrt.lib  |    [static]    | /MT    | _MT                     |      Y
 libucrtd.lib |    [static]    | /MTd   | _MT _DEBUG              |      N
 ucrt.lib     |    ucrt.dll    | /MD    | _MT _DLL                |      Y
 ucrtd.lib    |    ucrtd.dll   | /MDd   | _MT _DLL _DEBUG         |      N

Now, when the options are actually enabled in C/C++ code, the compiler will embed an appropriate directive for the linker to select the correct library and prevent the alternates from being pulled in. Since we do not do this in Swift currently, the user must correctly select the library to link against.

The question now is, is it acceptable to expose this control to the user? If so, what is the desired way to expose this to the user? At least for Windows developers, the /MT, /MTd, /MD, /MDd options are familiar. However, we also currently do not support /MT or /MTd configurations in the build (as I believe /MD and /MDd to be better and thus only bothered wiring that through).

CC: @jrose @graskind @Torust

2

I've got a few scattered thoughts on this, but not really any great solutions.

One option is to just build all six variants of the Swift/Foundation/Dispatch libraries: dynamic-Swift dynamic-CRT, dynamic-Swift dynamic-debug-CRT, static-Swift dynamic-CRT, static-Swift dynamic-debug-CRT, static-Swift static-CRT, static-Swift static-debug-CRT. That's obviously not great in terms of size of the toolchain, but does comprehensively address any need. As a possibility, the default toolchain could only include /MD, with a 'full' toolchain available/buildable containing every option. For driver options, I'd think something like a --windows-crt=MDd argument would be fine.

From a user perspective, I've always preferred /MT applications – installing endless variants of the Visual C++ redistributable has always felt clunky to me, particularly before I understood why it was necessary.

From a developer perspective, I'd be inclined to only support /MD (non-debug), since that's the most widely compatible variant, and crucially is the least likely to run into issues when linking to third-party libraries.

I do think a statically-linked Swift is valuable on Windows, although that doesn't necessitate /MT as I understand it. I don't think that's currently possible since all the dllimport semantics are currently unconditional, but that could be a good step in that direction if we do eventually want to support /MT or MTd.

3

The question of what the toolchain should distribute is a separate concern. This is more a question of how to expose this from the driver. I'm not a fan of the --windows-crt= option (least of which is because the single dash variants fit better into the existing options, but also the processing the flag this way requires additional diagnostics).

To the redistributables for the VC runtime, the redistributable for the standard library can include that.

Yes, /MD is the most widely compatible, which is why the nightlies are /MD and the build only supports /MD and /MDd.

Disabling the dll storage annotations requires that we have a way to specify whether the build is static or not. Having something more fine grained than everything or nothing requires a way to have a way to specify dll storage explicitly.

4

What would be your preference in that case? I was basing that suggestion off the existing --static-stdlib flag, although now that I think about it more perhaps --static-windows-crt and --debug-windows-crt might be better options. The /MDd etc. names may be familiar to Windows developers, but they’re nonobvious to others, and are also in my opinion too terse for such an impactful flag; I’d be in favour of a more explicit option.

1 Like
5

I am on the fence about /MD because it is confusing to me, particularly when spelt with the - form. Do you mean -MD the windows dynamic multithreaded standard library or do you mean the make dependencies?

I think that -- forms are terrible, and -static-windows-crt is a bit odd. If we are going that route, then I think copying clang's -static-libc++ and introducing -static-libc and -static-libc++ is nicer. There is no reason to limit this to Windws.

The thing that I don't like about this is that you now need to specify two flags: -static-libc -debug-libc … which feels error prone. The thing is that it is meaningful to combine a release swift standard library with the debug VC runtime (and in fact would make development significantly easier for the Windows port). I suppose that we could spell these:

  • -debug-static-libc [/MTd]
  • -static-libc [/MT]
  • -debug-shared-libc [/MDd]
  • -shared-libc [/MD]

Although they feel a bit long, they scale well to -[debug-]-[static|shared]-libc++ for the future.

6

I see the problems but I don't think I know enough about Windows dev to have a good solution. I think I'd recommend putting "debug" second in the name if you go for combo names, though, since it's not exactly a debug info option and it's definitely not a compiler-debugging option.

7

IMO, placing the debug in the middle makes the options even more unwieldy. Changing the build type now requires you to change the middle of the option. I think that really the problem here is coming up with a way to control the various libc variants with a single knob that gives us enough fine grained control. This is already a problem with Linux, and the need to switch between musl/glibc. On some distributions, this is handled cleanly via the triple, e.g. x86_64-unknown-linux-gnu vs x86_64-unknown-linux-musl.

IMO, the problem here is not the Windows aspect of it, but coming up with a good solution to expose the control through the driver.

8

I'm not sure I'd classify the environment part of the target triple as "cleanly". Target triples are great for cross-compiling, but for host compiling they force you to specify information you don't care about explicitly, which you may not have ever even learned how to spell.

(This applies to macOS deployment targets too, by the way, which also require the full target triple. I still think it's better than -min-*os-deployment-version, but mostly because Clang had one of those for each platform.)

9

I disagree with you on the use of the environment there. That is really the purpose of the environment field. Actually, for exherbo, host compilation is complicated too, because when you say host, which host are you talking about? I can have parallel installations of a musl, glibc, and uclibc environment on x86_64. Because they are co-installed, you could be building for any one of them! There is no good solution to identifying what is running currently, especially if you get a toolchain vended by a third party. When you start making libcalls, the behavioural differences of the environment become material, so I think that the "you don't care about" becomes the null set, unless you are in a freestanding environment, in which case, really, your triple should be x86_64-unknown-none and you should be specifying -ffreestanding. Because you no longer have a libc or any hosted environment that the compiler can reason about, the perceived unnecessary bits disappear as well! IMO, this is rather elegant. But, we can disagree on that.

To the issue on hand though - the variants of MSVC are not as easy to specify in the environment. Both cl and clang-cl require the /MT, /MTd, /MD, /MDd options, which translates into embedded linker directives to avoid the linker invocation needing to be altered to get the correct linkage. I think that one option may be to just use those spellings, with the explicit /-only spelling to differentiate them (cl and clang-cl will accept a - spelling just as well as the / spelling of these).

2 Likes
10

If you think it's the way to go, I'd be happy to see an environment field for MSVC triples that encapsulates this. That would need LLVM buy-in, though—we should not be setting our own standard here.

I'd be sad to see / options for swiftc, even if it were only for options taken verbatim from cl.

11

Right, and I suspect that will be far more challenging as that will be a huge effort to get that right. There is already the unspecified, required details about the target triple for MSVC - a properly constructed triple currently is of the form: x86_64-unknown-windows-msvc19.21.27619. It does not differentiate between the ucrt and MSVCRT, nor does it account for the platform or partition. There are a number of options which currently control that, and those options are designed to mirror Microsoft's behaviour. This effectively comes down to something like: clang -target x86_64-unknown-windows-msvc19.21.27619 /LD /MD /D_WINAPI_PARTITION_DESKTOP /D_USRDLL. The thing is that the defaults for WINAPI partition are desktop, which I think means that most people are relatively happy. /LD is spelt -shared. The MSVC toolset version (and consequently, the version of the UCRT) is specified by means of environment variables. _USRDLL is needed to correctly identify the shared build and specify the correct DLL storage. That really just leaves the /MD (and family) option. The slash'ed arguments are the best that I can currently see. The other spellings are challenging for various reasons.

Actually, thinking more about this, I think for the Windows case, this might be best as a flag even longer term - CMake recently introduced CMAKE_MSVC_RUNTIME_LIBRARY to control the flag. And thinking longer term, I think that we will need to figure out the C++ runtime variants as well. I guess, then the question is, would you be okay with adding the following set of flags:

  • -MT
  • -MD
  • -MTd
  • -MDd

since you do not care for the / variants.

12

I hate them but there's precedent and maybe that's more important.

13

This is by far preferable to me over keeping the historical flags. I would argue that the precedent that swiftc arguments actually mean something when you read them without context outweighs the precedent of the /MT /MTd /MD /MDd variants. I also personally think -debug-libc -static-libc composes fine (rather than -debug-static-libc), but I can see the argument for either option there.

14

We could also do something like the sanitizers, or Clang's C++ stdlib choice: -libc=debug-static-ucrt, with a possible alias of -libc=MD. I don't love this either, partly because it seems kinda verbose for an option that (it sounds like) MSVC users will use all the time, but it's another choice in the space.

15

I'd been operating on an assumption that by default the flag with be set to /MD since that's what most people will want most of the time, and that these options would be fairly rarely used as a result. @compnerd, is that a reasonable assumption or do people use /MT and MDd more than I thought?

16

Unfortunately, that assumption doesn't hold up as well as we would like I think. For Windows development you really do want to switch between /MDd/MTd and /MD//MT more often than not. I agree with @jrose that -libc= feels more verbose, but, I don't think that -libc=[MD,MDd,MT,MTd] is the worst possible option. So, if that is more preferable, I think that its better than the original 4 variants that we had. Furthermore, I really would prefer to match the defaults of MSVC rather than create our own. That established usage is fairly important to maintain IMO as it will confuse developers if it were the opposite of the cl behaviour.

17

That may be true for C/C++ programming, and I think that’s the perspective you’re coming from – large mixed-source codebases and experienced Windows developers. I don’t dispute that your approach may best fit that class of programming.

From my perspective, which is that of a Swift programmer wanting to port to Windows, the cl-style arguments are foreign and unintuitive, and I’m not particularly interested in relying on a debug build of the CRT to catch things like buffer overflows and memory leaks – instead, assertions in and features of safe Swift code catch those for me.

To me, having to add a -debug-libc or -static-libc flag in the rare cases I need something different from the default seems perfectly reasonable. Additionally, those really do seem like separate options – I can see toggling between debug and normal in some rare cases, and I can see switching from shared to static, but it feels like those changes would almost never happen simultaneously.

18

I think that I prefer that we make the option explicit on Windows (and required). Particularly with CMake, this becomes really simple: CMAKE_MSVC_RUNTIME_LIBRARY and the MSVC_RUNTIME_LIBRARY property can be passed to the driver and everything should work just fine that way.

19

The downside of any required options is that they increase friction. (This is the main win of build-script over cmake, for example…except that build-script with no arguments is a bad default.)

20

The problem with the default is that I think that we should match the behaviour of cl which is /MT to reduce friction and prevent surprise. However, this mode is currently unsupported in Swift, and more importantly does not work very well in practice (everything must be statically linked or you will fail at runtime with weird errors).

I just checked this with the latest VS toolset (19.21). This is the command line default. However, with Visual Studio 2015 and newer, Visual Studio will provide an overridden command line parameter to switch to /MD.

I think that the element of surprising behaviour and trying to reduce conflicts while making it easy to use is the problem here. I am happy enough to default to /MT to match cl.