Full Type Metadata On Windows

I was looking into the handling for the full type metadata for types. It seems that we have a case where the generated code cannot be represented in the PE/COFF relocation model.

Consider the following code:

class C {}

This generates a full type metadata for the class C. Since it derives from Builtin.Object being a swift type, the record contains a reference to the underlying witness value table. This is lowered as:

.data
$S1M1CCMf:
  .quad ($S1M1CCfD)
  .quad ($SBoWV)
  ...

Because $SBoWV is imported from a different module, this needs to be indirected through the synthetic import symbol.

What would be the best way to handle this?

CC: @John_McCall @Arnold @Slava_Pestov

You can create a copy of that value witness table for the current image, instead of trying to share the one from the runtime. The identity of value witness tables does not matter. Last time this came up, you mentioned that C++ compilers on Windows do something similar for vtables IIRC.

1 Like

Yes, C++ on Windows does in fact emit vtables everywhere. If the identity is not important, that does make this an easy way to deal with. Would you by any chance have some pointers for how I can get started on changing IRGen to emit the witness table everywhere for Windows?

In GenMeta.cpp, in ClassMetadataBuilderBase::addValueWitnessTable, the implementation currently assumes it can use one of the prefab value witness tables for Builtin.NativeObject or UnknownObject. If you’re targeting Windows, you could call emitValueWitnessTable instead, which will generate a value witness table specific to the type instead. You’ll probably want to further tweak the code inside emitValueWitnessTable (in GenValueWitness.cpp) so that it only generates one class value witness table with ODR linkage, reusing it if already emitted, instead of generating a table for every single class, though.

In fact, that’ll probably be required; otherwise we’ll just use external linkage, which will create linker conflicts.

Alternatively, maybe it makes sense to move this logic into getAddrOfValueWitnessTable. On ELF and Mach-O and similar platforms, we treat the LinkEntity for a value witness table to a known builtin type as having external linkage; for Windows, we could conditionally treat them as having ODR linkage instead and when one is referenced, arrange for emitValueWitnessTable to be called lazily to define them in the current IRGen job. That would allow most of the higher-level IRGen logic to remain the same, while more thoroughly catching other situations where we try to use common value witness tables, and then Windows could still get some benefit from sharing common value witness tables among types with equivalent layout.

That makes sense.

The part that I am struggling with is the actual emission of the value witness table for the base type (arranging for emitValueWitnessTable for the conformed protocol). Even if we change the SILLinkage (which is slightly annoying due to the inability to identify the target within a LinkEntity as you don’t have the IGM), we still do not have the value for the definition. I am not seeing where to get the value to actually emit the initializer for the constant.

It’s in the body of emitValueWitnessTable, after the early-exit logic to try to use a known value witness table:

  ConstantInitBuilder builder(IGM);
  auto witnesses = builder.beginArray(IGM.Int8PtrTy);

  bool canBeConstant = false;
  addValueWitnessesForAbstractType(IGM, witnesses, abstractType, canBeConstant);

  // If this is just an instantiation pattern, we should never be modifying
  // it in-place.
  if (isPattern) canBeConstant = true;

  auto addr = IGM.getAddrOfValueWitnessTable(abstractType,
                                             witnesses.finishAndCreateFuture());
  auto global = cast<llvm::GlobalVariable>(addr);
  global->setConstant(canBeConstant);

  return llvm::ConstantExpr::getBitCast(global, IGM.WitnessTablePtrTy);

This part may need to be factored out into an emitValueWitnessTableDefinition function, that could be reused to lazily define ODR value witness table definitions for known layouts.

I’m sorry, it seems that I am not understanding something. There are a couple of sites where getAddrOfValueWitnessTable is used. Those need to ensure that the VWT is emitted. However, the problem with always emitting the VWT is that the body of the emitValueWitnessTable calls getAddrOfValueWitnessTable which will then go and build the VWT and recurse (infinitely due to no base case). How do we determine that getAddrOfValueWitnessTable should emit or not?

Interestingly enough, this now is also preventing getting some of the IRGen tests working on Windows :smiley:

We have to follow a similar pattern for many things we lazily emit that you can follow, for instance getAddrOfTypeContextDescriptor. If you look at getAddrOfTypeContextDescriptor's implementation, you’ll see that it notes the use of the entity before producing the address of the declaration. The note adds it to a list of declarations to lazily emit (if it hasn’t already), which we walk later when finishing up IRGen to emit everything we needed.

I don't know how executable formats work, and how a vtable can be used by swift runtime/compiler, thus you can ignore my comment If you think it does not fit for the issue.

Windows executable format does have a resource section, which is used to hold icon of exe file. It can hold other types like strings for localization. MSVC uses it to store window design and Delphi uses it to store Forms (both equivalent to interface builder in Xcode). Even info.plist counterpart in Windows is stored in this section.

Resource section can be added or even modified after compiling. It can be easily modified by a pair of Win32 API functions.

Some parts of witness table (or whole) can be stored in this section, after compiling, and a Swift app can load this section when running, or use this section in a dll which compiled by Swift.

The advantage is they can be added even after compiling, the downside is they can be easily modified by user.

More information is here