How to link one witness table from stdlib

Development Compiler
1

In a SIL pass, if I want to emit a call to a method required by a stdlib protocol using witness_method + apply, e.g. FloatingPoint.init(_builtinIntegerLiteral:), how can I link just one witness table from the stdlib module?

Currently, only SILModule::linkAllWitnessTables works for me, but it's very inefficient.

4 Likes
2

It may be that there's no way to do that right now, but that ought to be relatively easy to fix.

3

Trying to directly use SerializedSILLoader now. Thanks.

4

I ended up having something like:

    auto *decl = conf->getDeclContext()->getAsNominalTypeOrNominalTypeExtensionContext();
    auto linkage = getSILLinkage(getDeclLinkage(decl), NotForDefinition);
    auto *newTable = module.createWitnessTableDeclaration(conf, linkage);
    witnessTables.insert({conf, newTable});
    newTable = silLoader->lookupWitnessTable(newTable);

However, the witness functions this declares have shared linkage by default. This causes the verifier to crash because the witness tables are public_external. Not sure if it's a bug or not.

To work around the crasher, I reassigned public_external linkage to each witness.

    // Update linkage for witness methods.
    // FIXME: Figure out why witnesses have shared linkage by default.
    for (auto &entry : newTable->getEntries())
      if (entry.getKind() == SILWitnessTable::WitnessKind::Method)
        entry.getMethodWitness().Witness->setLinkage(linkage);
5

You don't need to deserialize a SIL witness table to emit a witness_method instruction. The witness_method instruction references a type (which can also be concrete) and the protocol requirement's ValueDecl wrapped in a SILDeclRef. The devirtualizer pass can deserialize witness tables already when transforming witness_method calls to concrete function references. Also the SIL linker will automatically deserialize witness tables as needed. I don't think you should be attempting to devirtualize such method calls yourself.

Can you give more details about what you are trying to do?

6

Are the witnesses themselves public? SILGen emits witness thunks with shared linkage, which means you need their body if you're planning on referencing them directly, so this simple approach won't work and you should instead use the SIL linker.

However, see my other e-mail in this thread; I suspect you don't need to explicitly deserialize a witness table at all.

7

Thanks for the pointers. Perhaps there's a bug in SILModule::lookUpWitnessTable on the tensorflow branch, where the derializeLazily flag won't make it successfully serialize the witness table.

The witnesses themselves are public: I'm calling _ExpressibleByBuiltinIntegerLiteral::init(_builtinIntegerLiteral:). Currently SerializedSILLoader works perfectly. I'm yet to verify whether the simpler approach works on master yet.

8

Even if a witness is public, the witness thunk is not going to be public. Witnesses are always invoked through a witness thunk since protocol requirements use a special ABI distinct from non-protocol methods.

Why don't you instead emit a witness_method instruction to call the protocol requirement and allow the devirtualizer to turn it into a call of a concrete function later? I don't think it's a good idea for several parts of the compiler to perform devirtualization.

1 Like
9

Why don't you instead emit a witness_method instruction to call the protocol requirement and allow the devirtualizer to turn it into a call of a concrete function later?

That's what I tried initially, but it either crashed or failed verification (I can't remember). It's likely an issue in the current tensorflow branch, so I had to use the deserializer. I can verify later when we merge from upstream.

10

Please do. In general, if you come across some behavior that doesn't make sense and requires a hack to work around, make a note of it at least, so that we can sort out the underlying issue before the tensorflow work goes upstream.

Also there's a general strategy that works well for asking questions: instead of asking how you can do some specific thing, tell us what the overall problem you're trying to solve is. In this case, it would be the devirtualization crash. Manually deserializing a witness table is just a workaround that obscures what was really going on.

1 Like