Clang Importer does not import simple functions if they have int parameters or return types

Development Compiler
1

Hi,

Me again. My uSwift fun is progressing nicely. I have compiled a simple, minimalist stdlib and I'm able to use it to compile to SIL/LLVM on unusual targets (AVR in my case).

The next problem I'm seeing is the system module I use to do lots of helpful things, written in C, has many of its functions no longer visible in my swift code. This is true even if I target x86_64

I've confirmed that if I use exactly the same code but use an off the shelf swift toolchain/swift/stdlib then it all compiles fine.

Doing a bit of investigation in the REPL, I can import my system module "CAVR" and I can see some of the function definitions in it but only if they take and return void or simple pointer types.

There is some piece missing, and I have deduced it must be in my uSwift stdlib, that is necessary for the clang importer to work.

I think there must be something in the compiler that formalises the mappings of like unsigned short in C mapping to the Swift stdlib type UInt16. But I can't figure out what/where it is.

I think I'll need to add some bits to my stdlib but I'm not sure where to start. And the whole point of this is to (a) understand how the stdlib/compiler interaction works better and (b) to create an absolute bare bones, minimal stdlib for my purposes.

Can anyone give advice or pointers how the clang importer knows what types in the stdlib to import function definitions as? I can see the MappedTypes.def but I can't see why it's not working?

Thanks for any help/advice guys!

2

I only have vague knowledge of this area, but when I was working on the C type importing for Windows these are the files I had to touch to get the mappings working. One of those is CTypes.swift, which maps e.g. CUnsignedInt to UInt32; that might be what you're missing?

3

That looks interesting and definitely near to what is breaking in my stdlib.

The only thing is I'm compiling very simple linux style C code, which doesn't have fancy types like CUnsignedInt, just types like "unsigned short", which I understand to be pretty much standard c compiler intrinsic in all reasonable compilers, rather than fancy aliases.

Unfortunately I'm in work at the moment and can't post my C code here. I'll do another post tonight with the functions that are not importing.

In the meantime, another thought...

The ClangImporter has this macro in MappedTypes.def that looks like it might be involved in the interpretation of C code type imports...

#define MAP_STDLIB_TYPE(C_TYPE_NAME, C_TYPE_KIND, C_TYPE_BITWIDTH,  \
                        SWIFT_TYPE_NAME, CAN_BE_MISSING,            \
                        C_NAME_MAPPING)                             \
    MAP_TYPE(C_TYPE_NAME, C_TYPE_KIND, C_TYPE_BITWIDTH,             \
             "Swift", SWIFT_TYPE_NAME, CAN_BE_MISSING,              \
             C_NAME_MAPPING)

// MacTypes.h defines typedefs UIntXX, SIntXX, FloatXX.
MAP_STDLIB_TYPE("UInt8",  UnsignedInt, 8,  "UInt8",  false, DoNothing)
MAP_STDLIB_TYPE("UInt16", UnsignedInt, 16, "UInt16", false, DoNothing)
MAP_STDLIB_TYPE("UInt32", UnsignedInt, 32, "UInt32", false, DoNothing)
MAP_STDLIB_TYPE("UInt64", UnsignedInt, 64, "UInt64", false, DoNothing)

This is then used in getSwiftStdlibType, which looks like it might be the system that reads types from C and finds their matching type in stdlib

static std::pair<Type, StringRef>
getSwiftStdlibType(const clang::TypedefNameDecl *D,
                   Identifier Name,
                   ClangImporter::Implementation &Impl,
                   bool *IsError, MappedTypeNameKind &NameMapping) {
  *IsError = false;

  MappedCTypeKind CTypeKind;
  unsigned Bitwidth;
  StringRef SwiftModuleName;
  bool IsSwiftModule; // True if SwiftModuleName == STDLIB_NAME.
  StringRef SwiftTypeName;
  bool CanBeMissing;

  do {
#define MAP_TYPE(C_TYPE_NAME, C_TYPE_KIND, C_TYPE_BITWIDTH,        \
                 SWIFT_MODULE_NAME, SWIFT_TYPE_NAME,               \
                 CAN_BE_MISSING, C_NAME_MAPPING)                   \
    if (Name.str() == C_TYPE_NAME) {                               \
      CTypeKind = MappedCTypeKind::C_TYPE_KIND;                    \
      Bitwidth = C_TYPE_BITWIDTH;                                  \
      if (StringRef(SWIFT_MODULE_NAME) == StringRef(STDLIB_NAME))  \
        IsSwiftModule = true;                                      \
      else {                                                       \
        IsSwiftModule = false;                                     \
        SwiftModuleName = SWIFT_MODULE_NAME;                       \
      }                                                            \
      SwiftTypeName = SWIFT_TYPE_NAME;                             \
      CanBeMissing = CAN_BE_MISSING;                               \
      NameMapping = MappedTypeNameKind::C_NAME_MAPPING;            \
      assert(verifyNameMapping(MappedTypeNameKind::C_NAME_MAPPING, \
                               C_TYPE_NAME, SWIFT_TYPE_NAME) &&    \
             "MappedTypes.def: Identical names must use DoNothing"); \
      break;                                                       \
    }
#include "MappedTypes.def"

    // We did not find this type, thus it is not mapped.
    return std::make_pair(Type(), "");
  } while (0);

  clang::ASTContext &ClangCtx = Impl.getClangASTContext();

  auto ClangType = D->getUnderlyingType();

I ran swift in the debugger and put a breakpoint on getSwiftStdlibType to find out why the types are not being picked up, but the breakpoint is not being hit!

I've confirmed that other breakpoints on the executable (e.g. main) are being hit so lldb is working fine but it seems this code is not being hit.

I'm not sure where to look now?

4

You need the typealiases in CTypes.swift to be defined for the Clang importer to be able to map the C types. If you define the typealiases, then you should be able to start importing declarations using the corresponding types.

5

You guys are spot on. Thanks guys. :slight_smile:

For anyone reading this in the future, @Joe_Groff pointed out the problem. Stdlib must contain entries like:

public typealias CUnsignedChar = UInt8
public typealias CUnsignedShort = UInt16
public typealias CUnsignedInt = UInt32
public typealias CUnsignedLong = UInt
public typealias CUnsignedLongLong = UInt64
...

After a bit of investigation, I realised that simple types in C are interpreted by the clang importer as builtin types like clang::BuiltinType::UShort, then the class SwiftTypeConverter from ImportType.cpp has a method called VisitBuiltinType, which attempts to find a matching stdlib definition.

    // Given a loaded type like CInt, look through the name alias sugar that the
    // stdlib uses to show the underlying type.  We want to import the signature
    // of the exit(3) libc function as "func exit(Int32)", not as
    // "func exit(CInt)".
    static Type unwrapCType(Type T) {
      // Handle missing or invalid stdlib declarations
      if (!T || T->hasError())
        return Type();
      if (auto *NAT = dyn_cast<NameAliasType>(T.getPointer()))
        return NAT->getSinglyDesugaredType();
      return T;
    }
    
    ImportResult VisitBuiltinType(const clang::BuiltinType *type) {
      switch (type->getKind()) {
      case clang::BuiltinType::Void:
        return { Type(), ImportHint::Void };

#define MAP_BUILTIN_TYPE(CLANG_BUILTIN_KIND, SWIFT_TYPE_NAME)             \
      case clang::BuiltinType::CLANG_BUILTIN_KIND:                        \
        return unwrapCType(Impl.getNamedSwiftType(Impl.getStdlibModule(), \
                                        #SWIFT_TYPE_NAME));
          
#include "swift/ClangImporter/BuiltinMappedTypes.def"

In this particular case for example, "unsigned short" in the C code would be parsed in clang as clang::BuiltinType::UShort and that would be interpreted by this macro usage (from BuiltinMappedTypes.def): MAP_BUILTIN_INTEGER_TYPE(UShort, CUnsignedShort) as to be translated to the stdlib type CUnsignedShort. In the code above, if the definition of CUnsignedShort is missing (as it was in my case), then the code will silently fail to read that function definition and will elide it from the imported module or bridging header definitions:

      // Handle missing or invalid stdlib declarations
      if (!T || T->hasError())
        return Type();

In order to test this I put a breakpoint on this return statement:
break set -f ImportType.cpp -l 208
and sure enough, it was hit:

Process 34438 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 3.1
    frame #0: 0x0000000102a1985b swift`(anonymous namespace)::SwiftTypeConverter::unwrapCType(T=Type @ 0x00007ffeefbf1140) at ImportType.cpp:208
   205 	    static Type unwrapCType(Type T) {
   206 	      // Handle missing or invalid stdlib declarations
   207 	      if (!T || T->hasError())
-> 208 	        return Type();
   209 	      if (auto *NAT = dyn_cast<NameAliasType>(T.getPointer()))
   210 	        return NAT->getSinglyDesugaredType();
   211 	      return T;
Target 0: (swift) stopped.
...
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 3.1
  * frame #0: 0x0000000102a1985b swift`(anonymous namespace)::SwiftTypeConverter::unwrapCType(T=Type @ 0x00007ffeefbf1140) at ImportType.cpp:208
    frame #1: 0x0000000102a12cc0 swift`(anonymous namespace)::SwiftTypeConverter::VisitBuiltinType(this=0x00007ffeefbf1d40, type=0x0000000115012460) at BuiltinMappedTypes.def:34

Line 34 showed the missing definition, the first function parameter that could not be mapped:
MAP_BUILTIN_INTEGER_TYPE(UShort, CUnsignedShort)

Once I added typealiases such as:
public typealias CUnsignedShort = UInt16
to my fledgeling stdlib, the import worked and the C functions are visible to the swift. Yay! :grin:

2 Likes