Getting a better understanding of how the swift calling convention impacts the AVR ABI

Development Compiler
1

Hi, I've prodded around at this topic with PRs and discussions to do with calling conventions generally and the SwiftABIInfo in clang, but I realised I didn't really understand it properly. I wanted to ask some basic questions to try and make sure I get it properly as I realised there are subtleties. To get my head straight, I wanted to split the questions into three separate scenarios:

  1. pure clang building machine code sample back end (in my case AVR) from normal C code (e.g. header files/C files)
  2. swift building machine code for various back ends (in my case AVR) where it calls other swift functions
  3. swift calling C code for various back ends (in my case AVR)

Note: one thing to bear in mind is the AVR back end is rather crude compared to the major architectures, it only recently became mainstream and has had orders of magnitude less work done on it than back ends such as ARM/x86. It is definitely not guaranteed bug free. We will improve it but it's baby steps really, given our community size.

  1. In the first case, when clang/llvm is implementing the de-facto standard AVR ABI (avr-gcc - GCC Wiki), I realised I was missing subtleties. I'm familiar with the llvm AVR back end code, but that's only part of the picture. The AVR GCC ABI in its section on "calling convention" describes how to allocate registers and structure the stack for calls but with terms like "the first argument", "the second argument", "the return value", etc. it's talking about the end to end process from C call sites to machine code. In reality clang does some of that and LLVM does some. The part myself, Ben, Dylan, Ayke and others are familiar with is the code in llvm-project/llvm/lib/Target/AVR such as AVRTargetLowering::LowerCall. These actually lower from LLVM IR to machine code (register assignments, stack handling, call frame creation). To state the obvious, clang does the lowering from C code call sites to LLVM IR function definitions. I think I have always envisaged that as a one to one mapping, but it doesn't have to be. For complex calling conventions and/or types, clang might change the LLVM IR emitted somewhat.

From what I can see, the clang code does very little modification to the arguments as lowered from C and mostly passes them to LLVM unchanged. Also it doesn't seem to respect any calling convention flags, which probably accounts for some of the confused conversations between our team and the experienced clang/llvm people. Most of the code that decides on things like register allocations, pass on the stack, etc. is hard coded in the AVR Target layer, and largely ignores calling conventions, sticking to the de-facto standard calling convention for AVR (the GCC one from 20 years ago).

  1. By contrast, swift calling swift has a great deal of latitude choosing what LLVM IR to emit, when lowering complex structures, it is free to change the structure of the IR in varied ways. The AVR back end won't know this has happened and will dutifully lower the LLVM IR arguments it is passed, following the AVR-GCC convention on the simple LLVM IR types passed and returned.

To emphasise, this may not be efficient or ideal but it's probably the best we can do for now. We can definitely improve it as time goes on.

  1. Where Swift calls C, I expect things to get a bit more sensitive. It's important that argument types match. This is where I could see some issues sometimes. If our platform doesn't support the swift call convention and the clang importer imports a header file that has functions attributed with swift call and assumes it's working, that might cause an issue?

Anyway... I'm just trying to understand if the above is all correct. Given that our platform basically doesn't support any calling convention other than the standard AVR calling convention, I wonder if I should be adding SwiftABIInfo to the AVR parts of clang at all? I think I did it at one point to resolve a crash but it might not be needed at all at this time?

1 Like
2

Calling convention lowering in LLVM and its frontends is complicated. LLVM IR aims to abstract over many basic calling convention details like whether an argument is passed on the stack vs. in a register, for the convenience of both frontends and middle-ends. To do this, LLVM backends have to be able to make type-aware and target-aware decisions. But ABIs are written in terms of the source language's type system, and LLVM IR's type system is its own thing, even if it superficially looks a little like C; Clang cannot turn every C type into an LLVM type with enough fidelity that LLVM would be able to accurately lower calling conventions. (For example, it is common for _Complex to have special ABI treatment, but LLVM does not have a complex type.) So what actually happens is that frontends make type-aware and target-aware decisions to lower C types into LLVM types and attributes that it knows that the corresponding LLVM backend will then lower correctly. C's fundamental types — pointers, integers, and floats — can typically just be translated to their obvious analogues in the LLVM type system because the obvious thing for the backend to do when it sees those analogues (e.g. an i16 argument) is to reverse that translation (e.g. back to short) and then pass the argument however the C ABI says to pass it. (Although in the specific example of short, the frontend probably also needs to add a zeroext or signext attribute.) But that breaks down for more complex types like structs; if you write a function that passes a small struct by value, and you compile it on different targets, you'll see majorly different IR coming out of Clang.

The effective type of a call site / function declaration in LLVM IR for CC purposes is a combination of (1) the IR function type (e.g. i16 (i32)), (2) the named calling convention, and (3) certain argument and function attributes like sret and signext. It is also target-specific; for example, the SysV and MSVC ABIs on x86_64 are very different, and LLVM handles its side of this by just using different rules for different target triples.

There's a lot of latitude in all this to handle tweaks to the calling convention. The right way to do that depends a lot on the exact tweaks. The first question is whether the tweak is a compiler flag or a declaration attribute. The Clang AST supports calling convention attributes on arbitrary function types, and that's already thoroughly hooked so that targets can say that they do / do not support various conventions; but if it's a global setting, it's easy to make sure that ends up in the CodeGenOptions so that you can consider it during IR generation. The second question is whether the tweak is something that's easy to completely handle in the frontend, like changing the rules around sign extension, or changing whether particular arguments are passed by reference. If so, you should probably just make Clang IR generation emit different IR and then use uniform rules in the backend. If not, you need to communicate the difference down to the backend, which you can do either globally in the target configuration or locally in the on a particular call site / function, probably with some sort of attribute.

Swift uses Clang as a library for both the Swift calling convention and the C calling convention. If you teach Clang how to emit a call to your function correctly, Swift should use the same logic and do the right thing.

5 Likes
3

Thanks for the very comprehensive explanation John! I understand it a lot better now. I think the truth is when clang emits AVR it only ever supports the AVR gcc calling convention. So there’s a chance this patch just isn’t needed. I’m going to try and build a toolchain now without the patch and see if it compiles code to AVR just fine. I’ll report back a bit later.

4

So I've confirmed that if I don't add my patch to llvm before compiling our swift compiler, the compiler is unable to emit IR. The compiler builds, but when I run it on my standard library it traps due to the lack of SwiftABIInfo...

5  libsystem_c.dylib        0x00007ff808da7a49 abort + 126
6  libsystem_c.dylib        0x00007ff808da6d30 err + 0
7  swift-frontend           0x0000000109507881 clang::CodeGen::swiftcall::SwiftAggLowering::shouldPassIndirectly(bool) const (.cold.2) + 33
8  swift-frontend           0x00000001052eda83 clang::CodeGen::swiftcall::SwiftAggLowering::shouldPassIndirectly(bool) const + 387
9  swift-frontend           0x0000000102eaaf53 swift::irgen::NativeConventionSchema::NativeConventionSchema(swift::irgen::IRGenModule&, swift::irgen::TypeInfo const*, bool) + 115
10 swift-frontend           0x0000000103011ab6 swift::irgen::TypeInfo::nativeParameterValueSchema(swift::irgen::IRGenModule&) const + 54
11 swift-frontend           0x00000001030e769c isLargeLoadableType(swift::GenericEnvironment*, swift::SILType, swift::irgen::IRGenModule&) + 300
12 swift-frontend           0x00000001030fa77d (anonymous namespace)::LoadableStorageAllocation::allocateLoadableStorage() + 237
13 swift-frontend           0x00000001030e8813 (anonymous namespace)::LoadableByAddress::run() + 835
14 swift-frontend           0x00000001035b6a21 swift::SILPassManager::runModulePass(unsigned int) + 929

... I hadn't quite appreciated how Swift relies on clang here.

Clang and llvm don't support swiftcall for this target (AVR) though so it feels a bit tricky what to do. I wouldn't expect any swiftcall structured call sites to get emitted in normal clang usage when targeting the AVR platform at this time.

But it seems like Swift needs to activate this part of clang to "get an opinion" from it for ABI questions like "should this type be passed indirectly or directly?". Which seems a bit of a grey area unless I'm misunderstanding how it all works?

I don't think we (the AVR llvm community) are likely to add "full support" for the swiftcall calling convention to pure clang/llvm any time soon. AVR is pretty simple and as far as I know the AVR-GCC calling convention has remained universal. AFAIK our sister languages AVR-Rust and TinyGo for AVR also just use the same old GCC based calling convention, and I don't think there'd be much support from the wider AVR community for changing calling conventions that might change the register/stack behaviour that we all know.

Which might leave Swift feeling a bit unloved. :frowning:

Would it be possible/OK to just support the "high level" part of the swift calling convention for now? That is to say, allow the code in SwiftCallingConv.cpp to be used by Swift when targeting this platform, but not actually support the swiftcall lower part of the calling convention (which would involve more significant changes to clang on AVR and involve "breaking" the standard calling convention that everyone else uses)?

I don't think this would actually "close any doors" on Swift on AVR as from what I understand, we are only talking about optimisations with things like context parameters and error parameters (to help with things like avoiding "thunks" for translating thin to thick calls, with errors, default parameters functions, etc.)

I think Swift will still function without these optimisations on AVR and they can be added later in due course.

If that's acceptable/workable...

Taking a look at the SwiftABIInfo type, it seems the default for shouldPassIndirectly is "yes if the type would use more than four registers". That seems reasonable for AVR too... because more than 32 bits is really a "large" type on AVR and it would get inefficient trying to load it into and out of registers. It's relatively common to use 32 bit integers in our hardware libraries (for things like "ticks") and our llvm backend handles it well, but it's very rare to use 64 bit integers in our Swift based hardware libraries and client code for example.

So, having gone full circle, I am thinking that the default SwiftABIInfo class looks fairly reasonable to just use in a very simple implementation as my original patch did. The only thing I'm wondering, the implementation of occupiesMoreThan looks like it maybe considers integers/pointers to be the same size as registers on the platform? For AVR, registers are 8 bit and pointers are 16 bit. If need be, I can create a simple subclass AVRSwiftABIInfo in the AVR bit of clang, that takes account of this discrepancy?

Carl