Why the compiler doesn't just assume @inliable by default for all symbols in non-resilient modules/builds?

Development Compiler
1

Lots of Swift codes use @inlinable nowadays, a simple search in apple / swiftlang org shows a lot of its occurrences: Code search results · GitHub

Are there any downsides to have @inlinable/@useableFromInline by default enabled on all symbols, for non-resilient modules / builds? Can the compiler possibly implement that? Would help a lot in not having to litter our code with those attributes.

A quick search in swift-nio shows 1300+ @inlinable in Sources/**/*.swift and 500+ @usableFromInline.

To my knowledge there is no technical reason for not having @inlinable by default for these non-resilient modules, but well my knowledge isn't 100% complete.

I know cross-module optimization does some of these already, but I think it doesn't go as far as assuming everything is just @inlinable?

1 Like
2

Yes.

@inlinable use will increase the amount of code that is inlined, which will increase the size of your binary. It is also likely to increase compile times — firstly because the compiler must consider whether it is actually going to inline a function, and second because, having inlined it, the function into which it is inlined is larger and that will impact the time taken to analyse and optimise it.

Meanwhile, @usableFromInline means that symbols that could otherwise be private to a module end up needing to be exported from that module, increasing the size of the symbol table. The biggest concern here is probably on Windows if your code is going into a DLL, because there are rather small limits on the number of symbols that any given image can export — but increasing the size of the symbol table will also have some impact on compile and link times.

Note also that inlining is not automatically a performance win. Yes, it can allow the compiler to perform additional optimizations that it could not do if code were not inlined, and yes, it avoids subroutine calls. On the other hand, it increases the overall size of the code, hence the pressure on the instruction cache, and it may also increase register pressure within functions, making spills more likely.

FWIW, I suspect the reason NIO is so keen on @inlinable is that it will allow the compiler to specialize generics rather than running (much slower) unspecialized code, rather than because NIO's authors actually want it to inline much of the code that is marked that way. @FranzBusch can probably comment?

3 Likes
3

I've heard (and correct me if not) @inlinable is just taken as "hello compiler, ensure everyone can inline this if they want" (by e.g. ensuring the definition is visible to everybody) and is not a direct command to inline the function.
Then the compiler decides if it wants to do it at all or not.
If someone wants to make the compiler more inclined to inline a function, they can use @inline(always), or perhaps there can be a @inline(preferred).

So essentially I'd think this is compiler's issue. It should be able to tell when to inline what, and not need explicit @inlinables. I can understand if it'll take time and effort to implement and it's not "free", but yeah.

Also with @inlinable, the only thing that I care about is not actual function inlining.
There are other important stuff as well such as ARC, exclusivity checks, bounds checks etc....

So essentially the compiler, theoretically, should be able to optimize the calling function based on the info it has about another function from another module that is going to be called in it.
In most cases it'll require inlining, or maybe creating a second version of the called function that doesn't have ARC/exclusivity/bounds overheard that e.g. 10 different callers can then use since it can be common to not need to e.g. go through bounds preconditions since correct user code needs to already ensure the index is within bounds and then the compiler/LLVM can see that.

Right I'm aware of possible issues with larger binary sizes and CPUs having to fetch instructions from DRAM too often. However I meant that we should not need to nudge the compiler for that many functions so the compiler starts thinking about inlining them.

Perhaps what I'm asking for is for "cross-module optimizations" to just be more aggressive I guess? I think CMO does inlining already, but that hasn't managed to satisfy the needs of e.g. swift-nio folks?
I mean it hasn't been able to satisfy my needs either, in e.g. GitHub - swift-dns/swift-endpoint: A highly-optimized library containing types representing an endpoint, such as DomainName and IPv4/v6Address · GitHub, but let's talk about swift-nio since we can all trust swift-nio more.

FWIW, I suspect the reason NIO is so keen on @inlinable is that it will allow the ...

Off-hand I know there are multiple reasons that swift-nio does that.
Specializations are the most important one perhaps. But there are other concerns as well like I've mentioned, for example eliminating exclusivity checks that can cost a lot.

Of course sometimes the function is so small that not inlining it would not really be helpful to anything. I'm not sure if CMO already eliminates those well, and if swift-nio has gone for manually eliminating those (via more attributes) or not.

4

This is correct (it is also true of the inline keywords in many other languages, including C/C++), but it is worth noting that compilers historically have been rather bad at determining whether or not inlining a given function is a win, hence the existence of things like @inline(always).

Well, @inlinable is a signal to the compiler that it will need visibility of the function's definition when building other modules. Without it, it would have to dump the code for the function into the swiftmodule (or, for resilient modules, the swiftinterface) in all cases, and it will then have to pay a cost to decide whether or not to actually inline it whenever you call it in your code.

CMO is probably a better way forward here, rather than being @inlineable by default. Maybe the thing to do here is to raise Github Issues showing the cases where you were hoping CMO would help but didn't?

2 Likes
5

In the beginning, the most common case of the @inlinable usage in NIO and the broader Swift Server ecosystem was to guarantee generic specialization. However, over time we found that aggressive inlining in performance critical code is in many many cases a huge performance win. This is why repos like swift-certificates and swift-asn1 are also littered with @inlinable/useableFromInline.

For server use-cases the code size trade-off is often moot and we prefer faster performance in almost all cases. In addition to that, inlining can also have a positive effect on code size. In particular, I observed in async heavy code bases that inlining often reduces code size because the compiler optimizes away many of the async funclets up until the actual suspension point. This not only reduces the funclet indirection but across modules also the removes task stack allocations.

I agree though that it is use-case dependent, in my opinion it would be best if we could avoid libraries sprinkling these annotations and instead rely on a tunable cross-module-optimization and guaranteed generic specialization flags. Then end-user application can decide what level of inlining across modules they want. At best a sensible default is chosen.

9 Likes
6

Exactly, forgot to mention this myself.

I've been looking at a lot of C++ production code lately as well, and it almost looks like they don't even care about code size. Even when the libraries are used on e.g. phones (more restricted devices, not servers) as well. And these are super high-performance libraries, a bunch of them using lots of SIMD, and SIMD by itself usually adds to code/instruction sizes (one of the easier-to-notice cases were more instructions doesn't necessarily mean more cpu cycles.).

As long as you're not misbehaving by e.g. doing waay too much inlining, or e.g. using too many lookups tables, usually DRAM (and of course CPU caches) don't bite back (where inlined code or table lookups need to fetched from).
Now we (possibly me) should probably do statistical research about this, but yeah that's been the vibe that I'm getting.

8

Another interestingly subtle code size issue is that PC-relative jumps on ARM can only go +/- 128MB due to argument size limitations. You wouldn't think this would come up very often, but on platforms like iOS that combine all system libraries into one binary (the dyld shared cache), it's surprisingly relevant.

One reason servers care less about code size is that they don't tend to have to worry as much about paging in __TEXT, since the server process launches once and then stays resident. Servers are also typically not soft-realtime systems (which prefer not hitting pageins for obvious reasons), unlike audio processing, head tracking for AR/VR, or to a lesser extent animation and event processing.

Something we've discussed in the past is some sort of annotation for "always inline this unless the caller is a cold path", which would avoid some of the unnecessary code size issues while still allowing aggressive inlining where it's relevant.

3 Likes
9

This cuts both ways. It can discourage inlining to avoid pushing related code too far away, but it can also encourage inlining to avoid generating a trampoline.

3 Likes