When is inlining worth it?

Question

Modern compilers often inline functions when they decide it is worth it.

But here comes my question: how does one define if it is optimal to inline function at given time, or more important how to decide that it should be avoided?

Could you give a bit more context to this question? Are you writing a complier? JIT optimizer? Trying to decide the settings for using someone else's compiler or JIT? What language are you dealing with? — , Dec 01 '14 at 23:34
I'm just getting into language design lately, and questions like those pop into my head... At this time I had compiled languages in mind... I think inline is rather equivalent no matter the language used, so I guess there have to be some common parameters defining when to, and when not to inline... — zduny, Dec 01 '14 at 23:37
As its currently written, this is a rather broad question. There are books and research papers written on inline optimizations. Many of those are specific to a given language or structure. Without more context and a bit more research on your part, this question would be a bit too vague to really answer well. — , Dec 01 '14 at 23:42

score 8 · Accepted Answer · edited Dec 02 '14 at 17:25

8

It's a trade off between pushing the parameters and registers on the stack and a jump plus the jump back and restoring it all vs. code bloat.

Another factor is the possibility of other optimization available on the inlined code. Particularly when some of the parameters are known.

For example inlining printf when called with a static string will let the compiler eliminate parsing the string at runtime and emit the formatting code directly. In other words printf("%d", x); can get optimized to a itoa and a write.

edited Dec 02 '14 at 17:25

answered Dec 01 '14 at 23:41

ratchet freak

25,876

Could you also explain why "code bloat" is undesirable? – zduny Dec 01 '14 at 23:46
4

bloat means a larger executable which means more page faults as new code has to get loaded before executing, and more memory needing to be dedicated to the program – ratchet freak Dec 01 '14 at 23:50
7

I once had a significant performance decrease in a cryptographic hashfunction due to the increased code size when unrolling a loop, probably because it didn't fit into a CPU cache. CPU caches, especially the faster ones, are quite small compared with RAM. – CodesInChaos Dec 02 '14 at 13:49

Lutz Prechelt · Answer 2 · 2014-12-04T09:46:12.333

4

Unless the call is on a really really hot path (executed gazillions of times), the most relevant aspect is code size: If it shrinks, inline. If it grows, don't. Unless your routine is really small, non-inlined code may be more cache-friendly and thus faster for routines called in several places.
On the really hot path, the man issue will tend to be execution time: time no longer required for pushing arguments and return address, branching, retrieving result, returning. The most important aspect may be the improved branching prediction.

edited Dec 04 '14 at 09:46

answered Dec 02 '14 at 15:46

Lutz Prechelt

1,493

Regarding your second point: execution time can increase with code size due to inlining because of page faults. – Dec 02 '14 at 15:58

score 2 · Answer 3 · answered Dec 04 '14 at 11:17

It is also likely to help when arguments are constants, as you can often eliminate whole code paths. Simple example:

function foo(b:Bool) {
  if (b) {
    lots;
    of;
    code;
  }
  else {
    and;
    even;
    more;
    code;
  }
}

If b is a constant, that helps. Equally small int arguments that are loop bounds may allow for loop unrolling in the inlined body, and what not.

Furthermore, inlined functions make escape analysis easier, which can then again help you to avoid allocation.

The reduction of invocation cost is usually insignificant. The actual advantage that comes from function inlining are the subsequent optimization opportunities that may arise when you get rid of the indirection and can combine the information of the call site and the callee.

If any significant performance can be achieved by saving invocation cost, it is most likely achieved for JITs when they are able to inline virtual methods, because it gets rid of all the indirection involved with the vtable lookup. While you will need to cache more code, you don't need to load the vtable.

When is inlining worth it?

3 Answers3