Tune Code Before Your Garbage Collector

It's like caching, in kind but not in type. Once you add it, people will stop trying to be parsimonious with resources and just reach for the cache every time. They'll just lean into it. In a hot minute you will discover you can't turn it off because people have lost their brains and the data flow of the app is through the cache and not through the call tree.

If you tune for allocation patterns that are in the code, then you are cementing those as continuing in perpetuity. Better to cut the fat first, so that you can tune for the necessary complexity instead of the accidental. That will be self-correcting because any new misuses will be taxed with higher performance regressions.

In this post I look at a simple event to response latency benchmark, MarketDataSnapshot to NewOrderSingle at 50K/s for 30 minutes using JLBH to test Chronicle-FIX. The goal is to compare a system which is doing redundant work (in this case logging each message using SLF4J), compared with not logging (Chronicle-FIX records every message internally using Chronicle Queue) and how this changes the choice of Garbage Collector

Reading this gives me considerable pause - I can’t think of many classes within the codebase I work on that don’t have @Slf4j at the top…

Since there wasn’t a link to the source code in that post, can you help me understand this - for the SLF4J baseline is your logger impl a console appender, a file appender, or a network service like an OTel collector? Does any of that matter for GC context?

Garbage collector?

To quote Bjarne Stroustrup:

> I don't like garbage. I don't like littering. My ideal is to eliminate the need for a garbage collector by not producing any garbage. That is now possible.

I worked on Java code at AWS for a few years and nobody tried to optimize allocations. Then I changed jobs and started working on a Java MPP database and my first code review was brutal. You were expected to avoid allocations as much as possible (mostly by using the SoA pattern everywhere). At that scale no GC could save you from excessive allocations.

Reading this gives me considerable pause - I can’t think of many classes within the codebase I work on that don’t have @Slf4j at the top…

All common logging backends create a LogEvent or similar object for each logging call, and logging calls also typically construct new strings, which usually means a new StringBuilder object, its internal array (multiple ones if it grows), the final array it is copied to, and the String object that wraps that array.

These are typically short-lived objects and therefore cheap. Nevertheless, continually creating many such objects increases GC pressure, in particular if the logging happens in code that doesn't otherwise create many objects.

Garbage collector?

To quote Bjarne Stroustrup:

> I don't like garbage. I don't like littering. My ideal is to eliminate the need for a garbage collector by not producing any garbage. That is now possible.

Everything Bjarne Stroustrup has ever produced is garbage. No single product has harmed the field of computing more then C++.

Cheap, not free, and even pretty simple to accidentally fool the GC on the lifetime of these objects.

Consider, for example, if you have a log message like this

    logger.info("Hello {}", myOldObject);

if "myOldObject" is large enough or contains references to large things or has just been around for a while, it may be a part of OldGen at this point. And if that's the case, the LogEvent objects will end up automatically promoted to OldGen. Meaning the only time those can be be claimed is in an expensive major collection. The end result is that these things will ultimately fill up old gen and trigger more of the expensive old gen collections.

That's why it can be faster in some circumstances to write the more wordy

    if (logger.isInfoEnabled()) {
      logger.info("Hello {}", myOldObject.toString());
    }

Nothing saves you, however, if your string being logged is too long. It can be autopromoted to old gen if you are trying to log a 10mb string.

> All common logging backends create a LogEvent or similar object for each logging call, and logging calls also typically construct new strings, which usually means a new StringBuilder object, its internal array (multiple ones if it grows), the final array it is copied to, and the String object that wraps that array.

Which then gets discarded because that was a Log.verbose and your minimum log level in production is WARN.

Which is why many libraries have moved towards making your log message returned by a lambda. One constant lambda allocation (so, not a lot, an invokedynamic is absolutely fuck all.) that allows you to straight up skip allocating a full string that most likely is interpolating things and attempting to reach for context present on other threads is strictly better in 99.9% of the cases. The GC pressure is kept minimal and most importantly, constant.

I believe the whole string vs stringbuffer that later was made redundant by compiler contributed to that vision.

People started dismissing allocation discipline as a thing from the past because "that thing was solved a lot ago and the compiler now is smart enough".

Well, for string, yes, but not for arbitrary objects.

Cheap, not free, and even pretty simple to accidentally fool the GC on the lifetime of these objects.

Consider, for example, if you have a log message like this

    logger.info("Hello {}", myOldObject);

That's why it can be faster in some circumstances to write the more wordy

    if (logger.isInfoEnabled()) {
      logger.info("Hello {}", myOldObject.toString());
    }

Nothing saves you, however, if your string being logged is too long. It can be autopromoted to old gen if you are trying to log a 10mb string.

Which then gets discarded because that was a Log.verbose and your minimum log level in production is WARN.

I believe the whole string vs stringbuffer that later was made redundant by compiler contributed to that vision.

People started dismissing allocation discipline as a thing from the past because "that thing was solved a lot ago and the compiler now is smart enough".

Well, for string, yes, but not for arbitrary objects.

The most surprising allocation pressure I constantly run into is primitive boxing.

The JVM does heroics to try and avoid it as much as possible, but when you end up with some primitive boxing in a hotspot the amount of GC pressure that creates can be unreal.

The most surprising allocation pressure I constantly run into is primitive boxing.

The JVM does heroics to try and avoid it as much as possible, but when you end up with some primitive boxing in a hotspot the amount of GC pressure that creates can be unreal.

Yep, and sometimes just a small code change can flip on boxing.

Why You Should Tune Code Before Your Garbage Collector

Optimising your memory allocations in Java could make far more difference than your choice of Garbage Collector and may even change which is the best garbage collector.

For the p99 (worst 1 in 100) the choice of Garbage Collector makes a different on par with optimising how loggin is done

However, for the p99.99 (worst 1 in 10,000) optimsing how the logging is done is orders of magnitude more signifciant than the choice of Garbage Collector

Unoptimised Benchmark

This takes the optimised benchmark and adds one SLF4J log line of just the message to be sent. One log line might not sound like much but ding this on every message makes a big difference esp when the rest of the code is written for low latency.

Table 1. RTT latency distribution with slf4j logging
GC option	p99	p99.99
Parallel large Eden, no large pages	16.86	20,480
ZGC, no large pages	12.21	19,694
G1 + COH, no large pages	12.02	20,349
G1, 2 MiB large pages	13.94	20,021
Shenandoah generational, no large pages	12.30	19,235

Note	The p99.99 are thousands of microseconds or 19 to 20 milliseconds.

The p99.99 or worst 1 in 10,000 might sound rare, however at 50K/s that is 5 times per second or 300 times per minute

Based on these results, you might conclude that Shenandoah is a good option, and avoid Parallel GC.

How much difference is the disk IO making

IO is often a significant proportion of delays, and we can see that just moving where the logs are written. In this case to a tmpfs filesystem.

Table 2. RTT latency distribution with slf4j logging to /dev/shm
GC option	p99	p99.99
Parallel large Eden, no large pages	34.50	10,994
ZGC, no large pages	10.86	9,617
G1 + COH, no large pages	10.90	10,043
G1, 2 MiB large pages	10.64	11,682
Shenandoah generational, no large pages	10.32	9,552

While the p99 for parallel GC is possibly an outlier, you still might favour Shenandoah, however just moving where the log is being written makes far more difference.

Note	The p99.99 are thousands of microseconds or 9 to 10 milliseconds.

Remove the redundant logging

Chronicle-FIX already records every message using Chronicle-FIX so the slf4j log is redundant. For low latency coding we use Chronicle-Queue for all recording and almost no logging as it can be the biggest source of delays otherwise. As a policy, we don’t have info level logging after startup. ie. either its an error/warning and on by default, or a debug logging and off by default. ie. make a decision as to whether it’s really needed or not.

Table 3. RTT latency distribution with only Chronicle-Queue recording
GC option	p99	p99.99
Parallel large Eden, no large pages	7.14	8.94
ZGC, no large pages	7.82	9.36
G1 + COH, no large pages	8.30	12.4
G1, 2 MiB large pages	8.59	154.4
Shenandoah generational, no large pages	7.83	233.7

As you would expect, the p99 has dropped about 2 microseconds, however what is significant is that the p99.99 has dropped by three orders of magnitude. Also, the preferred garbage collector has flipped due to the change in work load, all for the sake of one log line. In this case, parallel GC looks best, and Shenandoah came out worst on the p99.99.

Note	The p99.99 are microseconds or more than a 1000x faster than logging to /dev/shm

Conclusion

The choice of the best Garbage Collector and how to tune it depends on your workload. It makes sense to ensure your workload has been optimised first because a) it can make more difference in some cases, b) it can change your preference as well.

Asking multiple AI to optimise the same code

As different AIs are implemented differently, they don't all provide the same answer, nor do they consistently outperform one another. The best approach is to use multiple AI and pick the one you like best. My goal here is not to declare a winner based on one example, but instead to show the variety of answers you can get with different AI. I asked each AI to Suggest how to implement this more optimally private static String formatOffset(int millis) { String sign = millis < 0 ? "-" : "+"; int saveSecs = Math.abs(millis) / 1000; int hours = saveSecs / 3600; int mins = ((saveSecs / 60) % 60); int secs = (saveSecs % 60); if (secs == 0) { if (mins == 0) { return sign + twoDigitString(hours); } return sign + twoDigitString(hours) + twoDigitString(mins); } return sign + twoDigitString(hours) + twoDigitString(mins) + twoDigitString(secs); } private static String twoDigitString(int value) { ...

Demystifying Java Object Sizes: Compact Headers, Compressed Oops, and Beyond

Introduction Measuring an object’s size in Java is not straightforward. The platform encourages you to consider references and abstractions rather than raw memory usage. Still, understanding how objects fit into memory can yield significant benefits, especially for high-performance, low-latency systems. Over time, the JVM has introduced optimisations like Compressed Ordinary Object Pointers (Compressed Oops) and, more recently, Compact Object Headers. Each of these can influence how large or small your objects appear. Understanding these factors helps you reason about memory usage more concretely. Measuring Object Sizes In principle, you can estimate an object’s size by creating instances and observing changes in the JVM’s free memory. However, you must neutralise certain factors to get consistent results. For example, turning off TLAB allocation ( -XX:-UseTLAB ) makes memory usage more directly observable. Repeated measurements and median calculations can reduce the im...

Updated Biography

Peter Lawrey is an Australian/British software engineer and entrepreneur best known for work on ultra-low-latency Java systems and for leading the open-source OpenHFT libraries. He is the founder and chief executive of Chronicle Software, a London-based company whose technology is used in trading and market-infrastructure workloads. Lawrey is also a recognised Java community figure: he was named a Java Champion in 2015, has been described by conference organisers as having provided the most answers for the Java and JVM tags on Stack Overflow, and writes the long-running Vanilla Java blog. ( Chronicle Software , javachampions.org , qconnewyork.com , blog.vanillajava.blog ) Career Lawrey founded and leads Chronicle Software, which builds enabling technology for event-driven trading and market-data platforms. The company states that its software underpins systems at several tier-one banks; a 2024 press announcement similarly described Chronicle as supplying "8 of the t...

Hacker Times