Low Latency

The vast majority of Java applications do not need to be highly optimized, but there are certain situations where you need to squeeze every last ounce of performance out of Java. There are a number of techniques for doing this, but they place significant restrictions on the code you can write.

Garbage Collection

The garbage collector is a fundamental features of the JVM. It is incredibly useful, but is also a black box and without careful tuning it can trigger the nightmare 'stop-the-world' garbage collection.

When optimizing a Java application for low latency, it is import to make every effort to minimise garbage collection.

Allocate Objects At Startup.

So our first optimization technique is to allocate the objects you need for your service into heap memory at startup. Garbage collection locates objects which are no longer referenced and deallocates them from heap memory. By allocating all objects during startup and maintaining strong references to them, the garbage collector can not collect them!

Object Pooling

The objects should be stored in a pool. When a new object is needed, an existing one is retrieved from the pre-allocated pool of objects. When that object is no longer needed, it is returned to the pool. This is a very unusual technique in Java, as the garbage collector is (usually) fundamental to Java development.

// During start up we create and populate the pool.
ObjectPool<T> pool = new ObjectPool<>();
for (int i=0; i<1000000; i++) {
   pool.add(...);
}

// When we need an object, we do NOT create a new one, we get it from the pool
T instance = pool.next();

// When finished with the object, we must release it back to the pool for re-use
// We must not allow it to be collected
pool.release(instance);

There is one other important aspect to this technique. The pre-allocated objects must be mutable. Working with primitive and enum fields is straight forward, but there will likely need to be an object pool for every other object type used!

This technique is not for the faint-hearted, and should only be necessary where data is being handled at exceptionally high volume (e.g. FX price updates).