Java Performance Tuning, Profiling, and Memory Management

Java application performance is an abstract word until you face its real implications. It may vary depending on your interpretation of the word 'performance'. This article is meant to give the developer a perspective of the various aspects of the JVM internals, the controls and switches that can be altered to optimal effects that suit your application. There is no single size that can fits all. You need to customize to suit your application.

You may be facing one of the issues listed below:

1. The dreaded java.lang.OutOfMemory Error
2. Your application is literally crawling.

Before we take the plunge into solving the issues, we first need to understand some of the theory behind the issues.

Theory

What does the JVM do? The Java Virtual Machine has two primary jobs:

1. Executes Code
2. Manages Memory  
   This includes allocating memory from  the OS, managing Java allocation including heap compaction,
   and removal of garbaged objects 

Besides the above, the JVM also does stuff like managing monitors.

Very Basic Java Theory

An object is created in the heap and is garbage-collected after there are no more references to it. Objects cannot be reclaimed or freed by explicit language directives. Objects become garbage when they’re no longer reachable from the root set (e.g static objects)

Objects inside the blue square are reachable from the thread root set, while objects outside the square (in red) are not.

The sequence of the garbage collection process is as follows:

Infant mortality in Java

Most of the objects (80%) in a typical Java application die young. But this may not be  true for your application. Hence there is a need to figure out this rough infant mortality number so that you can tune the JVM accordingly.

JVM flavors

• classic : disables the Hotspot JIT compiler.
• client (default): activates the Hotspot JIT for "client" applications.
• server: activates the "server" Hotspot JIT: it requires a fair amount of time to warm up, but delivers best performance for server.

• JVM begins by interpreting all code, but it monitors the HotSpot
• Fires off a background thread that compiles hotspot bytecode to native code
• Hotspot JVM is only compiling and optimizing the "hot spot". Hotspot JVM has more time than a traditional JIT to perform optimizations
• The Hotspot JVM keeps the old bytecodes around in case a method moves out of the hot spot.

Java Garbage Collector

The following describes what the Java Garbage Collector does. 

Sun Classic (1.1 JVM) ...for historical reasons

• Mark, Sweep & Compact
    Mark: identify garbage
    Sweep: Find garbage on heap, de-allocate it
    Compact: collect all empty memory together
• Eligibility for garbage collection is determined by walking across memory, determining reachability and then compacting the heap
• Compaction is just copying the live objects so that they’re adjacent in memory
• there’s one large, contiguous block of free memory
• The main problem with classic mark, sweep and compact is that all other threads have to be suspended while the garbage collector runs
• Pause time is proportional to the number of objects on the heap

• Sun improved memory management in the Java 2 VMs by switching to a generational garbage collection scheme.
• The JavaHeap is separated into two regions(we will exclude the Permanent Generation for the time being):
   New Objects
   Old Objects
• The New Object Regions is subdivided into three smaller regions:
   1. Eden , where objects are allocated
   2. Survivor semi-spaces:  From and To
•  The Eden area is set up like a stack - an object allocation is implemented as a pointer increment. When the Eden area is full, the GC does a reachability test and then copies all the live objects from Eden to the To region.
•  The labels on the regions are swapped
•  To becomes  From - now the  From area has objects.

JVM Heap:

Java Heap is divided into 3 generations: Young(Eden), Old(Tenured), and Permanent.

The diagram below shows how objects get created in New generation and then move to survivor Spaces at every GC run, and if they survive for long to be considered old, they get moved to the Tenured generation. The number of times an object need to survive GC cycles to be considered old enough can be configured. 

By default, Java has 2 separate  threads for GC, one each for young(minor GC) and old generation(major GC). The minor GC (smaller pause, but more frequent) occurs to clean up garbage in the young generation, while  the major GC (larger pause, but less frequent) cleans up the  garbage  in the old  generation. If the major GC too fails to free required memory, the JVM increases the current memory to help create new object. This whole  cycle  can go on till the current memory reaches the MaxMemory for the JVM (default is 64MB for client JVM), after which JVM throws OutOfMemory Error.