Terracotta's Perspective on the Java GC Problem

Caching is at the core for data management in applications, says Amit Pandey, CEO of Terracotta.  Too much time is spent tuning the size of the heap to keep application performance fast and predictable.  Garbage collection is to blame for making an application unpredictable for SLAs (Service Level Agreements).  When latency reaches unacceptable levels, a lot of work needs to be done that Terracotta sometimes gets pulled into.  They've also seen customers who, after months of development, integration testing, and functional testing, have had to spend additional months tuning and performance testing.  "In our experience, the majority of that work is driven by tuning around garbage collection.  When that heap size starts to exceed even 2GB, people start to see issues."  Along with Amit Pandey, Mike Allen, the head of product management, and Steven Harris, the Terracotta VP of Engineering, talked to DZone about the growing issues with Java garbage collection.

Industry changes like bigger applications are beginning to exponentially increase the problem of garbage collection, making the need for a solution in this space very critical as bigger and better applications will always require more memory as time marches on.  Web-based apps are getting very large and memory is getting cheaper, but garbage collection still bogs us down.  Some organizations won't buy servers with under 32GB memory, so applications that are tuned to run 2GB heaps have trouble working around that.  Another problem is that cloud and virtualized infrastructure is creating more pressure to get things into memory.  "In these infrastructures, going to disk is like committing suicide for the app.  Going to local memory is like going to your kitchen to get a cup of coffee, but going to disk is like going to the Moon to get a cup of coffee."

"In a language like Java, the developer has no control over memory, so the VM has to manage it.  Because the relationships between data in Java can be arbitrary, it becomes infinitely complex to decide when data is no longer needed.  Garbage collection is still a bit of a black art."  --Steven Harris, Terracotta VP of Engineering

                                          
                          

"A lot of Java customers are stuck on 32-bit operating systems because they can't use more than 4GB of memory in the heap.  A customer told me that GC is like a housekeeper that comes in and dusts off surfaces, but every now and again somebody arrives who knocks down your door, says 'everybody out', and locks you out of your own house while they clean.  That happens in Java.  So the customer can't live in a big house (with a large heap), they have to live in a small apartment (2GB) where someone won't come and break down their front door.  They have to own many small apartments (JVMs) rather than having all of their stuff in one place." --Mike Allen, Terracotta head of product management

                     

Azul Systems, another hotbed for Java innovation, has offered its Vega compute appliances for a long time, which had a software and hardware solution to the pauses of Java garbage collection.  More recently, they have initiated open source contributions to the Linux kernel (a reference implementation based on OpenJDK) that will mitigate garbage collection problems in Java and boost performance, scalability, and reliability.  They also unveiled the Zing Platform, which takes the algorithm Azul used to circumvent the GC issues, along with other software techniques for getting around locks, and makes them run on generic hardware.

For years, people have been able to work with these limitations in Java garbage collection, but the time is coming when it just won't be a viable option anymore.  Terracotta believes the time to move past GC issues is now.  Mike Allen says that once a more elegant solution for the problem emerges, we will enter "a new era of easy Java."