Nicolas Frankel is an IT consultant with 10 years experience in Java / JEE environments. He likes his job so much he writes technical articles on his blog and reviews technical books in his spare time. He also tries to find other geeks like him in universities, as a part-time lecturer. Nicolas is a DZone MVB and is not an employee of DZone and has posted 218 posts at DZone. You can read more from them at their website. View Full User Profile

Clustering Tomcat

02.24.2010
| 18969 views |
  • submit to reddit

In this article, I will show you how to use Apache/Tomcat in order to set up a load balancer. I know this has been done a zillion time before, but I will use this setup in my next article (teaser, teaser) so at least I will have it documented somewhere.

Apache Tomcat is the reference JSP/container since its inception. Despite a lack of full JEE support, it certainly has its appeal. The reasons behind using a full-featured commercial JEE application server are not always technical ones. With lightweight frameworks such as Spring being mainstream, it is not unusual to think using Tomcat in a production environment. Some companies did it even before that.

When thinking production, one usually think reliability and scalability. Luckily, both can be attained with Apache/Tomcat through the set up of a load-balancing cluster. Reliability is thus addressed so that if a Tomcat fails, following requests can be directed to a working Tomcat. Requests are dispatched to each Tomcat according to a predefined strategy. If the need be, more Tomcat can be added at will in order to scale.

In the following example, I will set up the simplest clustering topology possible: an Apache front-end that balances 2 Tomcat instance on the same physical machine.

Set up Apache

The first step is to configure Apache to forward your requests to Tomcat. There are basically 2 options in order to do this (I ruled out the pre-shipped load-balancer webapp):

  • use mod_jk, the classic Apache/Tomcat module
  • use mod_proxy, another Apache module

I’m not a system engineer, so I can’t decide on facts whether to use one or the other: I will use mod_jk since I’ve already used it before.

  • Download the mod_jk that is adapted to your Apache and Tomcat versions
  • Put it in the ‘modules’ folder of your Apache installation
  • Update your httpd.conf configuration to load it with Apache
    LoadModule jk_module modules/mod_jk-1.2.28-httpd-2.2.3.so
  • Configure Apache. Put these directive in the httpd.conf:
    JkWorkersFile	conf/worker.properties
    JkShmFile	logs/mod_jk.shm
    JkLogLevel	info
    JkLogFile logs/mod_jk.log
    JkMount		/servlets-examples/* lb

This configuration example is minimal but needs some comments:

Parameter Description
JkWorkersFile Where to look for the module configuration file (see below)
JkShmFile Where to put the shared memory file
JkLogLevel Module log level (debug/error/info)
JkLogFile Log file location. It is the default but declaring it avoid the Apache warning
JkMount Which url pattern will be forwarded to which worker

Since mod_jk can be used in non-clustered setups, there could be any JkMount, each forwarding to its own worker (see below). In our case, it means any request beginning with /servlets-examples/ (the trailing slash is needed) will be forwarded to the ‘lb’ worker .

Configure the workers

Workers are destination routes as viewed by Apache. They’re are referenced by an unique label in the httpd.conf and parameteirzed under the same label in the worker.properties file.
My workers.properties is the following:

worker.list=lb

worker.worker1.port=8010
worker.worker1.host=localhost
worker.worker1.type=ajp13

worker.worker2.port=8011
worker.worker2.host=localhost
worker.worker2.type=ajp13

worker.lb.type=lb
worker.lb.balance_workers=worker1,worker2

I define 3 workers in this file: lb, worker1 and worker2. The ‘lb’ worker is the load-balancing worker: it is virtual and it balances the latter two. Both are configured to point to a real Tomcat instance.

Now, with the Apache configuration in mind, we see that requests beginning with /servlets-examples/ will be managed by the load balancer worker which will in turn forward to a random worker.

Note: one can also put weight on workers hosted by more powerful machines so that these are more heavily loaded than less powerful ones. In our case, both are hosted on the same machine so it has no importance whatsoever.

Configure the Tomcat instances

The last step consist of the configuration of Tomcat instances. In order to do so, I shamelessly copied entire Tomcat installations (I’m on Windows). While editing the server.xml of the Tomcat instances, three points are worth mentioning:

  • The Engine tag has a jvmRoute attribute. It’s value should be the same as the worker’s name used in both httpd.conf and worker.properties. Otherwise, sessions will be recreated for each request
  • Look out for duplicated port numbers if all Tomcat instances are on the same machine. For example, use an incremental rule to configure every stream on a different port
  • Be sure that the tcpListenPort attribute of the Receiver is unique across all Tomcat instances

Use it!

With the previous set up, one can now start both Tomcat and Apache, then browse to the servlet-examples webapp, and more precisely to the Session page. Look there for Tomcat 5.5 and there for Tomcat 6. The servlet-example page page displays the associated session ID:

ID de Session: 324DAD12976045D197435033A67C025D.worker2
Crée le: Tue Feb 23 23:15:13 CET 2010
Dernier accès: Tue Feb 23 23:31:47 CET 2010

Notice that on my Tomcat instance, the worker’s name is part of the session ID.

If everything went fine, two interesting things should take place: first, when refreshing the page, the session ID should not change because of the sticky session (enabled by default). Morevoer, if I shutdown the Tomcat instance associated with the worker (the second in my case), and if I try to refresh the page, I still can access my application, but under a new session.

Thus, I lose all the information I stored under my session! In my following article, I will study how on can try to remedy to this.

To go further:

 

From  http://blog.frankel.ch

Published at DZone with permission of Nicolas Frankel, author and DZone MVB.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Nirav Purohit replied on Wed, 2010/02/24 - 11:29am

Hi Nicolas,

 Thanks for such an interesting article. Although I have been working with Tomcat since long time, I haven't explored this interesting perspective of it. Waiting for your next article...

Martin Grotzke replied on Sat, 2010/02/27 - 3:54am

If you're interested in tomcat clustering / session failover you might be interested in the memcached-session-manager.

I created this project for highly scalable tomcat cluster which is not limited by database scalability (like tomcat's own PersistentManager) and which does not do an all-to-all session replication like tomcat's DeltaManager (which is only applicable to smaller clusters).

I'd be happy if you find this useful.

Cheers, Martin

Nicolas Frankel replied on Sun, 2010/02/28 - 9:49am in response to: Martin Grotzke

Oh yes, I saw about your product when you followed me on Tweeter. Since my teaser post will be about sessions replication across cluster nodes, I intended to cite your product.

Martin Grotzke replied on Wed, 2010/03/03 - 3:15am

Cool, looking forward to your next post :-)

Btw, are going to use backup or delta replication? (Asuming that you're choosing tomcat 6 for your study)

Good work, cheers, Martin

Joseph Schmidt replied on Wed, 2010/03/03 - 5:35am

What I miss in all articles is how to cluster Tomcat without Apache HTTPD, i.e. to cluster it with a simple and pure Java solution.

Martin Grotzke replied on Wed, 2010/03/03 - 8:01am in response to: Joseph Schmidt

What exactly do you have in mind?

In either case you need some kind of loadbalancer that distributes incoming requests on one ip-address to the available tomcat instances either using sticky or non-sticky session. The other challenge is how sessions are distributed/replicated between the different tomcat instances so that sessions are available in more than one tomcat instance (session failover).

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.