Jason has posted 2 posts at DZone. View Full User Profile

Load Balance Your Web Application Servers

02.06.2009
| 20126 views |
  • submit to reddit

I began my adventure with the need to ensure a lot of uptime for a particular web service that was running on a tomcat server. I investigated a lot of different avenues and, eventually, ended up on a colleagues blog.

Most of the information on the web about clustering tomcat servers tells you that you need to use the mod_jk apache module. This is not true and, in some cases, not what you really want. One such example can be found at JDBC Tutorial, which isn't a bad way to go. I started doing that and then a colleague of mine directed me to one of his blog entries which is mainly geared toward clustering Glassfish servers but the concept and the process for configuring Apache is the same.

There are a couple of things that he left out, however.

1. You must make sure that you load the proxy load balancer. For instance, in your httpd.conf file there is a section where the apache modules are loaded. Make sure you have a LoadModule for the load balancer:

LoadModule proxy_balancer_module /modules/mod_proxy_balancer.so


2. When you are configuring your cluster as in Manfred's article, consider where you are putting these rules. They have to be included in the startup. So, either in the httpd.conf or maybe a rewrite rules file.

3. The and configurations in Manfred's article contain a couple of mistakes (sorry Manfred). The "Order" parameter does not allow spaces after the comma. So, then the configuration becomes:

Order deny,allow
Allow from all

Order allow,deny
Allow from all


4. Pay attention to what you have put in your tag. This is the context you will use to access the load balancer management console. More on that later....

5. The balancer is defined in the tag. This is referenced in the ProxyPass parameter. Also in the ProxyPass parameter is the context of the application that needs to be load balanced. In Manfred's example, /clusterjsp is the context of the application that is to be load balanced. There is a bunch of information about how to do different configurations for load balancer. A decent place to start for more information is here.

6. You define the nodes in your cluster in the section. The nodes are defined by the BalancerMember entries. These directives take the same parameters as the ProxyPass directive. So you can assign the route and the redirect path for your nodes to take. For instance:

BalancerMember http://localhost:8080 route=myroute1 redirect=myroute2
BalancerMember http://localhost:8081 route=myroute2 redirect=myroute3
BalancerMember http://localhost:8082 route=myroute3 redirect=myroute1



So, if any of your nodes fails, you can specify which route to take. Or, you can leave them off and let apache decide. This is sort of handy, though, nonetheless.

7. Once you have this all set up and working (you have restarted your apache server), the next thing to do is to take a look at the load balancer console. If you have set this up on your local machine, and we are still working from Manfred's example, the URL to this console would be this:

http://localhost/balancer-manager


This will load a page where you can enable/disable nodes in the cluster and set the failover node for a particular node. Click the link for one of the nodes. A new section of the page opens to allow you to enter information about the load factor (how much load should that node get), the route (can be anything you want), and the route redirect (which node to go to if this node becomes unavailable). The route redirect is the name of the node. So, if you call your first node "node1", then the other nodes can refer to this node as "node1" in the redirect route box.

Also on this page are statistics about errors, usage, status of a particular node. Some useful information about the health of your cluster.

Finally, if you want to disable a particular node (for some maintenance) you can do so from the load balancer console.

There is lots more stuff that you can do with load balancer. If you are the least bit interested, make sure you read the Apache documentation.

Enjoy!

References
Published at DZone with permission of its author, Jason Kilgrow. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Amir Laher replied on Mon, 2009/02/09 - 5:21am

Hi Jason, good article.

One other thing to think about, is to prevent the Apache proxy itself becoming a single point of failure. You can pool 2 servers using IP-address-level failover, using an open source tool called Linux-HA ( http://www.linux-ha.org/ ). The sotware runs on both servers, which then poll each other [plus the gateway], to check if the primary one has 'gone down'. In the event of the primary server going down, the secondary server takes on the IP address and starts up Apache to handle the traffic. You obviously need to keep the config in sync between the 2 servers, and it all works like magic.

We used it to pool an IP address across 2 instances of Apache+mod_proxy_balancer, thereby removing a single point of failure. Worth a look!

Cheers,

Amir

Jason Kilgrow replied on Mon, 2009/02/09 - 9:06am

Great idea, Amir! Thanks!

 

In fact, we are doing something like that in our production environment. However, I am now experiencing a new problem to tackle.

 

It seems that the heartbeat in the Apache load-balancer does not do a very good job of detecting when an app server goes comatose. There have been several times when my tomcat server was not responsive but I was still able to ping the ip address. I believe this is the heartbeat mechanism that load-balancer uses. Be careful!

 

Next up in my blog (http://developmentech.wordpress.com): how to secure your Apache load-balancer from outside usage.

 

Stay tuned and thanks for reading!

Raghavan Alias ... replied on Mon, 2009/02/16 - 1:56am

Thanks for the excellent write-up!

 

Cheers,

Raghavan alias Saravanan M.

Alexander Schwartz replied on Fri, 2009/02/20 - 2:48pm in response to: Jason Kilgrow

Hi,

maybe you want to have a look at the "ping" parameter. This might detect some if not all problems with hung tomcats. 

Alexander.

Frank Moreland replied on Tue, 2009/05/05 - 11:17am

Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4 /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} I agree with Amir and jkilgrow, I've learned the hard, distributing traffic with a dedicated purpose build hardware load balancer is just better, especially if you're looking for load balancing for performance and high availability, I can’t say I particularly like the APJ connector with mod_proxy its CPING approach to determine Tomcat health isn’t very intelligent, that combined with mod_proxy_balancer’s limitation to only three scheduling algorithms just aren't good enough.

You could always fix these short falls by using the HTTP connector instead of APJ and then use a dedicated hardware load balancer before the Apaches and also the Tomcats, you can still get all the session stickyness plus you can now load balance based on resource usage on the Tomcat servers.  There are a few good options at the value tier for purpose built load balancers, they really aren’t all that expensive especially for piece of mind, performance and site availability.  I personally like LoadMaster, cheap and good!  Plus the have these application delivery controller features that you can leverage for the facade.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.